The problem
The problem that many users have with Ethereum (in its current form) is that whilst its value proposition is clear it is oftentimes difficult for individuals coming from a non-technical background to use.
There may be awesome products and tools being developed on the platform but that is of no use if one cannot actually use those tools.
Having been aware of Ethereum since mid 2015 I have been able to watch its development, and the influx of interest that has resulted. I felt that I should do my bit, and thus in late 2016 set out to build EthTools.com
As with any new technology I started out by learning what it does, and how it works. I noted that ‘wallets’ are pretty important, and as such went about building functionality to create or load an Ethereum wallet. My knowledge seeking adventure resulted in a primitive attempt at building an index of the Ethereum blockchain, and then the Ethereum Name Service popped up on my radar.
Having a background in what I will refer to as the ‘traditional domain name industry’ the Ethereum Name Service was an extremely interesting addition to the already promising technological potential of the Ethereum network. I wanted to build tools for interacting with the ENS. The fact that I have a degree in Economics made the theoretical (auction theory) Economics behind the ENS registrar contract even more enticing an area of study.
I built some tooling for the ENS, and released it. The service was overwhelmed with users, but unfortunately (despite my best efforts) many of them struggled to use the tools.
The most interesting aspect of Economic theory (in my opinion) has always been rationality. Many concepts assume human rationality. It became apparent to me that as well as not necessarily being rational, human beings are often times not sensical. I had created a tool for interacting with something new, and some people were using it with absolutely no understanding of how ‘it’ worked, and what it did.
“Have you revealed your bid?” I replied to one of the hundreds of incoming support requests. “What is that?” was the response.
Another expletive filled email demanded that I returned the users ‘stolen’ funds. Needless to say, no funds had been stolen (EthTools.com is an interface to the Ethereum blockchain — we never see/touch/hold your funds in any capacity).
A further email ended in me querying if the user even knew what Ethereum was. I try not to be provocative, but the way the thread was going I was genuinely not sure.
It hit me that making interacting with the Ethereum blockchain (reasonable) user friendly was going to be tough, and that making it (unreasonable) user friendly was impossible.
There is oftentimes not widespread knowledge of new technology. When there is it tends to be news reports about how ‘Ransomware destroys world’. It intrigues me what would happen if there were to be a 15 minute segment on mainstream TV where someone sought to explain Ethereum in layman's terms.
Infrastructure
To provide functionality for interacting with the Ethereum Name Service it became apparent that a well designed index of the Ethereum block chain would need to be developed. Blocks are being mined all day every day, and these blocks contain the transactions that users are executing through EthTools.com as well as other services.
When a user starts an auction or bids for a domain name on EthTools.com, they are submitting a transaction to the Ethereum block chain. By developing a well functioning index of the chain, we can build user interfaces and experiences which make interacting with the chain easier. For example when you submit a bid you want to know if you bid was successfully submitted and if the function call executed successfully without error.
There has been a whole ‘period’ of recent Internet history whereby developers have focussed their time and energy on making everything quick and pretty. Ethereum dapps (given the nature of blockchain) can not be quick and pretty. After all, when I want to bid on an ENS name what I am really doing is telling thousands of computers around the world that I want to bid on a name.
The uneducated or those with nefarious intention will exclaim that Ethereum therefore sucks. It is distressing that human nature prevents the logical comeback of “You have missed the point” from resonating.
Some tech
Developing an index of the blockchain is not easy. Depending on your settings a Geth or Parity node which fully synchronises the Ethereum chain requires many gigabytes of storage space. As the chain grows, this will obviously increase. We however need an index of the chain that is searchable. If someone wants to see information about Block 1234567, we need to be able to search for it and display it quickly.
With 4 million blocks in the chain this becomes quite a challenge. As you can imagine, searching a database of 4 million records could take quite a bit of time. Fortunately computer processors and software offerings are extremely impressive in the modern day and age. SQL based databases such as mySQL can index the block number (in this case), and allow a search of this nature to return in a matter of milliseconds.
Sadly, this is the least of our worries. In each of those blocks are.. transactions. In these initial 4 million blocks, there are over 30 million transactions. It gets worse. Each of those transactions is made up of ‘steps’ — a transaction executes a number of steps on the Ethereum Virtual Machine. There have been over 200 million internal transaction steps. It is not quite so quick to search through that many rows. It is especially slow to execute complex queries such as searching for all transactions to/from a given address.
In my initial setup, searching for all transactions to the ENS Registrar contract took 40 seconds. Not only that, it put a lot of stress on our server resources and blocked other queries from being executed. I.E. If someone was ‘exploring’ the registrar contract, no-one else could do anything on the EthTools.com website. We would like to scale to more than one concurrent user :)
The first thing that I noticed involves the mySQL query cache. When initially exploring ways of improving query speeds I tried disabling the query cache. It sped up my queries by a factor of ten. This was bemusing as one would assume that a cache would speed things up. It turns out that as I was indexing the chain (constant DB writes) I was causing my cache to be constantly invalidated. This was resulting in access issues due to serialisation of requests. More can be read about this here.
In mySQL “index values are always stored in ascending order”. What I found was that it was more common for people to be interested in more recently executed transactions. This meant that if someone was searching for transactions at the head of the chain, even using an index mySQL would have to sort the results in the reverse direction prior to looking for the results required. This made things slower than necessary. Given mySQLs index order constraints there was a somewhat strange resolution to this — we store the block number associated with a transaction as a negative number. Block 123456 is stored as -123456 such that the results of these more common queries do not have to be sorted. This also improved query times by a factor of ten.
Unfortunately as we processed more and more transactions I noticed that getting transactions to/from the ENS Registrar contract was still too slow. We managed to resolve this with a mapping table, some appropriate indexes, and database partitioning.
The mapping table maps addresses to transaction IDs independent of whether the transaction was to/from that address. We no longer need to search combined indexes. Instead we find all rows for a given address, and then load the transactions with the specific IDs that we have found. This is quick.
Partitioning allows you to behind the scenes split your database table into multiple database tables. You can split a table such that rows with ID 1–1000 are in partition 1, 1001–2000 in partition 2 etc. The benefit of that is that rather than searching the whole table mySQL can optimise knowing that it only needs to search partition A.
The result..? Well, this is the ENS Registrar contract. It should load quickly.
How interesting
When I decided to build EthTools.com I was interested in investigating new technology. Perversely I learned a lot about old technology (as well).
I have spent a lot of time trying to build useful functionality that serves a purpose. I have opted to do less, but hopefully do it better.
The functionality
So EthTools.com offers:
- A fully featured and optimised blockchain explorer.
- Wallet tools for creating and loading a wallet generated by EthTools.com as well as any wallets generated by number of other tools.
- Tooling for registering domain names with the Ethereum Name Service — starting an auction, bidding, revealing, and finalizing your bid.
- Tooling for managing domain names registered through the ENS — setting resolver contracts, setting resolution addresses, and changing ownership.
- An ENS whois service, and a log of all ENS events.
- Tooling for submitting and verifying contracts on the Ethereum blockchain.
- Tooling for interacting with any deployed and verified contract.
Contracts !
One of the killer features of Ethereum is that you can write and deploy smart contracts.
One of the killer features of EthTools.com (in my opinion) is that you can interact with any deployed and verified contract on the blockchain. That is to say we provide an interface for all verified contracts.
I think this is cool enough that I gave it its own section.
Move quickly. Break things.
I do not have a big team behind me and/or any kind of funding. I built EthTools.com because I am aware of Ethereum’s potential to change the technological landscape moving forward.
If you encounter any bugs, or have any suggestions please let me know.
When I first released our tooling for the ENS I was overwhelmed with requests for support. Given that EthTools.com is available to users free of charge I am unable to guarantee personalised support. In light of this, I have set up the EthTools.com community — a discussion forum where I will post guidance to commonly asked questions.
In addition to this I hope that other individuals interested in Ethereum and EthTools.com will join the site and join the conversation. Shared support, technological discussion, and community are pretty great.
