Last week, seven of the eight Eth2 clients in active development achieved the major milestone of moving from single-client testnets to multi-client testnets in the “Interop Lock-in”. With this exciting success in the development of Eth2, we wanted to reflect on how we got to this point and what it means for the Ethereum network and ecosystem.
Anyone who has followed Ethereum in the last two years has probably been familiar with terms like “Ethereum 2.0”, “Eth2” or “Serenity”. Each of these refers to scheduled substantial upgrades to the Ethereum protocol that have been planned in some form since before the network went live in 2015.
In the early years of Ethereum, groundbreaking research was accomplished in parallel to the launch of the original chain (Eth1), while the massive growth of the Ethereum community that followed aided early adoption of decentralized applications. Still, the road from these early advances to a highly decentralized yet scalable proof-of-stake blockchain has been a long one. However, over the last 18 months, the research has finally stabilized into a cohesive and comprehensive vision for the next major upgrades known as Eth2.
As research progressed towards specifications towards the end of 2018, many teams (client teams) from across the community stepped forward to develop core implementations of the protocol (clients). Since then, there has been a dynamic game between the specification and the implementation. biweekly calls and a common spec repository organize communication and the exchange of ideas, but client teams have mostly worked in relative isolation, building and testing their implementations of the protocol.
While the specification was a moving target, customers could only dig into interoperability and optimizations, but once the Eth2 Phase 0 specification was considered “frozen” On July 1, 2019, customers made great progress and began to take concrete steps towards production.
interoperability
Jose Delong of pegasys had the wild idea of bringing together members of each of the customer’s engineering teams in a remote location for a week of interop work. The event was considered the “interop crash” or, as it is generally known, “interop”. With the spec freeze in the offing and DevWith on the horizon, Interop in September was an opportunity for all of these stakeholders to personally work through initial interoperability issues.
He primary purpose of the event was to have each participating customer achieve peer-to-peer interoperability with each other on small test networks — Lighthouse <-> Sagebrush, Polar Star <-> Lighthouse, Polar Star <-> Sagebrushetc.
Participating customer teams included:
Additional goals involved testing (1) larger networks in both the number of nodes and number of validators (2), (3) networks with more than 3 clients, (4) improved tools for monitoring and debugging Eth2 networks, and ( 5) other fun stuff like getting raspberry pis running and building fork visualizers.
Before the event, some goals seemed exaggerated, but the teams worked diligently to the deadline and made incredible progress. By the end of the week, the customer teams far exceeded the original expectations of having some peer-to-peer networks, instead of completing all of the peer-to-peer testing, building a small network of all 7 participating clients, and more.
The following represents a look at the highlights of customer successes, but is certainly not exhaustive:
Multi-client testnets
- The 7 participating clients achieved peer-to-peer interoperabilityand although an eighth, Swindlercould not be present, have started to work through this milestone as well.
- Many larger testnets were formed between 3+ clients, 3+ nodes, and above minimum validator count.
- the 7 clients in support were successfully run on a single network.
- The libp2p implementations of all participating languages are now interoperable after debugging some minor issues.
Network debugging and tools.
- Some consensus errors between clients were identified, debugged and logged as parts of the state transition that require further test coverage.
- Command line tools were created to better debug ssz objects and state transitions (zcli, pycliand similar tools built into the clients).
- Advances made in metrics dashboards, a branch visualizer, and other tools to better understand customers and networks
- Clients were containerized for large-scale network testing within the white block genesis platform.
And something else
- Customer teams served as the first alpha users of each other, resulting in a wide compile/run scripts and related documentation.
- Isolated load tests with Nimbus and Lighthouse drove over 2,000 validators on a single machine paired with similar full nodes over LAN.
- Multiple clients were built and tested on a small network raspberry pi.
And beyond
Interop marked a major turning point for Eth2. There is still a lot of work to be done before release, but engineering efforts will increasingly focus on testnets, optimizations, and usability, work that begins to put this software in the hands of users.
So what’s next for customer teams and eth2 development?
- Benchmarks and optimizations
- Test synchronization, stress test networks, etc.
- Public and incentivized testnets
- third party audits
- Polish the validator user experience
Finally, we owe special thanks to the ConsenSys team for helping organize, host, and provide the resources that made Interop possible.
