Developer Testnet 3

Our last Developer Testnet, Developer Testnet 2, started January 25th at 8am CST. It ended at 8:20am. What happened there?

DT2 had two releases. The first was released a few days before DT2 was scheduled to go live, and was proven stable. It’s notable not only for being stable, but also for not including peer finding. The second release was released a day before DT2 was scheduled to go live, and it did include peer finding.

The peer finding implementation was prioritized to make running a node as easy as possible. It passed all the existing tests we had and the test written to verify it. Unfortunately, when you connected a large amount of nodes together in a network, the network became too unstable to function. Nodes were shutting down on the scale of minutes.

We tried to get patches out, and wrote multiple fixes for identified bugs. Unfortunately, the problem boiled down to “Sync Mode”. Sync Mode was part of the network protocol which was designed to be simple and functional. One node said they needed data. The other node said they could provide the data. One node said they needed X, Y, and Z. The other node said here’s X and Y, but they don’t have Z.

While being simple and working for the time, Sync Mode had two flaws in it. The first is whoever started syncing became the only one who was able to request data. The second is that it interrupted normal networking traffic. These, combined with peer finding which had some level of unpredictability to an otherwise structured dialog, stopped the network from being able to exist stably.

A few hours later, after fixing some bugs but realizing the true problem, the network was officially shut down (while the seed nodes had been offline, the existing plan was to start them back up later that day). Work began on removing Sync Mode, and after multiple days of hard work, it’s gone. The new networking code is much more efficient, much cleaner, and most importantly, stable.

We have also fixed a few other issues related to Merit Removals, and added logging statements to Meros. We now feel confident about starting our next Developer Testnet, Developer Testnet 3. DT3 will launch Saturday, February 1st, at 8am CST, exactly one week after DT2 launched.

We’re very excited to have a network which will be able to exemplify what we’ve been working on for over a year now. While we are disappointed with how DT2 went, we are happy to have realized the problems with it. That was the purpose with DT2, as it is all our testnets; to find problems and fix them before mainnet.

Just like DT2, if you are interested in DT3, please join our Discord ask for the “Developer Testnet Node” role. We are very excited for these upcoming weeks and will make sure to keep you informed.