Miner signalled consensus upgrade

This is an update describing a new mechanism for hard-forking protocol upgrades that will be used after Lima.

Overview

All the hard forks have been scheduled so far. Lima hard fork is going to be the last scheduled hard fork, after that, there will be just community hard forks. It means that Aeternity will provide implementation of new features (some of which will be consensus breaking) and let miners decide whether they want to upgrade to a new consensus protocol or stay with the current one.

It is done in a form of voting where key blocks produced by the miners will include a certain signal showing whether they support the new consensus protocol. If the number of key blocks that include a predefined signal within a predefined signalling interval (between two block heights), a node will upgrade to the new consensus protocol at the end of the signalling interval.

--- [ ] --- ... --- [ ] --- ... --- [ ] --- [ ] ---
     H1              HS            HE - 1    HE

H1 is a height at which a node has protocol C1
HS is a signalling interval start (inclusive)
HE is a signalling interval end (exclusive)

The new consensus protocol must be strictly greater than C1. At height HS we assume that all the nodes willing to participate in the miner signalling are updated to the version able to signal and follow the new consensus protocol. HE - 1 is the last signalling block and fork happens (depending on the signalling outcome) at height HE. The hard fork happens per chain fork:

                            [ ] --- [ ] --- [ ] --- v5  <--- High difficulty
                             |
--- [ ] --- ... --- [ ] --- ... --- [ ] --- [ ] --- v4  <--- Low difficulty
     H1              HS            HE - 1    HE

If there was a fork within the signalling interval, there could be multiple competing chains with different signalling outcomes at HE. The decision whether the chain upgraded to the consensus protocol can be made after the chain reasonably settles (after HE).

Implementation

The work on miner signalled consensus started at the end of July/beginning of August. First, we had to identify where we tried to get a protocol version from block height. Since there were just scheduled hard forks, there was no doubt what protocol version was supposed to be effective at a certain height. However, this changed with community forks, now it’s not possible to determine which protocol is effective at a given height (after Lima), because it depends on the fork signalling outcome. Therefore, we had to refactor a lot of code that assumed a protocol version from height. The implementation was split into two parts: code refactoring and signalling outcome computation.

Code refactoring

The core of this work was mostly getting rid of the calls to aec_hard_forks:protocol_effective_at_height/1 function which returns a protocol at a given height. For example, this function had to be removed from the sync - imagine a situation where community fork may happen at height 1000 and the local node’s chain is at height 999, the sync receives a block with height 1010. The sync at this point doesn’t know what the protocol version will be at height 1010 so getting the protocol from height isn’t relevant.

The refactoring PRs: PR #2678, PR #2686, PR #2697, PR #2705, PR #2760, PR #2763, PR #2769, PR #2774, PR #2778, PR #2793, PR #2800, PR #2802, PR #2804, PR #2837, PR #2847. The result of the refactoring was that the function for getting the protocol from height can be used only when:

  • a new block is being added to the chain;
  • a new key block candidate is prepared.

Signalling outcome computation

At first we developed a mechanism that used asynchronous workers to compute the signalling outcome per chain fork. This implementation made it possible to change the configuration of fork signalling, restart the node and return the correct result per given configuration. Although, the rest of the developers decided it could be done another way when we don’t allow changes of the configuration for ae_mainnet and ae_uat so the users won’t make accidental mistakes.

The second mechanism for computing of the signalling outcome uses a separate database table where it aggregates the count of the predefined signal per key block within the signalling interval.

This part also includes changes of the configuration. For ae_mainnet and ae_uat, it’s possible to use fork_management > fork > enabled which can be set to true or false. It indicates that the node will follow the fork signalling outcome (true) or stay with the current protocol (false). If the configuration is missing, the default value is true. The other environments (different form the mainnet and testnet) may use the full configuration, for example:

enabled: true                                
signalling_start_height: 1000
signalling_end_height: 2000
# How many blocks at least must include signal to make new protocol active.
signalling_block_count: 900
# The signal value (it's part of key block header - info field).   
info_field: 1234
# The new protocol version.
version: 5

The PR that implements all of this is now in master (PR #2868). There is also a pending PR in the protocol repo (PR #385).

Conclusion

The signalling using the key block header field info is backward compatible, as the meaning of the value of such field is not under consensus.

The activation of the new consensus protocol based on miner signalling is meant to be a temporary measure during the time interval when the new protocol may be activated on the network while the node is running. Once the network reasonably settles on whether to activate the new consensus protocol, the user is meant to enforce such decision on the eventual node restart either by configuration or by upgrading the node to a new version.

Team

All the initial design was done by Luca Favatella with help of Juraj Hlista. The implementation was done by Juraj Hlista.

4 Likes

I wrote a paper about miner signalled consensus upgrade, it can be found here.

4 Likes

Thanks a lot for your contribution! I will read and feedback on this (for sure I will also have one or two questions).

Keep it up!

2 Likes