Hyperchains development updates

Hi!
My team is currently tasked with developing Hyperchains - this thread will contain periodic(probably biweekly) public updates on the progress of development.

Update #1

Hyperchain whitepaper has been released!

https://forum.aeternity.com/t/hyperchains-whitepaper/7587/7
This concluded a lot of R/D of the concept and actually defined hyperchains in an implementable form.

Initial implementation of the staking contract

Each hyperchain will be governed by a unique smart contract tasked with:

Plugable functionality in the core node


Hyperchain commitment DB

Hyperchain parent simulator

@radrow @erlmachinedev1.chain Let me know if I forgot to add something :wink: I omitted WIP tasks as amending the whitepaper for when this will be done.

5 Likes

Really liking the simple 4 step local deployment

will give it a try this weekend .

Update #2 [07 - 11 September]

Hi,
Short update on the development progress.
CC: @yani.chain @sergiimaksiuta.chain

What was done this week:

  • We finished and merged the initial Hyperchain parent DB
  • We finished and merged initial Hyperchain parent conector
  • Created a Chain Simulator parent connector
  • Sophia compiler refactoring for the necessary Erlang ACI helper library

What we are working on right now:

  • Sophia smart contract ACI helper
    While working on the Staking contract deployment in HC and running benchmarks of our Sophia compiler it turned out that it is infeasible to ship hyperchains together with the compiler for encoding/decoding call data as the compiler would introduce a nasty bottleneck in block validation times - somewhere around 2-3 seconds for validating a single header… The ACI library is the only sensible option for proceeding further and will replace the compiler in some places in the node test suite.
  • Chain simulator improvements
    The state channel test suite provided a chain simulator for testing various forking scenarios - after we started testing our Staking Contract it turned out that the only sensible way to test it is by using the chain simulator. Unfortunetelly as the Simulator was designed only for SC we are now in the progress of major refactorings to allow testing smart contract behavior during forking scenarios.
  • Connector DB integration
    Save the data obtained from the parent chain in our DB
  • Parent chain connector manager
    Manage parent chains and keep track of hard forks(changing the parent chain over the lifetime of the Hyperchain)

@radrow @erlmachinedev1.chain Did I miss something?
Keep in mind that if [Active] Aeternity node maintenance - iris hard fork release candidate doesn’t get approved then some more important tasks listed in the proposal will become part of the Hyperchains project(Fast Sync, Peer pool fix, etc…).

6 Likes

Update #2 [14 - 18 September]

Hi,
Short update on the development progress.

We are still working on tasks started last week but we made a lof of progress on them:

  • Chain Simulator improvements:
    A draft PR can be found here: https://github.com/aeternity/aeternity/pull/3358 - right now this draft PR breaks Q/A checks and needs to be polished

  • ACI helper:
    Progress can be tracked here https://github.com/aeternity/aesophia_aci_encoder
    Right now this library works and was tested against all contract tests in the core node(I’ve intercepted calls to the compiler which got me a LOT of test cases). Some polishing needs to be done before integrating it in the core node test suite and the compiler.

  • Parent chain manager and DB integration:
    We are still working on it, no draft PR yet.

@radrow @erlmachinedev1.chain Did I miss something?

4 Likes

Update #2 [21 - 25 September]

This week’s progress:

  • Parent chain manager:
    The manager is still in development but there is a draft PR:
    https://github.com/aeternity/aeternity/pull/3360
  • ACI library:
    The basic functionality required by the library was implemented - generic and quick call data encoding for Sophia contracts. The library went under extensive compatibility testing which revealed some bugs. Benchmarks show that this library is suitable for use in HC - Using the current version of the staking contract as a baseline:
    • Encoding using the compiler and the original contract: ~500 ms
    • Encoding using the compiler and an ACI stub: ~20ms
    • Encoding using the library: ~0.1ms
  • ACI library integration with the core node
    This is done, currently the PR is in the review process and changes were made to address review comments: https://github.com/aeternity/aeternity/pull/3361
  • Chain simulator improvements:
    We are still ironing out issues/bugs which cause our testing suite to fail.
  • Discovered a bug in the Sophia compiler:
    https://github.com/aeternity/aesophia/issues/281

Plans for next week:

Not all of the bellow tasks will be done next week but depending on the progress we will make the possible tasks for next week are:

  • Deploying the staking contract in the genesis block of each hyperchain
  • Possibly introducing FATE call data decoding to the ACI library/polishing error messages
  • Chain simulator improvements
  • Parent chain manager
  • Changing the block structure in the HC
  • Refactoring and changes to the mining conductor
4 Likes

Update #3 [28 September - 2 October]

This week’s progress:

  • The ACI library was integrated with the node
  • Rebar plugin was created for compiling any contracts used in hyperchains - a required part for the build system of hyperchains
  • The chain simulator got to an acceptable point - some slight polishing still needs to be done
  • The parent chain manager is looking better!

Plans for next week:

Not all of the bellow tasks will be done next week but depending on the progress we will make the possible tasks for next week are:

  • Deploying the staking contract in the genesis block of each hyperchain
  • Possibly introducing FATE call data decoding to the ACI library/polishing error messages
  • Writing extensive tests for the Sophia staking contract
  • Parent chain manager
  • Changing the block structure in the HC
  • Refactoring and changes to the mining conductor
4 Likes

Update #4 [5 October - 9 October]

This week’s progress:

Plans for next week:

  • Customizable consensus engine - the aim is to first introduce PoA consensus to the node in order to check the new engine and to possibly switch over the Aeternity testnet to PoA
  • Parent chain manager and embedding of an BTC/ETH light client for hyperchains
  • Depending on approval we might start working on improving the developer experience in the ecosystem - to improve the usability of the ecosystem and development speed of aepps.
3 Likes

So users can stake their ae tokens and receive interest?

4 Likes

Hopefully without running node/mining rig?

2 Likes

@ominous.chain Yes, There will be 2 ways to gain interest while securing the AE network:

  • Running and operating a node - when you get elected to produce blocks you will receive block rewards and transaction fees.
  • Delegating your coins to another delegate - when they become a validator then you will get a share proportional to your stake. All you will need to do is to enter a special Aepp and delegate your tokens :slight_smile:
2 Likes

Great 2 options there, thanks for the reply. Very much needed by the community IMHO. Some rewards back to token holders

1 Like

Update #5 [12 October - 16 October]

This week’s progress:

Plans for next week:

  • Customizable consensus engine
  • Chain simulator as a parent chain
  • Evaluation of ETH light clients and integration with one of them
  • Finalizing the tests for the staking contract
  • Sophia 2 Purescript transpiler
5 Likes

Update #6 [19 October - 23 October]

This week’s progress:

  • Hyperchain parent manager got merged
  • The work on the first real world parent connector has started - we are now in the process of choosing a BTC light client which will be embedded in the HC node
  • The upgraded chain simulator got merged
  • The testing suite for the staking contract is close to be merged
  • Customizable consensus layer in the node - this is still WIP and will be for some time. This also includes various sync performance optimizations - a bottleneck in the tx pool has been discovered which slows down sync.

Plans for next week:

  • BTC parent chain connector
  • Erlang to Purescript transpiler
  • Customizable consensus in the code, refactoring + performance improvements
7 Likes

Update #7 [26 October - 30 October]

This week we faced a lot of CI issues which slowed us slightly down.

This week’s progress:

Plans for next week:

  • Integration with the chosen BTC light client/implementation
  • Research towards the most cost/time efficient way to submit commitments on BTC
  • Customizable consensus engine - Consensus switching and moving the old mining mocks/instant mining under this new engine
  • Transcompilation of a subset of the erlang standard library

@erlmachinedev1.chain @radrow Did I miss something important?

5 Likes

Hei,bro when will the hyperchains lanch?

My 2 cents: I hope they will launch once they’re ready but not before that. It is a project that is modifying quite some internals so this needs small steps and heavy testing.

5 Likes

How long will it take?

Update #8 [02 November - 06 November]

@radrow @erlmachinedev1.chain

  • Sync time
    While working on the new consensus engine we removed as much bottlenecks in sync as possible - We still aren’t close to proper fast sync which is currently worked on by the AE foundation but It’s a decent improvement - After our fixes got merged the time to sync the first 20k generations on mainnet became reasonable.

  • We fixed and optimized our CI pipeline
    Currently both the testing and build times improved. By taking advantage of the rocksdb upgrade the build times went down from 10 minutes to 2 minutes. There were some issues with build caching which we fixed. Some dependencies now aren’t rebuild every time we run our tests. There is still a small issue with deploys which we will fix on monday. Related PR’s:

  • Instant mining for tests as part of the new consensus engine
    https://github.com/aeternity/aeternity/pull/3399
    This reduced build times and will soon be placed under the new consensus engine. This will later be used not only for testing but also for “dev” mode which will improve the Aepp ecosystem dramatically.

  • Consensus engine
    It’s still WIP - we hope to get it finished next week, right now we finalized user configuration and we started to write some tests for it.

  • Erlscriptem
    I think this “demo” will describe our progress:


    Currently we managed to transpile parts of the real erlang standard library and it actually works :slight_smile: @nikitafuchs.chain we aim to transpile aesophia in the next 2 weeks :wink: Right now we still need to implement some missing erlang features in Erlscriptem:

    • Exception handling
    • Complex binary pattern matching and comprehensions(right now we only handle some easy cases - we came up how to handle any kind of binary pattern matching or bitstring comprehension a long time ago but we decided to postpone the implementation after we get a working “baseline”)

    Right now Erlscriptem generates suboptimal code with regards to the final JS code size - we overestimated the ability of Purescript to optimise the Applicative typeclass - fortunately It’s trivial to change our code to generate friendly code for the Purescript optimizer :slight_smile:

  • Connecting to BTC
    We did a lot of research towards integrating a BTC light node in our software - we hope to finish the integration in the next 2 weeks :slight_smile:

5 Likes

Hi bro.I check your reply ‘ Hyperchains - a new consensus algorithm which will allow you to stake your coins and gain interest from it while securing the Hyperchain Network. ‘by tg.
So I want to know can I stake my ae coins at Hyperchains?

1 Like

When hyperchains will launch there will be 2 ways to gain interest:

  1. Delegate your coins to a delegate via a webapp - This is the simplest way, just go to a fully decentralized and trustless DApp(this is why we are building the erlps compiler), choose your given delegate and always when the delegate is chosen to process blocks you will gain interest from your stake. It’s cheaper and less profitable than 2) but the UX will be amazing :slight_smile:
  2. Become a delegate - you essentially will become a miner and will be obligated to process transactions and submit commitments to the parent chain - The cost of operating a mining rig will still need to be fully determined - you won’t need expensive hardware but you need to have the money for submitting commitments to a given parent chain. In return you will get block rewards + transaction fees.
3 Likes