[Completed] AE maintenance Q1 2022

aeternity Foundation Grant Applications
1

Application Status

Status: Completed on 01.04.2022, approved on the 17.01.2022, submitted on the 14.01.2022
Last updated: 14.1.2022
**Submited by Dimitar Ivanov, dimitar.chain
Team: Ulf Wiger, Hans Svensson, Dincho Todorov, Sean Hinde, Craig Everett, Fabian Krol, Chandru Mullaparthi, Michal Bargowski, Dimitar Ivanov
Approved Budget (in h):
Used Budget (in h):
Planned Delivery:

Specify the funding category

Open Source Development

Application Title

Core team application Jan-Mar 2022

Applicant

We are the Aeternity core devs.

Value Application

We are developing new features, improving the current state of the node and fixing bugs. During the next quarter we plan on focusing on HyperChains. There is a lot of work there to be do be done but it is a relatively narrow field. We expect 3 or 4 people to be working on it without stepping on each other’s toes. We do not expect them to be finished in the scope of this application. We will also be working on various ongoing tasks like:

  • GUI launcher
  • RocksDB refactoring
  • aeCanary improvements
  • docker support for M1
  • Rosetta API
  • Plugins
  • Advanced State Channels
  • MultiSig
  • Sync improvements

Definition of Terms

Here is a more detailed description of the mentioned features:

  • HyperChains - this is the new protocol we will implement. It will be developed in phases and tested heavily both in private and public testnets. It will require updating some of the tools in the ecosystem.
  • GUI launcher - a native application that compiles and runs the node. This is a really cool project we expect to be able to show soon
  • RocksDB refactoring - we are using a RocksDB plugin but so far we’ve been limited only to using a small subset of its API. There is a huge effort from our side - we had to refactor significant portions of our code. We also fixed some C++ bugs in the plugin itself. Stay tuned.
  • aeCanary - our monitoring tool needs some more work so we can automatically trace accounts and some other fancy features
  • docker support for M1 ARM architecture - currently the node builds on Apple’s M1 and runs great. We should extend our docker support - we currently publish x86 docker images and those do not work in the new ARM context
  • Rosetta API - we are using this as an onboarding project for Chandru, he just started working on it.
  • Plugins - as we now have support for proper plugins, a number of ideas have come up for example- and utility plugins.
  • Advanced State Channels - Develop the concept of State Channel markets and perhaps other advanced State Channels use cases. This may involve improving the client-side experience.
  • MultiSig - Implement and test MuSig2 for ed25519; this will produce transaction signatures indistinguishable from “normal” signatures. To test and showcase this we also need rudimentary wallet/key support which means AEX-3, AEX-10 and a couple of more things.
  • Sync improvements - there is some known low hanging fruit with regard to improving peer broadcasting and sync

Status Quo

None of this is present.

Required Work

Once we have enough features implemented, we will make official releases.

Estimate

We would be working full time on this, although some of the team (Hans, Dincho and Chandu) already have other commitments and would help us depending on their availability.

Known Limitations

We do not expect finishing the HC and GUI. Given Chandru’s availability, the Rosetta API might also not be fully implemented. Plugins and State Channel markets are evolving work.

Outlook

HyperChains have been a long standing goal of ours but due to polishing of the idea, there is still a lot of work left to be done. It would be great showing some of the progress we’ve made so far.

I personally think the GUI would be a game changer in the ecosystem: it will allow node operators to interact with their nodes in radically different ways and it would present the information in a user-friendly manner.

Publishment

We publish our work in the official AE repositories.

8 Likes
2

Hope the foundation can approve it in a faster way! @lydia

1 Like
3

Many work was planned in mid-2021; it is still planned; I don’t understand what the team plan is?Unfortunately; no long-term planning; Even if the project is planned; it is not completed. we Want to see the interaction like the middleware team（praise @marco.chain ）.Instead of a rosetta api planned for a year!。Allow me to joke; hyperchain from planning to disappearing
Many projects are not that necessary; aeCanary is well developed; but are there exchanges in use?The recharge of ae still requires a lot of confirmation.
Other public chains are growing; But ae is still in the plan.The community is powerless; because development and funding are in the hands of the team. 。I hope the foundation and founders plan well; the community should not be sad again and again. @lydia @YaniUnchained @erik.chain
I hereby accept the rebuttal from the core team @dimitar.chain

4

The Application is approved by AF Board.

Thanks to the core team for continuously keeping the blockchain safe and stable. Their extraordinary work during and after the 51% attacks helped Huobi to lock 90% of the lost tokens on their exchange. We are proud to have such excellent developers. The progress on the dev tools was only possible due to the work of the core devs too. The hyperchains development had several iterations of improvement and is ongoing. Due to personal changes in the original hyperchains team and the attacks the hyperchains research had a delay last year. However, the core team is growing and getting stronger to finally deliver the expected results on time. Iris HF was done and aeCanary is already used by some exchanges. AF believes in the bright future!

5 Likes
5

Let’s look forward to the punctual release of the super chain in February! Refuelling! @YaniUnchained

2 Likes
7

It sounds very exciting. But we need to be clear that there is not much time left for us. Most people have been following Aeternity for 3 years or more. Don’t let them down. Thank you for your reply and have a nice day.

2 Likes
8

Core team progress update:

Week 1 (3 Jan - 9 Jan)

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on migration scenarios for the new database backend plugin. Time spent: 18 hrs

Dincho Todorov (@dincho.chain)

Get familiar with state channels in a bit more details. Time spent: 4hrs

Michał Bagrowski

Invesigate and fix issues around CI/CD failing, started process upf updating CI/CD tools. Time spent: 9hrs

Dimitar Ivanov (@dimitar.chain)

Dimitar participated in another brainstorming session that changed some key details of HC. The goal was to lower the amount of commitments on the parent chain. This would make HC much cheaper to operate and much more profitable. Documented the impact of this on the current concept. Identified some unresolved questions. Dimitar also provided some statistics and analytics needed in communication with some exchanges. Time spent: 32h

Hans Svensson (@hanssv.chain)

Improved error message reporting in aesophia_cli, adding the option to print errors on a single line. Clarified the signature format for ecverify/ecrecovery in Sophia.
Time spent: 3h

Sean Hinde

Investigate and fix failures in the regularly run system tests GH3806. Time spent: 31hrs

Fabian Krol

Investigate exchange exposure discrepancies in ae_canary. Time spent: 30h

Michal Bagrowski

CI/CD bug fixes, updating terraform (IAC) to latest version. TIme spent: 9hrs

Week 2 (10 Jan - 16 Jan)

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on test suites and fault tolerance for the new database backend plugin. Time spent: 40.5hrs

Hans Svensson (@hanssv.chain)

Cleanup logs during sync + small speed improvements to sync. Fixed old style repository references in our build config. Provided a first implementation of AEX-3. Implemented BIP-0039 (Mnemonics).
Time spent: 38h

Sean Hinde

Investigate and fix failures in the regularly run system tests and start investigation into sync improvements. Time spent: 40hrs

Fabian Krol

Investigate exchange exposure discrepancies and refactoring of internal services implementation in ae_canary. Time spent: 52h

Dimitar Ivanov

Dimitar documented latest and greatest HC concepts. Then he started working on HC Alpha. He did initial scoping and started working on pluggable smart contract consensus. This is a new consensus that allows running a chain governed by a Sophia smart contract. Time spent: 40h

Michal Bagrowski

Droping ubuntu 16.04 support. Time Spend: 8.5hrs

Week 3 (17 Jan - 23 Jan)

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on test suites and fault tolerance for the new database backend plugin. Time spent: 42.5 hrs

Hans Svensson (@hanssv.chain)

Started looking at MuSig2. Spent time on FATE improvements w.r.t. gas and caching reads from the contract store.
Time spent: 32h

Sean Hinde

Worked on changes to the gossip protocol with a view to implementing Beam Sync or other faster initial sync mechanism. Investigated the potential use of the Gradualizer tool to improve code quality. Time spent: 40hrs

Fabian Krol

Refactoring of internal services implementation in ae_canary. Time spent: 21h

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. Time spent: 40h

Michal Bagrowski

Release v6.4.0 Time Spend, terrafrom (IAC) update: 21hrs

Week 4 (24 Jan - 30 Jan)

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on test suites and fault tolerance for the new database backend plugin. Time spent: 41 hrs

Hans Svensson (@hanssv.chain)

Back to Multisig/MuSig2. We need to be able to do key derivation according to AEX-10, so this is implemented in a standalone repository. Also got MuSig2 working, some testing remains and also need to make a rudimentary UI.
Time spent: 29h

Sean Hinde

Worked on initial design of beam sync. First phase to sync the spine of the chain - just keyblocks. Time spent: 34:30hrs

Fabian Krol

Refactoring of internal services implementation in ae_canary. Time spent: 36h

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. Initial dummy smart contract was developed and some code hooks were provided. The initial test framework is in place. Now keyblocks have set the appropriate leader in the beneficiary field. This is not yet validated while syncing. Time spent: 44h

Michal Bagrowski

Terrafrom (IAC) update, release fixes: 18hrs

Week 5 (31 Jan - 6 Feb)

Hans Svensson (@hanssv.chain)

Finished a prototype version of musig_cli - this repository is kept private for now, until we have settled on the exact API/UI.
Time spent: 11h

Sean Hinde

Worked on beam sync. Continued with spine syncing and implemented delayed tx pool sync. Time spent: 34:50hrs

Fabian Krol

Hyperchain alpha research. Time spent: 34.5h

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. More code hooks were provided to use the smart contract. In PoW context microblocks are signed by the generation leader’s keys. Those are created and discarded on every new generation. In the HyperChain’s Alpha context both the microblocks and the keyblocks must be signed by the appropriate leader. This requires keeping the leader’s keys for the time being. A new signing module was started. It is to be used for persisted keys. Time spent: 40h

Michal Bagrowski

Hyperchains testnet preparations, upgrading terrafrom (IAC) to lates: 16hrs

Week 6 (7 Feb - 13 Feb)

Hans Svensson (@hanssv.chain)

Wrote a simplistic staking contract for HC Alpha.
Time spent: 8h

Sean Hinde

Holiday

Fabian Krol

Hyperchain alpha research. Time spent: 34h

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on test suites and fault tolerance for the new database backend plugin. Time spent: 39 hrs

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. The new key handling module was finished. The existing code had been adapted to use a module depending on if the consensus is PoW or PoS. Time spent: 46h

Michal Bagrowski

Upgrading CI/CD tools (packer): 8hrs

Week 7 (14 Feb - 20 Feb)

Sean Hinde

Worked on beam sync. Continued with spine syncing and implemented delayed tx pool sync. Time spent: 34:50hrs

Fabian Krol

Refactoring of internal services implementation in ae_canary. Time spent: 33.5h

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on test suites and fault tolerance for the new database backend plugin. Time spent: 40.5 hrs

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. All the refactoring for key management is finished. Rewards are being redistributed by the smart contract. This exposed a flaw in the initial smart contract: on every generation certain portion of the fees go the BRI account. Then it can become a staker as well. The notion of offline stakers is being developed and tested. Now every staker can declare themselves as being offline or online. Time spent: 46h

Michal Bagrowski

Upgrading CI/CD tools (packer): 4hrs

Week 8 (21 Feb - 27 Feb)

Sean Hinde

Worked on beam sync. Time spent: 22hrs

Fabian Krol

Refactoring of internal services implementation in ae_canary. Review initial implementation of HC. Time spent: 32h

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on test suites and fault tolerance for the new database backend plugin. Time spent: 41.5 hrs

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. Work went in block production: keyblocks now have the appropriate miner being set - the one defined by the smart contract. Microblocks are being signed by the appropriate miner. All of this is being heavily tested. Time spent: 47.5h

Michal Bagrowski

Hyperchains testnet preparations, adding ubuntu 20.04 support to CI/CD: 11hrs

Week 9 (28 Feb - 6 Mar)

Hans Svensson (@hanssv.chain)

Add proper conversion for old DB-format in SC Force progress transactions.
Time spent: 4h

Sean Hinde

Worked on beam sync. Investigated an approach to sync the state trees directly. This approach was abandoned as it looked to be as slow as current Tx sync. Time spent: 15:50hrs

Fabian Krol

Review initial implementation of HC. Time spent: 20.5h

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on migration scenarios for the new database structure. Time spent: 40 hrs

Chandru Mullaparthi

Worked on Rosetta API read operations. Time spent: 12h

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. Dimitar combined Hans’ smart contract with the test smart contract. This added to his contracts election, online/offline staking validators and small tweaks. Consensus code had been adapted to accomodate the new contract entrypoints. The Genesis block is now seeded with the new contracts. Stakers are created in genesis. Time spent: 43.5h

Craig Everett (@zxq9.chain)

Craig was finally able to finally able to push a v0.1 Alpha release of the native GUI tool (and subsequent bug patch 0.1.1). Time spent: 33h

Michal Bagrowski

Hyperchains testnet preparations: 4hrs

Week 10 (7 Mar - 13 Mar)

Sean Hinde

Holiday

Fabian Krol

Review initial implementation of HC. Time spent: 42.5h

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on migration scenarios for the new database structure. Time spent: 42.5 hrs

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. A few bugs were cleared. Key block are now being singed as well. Since we keep the old block&header structure, the PoW’s evidence field is being recycled. It is bigger than the singature so it is padded with 0s. Initial steps are being made in block validation. Testnet is live. Time spent: 45.5h

Craig Everett (@zxq9.chain)

(Away working on a parallel Aeternity/Sophia project at his parent company.)

Michal Bagrowski

Hyperchains testnet preparations, Hyperchains alpha release. Expired SSL issue fix: 14.5hrs

Week 11 (14 Mar - 20 Mar)

Sean Hinde

Further worked on beam sync. Started work to establish block witness records to allow syncing nodes to fetch only affected records. Time spent: 22hrs

Fabian Krol

Review initial implementation of HC. Time spent: 38.5h

Dimitar Ivanov

Dimitar was working on HC Alpha exclusively. Investigated testnet and if everything is going according to expectations. There was a problem with the config of one of the nodes and it did not sync. Leader’s are now being validated that they are correct according to the smart contract. Initial steps were made in fees’ validation. Time spent: 37.5h

Chandru Mullaparthi

Worked on Rosetta API read operations. Time spent: 12h

Craig Everett (@zxq9.chain)

(Away working on a parallel Aeternity/Sophia project at his parent company.)

Michal Bagrowski

Hyperchains testnet preparations, preparing common CI/CD tools (circleci orb) for all terraform pipelines: 5hrs

Week 12 (21 Mar - 27 Mar)

Craig Everett (@zxq9.chain)

(Away working on a parallel Aeternity/Sophia project at his parent company.)

Michal Bagrowski

Hyperchains alpha release, + æternity hyperchains alpha demo app: 8.5hrs

Fabian Krol

Hyperchains demo UI implementation. Time spent: 64h

Sean Hinde

Continued work on beam sync. Working through block witness records for basic transaction types. Time spent: 33:15

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on integrating the new rocksdb plugin into the AE node, and debugging test suites. Time spent. He also gave a presentation at FunctionalConf 2022 about Aeternity’s work on the mnesia_rocksdb backend. Time spent: 43 hrs

Week 13 (28 Mar - 31 Mar)

Fabian Krol

Hyperchains demo UI implementation. Time spent: 29h

Ulf Wiger (uwigeroferlang.chain)

Ulf worked on integrating the new rocksdb plugin into the AE node, and debugging test suites. Time spent: 32.5 hrs

Sean Hinde

Continued work on beam sync. Established block witness records for basic transaction types. Time spent: 23:10

Craig Everett (zxq9.chain)

Fixed GUI to detect and work with R23 and R24 (v0.1.3), and began work on developer features for GUI (v0.2.0 branch).
spent: 8:19

4 Likes