[Completed] AE maintenance Q2-2022

Application Status

Status: Completed on 30.06.2022, approved on the 12.04.2022, submitted on the 01.04.2022
Last updated: Submitted on the 01.04.2022
Submited by: Fabian Krol
Team: Michal Bargowski, Craig Everett, Sean Hinde, Dimitar Ivanov, Fabian Krol, Chandru Mullaparthi, Hans Svensson, Dincho Todorov,Ulf Wiger
Approved Budget (in h): -
Used Budget (in h): -
Planned Delivery: -

Specify the funding category

Open Source Development

Application Title

Core team application April-June 2022

Applicant

Aeternity core developers team.

Value Application

We are developing new features, improving the current state of the node and fixing bugs.

During the second quarter, our main task will be to continue HyperChains implementation. However, We do not expect to finish it in the scope of this application.

We will also be working on various ongoing tasks:

  • GUI launcher improvements
  • RocksDB refactoring
  • aeCanary improvements and refactoring
  • Rosetta API
  • Plugins
  • MultiSig improvements
  • Beam sync
  • Advanced State Channels
  • Garbage collection

Definition of Terms

Here is a more detailed description of the mentioned features:

  • HyperChains - a HyperChains Alpha testnet is now up and running. We have made significant progress with the support for contract-based consensus, but much work still remains. Development will progress in phases and tested heavily both in private and public testnets. It will require updating some of the tools in the ecosystem.
  • GUI launcher - a native application that compiles and runs the node. New developer features will be added depending on Craig’s availability.
  • RocksDB refactoring - we are using a RocksDB plugin but so far we’ve been limited only to using a small subset of its API. There is a huge effort from our side - we had to refactor significant portions of our code. We also fixed some C++ bugs in the plugin itself. Remaining work is mainly about refactoring the core code to take full advantage of the new possibilities. Stay tuned.
  • aeCanary - our monitoring tool needs refactoring. We’re planning to release the tainted account feature.
  • Rosetta API - ongoing work on the implementation. Delivery will depend on Chandru’s availability.
  • Plugins - as we now have support for proper plugins, a number of ideas have come up for example indexing- and utility plugins. Documentation and carefully crafted APIs are needed.
  • MultiSig - Implement and test MuSig2 for ed25519; this will produce transaction signatures indistinguishable from “normal” signatures. To test and showcase this we also need rudimentary wallet/key support which means AEX-3, AEX-10 and a couple of more things. Most of ther work is done but it needs a bit more polish and documentation.
  • Beam sync - we have started initial work on a sync option that aims to deliver a secure working new node within a few hours rather than days by first syncing keyblocks then fetching state on the fly.
  • Advanced State Channels - Develop the concept of State Channel markets and perhaps other advanced State Channels use cases. This may involve improving the client-side experience.
  • Garbage collection - Once the RocksDb refactoring is merged, chain garbage-collection can be redesigned and extended, in order to significantly reduce the on-disk footprint of garbage-collected nodes.

Status Quo

None of this is present.

Required Work

Once we have enough features implemented, we will make official releases.

Estimate

We would be working full time on this, although some of the team (Hans, Dincho and Chandru, Michal, Craig) already have other commitments and would help us depending on their availability.

Known Limitations

We do not expect finishing the HyperChains.
Chandru’s availability: the Rosetta API might also not be fully implemented.
Craig’s availability: the scope of GUI launcher improvements will depend on availabilty

Outlook

HyperChains have been a long-standing goal, but there is still a lot of work left.

Beam sync will significantly improve the user’s experience by delivering a secure working node within a few hours rather than days.

Publishment

We publish our work in the official AE repositories.

I am not that deep into that. but I have a general question here that @hanssv.chain can probably answer. is it possible to prove to the public that the account requires a specific set of signers? I am thinking of usecases where I want to be able to show that a certain account is actually a multisig account

Yes and no. Just looking at an account A and a transaction T signed by A you can not tell that it is a multi-sig account. But - you can show that if you combine accounts X, Y and Z (in the right way according to MuSig2) you do get account A! Does that answer your question?

1 Like

yeah that’s what I mean. would be cool to have some tool to prove that to the public without revealing the private keys of course.

Haha, yes preferrably without revealing the private keys :joy:

1 Like

You’re saying that I might not develop Rosetta, right? How difficult is a technique that takes a long time or gives up?

Does the abandonment of Rosetta’s development mean giving up listing on Coinbase?

@forjuwon, we’re not abandoning Rosetta by any means.
We’re planning to deliver in Q2, but depending on Chandru’s availability(he’s not working full time for the core team), it might bleed into Q3.

The main complication with Rosetta is that it was not designed with Bitcoin-NG in mind.

But as already stated, we’re not dropping it.

3 Likes

Core team progress update:

Week 13+14 (1 Apr - 10 Apr)

Fabian Krol

Staking / unstaking via HC demo UI. Time spent: 46:45h

Ulf Wiger (ulfwiger.chain)

Ulf is finalizing the work on the rockdb backend rewrite. Time spent: 40 hrs

Sean Hinde

Worked on establishing block witness records for generic smart contracts. Time spent: 23:10

Dimitar Ivanov

Dimitar helped with the RocksDB refactoring. Time spent: 28 h

Dincho Todorov (@dincho.chain)

  • Buildx support in node repo
  • Add Set.set support to calldata lib
  • CI fork PRs fixes

Time spent: 42.5h

Week 15 (11 - 17 Apr)

Ulf Wiger (ulfwiger.chain)

Ulf worked on finalizing the rocksdb plugin, fixed a mempool-related bug, and started working on an improved and extended garbage collection system. Time spent: 35.5 hrs

Dimitar Ivanov

Dimitar rebased the hc_alpa brnch on top of latest master. Revisited and simplified the config for a HC node. Worked on contract call caches to improve syncing. Some time spent in planning next steps. Time spent: 37 h

Sean Hinde

Worked on the Rosetta API. Added balance change transaction witness for various transaction types. Time spent: 33:00

Fabian Krol

Staking / unstaking via HC demo UI. Time spent: 33:45h

Week 16 (18 - 24 Apr)

Ulf Wiger (ulfwiger.chain)

Ulf worked on improving the garbage collection. Time spent: 22.5 hrs

Dimitar Ivanov

Dimitar finished the caching of contract data. It is fork aware. Some more planning was done. He started working in taking the seed contract creates and calls out of the config. Time spent: 36.5 h

Sean Hinde

Worked on the Rosetta API. Added balance change transaction witness for more transaction types. Time spent: 40:00

Fabian Krol

Staking / unstaking via HC demo UI. Time spent: 33:25h

Week 17 (25 Apr - 1 May)

Ulf Wiger (ulfwiger.chain)

Ulf started working on generalizing and extending the chain garbage collector, making use of the new (not yet merged) mnesia_rocksdb backend. Time spent: 38.5 hrs

Dincho Todorov (@dincho.chain)

  • PR reviews and integration
  • Add BLS12_381 support to calldata lib

Time spent: 31h

Fabian Krol

Staking / unstaking via HC demo UI. Time spent: 39:30h

Dimitar Ivanov

Dimitar finished the config setup: now contracts are seeded in it and appropriate calls are being made (so initial stakers are set online). Some planning is being done. Improvement of HC test suite. Time spent: 33.5 h

Sean Hinde

Added an online CLI system to assist with node management. Time spent: 34:50

Week 18 (2 May - 8 May)

Ulf Wiger (ulfwiger.chain)

Ulf continued working on the chain GC, and also started re-applying some optimizations in the db layer, which were removed in the first integration of the new mnesia_rocksdb backend. Time spent: 20.3 hrs

Dimitar Ivanov

Dimitar was improving the HC test suite. Some small bugs are fixed. Time spent: 42.5 h

Sean Hinde

Finished the online CLI system and added additional transaction types to the Rosetta API block witness. Time spent: 41:10

Fabian Krol

Staking / unstaking via HC demo UI. Deployment. Time spent: 54:00h

Week 19 (9 May - 15 May)

Dimitar Ivanov

Dimitar was mostly on a vacation. Some improvment the HC test suite. Now we can have two chains running in parallel in a single test. This is imporant for testing HCs later on. Time spent: 17.5 h

Sean Hinde

Continued adding additional transaction types to the Rosetta API block witness and started work on Hyperchains parent conectors. Time spent: 38:25

Ulf Wiger

Ulf helped out a bit on the Hyperchains Alpha branch, prepared his presentation for Code BEAM EU, and worked on re-introducing some optimizations of the post-commit updates into the rocksdb tables. Time spent: 40 hrs

Fabian Krol

Staking / unstaking via HC demo UI. Deployment. Time spent: 12:45h

Week 20 (16 May - 22 May)

Dimitar Ivanov

Dimitar was mostly on a vacation. Time was spent in preparation for the AMA session, HC issue debugging and some nodes health checks. Time spent: 25h

Sean Hinde

Worked on Hyperchains parent connectors for both BTC and Aeternity parent chains. Time spent: 40:10

Ulf Wiger

Ulf presented Aeternity’s work on app_ctrl and plugins at the Code BEAM EU conference in Stockholm. In preparation for this, he also did some cleanup and documentation improvements of app_ctrl. Also, after seeing a presentation of Honeybadger BFT at the conference, he started looking into complementing the AE consensus with byzantine fault-tolerance protocols. Time spent: 47 hrs

Fabian Krol

HC tests. Canary refactoring. Time spent: 34:45h

Week 21 (23 May - 29 May)

Dimitar Ivanov

Running of HC testnet exposed an issue. Dimitar was searching for it, eventually ended up writing a new test suite to test specifically the consensus smart contract in isolation. Time spent 50.5h

Sean Hinde

Worked on Hyperchains parent connectors for both BTC and Aeternity parent chains with test simulators. Time spent: 40:15

Ulf Wiger

Ulf was off for part of the week, but did some code review of the Hyperchains Alpha branch and some experiments with consensus contracts. Time spent: 28.5 hrs

Fabian Krol

HC tests. Time spent: 32:45h

Week 22 (30 May - 5 June)

Sean Hinde

Worked on the hyperchains parent connector and adding support for Erlang/OTP-25. Time spent: 40h

Dimitar Ivanov

Adding more tests to the DPoS consensus contract. Time spent 27.5h

Ulf Wiger

Ulf worked on re-introducing db backend optimizations and code review of Hyperchains Alpha. Time spent: 33.8 hrs

Fabian Krol

HC tests. Demo UI improvements. Time spent: 34:40h

Week 23 (6 June - 12 June)

Sean Hinde

Finished adding support for Erlang/OTP-25, and worked on improving the CLI and configuration systems. Time spent: 32:10

Dimitar Ivanov

Hardning the tests of the DPoS consensus contract. Time spent 22.5h

Ulf Wiger

Ulf worked on db backend optimizations and consensus contract experiments. Time spent: 31.5 hrs

Fabian Krol

Demo UI improvements. Time spent: 22:00h

Week 24 (13 June - 19 June)

Sean Hinde

Worked on improving common test suite reliability with the goal of reducing time spent by all team members re-running CI. Time spent: 31:25

Dimitar Ivanov

Attended the Sozopol workshop. Helped fixing a small bug and supporting the second release. Adding more features to the DPoS contract. Time spent 36.75h

Ulf Wiger

Ulf spent some time prototyping an indexing plugin, then returned to completing the db backend optimizations, given that the new rocksdb backend would be much better for such plugin work. Time spent: 41 hrs

Fabian Krol

Demo UI improvements. Time spent: 27:15h

Week 25 (20 June - 26 June)

Sean Hinde

Worked on replacing the default rebar3 start script with one custom to our project. Picked up further development of the Rosetta API. Time spent: 40h

Dimitar Ivanov

Added some final tests of the DPoS consensus contract. Documented pending tasks for the next HC iteration. Tested locally the configs, had a call with Dincho and Michal regarding improving the release process. Started adding a new consensus for HCs. Time spent 44.5h

Ulf Wiger

Ulf worked on finishing the work on the new rocksdb backend. Time spent: 41.5 hrs

Fabian Krol

Demo UI improvements. Time spent: 32:15h

Week 26 (27 June - 3 July)

Sean Hinde

Worked on the Rosetta API and test suite reliability. Time spent: 19:45

Hans Svensson

Helped in debugging an aec_tx_pool crash on Testnet.

Time spent: 3h

Dimitar Ivanov

Dimitar was mostly working on the new consensus for HCs. He helped with making the emergency release 6.5.2. Time spent 46h

Fabian Krol

Demo UI improvements. Time spent: 29:30h

Ulf Wiger

Ulf worked on the new rocksdb backend. Some experiments running a full sync against Mainnet indicate significant speed improvements. Time spent: 41.2 hrs

2 Likes