[Completed] Aeternity node maintenance - iris hard fork release candidate

aeternity Foundation Grant Applications
,
1

Application Status

Status: Completed on 4.08.2021, Approved on the 25.06.2020
Last updated: 25.06.2020,

Prepared by
Sergii Maksiuta [email protected]
Project manager of Core Dev team

Team

Approved Budget (in h): 500h/Month for the first 3 months

Used Budget (in h): TBR (to be reported on a monthly basis)

Planned Delivery: 3-12 month depending on resources availability

Specify the funding category

Open Source Development

Application Title

Aeternity node maintenance - iris hard fork release candidate

Applicant

We are the core team that built the core technology. You can find our names and nicks in the contributors’ sections in the corresponding GitHub repos

Value Application

Aeternity blockchain is a distributed network of peers.
The node is a complex software that needs adjustments, tests and bugfixes.
There are some functionalities currently that are implemented but are not yet usable on aeternity mainchain as they
wait for the iris hard fork.

We propose iris hard fork candidate and let users vote in favour or against it.

Definition of Terms

Aeternity node must be maintained and this is vital to the Aeternity blockchain.
For this to happen, there must be a team of people that constantly tests and improves it.
Those people shall have a deep understanding of the underlying core software and its infrastructure.
Also the more people share that knowledge - the better security the codebase has.
We expect a long term commitment and a planned horizon ahead so we can do our job properly.

The proposed backlog is a result of continuously gathering feedback from the community.

It also contains bugs and points of improvement of the codebase. Those are publicly available as reported issues in GitHub Kanban board:

Issues are sorted to must-have-s, should-have-s and the coulds sections. The issues have their
descriptions and corresponding labels for getting a better overview.

High priority issues consensus breaking included in the iris hard fork candidate:

Status Quo

Plans for the iris hard fork candidate are to be settled after this proposal will be agreed and the feedback is gathered in an open discussion

Required Work

We will gradually start cleaning technical debt. Please consult the Kanban board

It is expected while cleaning up technical debt to find new bugs. If those are important to fix, esp. protocol level ones or API breaking ones, we will put them in the “musts” section and fix them.
It makes no sense finding bugs that would require yet another hard fork and postponing them.

Estimate

The backlog is already too big and resources are limited. Core team believes making estimates IS NOT an adequate solution in this particular case. The listed team is already engaged in related AE and other projects and propose maximum time allocation by team member.

  • Sergii Maksiuta 20h/month
  • Ulf Wiger: 80h/month
  • Hans Svensson: 32h/month
  • Ulf Norell: 32h/month
  • Karol Skocik: 80h/month
  • Daniela Ivanova: 80h/month
  • Artur Kratt: 80h/month
  • Dincho Todorov: 80h/month
  • Dimitar Ivanov: 80h/month

Progress presentation and time reports periods to be negotiated with the Foundation

Known Limitations

The core team proposes a hard fork release candidate.
Then the community vote will decide the need from it.

Performing the community vote is not in the scope of this proposal.
Ledger support for the governance app is not part of this proposal
Any additional governance app changes are not part of this proposal.

Examples for tasks depending on community input

  • Deprecating Windows support - we would ask in the forum if we shall continue supporting the Windows builds. We strongly suspect they are not required.

  • Extending the default name period - there must be a community vote for it.

    This had already been discussed that it should be done, some forum threads:

Outlook

Planned activities address current community pains and needs
Core team aims having Aeternity node green, bug-free and iris functionality enabled.

Publishment

All of our work on Aeternity node had always been transparent and open source. As long as we are the developers behind it - it will remain so.

Maintainance

This project is already of maintenance type.

9 Likes
2

I highly appreciate this effort!

for aenalytics.org we plan to provide some incentivisation mechanism in regards to the naming system which will allow users to extend other users names by providing some small reward. in order to be able to provide this we need the hardfork :smiley:

8 Likes
3

Thank you for the work of the core team, because of you, aeternity will be more powerful. As a fan of aeternity, we support your work. Looking forward to more changes. Thank you.

3 Likes
4

For all participants of the aeternity network this will be useful, there is no alternative to a well-maintained core node, moving to iris functionality will be a big help for various products, e.g. with the naming system improvements, sophia additions and PayForTx.

7 Likes
5

For the maintenance of core code, I think it is worth investment

2 Likes
6

The AF Foundation is happy to support the core developers to make the Iris major release. @coreTeam Please provide a weekly report on your development.

3 Likes
7

All sounds fair and reasonable. Keep up the work, always need rounding off the edges :handshake:

2 Likes
8

Good to hear the news!

1 Like
9

for the record: I am only supporting a HF after #hyperchains have been build (or for super critical things / emergencies). protocol should be as stable as possible and only be updated when its really worth it.

2 Likes
11

Dear core team,
please inform us how do you define your working package per month. Who will work on what and what is the time estimate for that. The general contract offer from the foundation was not accepted by the core team. Therefore we need to make new one on concrete working package for any developer. In case we can not agree how to proceed with the project within the next two weeks the maintenance application will be time out and you need to apply again.

12

we opened a poll and it seems like people are interested in such a service, see

https://twitter.com/aenalytics/status/1285836655119806465?s=20

do we already have a rough timeline for the hyperchain implementation? is there any sort of roadmap in regards to hardforking? we need to know how long we must wait for the new Sophia capabilities around AENS

3 Likes
13

@sergiimaksiuta.chain will this proposal be redefined?

14

Hi @marco.chain
We’re almost done with all the small details that had to be polished. Once this is done, we will publish the final distrubution of work.

3 Likes
15

Hi all,

Some updates on this proposal:

Most of the people mentioned in the proposal are working on other aspects of the AE ecosystem and will not be contributing to the proposal. The assumption was that they would be spending half of their time on this proposal but at the moment this is not possible. At the moment the proposal is reduced to @uwiger, @dincho.chain and me as Ulf and I would be working full time on resolving tthe reported issues and Dincho can dedicate up to half of his time to this. Given the manpower available to this proposal, the only sensible thing to do is revisit the scope of it: now we are focusing on fixing currently opened issues. The limit of this proposal is moved to 1 month - if we see we can do meaningful contributions in the current team - we can extend that. What I would like to see is @hanssv.chain and @ulfnorell joining us as soon as possible.

For the scope of 1 month, we will do our best to solve as many issues as possible. Our main target would be not a high number but we will rather focus on important or difficult ones. Below you can find a set of tasks each one of us would be working on. Some of those could end up being unexpectedly big and could not be finished in a month’s time. On the other hand - if we solve all listed issues and we still have some time budget, we would tackle some more from the opened issues. Since there are a lot of lose ends around state channels, they are the main focus. Once Hans and Ulf Norell join the proposal, they could solve some of the Sophia ones.

Ulf Wiger’s horizon of tasks

Upgrade OTP version

Currently the AE node is running on OTP 20 which is no longer supported. The same stands true for many of the node’s dependencies. The latest OTP is 23 but it might be wiser to move to 22 instead. Ulf will do some research there and upgrade to a newer version. The same stands true for the produced releases. IMO - from all open tasks, those two are one of the most important ones and one of them could turn out to be a time hog.

Make the chain simulator more generic

Ulf wrote a really neat chain simulator that could be reused everywhere in order to better test fork awareness of different components. Currently it is confined in the scope of the chain watcher. This is a requirement for some of the tasks to be fixed later on.

Relax restrictions for min_depth

We had discussed this so many times now: at the moment the user of a SC does not have any control over when a SC can consider a transaction to have enough confirmations on-chain. The FSM has full control over that. This was a shortcut we took a long time ago and we never had the time to address. This hurts usability for no particular reason.

Dimitar’s horizon of tasks

FSM shall not conflict on unilateral transactions

This one but it presents a small security risk in SC but yet I would rather not get into details in it. It does not require any protocol change and the fix is fully backwards compatible. It must be fixed ASAP.

Initiator leaves, responder crashes

Clearly a bug, and one could hurt service providers.

Leaking WebSocket connections

Yet another bug that could hurt the node’s performance.

Improve message handling on channel open

Checks are a bit suboptimal and should be revisited. This not only makes testing harder but also imposes a small security risk.

Allow responder to listen for all incoming connections

This is something we had been discussing for really long time now, basically this will improve greatly the user flow: currently both participants are expected to have already agreed upon the opening conditions, they both had initiated a FSM on their end and etc. Although this makes perfect sense for a P2P setup, it makes the UX really cumbersome for B2C use cases as the service provider must know in advance who the responder is and etc.

Dincho’s horizon

Dincho is already overloaded with work and his availability for this proposal is not clear. Basically he will produce a release at he end of it but he will also tackle some tasks. How many of those he can sqeeze in depends on his availability. You can see the tasks he would be working on here:

Bear in mind he will not solve all of those in the scope of this month :slight_smile:

6 Likes
16

Thank you very much @dimitar.chain for the update.
The proposal is approved by the foundation. :slightly_smiling_face:

The plan is three developers (@dimitar.chain, @uwiger and @dincho.chain) will be involved for the first month. To work on the above mentioned topics, they require maximum 264 hours in total.

As agreed, the weekly report about the progress will be published in the forum.

We are looking forward to getting it started!

3 Likes
17

Since this is officially approved, I can share our progress for the previous week (12-16 August:

@uwiger was working for 12 hours on upgarding OTP release (Github issues 3293 and3104); he reported a bug in OTP23.03 compiler

@dimitar.chain was fixing a bug in FSM - the conflicts on unilateral transactions a total of 14h

@dincho.chain had a limited availability and didn’t contribute to the project. He is having a limited availability this week as well.

This is from us, I will share this week’s progress on Monday. I will keep the regular Monday updates for the past week.

4 Likes
18

@dimitar.chain Thank you for the first report.
@uwiger Can you please inform us whether an upgrade to OTP 23 is possible and if not what are the reasons. Since the hyperchain team needs the OTP upgrade too it is necessary to coordinate the next upgrade/work between the teams.
@gorbak25 Please tell us if the hyperchain can build on the OTP 23?

1 Like
19

As reported in this issue comment (and in our progress report to be posted soon), Aeternity builds on both OTP 22 and 23.

Now that CircleCI is back, we can stress both builds a bit more. Personally, I feel that OTP 22 is the safer choice, and at least from a very limited first timing test, the significant performance boost seems to be in moving up to OTP 22.

But if there is a pressing reason to go for OTP 23 right now, that’ll probably work too.

2 Likes
20

The report for the past week:
@uwiger worked 34.5h:

  • did more investigation in OTP22-23 world, his progress can be seen in the GH issue
  • Issue #3329 - Break out chain simulator from chain watcher test suite
    This has been done, and the chain watcher test suite updated to use the standalone simulator. The PR has been reviewed. PR is waiting for tests pass and it is going to get merged.
  • Issue #3194 Make channel usable before min-depth confirmation
    Work has started, and some initial code changes have been made to prepare for the changed semantics. Currently thinking through the error situations that can arise, to try to work out a strategy for them.

@dimitar.chain worked 44.5h

  • Fix FSM bug: conflicts on unilateral transactions #3332
    This is basically done, doing some final test polishing. PR is waiting for tests to pass and to get some reviews
  • SC responder crash when initiator gone #3333
    Same as the above, PR can be seen here
  • SC: websocket is not closed on “invalid fsm id” #3163
    Started working on it, no PR yet

@dincho.chain had a limited availability and didn’t contribute to the project.

3 Likes
21

Hyperchains are strictly build upon the core node - if the core runs on OTP 23 so will Hyperchains.

3 Likes