[Completed] Aeternity node maintenance - iris hard fork release candidate

The PR exposing chain events during contract calls has been approved and merged.
Cc @karol.chain @Arthur @marco.chain @philipp.chain

6 Likes

Hi all,

Our progress for the past week

Ulf Wiger @uwiger

Ulf completed and merged the PR exposing chain events during contract evaluation. He started working on parallel evaluation of transactions, which is a very complex task, but could yield significant speedup, especially during sync. He also started looking into upgrading rocksdb to the latest version, as the erlang support library merged the changes needed to support OTP 23. Time spent: ca 37 hours.

Dimitar Ivanov @dimitar.chain

Dimitar identified two issues in the peer propagation and dead peers detection. Those are fixed in the corresponding PR. Thanks to @gorbak25 there are real life dashboards of two nodes syncing: one with the changes applied, and another without them. What stands out is that even provided with tons of unverified peers, the node recovers and those do not flood in the verified pool of peers. On the contrary, verified peers are aggressively checked and downgraded. Although there is a significant improvement of the outgoing connections, it is yet to be investigated why those do not reach the limit provided (100). Dimitar also spent some time working on persistence of peers. Time spent: 41.3 hours

Hans Svensson @hanssv.chain

Hans pushed PR with a Fix AENS.update signature check. He also did some code reviews and spend a total of 4 hours

Dincho ( @dincho.chain ) didn’t spend any time on this project last week. He will rejoin us this one.

5 Likes

Hi all,

Our progress last week:

Ulf Wiger @uwiger

Ulf spent most of last week working on the parallel eval of txs (#3355), but also moved the rocksdb upgrade forward (#3113). It should be ready to merge today. Time spent: 35.5 hrs

Dimitar Ivanov @dimitar.chain

Dimitar spent some time on persistence of peers. After some data had been aggregated regarding the improvement of peer handling last week, did some debugging based on that. Eventually with some help of @gorbak25 found yet another bug - there are duplicating entries for the same peers. Spent some time fixing that, not ready yet. Time spent: 38.6 hours

Hans Svensson @hanssv.chain

Initiated the AENS expire issue, did some discussions. Time spent: 1h

Dincho Todorov @dincho.chain

Worked on docker images with a strong focus on rocksDB changes. Time spent: 15 hours

5 Likes

Last week’s progress:

Ulf Wiger @uwiger

The Rocksdb upgrade PR ran into some CI-related issues, but has now been merged. Ulf spent most of his time working on the parallel evaluation of transactions. Time spent: 32.5 hours

Dimitar Ivanov @dimitar.chain

Finished the clean up of duplicating peers. Further improvement of the handling of peers had been tested and implemented. This shall finish the dead peers issue. Dimitar spent some time on persistence of peers. Time spent 40 hours.

Hans Svensson @hanssv.chain

Last week he spent some time on the AENS name expiry time. Time spent: 4 hours

Dincho Todorov @dincho.chain

Dincho fixed the CI with regards of Rocksdb. Time spent: 1 hour

6 Likes

Last week’s progress:

Ulf Wiger @uwiger

Ulf participated in PR reviews, and continued working on parallel evaluation of transactions. He also implemented support for passing configuration values via OS environment variables to ease testing and deployment (Issue #3298). The PR is ready for approvals. Time spent: 41 hours.

Dimitar Ivanov @dimitar.chain

Dimitar is doing a massive refactoring of the peer pool handling. This is not ready yet and once done would make peer support easier. This will unlock the peer persistence changes. Time spent: 40 hours.

Hans Svensson @hanssv.chain

Hans did not contribute to the project last week

Dincho Todorov @dincho.chain

Dincho participated in a few meetings and helped Ulf. Time spent: 2 hours.

4 Likes

Thanks to the Core Dev Team for the improvement of the node. The core dev contracts will we extended for the next months.
@Hyperchain Team Thank you for testing/improving the node!
@gorbak25 Thank you for coordinating the work between the hyperchain and core dev team.
A new release roadmap for the both teams will be presented soon.

8 Likes

Hi,

Our progress for the past week:

Ulf Wiger @uwiger

Ulf merged support for configuring Aeternity nodes via OS environment variables, and continued working on parallel evaluation of transactions. Time spent: 36.5 hrs

Dimitar Ivanov @dimitar.chain

Dimitar finished the refactoring of peers and resumed the work on persisted peers. This is almost done, only tests remain. Time spent: 41 hours

Dincho Todorov @dincho.chain

Dincho participated in a few synchronization calls. Time spent: 30 mins

Hans Svensson ( @hanssv.chain) could not contribute to the project last week.

1 Like

Hi,

Our progress from last week:

Ulf Wiger @uwiger

Ulf worked on the parallel evaluation of transactions, and started focusing on improving the presentation of trace output, not least to make Merkle Patricia Trees more readable. Time spent 41 hours.

Dimitar Ivanov @dimitar.chain

Dimitar was working on Peers’ persistence. While so, yet another issue had been found and fixed. The Peer’s persistence is in a PR. Time spent: 39.5h

Hans Svensson @hanssv.chain

Hans started working on AENS pointers. Time spent: 1h

Dincho Todorov @dincho.chain

Dincho improved the CI a bit. Time spent: 1.3h

4 Likes

Hi,

Our progress for the past week:

Ulf Wiger @uwiger

Ulf mostly completed the initial work on parallel evaluation of transactions. There is now a PR available for review. The code is run in the aehttp_contracts_SUITE and appears to work. More testing is required to find out how the implementation performs on real-world data, but initial small-scale performance testing looks promising. Time spent: ca 40 hrs

Dimitar Ivanov @dimitar.chain

Dimitar finished a few issues issue When syncing from backup, accept previous states in DB if they don’t differ#3311, Peer pool persistence #3356. Another task - meta_tx’s TTL #3056 is almost done and its PR is in review already. Dimitar also started working on Info p2p message #3107. Time spent: ca 40.5 hrs

Hans Svensson @hanssv.chain

Hans helped us with a some code reviews and discussions. Time spent: ca 1 hour

Dincho could not contribute to the project this week.

2 Likes

Our progress for the past week:

Ulf Wiger @uwiger

Ulf has been performing tests of block witness calculations on real-world data. The results have given some new ideas about how to proceed. Time spent: 40 hours

Dimitar Ivanov @dimitar.chain

The MetaTx’s TTL is done and merged. Dimitar fixed some intermittent test failures in CI. The huge chunk of time went into introducing a new P2P info message. It is backwards compatible and would be used to monitor the health of the network. This is in a PR and a corresponding protocol PR is yet to be prepared. Time spent: 40 hours

No progress from Hans Svensson and Dincho Todorov.

1 Like

Hi all. Our progress report is a bit overdue but it was an intense week. Here it goes:

Ulf Wiger @uwiger

Ulf made some progress with block witness calculation, and resolved to extend the existing garbage collector. He also assisted in analysis and mitigation of the network attack and analysed ways to protect against database corruption in the ae database. Time spent: 42 hours

Dimitar Ivanov @dimitar.chain

Dimitar spent his time mostly attending the emergency situation and providing the new p2p message in the noise protocol. The latter was merged only this week so you will also see it in the next update as well. Time spent: 40h

Dincho Todorov @dincho.chain

Dincho fixed the DB snapshots, he healed some seed nodes and did a proper maintenance to whole regions. He also spent some time doing monitoring checks. Time spent 15 hours.

Hans Svensson did not contribute to the project last week, we could expect more involvement from him after the holidays

5 Likes

Hi all. Our progress for the passed week. The highlight for sure would be that we produced a release candidate for 5.6. It includes a lot of changes, some visible for users, and some - not. Hopefully we will not see a corrupted database anymore. Please note that this is a pre-release and tests would be ran on it for us to make sure it is stable. Please do not update your nodes yet :slight_smile:

Ulf Wiger @uwiger

Ulf helped investigate causes of database corruption, supported in the post-attack analysis and continued working on the garbage collector and block witness production. Time spent: 32 hours

Dimitar Ivanov @dimitar.chain

Dimitar finished the new p2p info message which is merged. Dimitar fixed an inconsistency in the /status endpoint, also not yet merged. Dimitar merged some improvements of the spec of aec_conductor_sup, also merged but not in the scope of the release candidate. Speaking of which, Dimitar also produced the release candidate. Time spent: 44 hours

Dincho Torodov @dincho.chain

Dincho did a lot of work for the release: first he healed some nodes, he improved the CI, he did some tests, published the release candidate and deployed it on an integration node where it is left to sync the whole chain from the genesis block. Time spent: 21 hours

Hans is on his Christmas vacation, no update from him.

5 Likes

I forgot to mention that this is thanks to @gorbak25’s fix. Since he is not subject to this proposal, I did not get into details but later on felt like I missed him here :slight_smile: Kudos to @gorbak25!

4 Likes

Hi all. It is holidays time but we still managed to squeeze some work in between. You can see our report for the past week below.

Ulf Wiger @uwiger

Ulf continued working on refactoring the database support for extended garbage collection. This involves doing some adaptations to new features of rocksdb that should also improve robustness. Time spent: 26.5 hrs

Dimitar Ivanov @dimitar.chain

Mostly worked on a protocol changing bug fix for FP rollback of balances on failed call. Time spent: 6.5 hours

Hans and Dincho are enjoying their vacation time, no progress from them.

2 Likes

Hi @dimitar.chain , What time is the vacation going to finish?

I expect Monday, 4th Jan.

Hi @dimitar.chain @uwiger @philipp.chain @YaniUnchained @ lydia and all the AE team members,Wish you all a nice vacation,Thanks for your efforts in the past,we are looking forward to you AE team surprise in 2021,don’t let those people down who believe in AE.

7 Likes

@dimitar.chain @uwiger Happy New Year.Thanks for your works.

Thanks. It was an intense year and we all hope the next one is the year when AE will see the attention it deserves.

A small clarification: Ulf and I are here, contributing. We expect the rest of the team to come back on Monday :slight_smile:

6 Likes

Our progress for the past week:

Ulf Wiger @uwiger

Ulf kept working on the refactoring of the database support. Time spent: 25 hours

Dimitar Ivanov @dimitar.chain

Dimitar fixed a few bugs: FP balances rollback on error #3434, Make inner tx of PayingForTx non-valid #3437 (actually finalised this week) and Add Lima HF explicit settings #3438. The last one was of a particular interest for @marco.chain. Time spent: 41.5 hours

The rest of the maintenance team are now back from their vacation and the next weekly update will be more interesting :slight_smile:

6 Likes