[Active] Middleware Grant April - June 2023

Application Status

Status: Approved on the 04.04.2023, submitted on the 29.03.2023
Last updated: 29.03.2023
Submited by Sebastian ([email protected])
Team: Rogerio, Sebastian
Approved Budget (in h):
Used Budget (in h):
Planned Delivery:

Specify the funding category

Open Source Development

Application Title

Middleware team application April 2023 - June 2023


AEternity Middleware Development Team

Value Application

The upcoming grant will build upon the foundations established by the previous one. We will be leveraging the progress and insights gained from the previous grant to further advance our research and development efforts. This will involve building upon existing changes and exploring new ideas to continue improving Middleware.

Definition of Terms

The middleware is an indexing api for the aeternity core node, it exposed data in ways that the core node can not or is not supposed to be.

It is heavily used by various projects in the aeternity ecosystem such as the explorer, superhero wallet and dex, but available for everyone to consume or host themselves.

The overall goal of middleware is to expose all data in aeternity blockchain in a structured, filterable and developer friendly way.

Status Quo

Our work done during the previous grant includes the following:

  • New channels endpoint (/v2/channels/:channel_id) to retrieve a single channel information, including round details.
  • List state channels with balance updates.
  • Handle NFT contract and template limits.
  • New template tokens endpoint on /aex141/:contract_id/templrates/:template_id/tokens to allow tracking template NFTs, including edition, owner and token ID.
  • Add typespecs for database Models.
  • Improve indexes to include local indexes and internal contract function calls for better referencing. Use this index to include tx_hash and source_tx_type when it corresponds.
  • Handle liquidity addition on AEx9 balances.
  • Filter activities by roles and types by sending additional ?type= and ?is_owner=1 query parameters.
  • Render ga_meta and paying_for inner transactions likewise chain/regular ones…
  • Improved test coverage, including aex9, tx routes and channels.
  • Profile and restructure websocket subscriptions and make them more efficient when it comes to broadcasting messages.
  • Improve docker release building by using mix releases.
  • Add metrics and observability using telemetry and other profiling tools.
  • Allow filtering contract calls by function name prefix.
  • Adapt middleware behaviour for Hyperchains and newer versions of the node.
  • Update documentation to cover for some of the requests made by users.
  • Several bug fixes on the syncing logic and endpoints.

Required Work

Some of the upcoming work to be done is due to some changes on the node. These changes require a thorough analysis and implementation to ensure the smooth functioning of the system. Additionally, we will be covering some of the new features of æscan as they become available. This will involve exploring the new functionalities and providing a detailed overview of how they can be used to improve the efficiency and effectiveness of the system.

  • Adapt to HyperChains and index related data like commitments, parent chain info, AEX-N contracts, etc.

This will require Hyperchains to be fully implemented on the node and some coordination with the Node team to fully cover every new aspect provided by the Hyperchain features.

  • Decouple the middleware into two applications to scale the endpoints without syncing in every machine.

This can be done by separating the apps and spawning each one depending on configuration settings.

  • Full name history endpoint

Users would have the ability to track all changes (including proclaims and claims) made to a name, to be able to track the state of any given name. (Github issue #1202)

  • Improve endpoint performance after metrics provided by APM

Some of the metrics that are domain-specific would be helpful to track to provide a better understanding at what the syncing bottlenecks might be and how to improve them.

  • Sync new events and/or improve performance to fetch them.

There’s new delta events now provided by the latest node version which we’re currently ignoring. These events contain internal account transfers and other things that are not directly linked to function calls.

  • V3 endpoints

    • Removal of /blocks endpoint (in favour of /key-blocks and /micro-blocks)
    • Filtering of AExN tokens by query parameters
    • Fix display and nesting of names/auctions
    • Remove transaction index (txi) support from routes and responses
    • Tag the encoding of non-text binary response fields using the same encoding as the node
    • Add tx-encoded transactions to /v2/txs (Github issue #1187)
  • Removal of V1 endpoints

Users were notified about removing support for V1 endpoints, which had consistency and performance issues. Getting rid of these will involve some previous work of making sure no known-users are currently using it.

  • Add support for creating transactions on integration tests + using SDK for better integration tests

Currently, integration tests only test existing mainnet transactions. Adding support for creating our own chain with different kinds of transactions and edge-cases would improve the accuracy of tests with data directly returned by the node.

  • Review architecture documentation and include it on the repo documentation

The architecture docs are from legacy systems. We would update this, including diagrams of how the middleware syncing works.

  • Contract detail endpoints

Similar to how it was recently created with channels, we would include a channel detail endpoint. We would also add a nested paginated endpoint that would include all of the contract calls (both internal and external).

  • Move to newly exposed api module provided by the node

Since we’ve updated the Middleware to 6.8.0 version of the node, we can now use the new aeapi module, which provides a subset of functions that can be more easily maintained by the node team to avoid any breaking changes on external applications like Middleware.

  • Wealth tracking for getting the accounts with the highest balance

This might involve asynchronous tasks running to avoid impacting the performance when getting account balances.

Ability to filter activities by micro-blocks

Right now the endpoint only allows filtering by a range of generations, but not by a single micro-block.


We would be working full time on this, both Rogerio and Sebastian.

Known Limitations

Although we aim at delivering all proposed features as outlined, there may be some higher priority tasks coming up that need to be worked on by the middleware team, thus not every proposed item is expected to be delivered.


  • MDW client (JS/Elixir/Erlang)

Right now there’s no SDK/libraries available for using the middleware. Users have to build one themselves by interacting with the endpoints directly. Our plan would be to design and implement new clients on JS, Elixir or Erlang. This way we can also include new features or fixes faster, without having to notify external users directly.

  • Productization of MDW (users and request limits)

We would need to discuss a new way for users to access the middleware, by establishing some business plan that would charge those users that consume the most MDW requests.

  • GraphQL as an interface

At some point it would be worth reviewing alternative ways of returning the Middleware data. GraphQL was considered as a potentially good alternative for returning very specific subsets of aggregated and nested data.


During development there will be new releases published and deployed continuously.

We publish our work in the official AE repositories.


Maintenance is part of the proposal and included for the approved timeframe.


Weeks 1,2 (Apr 1 - Apr 9)


  • Update aex9 swagger docs 1h
  • Analyse aex9 balance case with missing emitted logs 3h
  • Investigate AEKnow total supply and sum hardfork account mintings consdering extra and contracts 10h
  • Count AEX-N contracts endpoint 8h
  • Track AEX9 holders count per contract 12h


  • Fix credo refactor errors. 4h
  • Add /oracles/:id/responses nested endpoint. 12h
  • Add /contracts endpoint to list contracts. 24h

Weeks 3 (Apr 10 - Apr 16)


  • Track aex9 contract supplies 20h
  • Track aex9 logs count 12h
  • AMA 4h


  • Adjust the order for contract call event mutations. 20h
  • Add contracts detail page. 20h

Weeks 4 (Apr 17 - Apr 23)


  • Wealth tracking and endpoint 36h


  • Adapt activities to new int transfers format. 24h
  • Integrate devmode and SDK for custom test txs. 16h

Weeks 5 (Apr 24 - Apr 30)


  • Track aex9 balances sorted by amount 36h


  • Integrate devmode and SDK for custom test txs. 16h
  • Add channels updates nested endpoint. 24h
1 Like

Weeks 6 (May 1 - May 7)


  • Integrate devmode and SDK for custom test txs. 16h
  • Delete inactive name owner deactivation records when activated. 16h


  • Create P2P Marketplace and analyse future API 16h
  • Make migrations available on new Elixir releases 20h

Week 7 (May 8 - May 14)


  • Integrate devmode and SDK for custom test txs. 8h
  • Include oracle responses on oracle queries endpoints. 16h
  • Reindex reward_oracle int transfers. 16h


  • Create P2P Marketplace 16h
  • Optimize account balance jobs 12h
  • Fix race condition on setting in-memory async tasks to done 12h

Week 8 (May 15 - May 21)]


  • Using the Mdw as P2P Marketplace API 20h
  • Add counters to ws block broadcast 4h
  • Generic configuration and encoding of custom event args 4h
  • Filter internal calls by contract and function 8h
  • Make docker volumes optional 4h


  • Index inner contract creations for /contracts endpoint. 24h
  • Add support for node 6.8.1. 16h

Week 9 (May 22 - May 28)


  • Using the Mdw as P2P Marketplace API 4h
  • Setup websocket inactivity timeout 2h
  • Aeternity Galaxy event 34h


  • Add support for node 6.8.1. 8h
  • Handle old oracle responses when migrating int transfers. 16h
  • Allow filtering channels by active/inactive. 16h

Can you share a few details and maybe also the presentation of the p2p marketplace api stuff? I think that might be interesting for some people :slight_smile:


Week 10 (May 29 - Jun 4)


  • Allow filtering channels by active/inactive. 8h
  • Include inactive channels on channels listing. 24h
  • Fix swagger docs to contemplate OpenAPI checks. 8h


  • Track internal calls for wealth 16h
  • Apply Node persists config to the Mdw 4h
  • Use count id by type also on /count 4h
  • Filter contract logs by function alias 4h
  • Add transactions count to websocket keyblock 4h
  • Upgrade swagger spec to OAS3 4h
1 Like

Week 11 (Jun 5 - Jun 11)


  • Fix swagger docs to contemplate OpenAPI checks. 16h
  • Update architecture documentation. 24h


  • Remove mock from contracts sync 4h
  • Contract call indexation and filtering by entrypoint 8h

Week 12 (Jun 12 - Jun 18)


  • Include aexn contracts with special chars 8h
  • Consider all oracles expirations 6h
  • Include minified swagger UI 6h
  • Count only aexn contracts with valid meta info 8h
  • Add transactions count to websocket keyblock 12h


  • Update architecture documentation. 8h
  • Add approximate_expiration_time to oracles. 16h
  • Add approximate_expiration_time to names. 16h

Week 13 (Jun 19 - Jun 25)


  • Display approximate expiration time for auctions. 16h
  • Create contract call event tx when Chain events. 16h
  • Add devmode Contract.create call test. 8h

Week 14 (Jun 26 - Jun 30)


  • Add micro block time to oracle nested resources. 16h
  • Add support for swagger using JSON format. 8h
  • Add block_time to all activities. 16h
1 Like