Visualizing coinbase transaction in the æternity node / Middleware

Protocol Features & Updates
1

I had a discussion today with @Kryztoval who was wondering why is it possible for the balance of an AE account to go up without there ever being any transaction associated with it.

For example, the BRI account is constantly increasing in value, but no transactions are associated with that.

Here is what Kryztoval thinks:

If I look at an address, I can’t really tell how is the balance created and recollecting all the transactions i can’t get the balance either… so I can’t do a running balance column :frowning:

Maybe it made sense from a development stand point… and i say maybe because i am a developer and i don’t understand that. But, how do I explain a simple person: Look, this account has magically gathered N coins, because of it’s height and position and looking up at this table we can see that it should be, well, that amount.

Why can’t I just say: Oh! Look, this coin had "N"coins as a block reward on this date. Period!
And if you sort all transactions you will get the current account balance. Of course. Cause every accountant knows that a ledger sum of all ins minus the sum of all outs is the current balance.

It needs to be in the node or in the middleware, the visualization layer is not the best place. @outcrop did a good job making a “mining” tab, that lists all the blocks that account has mined. and then goes to the table we are talking about, STATIC table from SQL and calculates the benefit, but this is all a calculation and if the team or a vote changes it it needs to be updated BY HAND, this is no where optimal or good at all. and it is most definitively not shown on transaction list.

I agree that visualizing these transactions in some way is a good idea (makes sense).

Best,
Vlad

2

Yeah, the BRI account is a good example, it has no mined blocks, it has no transactions, yet it increases it value. Why? How? where do we see it? how do we query it?

I think the information needs to readily available to be reviewed and audited at any given point in time, specially via automated means. it serves no purpose if it has to be done by a human.

3

Hey Kryztoval!

So there are several ways an account balance can change without leaving a transaction on the chain. It can gain it either in the genesis block, or on a hard fork, as we do with token migrations. Or it can be the recipient of funds from a contract call, which also leaves no trace.

I agree that this is not optimal. For the hard forks, we could import the JSON files which feed the balances into the middleware, which isn’t ideal but isn’t so bad either. For the contract side-effects, we had previously talked about having an instrumented node which would emit the side effects of contract calls for software like the middleware(s) to read. I think these two cases would cover everyone, but I suspect someone will correct me here…

The contract side-effect part is waiting for resource to become available, but it’s still on the agenda, as far as I am aware.

HTH,

j

3 Likes
4

The node has to know in some way or fashion how and when to insert the values from the json file. if it has it already loaded how do we know that? why is it not returning those transactions?

Contracts are transactions and the call does return values doesn’t it? it has to be decoded if I recall correctly, otherwise you can;t know if this happened or not. and also, the node knows it, otherwise it can’t tell the balance at an specific height. I do think that adding another node to emit the side effects of contracts in another node (the base node has the effects) would defeat the purpose of this, it should exist either in the middleware (with an API that is accessible via the sdk tools) or in the main node (via the provided restful interface for public information)

Does that make sense? I mean, the middleware is meant to have the transactions and show them, it should be able (ideally) to show the running balance of any account in a quick and accessible way, but so far we can only see a bunch of transactions and several things are hidden from the node.

I think right now it is possible to have an account with balance that has 0 transactions in the blockchain and the only way to see the the balance is to ask the node

I don’t know but this does not make sense. :frowning:
I think a trustworthy coin needs to have auditable transactions

1 Like
5

I’ll try to explain why it is more problematic to track the receivers than the spenders, just to see why this isn’t already there in the node.

There are two ways tokens enter the system:

  1. Through migration (i.e., in the genesis block and in the hard forks).
    This is accounted for in the json files associated with the migration.

  2. Through mining (i.e., the beneficiary rewards)
    This is accounted for with the coinbase reward for each height.

Migrated tokens enter the system only at the first key block of a consensus protocol version:

  1. The genesis block.
  2. The hard fork blocks with associated token migration. All hard forks so far have an associated token migration.

Reward tokens enter the system at each key block (except for the genesis block), and are immediately put into the lock account where they stay for the reward delay period.

So, to sum this part up:

  1. Migrated tokens end up in their accounts in the json file.
  2. Reward tokens end up in the lock account.

The more interesting (and difficult) part is the distribution of tokens from one account to another.

All transactions are associated with the nonce of exactly one account. In a few cases (e.g., state channels) more than one account needs to sign the transaction, but the transaction itself is always associated with exactly one account, through the account nonce. So, all transactions has a unique combination of account and nonce.

However, the transactions can distribute tokens from multiple accounts, and to multiple accounts.

Even the simplest transaction, the spend transaction, decreases tokens from the sender account (amount and fee), and increases tokens for two accounts (the amount to the receiver, and the fee to the lock account for the duration of the reward delay).

And a contract call transaction can spend tokens to any number of accounts, and even from any number of accounts either through calls to other contracts, or by signature delegation for naming service or oracle operations).

So, even for the actual transactions, it is not simple to see how a transaction distributes tokens from and to accounts.

Then we have the distribution that happens without explicit transactions.

Again, the introduction of migrated tokens are pretty straightforward, and can be seen in the json files, so I won’t spend more time on that now.

After the reward delay, the coinbase and fees are distributed from the lock account according to this:

  1. The coinbase of a key block goes to the beneficiary in that key block.
  2. The fees of a generation of micro blocks are split between the beneficiaries of the opening and closing key blocks of that generation.
  3. If there was a report of fraud (Proof of Fraud), the fraudulent miner doesn’t get anything, the reporter gets a fraction, and the remaining sum is given to the lock account.

After the BRI entered the protocol, a part of the previously described rewards goes to the BRI account.

In the beginning of development, we did have coinbase transactions for rewards, but they caused more difficulty than the problem they solved. I do agree that for the purpose of accounting, the transactions would have been nice, but they would also have increased the size of the chain, and they would not have solved the whole problem (e.g., the distribution from the ordinary transactions).

One thing we have discussed, and it even went as far as a couple of weeks of development, is to record the side effects of each transaction. The effort was cancelled since it turned out to be harder than expected, and we had more pressing matters to attend to. I personally think we should have pursued this effort, but it is a matter of priorities.

The side effects could be of different nature, but the most straightforward one would be to record every increase and decrease in balances of accounts, tied to every transaction. The same could also be done for each key block. The numbers are of course there already in the node, but collecting them isn’t as easy as it might seem. In particular, doing it without significantly increasing the processing time is hard.

The problem can be solved by having a specialized node which only validates the chain and records the side effect. And this was also once the ambition. The current middleware initiative is what came out, and while it doesn’t give all the information one could whish for, it certainly helps out with some solid information.

The only proper solution (note: my personal view) to this problem is to have an instrumented node, with proper regression testing etc, where the instrumentaion can be removed in compile time to not inflict any decrease in execution performance if you are not interested in the information.

Such a node could have an API for proper accounting of all token transfers. And then a third party software could audit these token transfers and check them against the balances and root hashes.

I hope I at least shed some light into why the information you want is not there (yet). I do agree that this information should be as transparent as possible, and accesible to more people than the core developers.

Please keep pushing for it, and we will hopefully get it there at some point.

3 Likes
6

After consulting the code I need to correct myself.

The pending rewards and fees are not in the lock account. The reward is not introduced until the delay is over, and the fees are taken from the accounts when the transactions are applied, and reintroduced after the delay.

This just goes to show that an external service is indeed needed.

3 Likes
7

Yeah, I understand your explanation and I see you understand why the availability of this information is a requirement.

I will keep pushing this because having such important information on a json file is not something I want to explain the grandkids of crypto era. You can count on this.

An instrumented node would solve the issue at hand but would also create a new node type. I believe this has to be either a config of main node (just like what middleware was meant to be, a plug in into the node that you could be enable or disable) or integrated into the API so the middleware can do the requests.

If I knew erlang I would have made the middleware reuse the original databases and just create a few index and tables to link the information on top of it, but I don’t know erlang and I didn’t quite understand rocksdb.

8

@YaniUnchained Someone found my middleware, and decided to give it a go, and the first question i got from that person was: “In this account, I see it is the beneficiary of a block, but when I look at the transaction list there is only one transaction but no increase of value from the mined block, why is that? is the middleware missing something?”

I can’t come back and say: Oh, to calculate that you need to get the file from the full node, then check the height, then get the value from the table, then see if it is a bri block or not, if it is substract the value, and hope that you didn’t mess up the transaction because, well, you have no way of asking the node if it was indeed applied on that block or not.

9

Sure, but this is not an immutable file and I have no way to be sure the file has not been tampered with. nor is there a way to see if this was actually done. so this still sounds incredibly uncertain. what happens if my node has the wrong json file? what if someone tamper with it?

Is it time to revisit this? Is there another way to see this? from an accounting point it makes sense to see all the ins and outs, surely the blockchain doesn’t need all this information to function but can it be calculated/generated/induced/deducted? most other things can be done (starting by the indexing of accounts transactions, passing by contract creation/calls/responses. etc.

A fully instrumented node seems like a good fit. Now that I have seen the blockchain size is increasing over time I have to wonder if there is an intention to trim the blockchain at some point or some way to slim it down. But maybe this goes in a different post.