En-/Decoding - a currently fatal design flaw in æternity?

And I guess there will be a problem to implement AE protocol in another language. Is the en/decoding lang agnostic or coupled to Erlang Beam VM?

I don’t expect there to be any major issues re-implementing encoding/decoding of calldata. Porting the compiler should also be straightforward - there shouldn’t be anything that is coupled with the BEAM but certainly a tad bit of Erlang knowledge will help the translation :slight_smile:

3 Likes

I think compiler is not an issue, because developers mostly will write, test and compile programs on the computers they control so there is no problem or bottleneck for that.
But decoupled encoding/decoding calldata from compiler is a different issue and there might be some benefits for that, especially if competent blockchains offered it from the beginning where you can use a JavaScript library to do it (web3, ether.js, etc I guess for Ethereum).

2 Likes

If you want to do encode/decode properly for an unknown contract I think you will need substantial parts of the compiler. But encode/decode for a known contract (and also unknown contracts with simple types) is not a huge effort.

2 Likes

So maybe one time there will be a JavaScript library that will let you do it at least based on generated ABI when you compile the contract.

4 Likes

I knew this. I saw this when I built my version of the middleware and realized there was information from contracts that I couldn’t read unless it was decoded too, which in turn meant making a connection to another server, calling another program or building an extensive decoder that I kept thinking should be part of the sdk but for some reason wasn’t there.

I think adding a decoder to the node is not a good idea because compilers are historically known for having vulnerabilities and exposing this port publicly could be a new attack vector for the network (this threat to me is more real than the flood of the network or the lack of miners, both of which never happened) . I guarantee that adding a port or endpoint that has access to a cpu intensive function will result in abuse and DDoS on the aeternity nodes.

2 Likes

I found this out quite recently and I feel like I am missing something.

For me a compiler is something you use before you start using your program. In this sense it makes 0 sense to me to use a compiler for anything during the workings of a program.

Imagine a golang compiled program to constantly ask the go compiler to do stuff for it to run. It is both slow and introduces another point of failure.

I do understand the reasons why this has been done, but my strong opinion is that everyone should be very clear that this is just a temporary workaround until it is implemented in the SDKs, and this should be at the very top of the SDKs backlog. Compilers are for developers.

4 Likes

I feel there is a misconception in communication here. You don’t need a compiler for anything else than compiling a smart contract. But currently the only implementation of a encoding/decoding helper is in the compiler due to the ease of doing it in Erlang. There is nothing that keeps a js developer away from implementing this in the sdk.

5 Likes

Yeah, my main point is that we need to have this implemented in the SDK asap, so that we do not have to work around it with the compiler :wink:

5 Likes

I’m curious though what we are more likely to see first: Someone with the skills in theoretical computer science, Erland and JS, or finally some actually working web assembly that is being hyped upon for years already, in which we can make the compiler run in a Rust implementation of the Erlang VM… sadly, I’m not really confident about any of these options, someone correct me, please :confused:

Well, if I get taught Erlang I would make an informed decision whether or not to do a JS implementation of it

2 Likes

The problem is not the Erlang, but to understand what’s actually going on in there. Therefore you’d need to spend a lot of time with our Swedishes I guess… :blush:

2 Likes

To be fair, it only failed for people that relied on hosted services they trust instead of setting up a trust-less environment by themselves (what every professional app developer should do). Nevertheless, developers who learn and haven’t reached a professional level yet to self host all the things will judge aeternity on reliable hosted services (therefore i agree, i think it is very important to provide a reliable testing environment with hosted services for developers right now).

4 Likes

@nikitafuchs.chain I will hopefully start on this soon (this week)

First goal is to have ability to construct calldata directly in JS, via WASM.

That could help to get the ball rolling a bit.

1 Like

Does a problem about this issue is because Aeternity VM is not programming language agnostic (so relying on Erlang and it’s BEAM too much)? If so, that problem will occur if there will be a need to create alternative Aeternity nodes for example in GO or JAVA language. Ethereum has many protocol implementation in many programming languages and it’s protocol doesn’t rely on any particular language, it is language agnostic and the basics are clear enough to implement the interactions with contracts in any lang.

That flaw would be immediately recognized if for example the Aeternity protocol would be implemented in two inherently different languages in parallel from the beginning, for example Erlang and Go or Java instead of choosing Erlang and Elixir which both relies on BEAM.

We just discussing about this with @ae-omar yesterday.

First, I also agree the compiler is a temporary solution.

Implementing the encoding/decoding per SDK means maintaining lots of sdks.

Since aeternity and the compiler are both created in erlang, what about separate the encoding/decoding part into a library, and include it in both, the compiler and the aeternity node. Then, create a new set of v2 endpoints, that encodes and decodes the calldata in the node.

That will fix the actual problem, and also, easier to maintain.

First this one. Yes, it will be very important to have different implementations of the aeternity node in the future. Right now the core team is working in such a high pace, that it would be impossible for anyone to catch up.

If this slows down a bit, creating alternative implementations will get easier and if it happens, it will make the whole Blockchain more robust.

It is also not only about bugs in the protocol but also potentially in the erlang/beam itself. Luckily Erlang is a very reliable piece of software though :wink:

A part of this is also “bringing aeternity to mobile” or parts of it, like the compiler or the VM. It is not necessary to have a full node with all it’s features everywhere, that would be overkill for many simple use cases.

Hi, it is impossible to maintain and provide SDKs for every language. That will not happen. Most of the resources are currently towards JavaScript and Python. Go and Elixir very small projects / teams.

Your suggestion already gets discussed. Looking at my previous answer we need to decide on a few feasible solutions that later can also be maintained “easily”. Some parts will always be behind and won’t provide all functionality.

The compiler is already seperated from the aeternity node. Overall the whole architecture is quite modular and components are well seperated from each other. The major challenge is bringing this to mobile / client side IMHO.

Also agree to this a lot. Looking at other projects and blockchains, everyone I see is using trusted services somewhere in their stack an that is totally fine.

The important thing here is beeing able to verify, if you don’t trust.

Transparency is more important than full trustless decentralization.

I expect a future where Blockchain aepps have millions of users, maybe 99% trust hosted services like a hosted node, a backend API server like the middleware or a hosted compiler.

1% though will run all the infrastructure themselves and if they spot a flaw or misbehaviour they will report it to the community and userbase.

A business that cheats it’s users will loose reputation and it will be easy to prove that they are not trust worthy.

Today in Bitcoin, 99% are using trusted nodes. It’s exactly the same. Using your samurai or ledger wallet but not connecting it to your own node at home will put you into the position that you need to trust that node.

Important is, that every major module is open source, well documented and easy to host. If this is given, there will be always someone invested who will do checks.

Great user experience is as important as being fully trustless. I would even say it’s more important.

Well i hope this get resolved somehow and the core team have this in their sights.

Since a native node on mobile is not in the immediate road map.

The issue with the compiler requirement by mobile apps is not a good solution, it multiplies almost by 3 every smart contract call, and on a mobiles platform that means latency issues, delays for the user, and we’re multiplying by 3 the possibility of an error, one must remember that not everyone have perfect mobile connections…, just been in the states recently :wink:

1 Like