There is no "one-size-fits-all" for any technology and everything comes down to "what we are trying to solve?". For Web3, we are dealing with distributed applications working on distributed software infrastructure (P2P networks). In business terms this means we are going to lose money if we don't get the right trade-offs in our technology stack. For startups these technical choices and trade-offs will be crucial in their product development and ability to scale. Having a good idea is not enough for a startup to thrive.
For the Web3 world, type safety (verifies and enforces type constraints at compile-time), high-performance async (good ecosystem of non-blocking I/O libraries and runtimes), automatic memory management and memory safety without garbage collection (ownership model) were the main reasons that made Rust my first choice. For instance, writing smart contracts that doesn't have memory bugs and consumes less storage on the blockchain is a massive advantage. On the other hand for user interface (UI) development, React is still a better choice due the rich UI features offered.
The examples developed, will be focused on the back-end services and distributed software infrastructure demonstrating the key points of using Rust with React and Rust with IPFS for the Web3 world.
- Web Application
- RESTful Web Service (library-service)
- User Web Interface (library-ui)
- InterPlanetary File System (IPFS)
- Conclusion
Our main objective is to understand how to build a distributed web application that are safe, efficient, highly performant, and do not "break the piggy bank" to operate and maintain. We are going to look at two type of storages a central storage(Postgres) and a distributed storage (IPFS).
With that in mind two key components were chosen to achieve those objectives:
- Actix is a modern, light-weight web framework written in Rust.
- SQLx is a Rust crate that provides asynchronous database access in our case to postgres.
When considering open-source software my first look is on the team building the software and the community using and maintaining it. (Makes no point selecting a state of the art software that is supported by a one man show. Believe Me or not, this is one of the most common errors startups do.) After saying that, having asynchronous support, ability to scale and maturity is essential in any startup product development. Nevertheless, there are some very good alternatives options to the ones used for this demo. For a Rust web framework axum is also a sound choice and for the database access SeaORM (SeaORM is an object-relational mapper which SQLx is not).
We have the following environment setup using kubernetes for our web application:
Our web application is composed of a Rust Rest API connected to a postgres database which forms a Microservice and a React user web interface served by the Rust Rest API.
The project code is organized with separate and clearly marked areas to store code for handlers, database access functions (dal), data models and database scripts.
Each route has a handler function and normally a database access function. The main purpose of structuring our code is to make it easier for other people to read and support a CI/CD pipeline. For instance for this project all integration tests are concentrated in the handler section.
Key points:
- Actix uses async I/O, which enables an Actix web application to perform other tasks while waiting on I/O on a single thread. Actix has its own Async runtime that is based on Tokio(async library in Rust).
- Actix allows to define custom application state, and provides a mechanism to safely access this state from each handler function. Since each application instance of Actix runs in a separate thread, Actix provides a safe mechanism to access and mutate this shared state without conflicts or data races.
- SQLx is built from the ground-up using async/await for maximum concurrency.
- The Postgres driver is written in pure Rust using zero unsafe code.
Service Handlers:
- post_add_book Insert one book into the table books.
- post_bulk_insert Insert books in bulk mode into the table books.
- get_books Read all books from the table books.
- get_book_by_id Read a book by id from the table books.
- put_book_by_id Update a book by id from the table books.
- delete_book_by_id Delete a book by id from table books.
Before runing the integration tests we need to create the table books and run the script test data using pgAdmin for instance.
The project code is organized with separate and clearly marked areas to store code for API functions, components and types.
The React version 18 was used to build this user interface(UI), because of the concurrent mode that comes with this version. This new feature allows React to work on several state updates concurrently. For providing asynchronous state management, server-state utilities and data fetching, TanStack Query was used. Together with React Router, which allow this UI to update the URL from a link click without making another request for another document from the server.
By integrating these two libraries and the use of the React V18, we get the following key benefits:
- Ability to prepare multiple versions of our UI at the same time.
- React Router’s data loader prevents an unnecessary re-render when data is loaded onto a page.
- React Query’s cache prevents unnecessary calls to the REST API.
SolidJS vs React
SolidJS is an open source, reactive declarative JavaScript library with an API similar to React. There is no such thing "one better than the other". Its always about tradeoff (js-framework-benchmark), we have to give up something to gain something else. In every decision made, as always, we should select two paths chose one and keep an eye on both.
Comparison Table
| Feature | SolidJS (2021) | React (2016) |
|---|---|---|
| TypeScript support | = | = |
| Declarative nature | = | = |
| JSX Support | = | = |
| Highly performant | + | - |
| Direct manipulation of the DOM | Yes | No |
| Server-side rendering | = | = |
| Conditional rendering | = | = |
| Concurrent rendering | = | = |
| Community and ecosystem | - | + |
Key points:
- React is a popular library for creating component-based frontends. React has the largest ecosystem out of any UI library, with very talent people supporting it.
- TypeScript provides a much richer type system on top of JavaScript. TypeScript uses the type system to allow code editors to catch type errors as developers write code.
- Concurrent React. Until React 18, the React render process was synchronous and non-interruptable. As a result, the UI would lock during long render processes and not respond immediately to user input.
- React Router reduces the number of re-renders.
- React Query(TanStack Query) provides a client-side cache of the data.
- SolidJS is a very solid alternative to React.
Running the React UI
The table books needs to be create and run the script test data using pgAdmin for instance.
InterPlanetary File System (IPFS) is a peer-to-peer distributed file system, which connect all computing devices with the same system of files. IPFS is the combinational technology of the version controlling system and peer to peer network spreadover global namespace.
In contrast to a central storage(like PostgreSQL), IPFS works on a decentralized system in which every user in the network holds a portion of the overall data, thus creating a resilient system for storage and sharing over the globe. Any user in the network is able to share a file and it will be accessible to everyone in the network by requesting it from a node which possesses it using the Distributed Hash Table (DHT).
Three fundamental principles to understanding IPFS:
- Unique identification via content addressing(CID)
- Content linking via directed acyclic graphs (DAGs)
- Content discovery via distributed hash tables (DHTs)
The examples provided were developed using the Kubo RPC API together with the actix-web, awc and actix-multipart-rfc7578 crates.
Note: A Rust implementation of a IPFS node (iroh) is currently being developed. But unfortunately is not mature enough for the examples needed.
The CID is a label used to point to material in IPFS. Import the book1.json using the Rust project api-v0-add file and copy the CID.
Search for file using the CID "QmTN78XgBo6fPaWrDhsPf6yzJkcuqpEUBqVRtHu3i5yosL" in the kubo node webui(http://demo:32546/webui/).
We can access the file directly via browser https://ipfs.io/ipfs/QmTN78XgBo6fPaWrDhsPf6yzJkcuqpEUBqVRtHu3i5yosL using the IPNS name. We can also use a DNSLink address which looks like an IPNS address, but it uses a DNS name in place of a hashed public key.
Directed acyclic graphs (DAGs) are a hierarchical data structure.
Import the file book2.json.
Import the file book3.json.
Import the DAG node file library.json using the the Rust project api-v0-dag-put.
Using the three CIDs of the files already imported(book1.json,book2.json and book3.json) to create a DAG in IPFS.
[
{ "/":"QmTN78XgBo6fPaWrDhsPf6yzJkcuqpEUBqVRtHu3i5yosL"},
{ "/":"QmYqo1Ack8g2rDX6TEoPA14oNASJrXEVB4oTEKv8So6Ect"},
{"/":"QmUfV4m2PUM559LSvDsJkoz1KofTVq25RDXwW5uMdjNb4u"}
]Copy the CID.
Using the CID "bafyreihw63bea7teb7araypl6sdhhhv57vohawroks4nxogorc2jx7b5oi" of the DAG node created. Search for the DAG node in the IPFS Kubo webui (http://demo:32546/webui/).
Distributed Hash Tables are a form of a distributed database that can store and retrieve information associated with a key in a network of peer nodes that can join and leave the network at any time. The nodes coordinate among themselves to balance and store data in the network without any central coordinating party.
DHTs have the following properties:
- Decentralised & Autonomous: Nodes collectively form the system without any central authority.
- Fault Tolerant: System is reliable with lots of nodes joining, leaving, and failing at all times.
- Scalable: System should function efficiently with even thousands or millions of nodes.
How does DHT work ?
As a simple example the diagram below represents a ring overlay network (logical network implemented on top of some underlying network) with 6 nodes. To find out which node (peer) will get to store a specific key we just need to hash that key. Each node will have a hash value (Peer Id) and will store 5 keys. Making each node an independent hash table bucket in that ring overlay network.
For this example we are using the DHT to map a data identifier to a peer; this is a "Provider records" type of key-value pairing. It's used to find and advertise content. There are other two main types, "IPNS records" (map an IPNS key to a IPNS record) and "Peer records" (map a Peer Id to a set of multi addresses at which the peer may be reach)
Most of the DHTs implementations support the following 3 basic functions:
- put (key, value) We are going to use the Kubo RPC api-v0-routing-put (Write a key/value pair to the routing system.)
- get (key) We are going to use the Kubo RPC api-v0-routing-get (Given a key, query the routing system for its best value.)
- provide (key) We are going to use the Kubo RPC api-v0-routing-provide (Announce to the network that we are providing given values.)
How to create an IPNS record ?
Execute the GO(go1.20.2) script ipnsRecord.go.
Copy the private-key-ipns-record.bin file to the src path and run the Rust project api-v0-key-import.
/ipns/k2k4r8lpp59iv154i7dfnd5m99tke25rqhqaybpssnk3ds5h5t5boe8jCopy the signed-ipns-record.bin file to the src path and run the Rust project api-v0-name-inspect against the IPNS name /ipns/k2k4r8lpp59iv154i7dfnd5m99tke25rqhqaybpssnk3ds5h5t5boe8j.
/ipns/k2k4r8lpp59iv154i7dfnd5m99tke25rqhqaybpssnk3ds5h5t5boe8j{
"Entry": {
"Value": "/ipfs/QmUfV4m2PUM559LSvDsJkoz1KofTVq25RDXwW5uMdjNb4u",
"ValidityType": 0,
"Validity": "2023-04-03T13:29:58.128901162Z",
"Sequence": 0,
"TTL": 60000000000,
"PublicKey": "mCAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDSVhekFyk3/EIdJW530Zip/MeliHGDumXpbT6dBB/BWTP8wv7ioRXAiH0fs9v6Uflglw1VqN+08gs4ScbWkeRaVP1Q2d+lzffeTwiG4eZm/O9OGmdPQfXP/nSOVsKjDITxrVnYLoLLXnHpRcSMuueqpULMpxKj1aT3tSetn7YgrC1TuT3A40RVtwi8ODly9dMkzYH1lqzqMNtqHFumAyPN1hgEDHN2awbQs7KDLsvHF/LFUVlq/xZb6mbmiZBhIfv/YpvTNA9sYZVvL02Q1NqqOvcg0T5SY/lLNv8w1PZuePV58ZMJOHYHgP22MR5hnmrSYOlpebbU8A3EJCgG9FGvAgMBAAE",
"SignatureV1": "meWDwdTYzNsaCrn/rP6IVaa5cnmqs5tYA6OKN1p647UcoGGDrurSgVXvIKmjIlKr6JigJB6NL2cnRksDdX+vYQdSNejb2bqtne+bG7jmc2hpj4xQHMWpVdgKdN1GwG1rJdcrEJnzqt0eNhiwKOMmhC8ynlx7B/8vzw+70unxCyBaWeXnumJYKXzymSRIe2UdI87WRzu6NEJh7QJuIBOuwlGjAyRepCAodUCVqr/Fl2hAtdlyRGmavtwGda6TmVp9Xi2F+o2qOwLJEvA6fMiR5uH0YoXnMdN7DY3IVr0+BmKnPyCsoCx+X5OsVofCkWpEKFvJKfY9ILeqxEp/7DbvabQ",
"SignatureV2": "modrb+wC4mBtUahM/KhSm1tYWhUj73fENvmceLAklbYYVfWwnmAHl1VGrlOmDVrdJv40+7gQgQcMUTwfk1S9XWqwopbBHhfKZxb0RFIoGy/Fmpay0rHpYBzLJBM5ah+Xcz6rnfo0FsNSjLz4PFl/dG4gNPNL1Pv9W01G3abjTEcZoTm8BOh7wRkbagG1eO/UayKaXeCRVgd/9xfKF9XJs0jo3RIbbf4lXbWme1j7k8vO861OCEp+bky9P5eQ/dnYnjxhysE6HkvbofqJxg9VVfyhe7DC46ZXu0FVmw2PznrsHz+mPRogZwawVrqxh+6YBkbJTJhd4YoEZdd5MEnZUaA",
"Data": {
"Sequence": 0,
"TTL": 60000000000,
"Validity": {
"/": {
"bytes": "MjAyMy0wNC0wM1QxMzoyOTo1OC4xMjg5MDExNjJa"
}
},
"ValidityType": 0,
"Value": {
"/": {
"bytes": "L2lwZnMvUW1VZlY0bTJQVU01NTlMU3ZEc0prb3oxS29mVFZxMjVSRFh3VzV1TWRqTmI0dQ"
}
}
}
},
"Validation": {
"Valid": true,
"Reason": "",
"PublicKey": "QmUYNURRiUgNo1YevHW7XH8dDTqw3ywKdUZN6PiVP1deJe"
}
}Copy the signed-ipns-record.bin file to the src path and run the Rust project api-v0-routing-put.
To get the new route created just, run the Rust project api-v0-routing-get.
To resolve the IPNS name k2k4r8lpp59iv154i7dfnd5m99tke25rqhqaybpssnk3ds5h5t5boe8j we will use the Rust project api-v0-name-resolve.
IPFS CID
/ipfs/QmUfV4m2PUM559LSvDsJkoz1KofTVq25RDXwW5uMdjNb4uNote: IPNS names are mutable pointers to immutable pointers IPFS CIDs (immutable because they're derived from the content).
Resolve the IPNS Name in the browser (https://ipfs.io/ipns/k2k4r8lpp59iv154i7dfnd5m99tke25rqhqaybpssnk3ds5h5t5boe8j)
Resolve the IPFS CID in the browser (https://ipfs.io/ipfs/QmUfV4m2PUM559LSvDsJkoz1KofTVq25RDXwW5uMdjNb4u)
To announce the new IPFS CID to the overlay network just, run the Rust project api-v0-routing-provide.
List of the peer Ids.
In a life span of 5 years, 90% of technology startups fail. Despite of not being the only reason, the common denominator present in all of those, was/is the wrong technical choices made. Those choices limited the time to market, the ability to scale and most important time to innovate. The balancing between profitability and growth is extremely difficult to managed by itself. Without having to be limited in our actions by a software that does not scale, was poorly implemented and have an expensive operational cost.
References:
Actix Web framework
SQLx
PostgreSQL
Asynchronous Programming in Rust
Testing
React TypeScript Cheatsheets
Type Script Language
IPFS