Forem: xutweety

Build a decentralized CRUD application using Typescript, React, and DB3.js

xutweety — Fri, 03 Mar 2023 10:02:32 +0000

Introduction

DB3.js is a JavaScript API development toolkit that complements Web3.js in dApp development. With it, you can write data into the DB3 Network (a decentralized database), query data, and more.

To help more front-end developers seamlessly switch from the web2 world to the web3 era and quickly build decentralized applications, DB3.js adopt the philosophy of the Firebase SDK in mind because DB3 Network aims to become a decentralized alternative to Firebase Firestore.

Firebase is one of the most popular backend-as-a-service options in modern technology stacks, suitable for multiple platforms, including mobile, Unity, Java, C++, and web. The drawback of the past SDK V8 version was its large size, which caused slow network connections. V9 version made improvements such as functional programming style, no-side-effect imports, etc. These more advanced design concepts reduced the size of the V9 version by 80%. How to design DB3.js to be more lightweight and help decentralized applications improve their user experience on mobile devices, etc., is also one of the directions in which DB3.js needs to learn from Firebase.

Using DB3.js to Show the CRUD Process of Decentralized Applications

This article will transform a classic demo example of TodoMVC into a decentralized TodoMVC using DB3.js. We will show how to use DB3.js to perform CRUD operations and demonstrate the process of writing, reading, modifying, and deleting data in decentralized applications. We will complete a to-do list application as shown below. You can click here to view the demo code.

During the process, we will display the following CRUD operations:

Create: Enter a new to-do item, add the to-do item to the list, and complete the addition process by pressing Enter.
Read: Listen to the document in the collection and display the list, including all to-dos, active to-dos, and completed to-dos.
Update: Modify the content of the to-do item; when it is done, click it to mark it as completed.
Delete: Delete the to-do items.

(Please note that there may be some instability as we connect to a test node in our network. If you encounter any problems, you can try again later or submit a bug on GitHub)

Running the Demo Directly

Step 1: Start DB3 Network

git clone https://github.com/dbpunk-labs/db3.git
cd db3 && bash install_env.sh && cargo build
# start localnet
cd tools &&  sh start_localnet.sh

After starting, you can see network activity similar to the following:

Step 2: Start DB3 CLI, Create Database, and Collection

The database address 0x7720f39c6bf077582ab89797c90ca7701ea82ea9 in the codeblock below is the database address for the todo application to be created next. Then, we create new collection under its database address, named. todos.

(base)  ✘ xuman@192  ~/Documents/GitHub/dbpunkslabs/db3   main  ./target/debug/db3 console       

██████╗ ██████╗ ██████╗ 
██╔══██╗██╔══██╗╚════██╗
██║  ██║██████╔╝ █████╔╝
██║  ██║██╔══██╗ ╚═══██╗
██████╔╝██████╔╝██████╔╝
╚═════╝ ╚═════╝ ╚═════╝ 
@db3.network🚀🚀🚀
db3>-$ new-db
 database address                           | transaction id 
--------------------------------------------+----------------------------------------------
 0x7720f39c6bf077582ab89797c90ca7701ea82ea9 | dw44D6TeBRRZJ8U6fCiVvLPuAvi/B/xBSOAtYMJgUBk=
db3>-$ new-collection --addr 0x7720f39c6bf077582ab89797c90ca7701ea82ea9 --name todos --index '{"id":0, "name":"owner_idx","fields":[{"field_path":"owner","value_mode":{"Order":1}}]}'
send add collection done with tx
o6m7DHogWxaFb6jLQRu5aPzPSwgAP5JK1kBYohNiq7c=

Step 3: Running the Todo Application

Download the todo application code and open the corresponding demo folder.

git clone <https://github.com/dbpunk-labs/db3.js.git>
cd db3.js/examples/todomvc

Next, build:

yarn
yarn start

Then use a browser to open http://localhost:3000/ to see the todo application.

Code Overview

Next, we will explain the project's creation process and provide code examples of key parts.

File Overview:

├── src
│   ├── App.tsx
│   ├── base.css
│   ├── context.ts
│   ├── footer.tsx
│   ├── header.tsx
│   ├── index.tsx
│   ├── link.tsx
│   ├── logo.svg
│   ├── main_section.tsx
│   ├── reducer.ts
│   ├── todo_input.tsx
│   ├── todo_item.tsx
│   └── todo_list.tsx

Briefly explain the functions of the above files:

header: displays the application's top "what needs to be done", including the form addTodo for creating a new to-do item by calling operations.
footer: displays aggregation information and controls, as well as a menu classification bar "all/active/completed."
main_section: displays the middle part of the to-do list.
reducer and todo_item: define the operations of CRUD and the update of operation status, which is the focus of this article.
todo_input: mainly defines the content input and the event monitoring of Enter.

Component Overview

The referencing relationship is quite complex, but don't worry; we can see that almost all other components are imported from the reducer, mainly because the reducer file defines almost all states, operations, and state change processes. The CRUD process that we focus on this time is also mainly described by the Reducer.ts file.

State, Operations, and Filtering Processes in Reducer

State and Content:

In concept, this Todo application mainly needs to define the following two parts of content:

A list of to-do items
The current filtering option (all/todo/complete)

Each to-do item needs to store the following information:

The text content of the to-do item (line 17)
A boolean value indicating whether it has been completed (line 18)
The owner of this to-do item data (line 19) (The owner concept is unique to the DB3 Network, and you can view the issue.)

The filtering process needs to be determined based on the status information of the to-do items:

An array of document lists (line 23)
A boolean value indicating whether it is loaded (line 24)
The name of the collection is stored in DB3 (line 26)
The user's wallet address (line 27) (PS. The wallet and its corresponding signature are automatically generated. For quick demo purposes, these pieces of information are currently stored directly in the browser. In the future, we will support the MetaMask wallet.)
Where to render and display under which filtering menu (line 28)

CRUD Operations

In the CRUD process, the first step is to enumerate the CRUD action (line 31-36): INSERT / UPDATE / QUERY / DELETE; the second step is that since we are not updating synchronously here, that is, the TodoState will not be updated immediately when dispatching the action. Therefore, we need to define which information changes asynchronously during this process (line 38-47).

Then, we can trigger asynchronous operations according to the CRUD action (line 53-99):

INSERT will trigger the collection and addDoc methods of DB3.js, complete adding the addDoc to the corresponding db and collection, and then trigger the QUERY operation accordingly;

UPDATE will trigger the updateDoc method of DB3.js, and the data in DB3Store will be updated accordingly, and the size and field of the doc will also be updated accordingly. After completion, the QUERY operation will also be triggered;

DELETE will trigger the deleteDoc method of DB3.js, and the data in DB3Store will be updated accordingly. After completion, the QUERY operation will also be triggered;

QUERY will trigger the getDocs method of DB3.js to list all docs, and at the same time, it will verify whether the owner of each to-do item is the current userAddress to filter the to-do items that belong to the current userAddress. After filtering, it will be refreshed;

All the above-mentioned DB3.js methods (except for getDocs) are collectively called mutations in DB3 Network. They will communicate with DB3 Network; all operation results will be recorded on the chain and cannot be tampered with.

Filtering and State Updates

As mentioned earlier, defining the filtering process is another key notion this application must fulfill. The runfilter method (lines 103-116) is defined to filter the display based on the status of each to-do item.

The main function of the reducer is to describe the process of generating nextState based on (previousState, action). Therefore, after the asynchronous operation is completed, when REFRESH is triggered, nextState remains unchanged, but the new todoList: resultSet is displayed; when LOADING and UNLOADING are triggered, nextState remains unchanged.

The above is from the sample code of the TodoMVC. As mentioned earlier, we will add two features in the future:

Adding support for MetaMask wallet addresses
Add network operation logs in UI

If you find our content helpful, please give us a Github ⭐️.

Reference

https://dev.to/chroline/why-the-new-firebase-web-v9-modular-sdk-is-a-game-changer-nph

https://zhuanlan.zhihu.com/p/551650121

https://redux.js.org/tutorials/fundamentals/part-3-state-actions-reducers

Create An EVM Compatible Blockchain Bridge——DB3 Network build in public weekly update (20230213-20230218)

xutweety — Sun, 19 Feb 2023 16:40:28 +0000

Brief intro

We began to refactor our bill and gas system based, use the EVM chain to settle, and prepare for our network's economic model and the token release.

Besides, more CLI and db3.js features are implemented, such as:

release a new version
Feat/delete doc in CLI
Create wallet/database/collection using db3.js

Brief pic about the EVM chain settlement design:

Key PRs:

use u64 to store the balance of bill and credit

Code:

db3>-$ new-db
 database address                           | transaction id 
--------------------------------------------+----------------------------------------------
 0xa15c58bf6ee75980b753396adb4d878129a47468 | qx7NsYyUOKondVHQjG0/itneIAujJofKsUc39Sy0Bpc= 
db3>-$ show-account
 address                                    | bills        | credits       | storage_used | mutations | session | nonce 
--------------------------------------------+--------------+---------------+--------------+-----------+---------+-------
 0x539a6192e4fcd61f53772a3284c7734bc0f6f1d6 | 0.000000 db3 | 10.000000 db3 | 102.00       | 1         | 0       | 0 
db3>-$ show-state
 name       | state 
------------+---------
 database   | 1 
 collection | 0 
 documemt   | 0 
 account    | 1 
 mutation   | 1 
 session    | 0 
 storage    | 102.00  
db3>-$

Detail:

Show network status in CLI; CLI could show the status of the following concept:

DB-related concepts:

Database
Collection: aka 'table'
Document: aka 'record'
Mutation: aka 'write into / update'
Session: aka ‘query’
Storage: the storage size of your data

Account-related concepts:

Address:
Bills: bills are generated by the consumption of data mutation and storage
Credits: The credits are set by the initial number 10

refactor the bill and gas system

refactor the bill and gas system, 1 MB data storage will cost 1DB3, you can experience it in our CLI tool.

Add EVM chain bridge

Key issues opened:

refactor the bill and gas system

We began to refactor our bill and gas system and prepare for an economic model and the token release.

Help spread the work by giving us a star🌟 on Github

Helping Firebase Developers Integrate with Web3 and Build a Decentralized Social Data Model on the DB3 Network

xutweety — Sat, 11 Feb 2023 14:33:38 +0000

Currently, most of the web3 developers still start their developers's careers with exposure to web2's technology stack. For example, Solidity language is like the combination of C++ and Javascript. The node providers are often compared to Amazon Web Services (AWS). So finding a similar developing stack to cut in lets web2 developers seamlessly to get familiar with web3 quickly.

Today we will use a decentralized firebase_DB3 Network, to help the developer to write data into a decentralized database.

In the next 5 minutes we will walk you through the social dApp data model and show you how to build with DB3 Network.

We have three core concepts: database, collection, document. And we use these concepts to build a social data model.

To create this data model in DB3 Network we can use the CLI and db3.js. Let’s use the CLI for now.

Build

git clone https://github.com/dbpunk-labs/db3.git
cd db3 && bash install_env.sh && cargo build
# start localnet
cd tools &&  sh start_localnet.sh

Use Console

Start DB3 console

(base) ~/Documents/GitHub/dbpunkslabs/db3   main  ./target/debug/db3 console

██████╗ ██████╗ ██████╗ 
██╔══██╗██╔══██╗╚════██╗
██║  ██║██████╔╝ █████╔╝
██║  ██║██╔══██╗ ╚═══██╗
██████╔╝██████╔╝██████╔╝
╚═════╝ ╚═════╝ ╚═════╝ 
@db3.network🚀🚀🚀
db3>-$ help
Usage: <COMMAND>

Create a new database

db3>-$ new-db
 database address                           | transaction id 
--------------------------------------------+----------------------------------------------
 0x3b3b3570100a7a02aa3efbad065e2eefb3f46bc7 | cg4flp3sIyt2MJuIqY1TRlFh8MnlSqNQfKnC7TxQnOU=

Show database

db3>-$ show-db --addr 0x1775b12ff28b48cdf66c2201d944c737514fe31f
 database address                           | sender address                             | releated transactions                        | collections 
--------------------------------------------+--------------------------------------------+----------------------------------------------+-------------
 0x3b3b3570100a7a02aa3efbad065e2eefb3f46bc7 | 0x72a2920c4f4b81a665aafb1f5f6e79e14bb9ce75 | 84Qs3vTO2Dp+ZCcIzeKWZ+5nGTycoCJRlAIh362KUjc= |

Add a collection to database

Collection is aka data table.Then we create a userinfo collection that records the social user information and set the indexers.

db3>-$ new-collection 
--addr 0x1775b12ff28b48cdf66c2201d944c737514fe31f 
--name userinfo 
--index '{"id": 1,"name": "idx1","fields": [{"field_path": "name","value_mode": {"Order": 1}},{"field_path": "bio","value_mode": {"Order": 2}}]}'
send add collection done with tx
tQ4dU+46ZkvsM0s2pbjupWx2u8SLbRZmCkG6leG39Us=

db3>-$ show-collection --addr 0x1775b12ff28b48cdf66c2201d944c737514fe31f
 name     | index 
----------+-------------------------------------------------------------------------------------------------------------------------
 userinfo | {"name":"idx1","fields":[{"field_path":"name","value_mode":{"Order":1}},{"field_path":"bio","value_mode":{"Order":2}}]}

Add a Document to the Collection

Add a document (aka record) to the Collection. Here we add a user profile as a document.

db3>-$ new-doc
    --addr 0x3b3b3570100a7a02aa3efbad065e2eefb3f46bc7 
    --collection-name userinfo 
    --documents '{"name": "Bob","age": 43,"bio": "a virtual person in Blockchain"}'
send add document done with tx
+O9cK2cHLexZQvIITk4OTm8SxBhq7Yz7g+xZYiionWo=

Next, we will add more features to our CLl and db3.js. And we will explain the permission relationships helps to claim the ownership of your social data. And more features in a social dApp could be implementes:

Following/follower relationships
Deleting posts.

As a web3 open source project, we are looking for your support.
GitHub: Give us a star

When you plan to build a dApp, you need DB3 Network

xutweety — Fri, 13 Jan 2023 13:18:06 +0000

The Background

Ethereum has defined the novel dapp data architecture successfully.

The novel dapp data architecture has the following improvement:

immutable, when the protocol has been deployed, all the codes are immutable
transparent, the smart contract can be verified by anyone and any organization
Permissionless, everyone can use the protocol in the world
but Ethereum has some limitations e.g. scalability problems and high storage cost and it can not cover the requirement of the storage-heavy dapps. So a hybrid data architecture so called web2.5 is used for the storage-heavy dapps.

The problem in the above is the hybrid data architecture will break the improvement that the web3 native application data architecture brings to us and the db3 network is trying to resolve the problem

What’s the DB3 Network

DB3 is a community-driven layer2 decentralized database network with the following core features:

Infinite Storage Space

Scalability is the key to the web3 explosion, db3 will use the following strategies to achieve web3 scale. PC can meet the minimum system requirements so everyone can join the db3 network to provide storage space. Using dynamic sharding to achieve scale out. when a storage shard chain has not enough space to store mutation, it will split itself into two subchains.Using cold data archive to recycle storage space. history cold blocks and cold state data will be archived to FileCoin and the storage node will always has storage space to store new data.

Blazed Fast and Provable On-chain Query

Currently, decentralization means bad performance but db3 is trying to make a big improvement in performance. Merkdb 2 is the storage engine of db3 network and it not only has high performance but also the fast-proof generation.Geo distribution, the nodes in every storage shard are geo-distributed and the clients can execute querys against the nearest storage node. Query session, the first decentralized query protocol to resolve performance and incentive perfectly

Crypto Native Data Ownership

In the decentralized network, only the private key owners can update their data and they can keep privacy by encrypting their data with the public key

On-chain Programmable

Dapp developers can develop data processing contracts and deploy them to the db3 network just like developing data backend in web2

Ethereum Guarded Security

DB3 network is a layer2 network on Ethereum and all the assets are guarded by Ethereum.

db3.js-Developer-friendly SDK

db3.js is the db3 network 2 javascript API and you can write and query JSON documents against the db3 network with db3.js and you can combine the web3.js and db3.js to build a fully decentralized storage-heavy application. the web3 developers can use this API to put and get documents with the db3 miner node directly.

request.onsuccess = function () {
  console.log("Database opened successfully");
  const db = request.result;
  // 1
  const transaction = db.transaction("cars", "readwrite");
  //2
  const store = transaction.objectStore("cars");
  const colourIndex = store.index("cars_colour");
  const makeModelIndex = store.index("colour_and_make");
  //3
  store.put({ id: 1, colour: "Red", make: "Toyota" });
  store.put({ id: 2, colour: "Red", make: "Kia" });
  store.put({ id: 3, colour: "Blue", make: "Honda" });
  store.put({ id: 4, colour: "Silver", make: "Subaru" });
  //4
  const idQuery = store.get(4);
  const colourQuery = colourIndex.getAll(["Red"]);
  const colourMakeQuery = makeModelIndex.get(["Blue", "Honda"]);
  // 5
  idQuery.onsuccess = function () {
    console.log('idQuery', idQuery.result);
  };
  colourQuery.onsuccess = function () {
    console.log('colourQuery', colourQuery.result);
  };
  colourMakeQuery.onsuccess = function () {
    console.log('colourMakeQuery', colourMakeQuery.result);
  };
  // 6
  transaction.oncomplete = function () {
    db.close();
  };
};

The overview of db3 network architecture

Why does DB3 Network focus on the storage-heavy dapps

image

The more dynamic data the dapp used , the more users that the dapp will onboard.So we think the storage-heavy dapps are the future of web3

Other Decentralized Databases

the graph, a decentralized on-chain indexer
Locutus, a decentralized key-value database
ceramic network, a decentralized data network that brings unlimited data composability to Web3 applications
kwil, the first permissionless SQL database for the decentralized internet
spaceandtime, a decentralized data Warehouse

if you are interested in the db3 network, please give us a star on GitHub

How to store NFT metadata decentralized and dynamically

xutweety — Tue, 20 Dec 2022 08:01:28 +0000

image from https://blog.chain.link/what-is-a-dynamic-nft/

It is difficult for the metadata in dNFT(dynamic NFT) to achieve “decentralization” and “dynamic change” at the same time.

Decentralized metadata storage

About 48% of NFT (most NFTs are static) metadata is decentralized and stored on IPFS. The format suffix is ipfs/your_collection_file_hash/tokenid, where your_collection_file refers to the NFT collection, where tokenid is a number, such as “26”. An example of such a project is Bored Ape Yacht Club, the metadata of which is stored in IPFS, although the BaseURI used for IPFS can be changed by the contract owner.

Suppose the platform wants to provide NFT customization capabilities and put metadata in decentralized storage. In that case, developers need to exhaust all possible combinations of attributes, put all the varieties of metadata on IPFS, and then index according to the combination of attributes to find the tokenURI in IPFS to get the metadata.

The disadvantages of this approach are:

There are many metadata combinations, leading to much storage cost in advance. You have to store them all in advance.

If the platform wants to add a Smartglasses option to the feature of Glasses, or when it provides the ability to change dynamically, it can change the metadata information in the NFT about the game level and other metadata information in the NFT according to the goals of the World Cup during a match. At this point, the timeliness requirements for metadata changes are higher. For example, this dynamic change almost equals replacing a new NFT, and the cost of changing NFT metadata will rise sharply.

Dynamically update first

The second method is to abandon the decentralized metadata storage and only put the ownership on the Chain to ensure timely and rapid dynamic changes. Some projects generate metadata through a centralized server, and the metadata information of the customized NFT will be combined and generated in a centralized manner.

To be able to display different houses in 2022 and 2027, NFT minter should first put other roofs on the Chain, store metadata off the Chain, and the URL of a centralized service to do the correlation, pointing to one of them in 2022, and replacing it in 2037 to another one.

However, they store metadata on their central storage, and these servers can fail in the blink of an eye, costing NFT owners millions worldwide.

More importantly, it isn’t easy to guarantee that such storage will not be tampered with. Especially for some game NFTs, their value does not lie in the displayed pictures but in the information that is crucial in the game, such as force values, levels, etc. This information is stored in metadata. It is impossible to guarantee that only the owner of the NFT owner who can change the metadata based on centralized storage.

And when the metadata storage is separated from data on the Chain, the dynamic metadata history does not leave traces on the Chain, and the two are unrelated, so it isn’t easy to carry out practical analysis and insight.

Currently, only the first two columns of data are kept on-chain, which you can see directly from platforms like OpenSea. But you only know that the price is increasing over time, but you need to understand why. The last data column is in the platform’s centralized server, which is unreliable. For example, a sharp price increase in 2029 is due to the repair of a new roof — the information about this is in the third column of metadata. Because the metadata is changing and is not stored on the Chain, it may be tampered with, so you cannot know, nor is it challenging to analyze and understand what a more reasonable price in 2037 is. These problems may lead to a poor display experience of dNFT in the NFT trading market, making it difficult for dNFT to circulate and prosper in the secondary market.

For example, in OpenSea, a dNFT that changes with the daily sunshine conditions rotates the animation every 8 hours, the change history of metadata is not presented on the website, and the dynamic feature is completely invisible when you first enter it. You have no way of knowing this is dynamic.

Put all the things on the Chain.

Therefore, it is best to put all metadata, images, transaction information, etc., on the Chain and can add the metadata and change it conveniently and dynamically. Your signature should perform the changes, and the NFT platform as an intermediary minimizes the direct touch of data (reading and writing, caching, etc.).

DB3 network

Here, let me introduce the DB3 network, a decentralized storage solution that adopts the key-value storage paradigm and has advanced database properties, such as mutation and programmability.

Change metadata easily based on a decentralized storage.

DB3 can support convenient CRUD-like operations, insert, update, read and delete stored data. This is easy to achieve for centralized storage, but it is challenging to accomplish for decentralized storage. The reason is that the update operation conflicts with the immutable characteristics of the blockchain. But in use cases such as the dynamic NFT mentioned above, immutable storage is important from a security point of view, but it limits the number of possible applications (such as the evolutionary ability of NFT).

A mutation is defined in the DB3 network as our core concept. All operations on data modification are defined as mutation. You can use mutation to insert, update and delete data. For example, you can easily complete the data renewal when you want to modify roof color meatada.

>put house roofcolor offwhite
submit mutation to mempool done!
>get house roofcolor
roof -> offwhite

At the same time, mutation can perform data update operations, and the other three: insert/update/delete and the hash returned by these operations will be submitted to the block.

Analyze metadata history onchain
Every metadata change of dNFT will be recorded on the chain like a snapshot. At the same time, because we have optimized the efficiency of our storage engine, we can achieve high-performance queries, not inferior to the web2 database. Therefore, in the future, analysis of dNFT will be very easy to read and display a more credible metadata history.

Claim metadata ownership

Secondly, the description information of dNFT will be stored in a concept similar to the “table” in web2. Since the metadata defines the asset, securing metadata is as important as securing ownership. The operation and modification of this table need to obtain the user’s permission, which is completely controlled by the user, ensuring the ownership of dNFT. The centralized service should barely touch metadata.

Our Github：

Browse some design docs;
Start coding from goodfirstissue;
Discuss with us according the WIPissue.