Forem: megared05

Chainsight Hands-on: Subscribe to On-Chain Events & Collect

megared05 — Tue, 27 Feb 2024 05:02:09 +0000

So far we have introduced three components focused on the purpose of "collecting data. The component we present here, Event Indexer, follows the same objective, but with a slightly different means. It subscribes to events issued on-chain and stores the confirmed event information in the component.

The Event Indexer specifies events issued from EVM-compatible blockchain contracts, and subscribes to and stores those events. It is possible to specify a block number to start subscribing to and collect all events issued in subsequent blocks. This component allows monitoring of the various actions that occur in the contract itself, which is difficult to collect with Snapshot Indexer. To learn more about Events in Ethereum, please see below.

https://ethereum.org/developers/docs/smart-contracts/anatomy#events-and-logs

Let's build an Event Indexer that subscribes to ERC-20's Transfer event, which is typical of smart contracts.

https://eips.ethereum.org/EIPS/eip-20#events

Completed Component Definition

Target the DAI of ERC20 token, Ethereum's stable coin, and build an Event Indexer that collects DAI transfers.

Prerequisite

The Chainsight CLI requires several tools and a csx installation. Please refer to the following articles for this preliminary preparation.

https://dev.to/hide_yoshi/step-by-step-creating-an-evm-price-oracle-with-chainsight-469g

Create a project & add a manifest template

As before, build from an empty project; use csx new --no-samples to create a template.

% csx new --no-samples event_indexer_pj 
Feb 13 12:56:29.488 INFO Start creating new project 'event_indexer_pj'...
Feb 13 12:56:29.489 INFO Project 'event_indexer_pj' created successfully
Feb 13 12:56:29.489 INFO You can add components with:

  cd event_indexer_pj && csx add

Then use csx add to add the Event Indexer, selecting the event_indexer type.

% cd event_indexer_pj
% csx add
✔ Please input Component Name to add · dai_transfer_subscriber
✔ Please select Component Type to add · event_indexer
Feb 13 12:57:22.481 INFO Start creating new component 'dai_transfer_subscriber'...
Feb 13 12:57:22.484 INFO EventIndexer component 'dai_transfer_subscriber' added successfully

From here, we will update the manifest so that DAI Transfer Events can be collected. The fields to be configured are a bit similar to those in the Snapshot Indexer EVM. The datasource field is important for setting the events to be subscribed to. Let's take a look at the generated manifest.

datasource:
  id: '0000000000000000000000000000000000000000'
  event:
    identifier: EventIdentifier
    interface: null
  network:
    rpc_url: https://polygon-mumbai.infura.io/v3/${INFURA_MUMBAI_RPC_URL_KEY}
    chain_id: 80001
  from: 37730337
  contract_type: ERC-20
  batch_size: null

The parameters that need to be modified this time are the id, event, network and from fields. Specify the blockchain network and its contract to be subscribed to in id, network, the RPC URL of the Ethereum blockchain, the chain ID, and the contract address of the DAI.

datasource:
  id: 0x6B175474E89094C44Da98b954EedeAC495271d0F
  event: ...
  network:
    rpc_url: https://polygon-mumbai.infura.io/v3/${INFURA_MUMBAI_RPC_URL_KEY}
    chain_id: 80001
  ...

Specify the event to be retrieved in event, the name to identify the event, and the ABI of the contract. Obtain the ABI for ERC20, which will be called this time, from the link below and create interfaces/ERC20.json.

https://etherscan.io/token/0x6b175474e89094c44da98b954eedeac495271d0f#code

Then from specifies the block at the start of the subscription.

Finally, event and from should be specified as follows.

datasource:
  id: ...
  event:
    identifier: Transfer
    interface: ERC20.json
  network: ...
  from: 17660942

The final manifest is as follows

version: v1
metadata:
  label: Dai Transfer Subscriber by Hands-on
  type: event_indexer
  description: 'Collect Transfer Event of DAI token. This is a sample of the components that will be built in the Hands-on article.'
  tags:
datasource:
  id: 0x6B175474E89094C44Da98b954EedeAC495271d0F
  event:
    identifier: Transfer
    interface: ERC20.json
  network:
    rpc_url: https://eth-mainnet.g.alchemy.com/v2/${ALCHEMY_KEY}
    chain_id: 1
  from: 17660942
  contract_type: ERC-20
  batch_size: null
interval: 1800 # 30min
cycles: null

Generate a code & build a module

Let's generate the canister code with csx generate. If you actually run this command, you will get the following output.

% csx generate
Feb 15 01:24:21.749 INFO Start code generation for project 'event_indexer_pj'
Feb 15 01:24:21.749 INFO Load env file: "./.env"
Feb 15 01:24:21.752 INFO [dai_transfer_subscriber] Start processing...
Feb 15 01:24:21.752 INFO [dai_transfer_subscriber] Interface file 'ERC20.json' copied from builtin interface
Feb 15 01:24:21.753 INFO [dai_transfer_subscriber] Generate interfaces (.did files) ...
Feb 15 01:24:23.007 INFO [dai_transfer_subscriber] Succeeded: Generate interfaces (.did files)
Feb 15 01:24:23.007 INFO Project 'event_indexer_pj' codes/resources generated successfully

There is no need to implement any additional logic in Event Indexer. The build process can be executed as is.

You can generate modules from code with csx build. Let's run it.

% csx build          
Feb 13 13:17:44.320 INFO Start building project 'event_indexer_pj'
Feb 13 13:17:44.320 INFO Load env file: "./.env"
Feb 13 13:17:44.321 INFO Start code generation for project 'event_indexer_pj'
Feb 13 13:17:44.321 INFO Load env file: "./.env"
Feb 13 13:17:44.321 INFO [dai_transfer_subscriber] Start processing...
Feb 13 13:17:44.321 INFO [dai_transfer_subscriber] Interface file 'ERC20.json' copied from builtin interface
Feb 13 13:17:44.389 INFO [dai_transfer_subscriber] Generate interfaces (.did files) ...
Feb 13 13:18:12.780 INFO [dai_transfer_subscriber] Succeeded: Generate interfaces (.did files)
Feb 13 13:18:12.780 INFO Project 'event_indexer_pj' codes/resources generated successfully
Feb 13 13:18:12.780 INFO Start building...
Feb 13 13:18:12.781 INFO Compile canisters...
Feb 13 13:18:51.182 INFO Succeeded: Compile canisters
Feb 13 13:18:51.182 INFO Shrink/Optimize modules...
Feb 13 13:18:51.304 INFO Succeeded: Shrink/Optimize modules
Feb 13 13:18:51.304 INFO Add metadata to modules...
Feb 13 13:18:51.416 INFO Succeeded: Add metadata to modules
Feb 13 13:19:14.065 INFO Project 'event_indexer_pj' built successfully

If you get logs like the above for this one as well, you are done. Finally, deploy and start the component!

Deploy a component

Deploy this component on the Chainsight Platform with csx deploy and csx exec as in previous hands-on sessions.

csx deploy --network ic --component dai_transfer_subscriber
csx exec --network ic --component dai_transfer_subscriber

When you run these, you should get the following logs.

% csx deploy && csx exec
Feb 13 13:19:15.970 INFO Checking environments...
Feb 13 13:19:15.970 INFO Running command: 'dfx ping http://127.0.0.1:4943'
Feb 13 13:19:16.033 INFO Suceeded: dfx ping http://127.0.0.1:4943
Feb 13 13:19:16.033 INFO Running command: 'dfx identity whoami'
Feb 13 13:19:16.093 INFO Suceeded: dfx identity whoami
Feb 13 13:19:16.093 INFO Running command: 'dfx identity get-principal'
Feb 13 13:19:16.167 INFO Suceeded: dfx identity get-principal
Feb 13 13:19:16.167 INFO Running command: 'dfx identity get-wallet'
Feb 13 13:19:20.479 INFO Suceeded: dfx identity get-wallet
Feb 13 13:19:20.479 INFO Checking environments finished successfully
Feb 13 13:19:20.479 INFO Start deploying project 'event_indexer_pj'...
Feb 13 13:19:20.480 INFO Running command: 'dfx canister create --all'
Feb 13 13:19:23.138 INFO Suceeded: dfx canister create --all
Feb 13 13:19:23.138 INFO Running command: 'dfx build --all'
Feb 13 13:19:23.402 INFO Suceeded: dfx build --all
Feb 13 13:19:23.402 INFO Running command: 'dfx canister install --all'
Feb 13 13:19:28.884 INFO Suceeded: dfx canister install --all
Feb 13 13:19:28.885 INFO Running command: 'dfx canister update-settings --add-controller 7fpuj-hqaaa-aaaal-acg7q-cai dai_transfer_subscriber'
Feb 13 13:19:31.795 INFO Suceeded: dfx canister update-settings --add-controller 7fpuj-hqaaa-aaaal-acg7q-cai dai_transfer_subscriber
Feb 13 13:19:31.981 INFO Current deployed status:
Canister Name: __Candid_UI
  Canister Id: {"local": "b77ix-eeaaa-aaaaa-qaada-cai"}
  Controllers: k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe
  Module Hash: 0x934756863c010898a24345ce4842d173b3ea7639a8eb394a0d027a9423c70b5c

Canister Name: dai_transfer_subscriber
  Canister Id: {"local": "bw4dl-smaaa-aaaaa-qaacq-cai"}
  Controllers: 7fpuj-hqaaa-aaaal-acg7q-cai br5f7-7uaaa-aaaaa-qaaca-cai k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe
  Module Hash: 0x80b8cbaa5c666cbd3dce384cf964d883828d78ac2ee2f2ffa1f9287b521499f8

Feb 13 13:19:32.030 INFO Execute canister processing...
Feb 13 13:19:32.030 INFO Load env file: "./.env"
Feb 13 13:19:32.030 INFO Start processing for commands generation...
Feb 13 13:19:32.031 INFO Script for Component "dai_transfer_subscriber" generated successfully
Feb 13 13:19:32.031 INFO Entrypoint Script generated successfully
Feb 13 13:19:32.031 INFO Start processing for commands execution...
Feb 13 13:19:32.031 INFO Run scripts to execute commands for deployed components
Feb 13 13:20:02.789 INFO Executed './scripts/entrypoint.sh' successfully
Feb 13 13:20:02.789 INFO Project "event_indexer_pj" canisters executed successfully

The deployment and operation start instruction is now complete! Now all you have to do is wait for the interval of the automatic execution and you can refer to the data.

Event Indexer has the following interface to retrieve data. Try using these to call components.

events_from_to : (nat64, nat64) -> (vec record { nat64; vec Transfer }) query;
events_latest_n : (nat64) -> (vec record { nat64; vec Transfer }) query;
get_last_indexed : () -> (nat64) query;

This concludes our hands-on with Event Indexer. We will continue to create and publish new hands-on sessions in the future, so please look forward to them!

Written by Megared@Chainsight

Chainsight Hands-on: Generate original indicators from data in indexers

megared05 — Thu, 22 Feb 2024 06:57:59 +0000

We have released three hands-on sessions so far, all of which involved creating components to "collect data.” By using the Snapshot Indexer in the Chainsight Platform, we were able to retrieve all kinds of data from various locations. In this hands-on session, we will build a component that generates a new index/indicator by performing a unique calculation using the data stored by the Snapshot Indexer as input.

This project uses Ethereum price data collected by multiple Snapshot Indexers and calculates their average value to calculate a reliable Ethereum price. The component used as the calculator is Algorithm Lens. In this case, we are collecting Ethereum price data from three Snapshot Indexers. By using different locations for each of these data sources, we aim to improve the reliability of the calculation results.

Completed Component Definition

Algorithm Lens, which performs the calculation, obtains the latest Ethereum price data from the three Snapshot Indexers and performs the calculation using those data as input. The configuration diagram is shown below.

Prerequisite

The Chainsight CLI requires several tools and a csx installation. Please refer to the following articles for this preliminary preparation.

https://dev.to/hide_yoshi/step-by-step-creating-an-evm-price-oracle-with-chainsight-469g

Create a project

As before, build from an empty project; use csx new --no-samples to create a template.

% csx new --no-samples lens-project
Feb 06 02:00:18.697 INFO Start creating new project 'lens-project'...
Feb 06 02:00:18.698 INFO Project 'lens-project' created successfully
Feb 06 02:00:18.698 INFO You can add components with:

  cd lens-project && csx add

Add Snapshot Indexers

This time, we will build three Snapshot Indexers, two of which are Snapshot Indexer EVMs. Two of them are Snapshot Indexer EVMs that use Mainnet's chainlink contract and Arbitrum One's API3 contract as data sources. The last one is Snapshot Indexer HTTPS, which uses coingecko's API as its data source. Please add the following manifest and update project.yaml.

# components/chainlink_eth_indexer.yaml
...
datasource:
  location:
        # Chainlink: ETH/USD Price Feed
    id: 5f4eC3Df9cbd43714FE2740f5E3616155c5b8419
    args:
      network_id: 1
      rpc_url: https://eth-mainnet.g.alchemy.com/v2/${ALCHEMY_MAINNET_KEY}
  method:
    identifier: latestRoundData():(uint80,int256,uint256,uint256,uint80)
    interface: EACAggregator.json
    args: []
interval: ${INTERVAL}
cycles: null

# components/api3_eth_indexer.yaml
...
datasource:
  location:
        # Api3ServerV1
    id: 709944a48caf83535e43471680fda4905fb3920a
    args:
      network_id: 42161
      rpc_url: https://arb-mainnet.g.alchemy.com/v2/${ALCHEMY_ARBITRUM_ONE_KEY}
  method:
    identifier: dataFeeds(bytes32):(int224,uint32)
    interface: Api3ServerV1.json
    args: [0x4385954e058fbe6b6a744f32a4f89d67aad099f8fb8b23e7ea8dd366ae88151d]
interval: ${INTERVAL}
cycles: null

# components/web_eth_indexer.yaml
...
datasource:
  url: https://api.coingecko.com/api/v3/simple/price
  headers:
    Content-Type: application/json
  queries:
    type: static
    value:
      ids: ethereum
      vs_currencies: usd
      include_market_cap: true
      include_24hr_vol: true
      include_24hr_change: true
      include_last_updated_at: true
      precision: 18
interval: ${INTERVAL}
cycles: null

# project.yaml
version: v1
label: eth-lens-project
components:
- component_path: components/chainlink_eth_indexer.yaml
- component_path: components/api3_eth_indexer.yaml
- component_path: components/web_eth_indexer.yaml

I will add a supplement to the manifest you added.

In chainlink_eth_indexer, the Price Feed Contract for Mainnet's Chainlink is specified. Since the ABI used in the interface field declared in the manifest is required, copy the ABI from the link below, create interfaces/EACAggregator.json, and paste the copied contents directly into the file to create the ABI file.

https://etherscan.io/address/0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419#code

The api3_eth_indexer specifies the Contract for API3 of Arbitrum One. The ABI file must be placed here as well. Copy the ABI from the link below and create an ABI file interfaces/Api3ServerV1.json.

https://arbiscan.io/address/0x709944a48caf83535e43471680fda4905fb3920a#code

web_eth_indexer specifies Coingecko's API.

The Snapshot Indexer EVM manifest also sets up an RPC URL to communicate with the blockchain, and in most cases when using RPC, the private RPC endpoint is generated using the Platform associated with Web3 and used. This is accompanied by a special key, which cannot be made public. To enable such RPCs without problems, environment variables are available in the Chainsight manifest. For the above manifest, generate the following .env file.

INTERVAL=60
ALCHEMY_MAINNET_KEY=(YOUR_KEY)
ALCHEMY_ARBITRUM_ONE_KEY=(YOUR_KEY)

Variables declared in this .env file are expanded when the manifest is read.

You have now completed creating the manifest for Snapshot Indexer! Make sure your folder structure looks like this

% ls
artifacts   components  interfaces  project.yaml    src
% ls -1 components
api3_eth_indexer.yaml
chainlink_eth_indexer.yaml
web_eth_indexer.yaml
% ls -1 interfaces 
Api3ServerV1.json
EACAggregator.json

Once you have done this, you can build it by running csx build.

% csx build
Feb 11 01:38:07.827 INFO Start building project 'lens-project'
Feb 11 01:38:07.828 INFO Load env file: "./.env"
Feb 11 01:38:07.828 INFO Start code generation for project 'lens-project'
Feb 11 01:38:07.828 INFO Load env file: "./.env"
Feb 11 01:38:07.832 INFO [chainlink_eth_indexer] Start processing...
Feb 11 01:38:07.833 INFO [chainlink_eth_indexer] Interface file 'EACAggregator.json' copied from user's interface
Feb 11 01:38:08.387 INFO [chainlink_eth_indexer] Generate interfaces (.did files) ...
Feb 11 01:38:09.684 INFO [chainlink_eth_indexer] Succeeded: Generate interfaces (.did files)
Feb 11 01:38:09.684 INFO [api3_eth_indexer] Start processing...
Feb 11 01:38:09.685 INFO [api3_eth_indexer] Interface file 'Api3ServerV1.json' copied from user's interface
Feb 11 01:38:09.778 INFO [api3_eth_indexer] Generate interfaces (.did files) ...
Feb 11 01:38:10.533 INFO [api3_eth_indexer] Succeeded: Generate interfaces (.did files)
Feb 11 01:38:10.533 INFO [web_eth_indexer] Start processing...
Feb 11 01:38:11.203 INFO [web_eth_indexer] Generate interfaces (.did files) ...
Feb 11 01:38:12.087 INFO [web_eth_indexer] Succeeded: Generate interfaces (.did files)
Feb 11 01:38:12.087 INFO Project 'lens-project' codes/resources generated successfully
Feb 11 01:38:12.087 INFO Start building...
Feb 11 01:38:12.089 INFO Compile canisters...
Feb 11 01:38:15.098 INFO Succeeded: Compile canisters
Feb 11 01:38:15.098 INFO Shrink/Optimize modules...
Feb 11 01:38:15.389 INFO Succeeded: Shrink/Optimize modules
Feb 11 01:38:15.389 INFO Add metadata to modules...
Feb 11 01:38:15.640 INFO Succeeded: Add metadata to modules
Feb 11 01:38:15.640 INFO Project 'lens-project' built successfully

When "built successfully" is displayed as shown above, it is OK. The implementation of Snapshot Indexer is now complete.

Algorithm Lens: Add a manifest

Here's the heart of this hands-on: let's build a component in Algorithm Lens that serves as a calculator to calculate the price of Ethereum by averaging multiple data sources.

Add the component with csx add, but this time select the algorithm_lens type.

% csx add
✔ Please input Component Name to add · eth_average_price_calculator
✔ Please select Component Type to add · algorithm_lens
Feb 11 06:16:01.723 INFO Start creating new component 'eth_average_price_calculator'...
Feb 11 06:16:01.727 INFO AlgorithmLens component 'eth_average_price_calculator' added successfully

If this command completes without problems, you will see the following manifest in your project.

# components/eth_average_price_calculator.yaml
...
datasource:
  methods:
  - id: sample_snapshot_indexer_icp
    identifier: 'get_last_snapshot : () -> (record { value : text; timestamp : nat64 })'
    candid_file_path: null
...

Edit this manifest to create a component that can retrieve data from the three Snapshot Indexers built in the previous section. Change the datasource field as follows

datasource:
  methods:
  - id: chainlink_eth_indexer
    identifier: 'get_last_snapshot_value : () -> (record { nat; text; text; text; nat })'
    func_name_alias: chainlink_eth_indexer
  - id: api3_eth_indexer
    identifier: 'get_last_snapshot_value : () -> (record { text; nat32 })'
    func_name_alias: api3_eth_indexer
  - id: web_eth_indexer
    identifier: 'get_last_snapshot_value : () -> (SnapshotValue)'
    func_name_alias: web_eth_indexer

Create a record for each Snapshot Indexer.

In the id field, enter the id of the Snapshot Indexer. (The id is determined from the filename of the YAML manifest that defines the component.)

The identifier is a Candidate for the interface of the function when calling the component. get_last_snapshot_value is one of several fixed interfaces built into Snapshot Indexer. The type of the return value depends on the data structure managed by the component. For further understanding, please refer to the following articles.

Behind Chainsight: A Look Into Snapshot Indexer Component | by Chainsight | Feb, 2024 | Medium

func_name_alias is an optional parameter, but is set for code clarity in the logic to be implemented later. The value specified here will be used as the automatically generated function name.

Algorithm Lens: Generate a template code & make it into your logic

Let's generate code with csx generate.

% csx generate
Feb 11 14:02:32.560 INFO Start code generation for project 'lens-project'
Feb 11 14:02:32.560 INFO Load env file: "./.env"
Feb 11 14:02:32.562 INFO [chainlink_eth_indexer] Start processing...
...
Feb 11 14:02:34.298 INFO [eth_average_price_calculator] Start processing...
Feb 11 14:02:34.380 INFO [eth_average_price_calculator] Generate interfaces (.did files) ...
Feb 11 14:02:36.470 INFO [eth_average_price_calculator] Succeeded: Generate interfaces (.did files)
Feb 11 14:02:36.470 INFO Project 'lens-project' codes/resources generated successfully

When the above generation is complete, the following template code is generated in the eth_average_price_calculator logic code file.

// src/logics/eth_average_price_calculator/src/lib.rs
use eth_average_price_calculator_accessors::*;
#[derive(Clone, Debug, Default, candid :: CandidType, serde :: Deserialize, serde :: Serialize)]
pub struct LensValue {
    pub dummy: u64,
}
pub async fn calculate(targets: Vec<String>) -> LensValue {
    let _result = get_chainlink_eth_indexer(targets.get(0usize).unwrap().clone()).await;
    let _result = get_api3_eth_indexer(targets.get(1usize).unwrap().clone()).await;
    let _result = get_web_eth_indexer(targets.get(2usize).unwrap().clone()).await;
    todo!()
}

There are several module imports, one structure (struct LensValue) and one function (fn calculate). You can customize these structs and functions to turn them into your own logic. It may be easier to understand if you think of it as a fixed interface whose contents can be customized.

The structure LensValue is the output type of the logic used by the Algortihm Lens. In this case, a scalar value, Float, will be fine since it returns the result of calculating the average value. Modify as follows

pub struct LensValue {
-    pub dummy: u64,
+    pub value: f64,
}

We will then touch on the main calculation logic. As you can imagine from the function name, the function declared from the beginning is the function to retrieve data from the data source.

...
    let _result = get_chainlink_eth_indexer(targets.get(0usize).unwrap().clone()).await;
    let _result = get_api3_eth_indexer(targets.get(1usize).unwrap().clone()).await;
    let _result = get_web_eth_indexer(targets.get(2usize).unwrap().clone()).await;
...

Let's use the values obtained here as inputs to the calculation to compute the average. After processing each input to f64 (64-bits float), adjust the number of digits in each data and calculate the average value.

use std::str::FromStr;
...
pub async fn calculate(targets: Vec<String>) -> LensValue {
    let result = get_chainlink_eth_indexer(targets.get(0usize).unwrap().clone()).await;
    let price_from_chainlink = result.unwrap().1;

    let result = get_api3_eth_indexer(targets.get(1usize).unwrap().clone()).await;
    let price_from_api3 = result.unwrap().0;

    let result = get_web_eth_indexer(targets.get(2usize).unwrap().clone()).await;
    let price_from_web = result.unwrap().ethereum.usd;

    let prices = vec![
        f64::from_str(&price_from_chainlink).unwrap() / 10f64.powi(8),
        f64::from_str(&price_from_api3).unwrap() / 10f64.powi(18),
        price_from_web,
    ];
    let average = prices.iter().sum::<f64>() / prices.len() as f64;

    LensValue { value: average }
}

Since the chainlink price data is adjusted to 8 digits and the API3 data is adjusted to 18 digits, we have set this back to an unscaled value. In order to see the actual values of the input data, you can use explorer for blockchain data or make API calls for web data. If you have already deployed Snapshot Indexer, you can also check the data by calling the implemented reference functions.

Source: https://etherscan.io/address/0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419#readContract

Source: https://arbiscan.io/address/0x709944a48caf83535e43471680fda4905fb3920a#readContract

This completes the logic construction. The final code should look like this If this is verified locally to you, development is complete!

// src/logics/eth_average_price_calculator/src/lib.rs
use eth_average_price_calculator_accessors::*;
use std::str::FromStr;
#[derive(Clone, Debug, Default, candid :: CandidType, serde :: Deserialize, serde :: Serialize)]
pub struct LensValue {
    pub value: f64,
}
pub async fn calculate(targets: Vec<String>) -> LensValue {
    let result = get_chainlink_eth_indexer(targets.get(0usize).unwrap().clone()).await;
    let price_from_chainlink = result.unwrap().1;

    let result = get_api3_eth_indexer(targets.get(1usize).unwrap().clone()).await;
    let price_from_api3 = result.unwrap().0;

    let result = get_web_eth_indexer(targets.get(2usize).unwrap().clone()).await;
    let price_from_web = result.unwrap().ethereum.usd;

    let prices = vec![
        f64::from_str(&price_from_chainlink).unwrap() / 10f64.powi(8),
        f64::from_str(&price_from_api3).unwrap() / 10f64.powi(18),
        price_from_web,
    ];
    ic_cdk::println!("{:?}", &prices);
    let average = prices.iter().sum::<f64>() / prices.len() as f64;

    LensValue { value: average }
}

Algorithm Lens: Build & Deploy

Now let's build, generate modules, and deploy as before. This time, be careful about the amount of Cycles remaining since there are many canisters to deploy. By default, about 6T Cycles are required per component.

csx build
csx deploy --network ic
csx exec --network ic

Execute the above command and if the following log is obtained, execution is complete!

% csx build
Feb 11 23:13:55.148 INFO Start building project 'lens-project'
Feb 11 23:13:55.148 INFO Load env file: "./.env"
Feb 11 23:13:55.148 INFO Start code generation for project 'lens-project'
Feb 11 23:13:55.148 INFO Load env file: "./.env"
Feb 11 23:13:55.150 INFO [chainlink_eth_indexer] Start processing...
...
Feb 11 23:13:55.758 INFO [api3_eth_indexer] Start processing...
...
Feb 11 23:13:56.376 INFO [web_eth_indexer] Start processing...
...
Feb 11 23:13:56.973 INFO [eth_average_price_calculator] Start processing...
...
Feb 11 23:13:58.084 INFO Project 'lens-project' codes/resources generated successfully
Feb 11 23:13:58.084 INFO Start building...
...
Feb 11 23:14:05.212 INFO Project 'lens-project' built successfully

% csx deploy --network ic
Feb 11 23:14:09.194 INFO Checking environments...
...
Feb 11 23:14:53.726 INFO Current deployed status:
Canister Name: api3_eth_indexer
  Canister Id: {"local": "br5f7-7uaaa-aaaaa-qaaca-cai"}
  Controllers: 7fpuj-hqaaa-aaaal-acg7q-cai be2us-64aaa-aaaaa-qaabq-cai k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe
  Module Hash: 0xf11c7d0d3f0cfec06ea1d7d7d6ac2ddc0219f964f65666461d758a9cce46e85f

Canister Name: chainlink_eth_indexer
  Canister Id: {"local": "bw4dl-smaaa-aaaaa-qaacq-cai"}
  Controllers: 7fpuj-hqaaa-aaaal-acg7q-cai be2us-64aaa-aaaaa-qaabq-cai k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe
  Module Hash: 0x1832aa2b92faa97065146047f87c503bd84a638ebd6073cb1c88606c663526f9

Canister Name: eth_average_price_calculator
  Canister Id: {"local": "b77ix-eeaaa-aaaaa-qaada-cai"}
  Controllers: 7fpuj-hqaaa-aaaal-acg7q-cai be2us-64aaa-aaaaa-qaabq-cai k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe
  Module Hash: 0xbbd87ce2d215bcc65b130cf13ca21d99dfbb2d9827dd5c8c2f8d10b0f8e8f337

Canister Name: web_eth_indexer
  Canister Id: {"local": "by6od-j4aaa-aaaaa-qaadq-cai"}
  Controllers: 7fpuj-hqaaa-aaaal-acg7q-cai be2us-64aaa-aaaaa-qaabq-cai k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe
  Module Hash: 0x6feb60c3a47517392b08b06f42a32ed544d00660137a35290ef43de5d589d14d

% csx exec --network ic
Feb 11 23:14:53.778 INFO Execute canister processing...
Feb 11 23:14:53.778 INFO Load env file: "./.env"
Feb 11 23:14:53.778 INFO Start processing for commands generation...
Feb 11 23:14:53.779 INFO Script for Component "chainlink_eth_indexer" generated successfully
Feb 11 23:14:53.779 INFO Script for Component "api3_eth_indexer" generated successfully
Feb 11 23:14:53.779 INFO Script for Component "web_eth_indexer" generated successfully
Feb 11 23:14:53.779 INFO Script for Component "eth_average_price_calculator" generated successfully
Feb 11 23:14:53.780 INFO Entrypoint Script generated successfully
...
Feb 11 23:16:47.932 INFO Project "lens-project" canisters executed successfully

Calculations in Algotirhm Lens are performed at each call. It is not executed on a regular basis. Therefore, you can check the calculation results with the following command. (You can also use IC Dashboard)

% cd artifacts
% dfx canister call eth_average_price_calculator get_result "vec {\"$(dfx canister id chainlink_eth_indexer)\"; \"$(dfx canister id api3_eth_indexer)\"; \"$(dfx canister id web_eth_indexer)\"}"
(record { value = 2502.2185809524067 : float64 })

If you see this command executed, you have deployed an Algorithm Lens that correctly reflects the original logic you have implemented! Thank you for your patience up to this point.

This hands-on session was more difficult than previous ones due to the large number of components to be built. We hope that through this hands-on, you will be able to experience how Chainsight can reflect the user's customization. This is the end of the hands-on on Algorithm Lens. We will continue to create and publish new hands-on sessions in the future, so please look forward to them!

Written by Megared@Chainsight

Chainsight Hands-on: Collecting Data from Internet Computer

megared05 — Mon, 19 Feb 2024 12:24:53 +0000

Last time we created a hands-on article on creating a component to get Ethereum price data from an Ethereum contract. The Snapshot Indexer EVM uses ABI as its interface to retrieve data from EVM-compatible chains. In this article, we will try to retrieve data from Internet Computer by using Candid, an interface on the Internet Computer where Chainsight is running. In this article, we will try to create a component that collects data in Internet Computer using Snapshot indexer ICP.

This project is a Price Snapshot Indexer ICP that collects ICP prices using a DEX contract on Internet Computer. As of February 2024, 1 ICP is trading at around $12.

Completed Component Definition

ICPSwap, one of the well-known DEXes on the Internet Computer, is used to obtain ICP prices. The ICPSwap contract (canister) is called periodically to collect ICP price data. Snapshot Indexer is constructed with a collection interval of 1 hour.

Prerequisite

The Chainsight CLI requires several tools and a csx installation. Please refer to the following articles for this preliminary preparation.

https://dev.to/hide_yoshi/step-by-step-creating-an-evm-price-oracle-with-chainsight-469g

Create a project & add a manifest template

As before, build from an empty project; use csx new --no-samples to create a template.

% csx new --no-samples sample_icp
Feb 06 02:00:18.697 INFO Start creating new project 'sample_icp'...
Feb 06 02:00:18.698 INFO Project 'sample_icp' created successfully
Feb 06 02:00:18.698 INFO You can add components with:

  cd sample_icp && csx add

Then use csx add to add the Snapshot Indexer EVM. snapshot_indexer_icp type should be selected.

% csx add
✔ Please input Component Name to add · icp_snapshot_indexer
✔ Please select Component Type to add · snapshot_indexer_icp
Feb 06 02:01:11.560 INFO Start creating new component 'icp_snapshot_indexer'...
Feb 06 02:01:11.560 INFO SnapshotIndexerICP component 'icp_snapshot_indexer' added successfully

Let's update this manifest that we have added so that we can call Canister already deployed on Internet Computer. In this case, we will use the Canister used in ICPSwap.

CanisterId: 2ixw4-taaaa-aaaag-qcpdq-cai

Source: https://info.icpswap.com/swap/token/details/ryjl3-tyaaa-aaaaa-aaaba-cai

In the datasource field, specify the canister and its function to call this time.

Specify the ID of the canister to call in the location.id field. In this case, set 2ixw4-taaaa-aaaag-qcpdq-cai. Next, in datasource.method, specify the canister function to call. In method.identifier, specify the function in Candid format, which is the interface notation of Internet Computer.

Candid is an interface description language. Its primary purpose is to describe the public interface of a service, usually in the form of a program deployed as a canister that runs on the Internet Computer. One of the key benefits of Candid is that it is language-agnostic, and allows inter-operation between services and frontends written in different programming languages, including Motoko, Rust, and JavaScript.

Source: https://internetcomputer.org/docs/current/developer-docs/backend/candid/candid-concepts

Internet Computer makes it easy to see the interface of the canister by using the Dashboard.

Source: https://dashboard.internetcomputer.org/canister/2ixw4-taaaa-aaaag-qcpdq-cai

This is the canister's Candid Interface that we will call. The declarations in the service section are the interfaces provided by the canister, and in this case we will use the get_exchange_rates function described here. There are two things we need to do to specify this function.

Specify a function declaration in method.identifier that includes arguments and response information: get_exchange_rates: () -> (vec ExchangeRate__1). Since this declaration uses a custom type that is not a Candid builtin, create a .did file that includes the Candid Interface so that it can be referenced, and specify that file in method.interface. With Candid selected in the Canister Interface of the Dashboard screen mentioned earlier, click "Copy" in the upper right corner to copy the declaration with the interface. Then, create target_interface.did under the interfaces folder and paste what you just copied into its contents. The actual folder and file will look like this

% ls interfaces 
target_interface.did
% cat interfaces/target_interface.did 
type ExchangeRate__1 = 
 record {
   base_class: text;
   base_symbol: text;
   decimals: nat32;
   error: opt Error;
   quote_class: text;
...

Once the file is created, specify interfaces/target_interface.did in method.interface. At this point, the datasource field should look like the following

datasource:
  location:
    id: 2ixw4-taaaa-aaaag-qcpdq-cai
  method:
    identifier: 'get_exchange_rates: () -> (vec ExchangeRate__1)'
    interface: interfaces/target_interface.did
    args: []

Finally, add a setting for this component to call canisters outside of Chainsight Platform. is_target_component should be set to false. If this flag is not set and is true, the component is to call a component within Chainsight Platform. This completes the update of the manifest. The completed manifest is as follows

version: v1
metadata:
  label: ICP Snapshot Indexer by handson
  type: snapshot_indexer_icp
  description: 'Collect ICP prices. This is a sample of the components that will be built in the Hands-on article.'
  tags:
  - Price Snapshot Indexer
  - ICP
  - "Hands-on Sample"
datasource:
  location:
    id: 2ixw4-taaaa-aaaag-qcpdq-cai
  method:
    identifier: 'get_exchange_rates: () -> (vec ExchangeRate__1)'
    interface: interfaces/target_interface.did
    args: []
is_target_component: false
lens_targets: null
interval: 3600
cycles: null

For metadata, any arbitrary setting is fine. Check the datasource field and is_target_component again. If you have successfully modified the code up to this point, let's generate the code immediately.

Generate a code & build a module

Let's generate the canister code with csx generate. If you actually run this command, you will get the following output.

% csx generate
Feb 06 07:22:01.688 INFO Start code generation for project 'sample_icp'
Feb 06 07:22:01.691 INFO [icp_snapshot_indexer] Start processing...
Feb 06 07:22:01.694 INFO [icp_snapshot_indexer] Generate interfaces (.did files) ...
Feb 06 07:22:14.300 INFO [icp_snapshot_indexer] Succeeded: Generate interfaces (.did files)
Feb 06 07:22:14.300 INFO Project 'sample_icp' codes/resources generated successfully

As with the Snapshot Indexer EVM, no additional customization is required for this interface in the Snapshot Indexer ICP. The build process can begin without any additional customization.

You can generate modules from code with csx build. Let's run it.

% csx build
Feb 07 14:28:14.403 INFO Start building project 'sample_icp'
Feb 07 14:28:14.403 INFO Start code generation for project 'sample_icp'
Feb 07 14:28:14.406 INFO [icp_snapshot_indexer] Start processing...
Feb 07 14:28:14.410 INFO [icp_snapshot_indexer] Skip creating logic project: './src/logics/icp_snapshot_indexer' already exists
Feb 07 14:28:14.411 INFO [icp_snapshot_indexer] Generate interfaces (.did files) ...
Feb 07 14:28:19.094 INFO [icp_snapshot_indexer] Succeeded: Generate interfaces (.did files)
Feb 07 14:28:19.094 INFO Project 'sample_icp' codes/resources generated successfully
Feb 07 14:28:19.094 INFO Start building...
Feb 07 14:28:19.099 INFO Compile canisters...
Feb 07 14:28:25.620 INFO Succeeded: Compile canisters
Feb 07 14:28:25.620 INFO Shrink/Optimize modules...
Feb 07 14:28:25.693 INFO Succeeded: Shrink/Optimize modules
Feb 07 14:28:25.693 INFO Add metadata to modules...
Feb 07 14:28:25.755 INFO Succeeded: Add metadata to modules
Feb 07 14:28:25.755 INFO Project 'sample_icp' built successfully

The build will be completed in less than a few minutes. This command generates the WASM module and candid interface. For a more in-depth understanding of these behaviors, please refer to the following articles

Behind Chainsight: How Modules are Created Pt. 2 | by Chainsight | Feb, 2024 | Medium

Finally, deploy and start the component!

Deploy a component

Deploy this component on the Chainsight Platform with csx deploy and csx exec as in previous hands-on sessions.

csx deploy --network ic --component icp_snapshot_indexer
csx exec --network ic --component icp_snapshot_indexer

When you run these, you should get the following logs.

% csx deploy --network ic --component icp_snapshot_indexer
Feb 07 14:43:27.188 INFO Checking environments...
Feb 07 14:43:27.188 INFO Running command: 'dfx ping ic'
Feb 07 14:43:27.828 INFO Suceeded: dfx ping ic
Feb 07 14:43:27.828 INFO Running command: 'dfx identity whoami --network ic'
Feb 07 14:43:27.910 INFO Suceeded: dfx identity whoami --network ic
Feb 07 14:43:27.910 INFO Running command: 'dfx identity get-principal --network ic'
Feb 07 14:43:27.987 INFO Suceeded: dfx identity get-principal --network ic
Feb 07 14:43:27.987 INFO Running command: 'dfx identity get-wallet --network ic'
Feb 07 14:43:28.063 INFO Suceeded: dfx identity get-wallet --network ic
Feb 07 14:43:28.063 INFO Checking environments finished successfully
Feb 07 14:43:28.063 INFO Start deploying project 'sample_icp'...
Feb 07 14:43:28.063 INFO Running command: 'dfx canister create --network ic icp_snapshot_indexer'
Feb 07 14:43:32.757 INFO Suceeded: dfx canister create --network ic icp_snapshot_indexer
Feb 07 14:43:32.757 INFO Running command: 'dfx build --network ic icp_snapshot_indexer'
Feb 07 14:43:32.945 INFO Suceeded: dfx build --network ic icp_snapshot_indexer
Feb 07 14:43:32.945 INFO Running command: 'dfx canister install --network ic icp_snapshot_indexer'
Feb 07 14:43:43.151 INFO Suceeded: dfx canister install --network ic icp_snapshot_indexer
Feb 07 14:43:43.151 INFO Running command: 'dfx canister update-settings --add-controller xkzfe-fiaaa-aaaal-qcnuq-cai --network ic icp_snapshot_indexer'
Feb 07 14:43:50.150 INFO Suceeded: dfx canister update-settings --add-controller xkzfe-fiaaa-aaaal-qcnuq-cai --network ic icp_snapshot_indexer
Feb 07 14:43:51.507 INFO Current deployed status:
Canister Name: icp_snapshot_indexer
  Canister Id: {"ic": "lqf7x-4yaaa-aaaal-qdg7q-cai"}
  Controllers: 5naws-oiaaa-aaaal-qbxuq-cai k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe xkzfe-fiaaa-aaaal-qcnuq-cai
  Module Hash: 0xfa98d0d929e40968a31f9da12485805b512ed538ae6b74ec41364057546d1ce3

% csx exec --network ic --component icp_snapshot_indexer
Feb 07 14:43:59.998 INFO Execute canister processing...
Feb 07 14:43:59.998 INFO Start processing for commands generation...
Feb 07 14:44:00.001 INFO Script for Component "icp_snapshot_indexer" generated successfully
Feb 07 14:44:00.001 INFO Entrypoint Script generated successfully
Feb 07 14:44:00.001 INFO Start processing for commands execution...
Feb 07 14:44:00.001 INFO Run scripts to execute commands for deployed components
Feb 07 14:44:00.001 INFO Selected component is 'icp_snapshot_indexer'
Feb 07 14:44:44.155 INFO Executed './scripts/entrypoint.sh' successfully
Feb 07 14:44:44.155 INFO Project "sample_icp" canisters executed successfully

The deployment and operation start instruction is now complete! Now all you have to do is wait for the interval of the automatic execution and you can refer to the data.

The components that can be created in this hands-on session are also actually running! (We have changed the description and execution interval a bit.)

Source: https://beta.chainsight.network/explore/components/lqf7x-4yaaa-aaaal-qdg7q-cai

This concludes our hands-on with the Snapshot Indexer EVM. We will continue to create and publish new hands-on sessions in the future, so please look forward to them!

Written by Megared@Chainsight

Chainsight Hands-on: Collecting Data from Onchain

megared05 — Mon, 12 Feb 2024 06:40:36 +0000

Previously we created a hands-on article on creating a component to collect bitcoin price data from the web. We have built Snapshot Indexer HTTPS to collect data from the web with reliability, but now let's try to get data on-chain. In this article, we will create a component that collects data from Ethereum using the Snapshot Indexer EVM.

In this project, let's create a Price Snapshot Indexer EVM that can get the price of a specified currency using the Price Aggregator Contract on Ethereum.

Completed Component Definition

Let's create a Snapshot Indexer EVM that collects Ethereum prices. In order to get the Ethereum price, we will use the ETH/USD Aggregator Contract in the Ethereum Blockchain to collect the price data. This component is constructed with a collection interval of 1 hour.

Prerequisite

The Chainsight CLI requires several tools and this CLI installation. Please refer to the following article for this preliminary preparation.

Step-by-step creating an EVM Price Oracle with Chainsight - DEV Community

Create a project & add a manifest template

As before, build from an empty project; use csx new --no-samples to create a template.

% csx new --no-samples sample_evm
Jan 30 13:24:51.690 INFO Start creating new project 'sample_evm'...
Jan 30 13:24:51.691 INFO Project 'sample_evm' created successfully
Jan 30 13:24:51.691 INFO You can add components with:

  cd sample_evm && csx add

Then use csx add to add the Snapshot Indexer EVM. snapshot_indexer_evm type should be selected.

% cd sample_evm 
% csx add
✔ Please input Component Name to add · sample_component
✔ Please select Component Type to add · snapshot_indexer_evm
Jan 30 13:25:38.714 INFO Start creating new component 'sample_component'...

Let's update this manifest that we have added and generate a contract that retrieves data from the Aggregator Contract. The contract to be called this time is as follows. Use the Aggregator Contract that Chainlink deploys on Ethereum.

Source: https://docs.chain.link/data-feeds/price-feeds/addresses?network=ethereum&page=1#ethereum-mainnet

In the datasource field, specify the contract and its function to be called this time, setting location.id to 5f4eC3Df9cbd43714FE2740f5E3616155c5b8419 without prefix and method.identifier to latestAnswer():(int256) to get the latest price data from this contract. Finally, an ABI file in JSON format must be obtained, placed in the interface folder, and specified in method.interface. The contents of the ABI can be getted from the following url.

https://etherscan.io/address/0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419#code

The final modified datasource field will look like this

datasource:
  location:
    id: 5f4eC3Df9cbd43714FE2740f5E3616155c5b8419
    args:
      network_id: 1
      rpc_url: https://mainnet.infura.io/v3/${INFURA_MAINNET_RPC_URL_KEY}
  method:
    identifier: latestAnswer():(int256)
    interface: Aggregator.json
    args: []

Once you have the above modifications at hand, let's get started with the code generation!

Generate a code & build a module

csx generate will generate canister code with the manifest in your project as input. Now let's run it to generate the code for the Snapshot Indexer EVM we mentioned earlier.

% csx generate
Feb 02 13:24:12.258 INFO Start code generation for project 'sample_evm'
Feb 02 13:24:12.259 INFO [sample_component] Start processing...
Feb 02 13:24:12.259 INFO [sample_component] Interface file 'Aggregator.json' copied from user's interface
Feb 02 13:24:12.411 INFO [sample_component] Generate interfaces (.did files) ...
Feb 02 13:24:13.039 INFO [sample_component] Succeeded: Generate interfaces (.did files)
Feb 02 13:24:13.039 INFO Project 'sample_evm' codes/resources generated successfully

If the log is displayed like this, the command has successfully completed. Depending on some Component and its parameters, it is possible to make customizations and adjustments to this generated canister code, but in the case of this Snapshot Indexer EVM, this is not necessary. This is the completed canister code. Therefore, this is all that is needed to begin the build process.

csx build builds the generated code into modules and generates what is needed to deploy them as components of Chainsight Platform. Let's run it immediately.

% csx build
Feb 02 13:28:31.518 INFO Start building project 'sample_evm'
Feb 02 13:28:31.518 INFO Start code generation for project 'sample_evm'
Feb 02 13:28:31.519 INFO [sample_component] Start processing...
Feb 02 13:28:31.519 INFO [sample_component] Interface file 'Aggregator.json' copied from user's interface
Feb 02 13:28:31.524 INFO [sample_component] Skip creating logic project: './src/logics/sample_component' already exists
Feb 02 13:28:31.524 INFO [sample_component] Generate interfaces (.did files) ...
Feb 02 13:28:32.443 INFO [sample_component] Succeeded: Generate interfaces (.did files)
Feb 02 13:28:32.443 INFO Project 'sample_evm' codes/resources generated successfully
Feb 02 13:28:32.443 INFO Start building...
Feb 02 13:28:32.443 INFO Compile canisters...
Feb 02 13:29:10.484 INFO Succeeded: Compile canisters
Feb 02 13:29:10.484 INFO Shrink/Optimize modules...
Feb 02 13:29:10.590 INFO Succeeded: Shrink/Optimize modules
Feb 02 13:29:10.590 INFO Add metadata to modules...
Feb 02 13:29:10.685 INFO Succeeded: Add metadata to modules
Feb 02 13:29:10.685 INFO Project 'sample_evm' built successfully

The build process takes less than a few minutes and the artifacts generated by the csx build are placed in the artifacts folder. If you look in that folder, you will see WASM modules and candid interface files that actually have component names as their file names. For a more in-depth understanding, please refer to the following article.

Behind Chainsight: How Modules are Created Pt. 2 | by Chainsight | Feb, 2024 | Medium

Finally, deploy and start the component!

Deploy a component

If you can deploy this module as a component of the Chainsight Platform, all is done.

With csx deploy we can actually deploy to the Chainsight Platform on Internet Computer, and with csx exec we can send the commands to start working. Let's run the command with some parameters to specify the component and execution network.

csx deploy --network ic --component sample_component
csx exec --network ic --component sample_component

If you run these commands and they complete without problems, you should see the following output! The log here has slightly changed the names of the components, so just read and check that point.

% csx deploy --network ic --component eth_snapshot_indexer
Feb 02 13:35:53.077 INFO Checking environments...
Feb 02 13:35:53.077 INFO Running command: 'dfx ping ic'
Feb 02 13:35:54.957 INFO Suceeded: dfx ping ic
Feb 02 13:35:54.957 INFO Running command: 'dfx identity whoami --network ic'
Feb 02 13:35:55.044 INFO Suceeded: dfx identity whoami --network ic
Feb 02 13:35:55.044 INFO Running command: 'dfx identity get-principal --network ic'
Feb 02 13:35:55.121 INFO Suceeded: dfx identity get-principal --network ic
Feb 02 13:35:55.121 INFO Running command: 'dfx identity get-wallet --network ic'
Feb 02 13:35:55.201 INFO Suceeded: dfx identity get-wallet --network ic
Feb 02 13:35:55.201 INFO Checking environments finished successfully
Feb 02 13:35:55.201 INFO Start deploying project 'sample_evm'...
Feb 02 13:35:55.201 INFO Running command: 'dfx canister create --network ic eth_snapshot_indexer'
Feb 02 13:36:02.966 INFO Suceeded: dfx canister create --network ic eth_snapshot_indexer
Feb 02 13:36:02.966 INFO Running command: 'dfx build --network ic eth_snapshot_indexer'
Feb 02 13:36:03.218 INFO Suceeded: dfx build --network ic eth_snapshot_indexer
Feb 02 13:36:03.218 INFO Running command: 'dfx canister install --network ic eth_snapshot_indexer'
Feb 02 13:36:17.031 INFO Suceeded: dfx canister install --network ic eth_snapshot_indexer
Feb 02 13:36:17.031 INFO Running command: 'dfx canister update-settings --add-controller xkzfe-fiaaa-aaaal-qcnuq-cai --network ic eth_snapshot_indexer'
Feb 02 13:36:22.852 INFO Suceeded: dfx canister update-settings --add-controller xkzfe-fiaaa-aaaal-qcnuq-cai --network ic eth_snapshot_indexer
Feb 02 13:36:24.305 INFO Current deployed status:
Canister Name: eth_snapshot_indexer
  Canister Id: {"ic": "5saj5-jqaaa-aaaal-qdfea-cai"}
  Controllers: 5naws-oiaaa-aaaal-qbxuq-cai k245a-2cvyi-rcnzj-cpq6h-dcy7o-qrg4b-325bv-5c3om-jfceo-ikmjz-aqe xkzfe-fiaaa-aaaal-qcnuq-cai
  Module Hash: 0x6c5ee5bdde96e2ef3c49d635d44086538bc38caf43e878a0651cdbbd0584c4ba

% csx exec --network ic --component eth_snapshot_indexer
Feb 02 13:36:34.713 INFO Execute canister processing...
Feb 02 13:36:34.713 INFO Start processing for commands generation...
Feb 02 13:36:34.713 INFO Script for Component "eth_snapshot_indexer" generated successfully
Feb 02 13:36:34.713 INFO Entrypoint Script generated successfully
Feb 02 13:36:34.713 INFO Start processing for commands execution...
Feb 02 13:36:34.713 INFO Run scripts to execute commands for deployed components
Feb 02 13:36:34.713 INFO Selected component is 'eth_snapshot_indexer'
Feb 02 13:37:15.705 INFO Executed './scripts/entrypoint.sh' successfully
Feb 02 13:37:15.705 INFO Project "sample_evm" canisters executed successfully

The deployment and operation start instruction is now complete! Now all you have to do is wait for the interval of the automatic execution and you can refer to the data.

The components that can be created in this hands-on session are also actually running! (We have changed the description and execution interval a bit.)

Source: Detail lcdio-qiaaa-aaaal-qdg4q-cai| Chainsight

This concludes our hands-on with the Snapshot Indexer EVM. We will continue to create and publish new hands-on sessions in the future, so please look forward to them!

Written by Megared@Chainsight

Chainsight Hands-on: Collecting Data from Web

megared05 — Thu, 08 Feb 2024 07:31:47 +0000

So far, in medium.com we have delved deeply into the behind-the-scenes of components and SDKs, as well as showcases that combine various components. This time, we will actually go hands-on to create our own component. In this article, we will try to create a component that actually collects data from the Web using Snapshot Indexer HTTPS.

Completed Component Definition

Completed Component DefinitionThis time, let's create a Snapshot Indexer HTTPS that collects bitcoin prices. There are several APIs that provide bitcoin prices, but we will use coingecho's API to collect the current price. Let's construct Snapshot Indexer with a collection interval of one hour.

Prerequisite

The Chainsight CLI requires several tools and this CLI installation. Please refer to the following article for this preliminary preparation.

Step-by-step creating an EVM Price Oracle with Chainsight - DEV Community

Create a project

The first step is to build a project template. In this case, we only need one manifest, so let's proceed by adding a manifest of the type we want to create from an empty project. The command to create a project is csx new. Give the -no-sample option to make the project empty.

Looking at the created project, the manifest folder, components, is empty.

Now let's run csx add to add the manifest as instructed in the log.

Add a manifest

You can use csx add to interactively add a manifest of the type you wish to add. Let's actually run csx add with the name btc_snapshot_indexer and the type snapshot_indexer_https.

This will generate a manifest (.yaml) for btc_snapshot_indexer and update project.yaml to include btc_snapshot_indexer in the project itself.

Then update this generated manifest so that the bitcoin price can be retrieved. Get the bitcoin price by calling the /simple/price endpoint of the coingecko API.

Crypto API Documentation | CoinGecko

The parameters for actually taking this data are as follows. The id to retrieve bitcoin data is bitcoin. Let's also specify some parameters to extend the data available at this endpoint.

https://api.coingecko.com/api/v3/simple/price
  ?ids=bitcoin
  &vs_currencies=usd,eur,btc,eth
  &include_market_cap=true
  &include_24hr_vol=true
  &include_24hr_change=true
  &include_last_updated_at=true
  &precision=18

Modify datasource field to specify this parameter in the snapshot_indexer_https manifest. Specify the endpoint in the url field. Since the query parameters are static and immutable this time, set queries type to static and value field to the respective key and value.

datasource:
  url: https://api.coingecko.com/api/v3/simple/price
  headers:
    Content-Type: application/json
  queries:
    type: static
    value:
      ids: bitcoin
      vs_currencies: usd,eur,btc,eth
      include_market_cap: true
      include_24hr_vol: true
      include_24hr_change: true
      include_last_updated_at: true
      precision: 18

The difference between the template generated by csx add and the updated manifest is as follows

9,10c9,11
<   - DAI
<   - USD
---
>   - BTC
>   - bitcoin
>   - "Hands-on Sample"
18,19c19,25
<       ids: dai
<       vs_currencies: usd
---
>       ids: bitcoin
>       vs_currencies: usd,eur,btc,eth
>       include_market_cap: true
>       include_24hr_vol: true
>       include_24hr_change: true
>       include_last_updated_at: true
>       precision: 18

Use this manifest to move on to the next step.

Generate a code

To generate the canister code for a component from its manifest, run csx generate with the project.yaml declaring the path to its manifest.

csx generate generates a project to generate the canister module under the src folder. If no customization is required, module generation can be performed immediately by running csx build in this state.

Since this Snapshot Indexer is intended to collect only bitcoin prices, not all data that can be retrieved from the API is required. Also, the more data content, the more expensive it is to retain, and the higher the probability of consensus generation failure when retrieving data via HTTPS outcalls. For more information, please refer to the following article.

Behind Chainsight: How Modules are Created Pt. 1 | by Chainsight | Jan, 2024 | Medium

Therefore, in order to avoid storing unnecessary data, modifications are made to narrow down the data to be collected.
To make customizations to the generated code, modify logics/(canister-name)/src/lib.rs in the src directory. For any type of canister, you can add your own customizations by modifying the file replacing canister-name with the name of the component of interest. If you want to know more about the project generated by csx generate, please refer to this article.

Behind Chainsight: How Modules are Created Pt. 1 | by Chainsight | Jan, 2024 | Medium

Now let's get back to the main issue. Let's open logics/btc_snapshot_indexer/src/lib.rs in your project. The code should look something like this

// Auto-generated code from manifest.
// You update the structure as needed.
// The existence of the SnapshotValue structure must be maintained.
use candid::{Decode, Encode};
#[derive(Debug, Clone, candid::CandidType, candid::Deserialize, serde::Serialize, chainsight_cdk_macros::StableMemoryStorable)]
pub struct SnapshotValue {
    pub bitcoin: Bitcoin,
}

#[derive(Debug, Clone, candid::CandidType, candid::Deserialize, serde::Serialize, chainsight_cdk_macros::StableMemoryStorable)]
pub struct Bitcoin {
    pub usd: f64,
    pub usd_market_cap: f64,
    pub usd_24h_vol: f64,
    pub usd_24h_change: f64,
    pub eur: f64,
    pub eur_market_cap: f64,
    pub eur_24h_vol: f64,
    pub eur_24h_change: f64,
    pub btc: f64,
    pub btc_market_cap: f64,
    pub btc_24h_vol: f64,
    pub btc_24h_change: f64,
    pub eth: f64,
    pub eth_market_cap: f64,
    pub eth_24h_vol: f64,
    pub eth_24h_change: f64,
    pub last_updated_at: i64,
}

This is an implementation for mapping the data obtained by calling the endpoints you specify. The trait you are using has the necessary functionality for the structure, so understand that that is the way it is and don't worry about it. The point of concern is the fields of the structure. Fields for a number of elements have already been declared. Where this code, which is implemented from the beginning, comes from is that the CLI automatically generates an implementation that matches the response from the endpoint you have set up. The actual response data will give you a sense of the implementation for mapping.

{
  "bitcoin": {
    "usd": 42451.59160065723,
    "usd_market_cap": 832331572531.3098,
    "usd_24h_vol": 12740914009.764158,
    "usd_24h_change": 0.7450874696750942,
    "btc": 1.0,
    "btc_market_cap": 19611887.0,
    "btc_24h_vol": 300160.8233960758,
    "btc_24h_change": 0.0,
    "eth": 18.650627839906598,
    "eth_market_cap": 365799851.323311,
    "eth_24h_vol": 5597576.829903464,
    "eth_24h_change": 0.9213166048247731,
    "last_updated_at": 1706500074
  }
}

In this case, we only need to collect bitcoin price information. In other words, only bitcoin.usd data needs to be collected. Let's remove all but the usd field of the Bitcoin structure.

pub struct Bitcoin {
    pub usd: f64,
-    pub usd_market_cap: f64,
-    pub usd_24h_vol: f64,
-    pub usd_24h_change: f64,
-    pub eur: f64,
-    pub eur_market_cap: f64,
-    pub eur_24h_vol: f64,
-    pub eur_24h_change: f64,
-    pub btc: f64,
-    pub btc_market_cap: f64,
-    pub btc_24h_vol: f64,
-    pub btc_24h_change: f64,
-    pub eth: f64,
-    pub eth_market_cap: f64,
-    pub eth_24h_vol: f64,
-    pub eth_24h_change: f64,
-    pub last_updated_at: i64,
}

Let's update the logic in this way and run csx generate again. You have the template logic generated once you run generate, and you have placed code that modifies it. Therefore, if you run csx generate again, you will see the log output that the execution has completed, skipping the logic generation.

Build a module

Generate a module from the code created in the previous processes. Let's run csx build immediately.

That's all it took to generate a module for Chainsight from your code! Let's actually check out the artifact.

The artifacts generated by csx build are placed in the artifacts folder. If you look in that folder, you will see WASM modules and candid interface files with the actual component names as their file names. For a more in-depth understanding, please refer to the following articles

Behind Chainsight: How Modules are Created Pt. 2 | by Chainsight | Feb, 2024 | Medium

That's all there is to the build. Now all you have to do is deploy and get the component up and running.

Deploy a component

Let's actually deploy the generated module in Chainsight. Components are deployed with csx deploy, and after deployment is complete, csx exec is executed to send a command to the component to start execution. Let's run through some options. Let's deploy to Internet Computer using --component to explicitly specify the component to process and --network to specify the network to run on.

csx deploy --network ic --component btc_snapshot_indexer
csx exec --network ic --component btc_snapshot_indexer

The deployment and operation start instruction is now complete! Now all you have to do is wait for the interval of the automatic execution and you can refer to the data.

Please take a look at the component that can be created in this hands-on session, which is actually running as a sample. (The description and execution interval have been slightly changed.)
Using the Chainsight UI, you can check the acquired data, the execution cycle, and the state of the cycles that fuel it!

Source: https://beta.chainsight.network/explore/components/wqsbs-fyaaa-aaaal-qdezq-cai

This concludes our hands-on with Snapshot Indexer HTTPS. We will continue to create and publish new hands-on sessions in the future, so please look forward to them!

Written by Megared@Chainsight