Skip to main content

Index canisters

Intermediate
Tutorial

Overview​

On ICP, an index canister is used in conjunction with a ledger canister to provide the ability for transactions to be queried for a specific account. Without an index canister, a ledger's transaction history cannot easily be parsed and used by applications, as the entire transaction history of a ledger would need to be processed.

An index canister can be used to:

  • Fetch blocks from the ledger canister.

  • Query account-specific transaction history.

  • Query the status of a transaction.

If you are working in a local development environment, you can't access the mainnet ledgers nor the mainnet index canisters. If your application is using a mainnet index canister and you want to test it, you can setup the index and ledger canisters locally. Neither of the two canisters will have any information about the state of the ledger on the mainnet. You will have to create your own transactions on the ledger so that the index can serve them through its endpoints.

This guide displays how to deploy an index canister locally and use it in conjunction with a ledger canister. Instructions for both an ICP index and an ICRC index are included.

Prerequisites​

  • Read the guide on deploying a local ledger, then continue with this guide. This guide will assume that you have setup up a local ledger pertaining to your intended token type (ICP or ICRC), and that all prerequisites are fulfilled.

Wasm and Candid files​

Go to the releases overview and copy the latest replica binary revision (IC_VERSION). At the time of writing, this is b9a0f18dd5d6019e3241f205de797bca0d9cc3f8.

Wasm URL: https://download.dfinity.systems/ic/$IC_VERSION/canisters/ic-icp-index-canister.wasm.gz

Command to download this Wasm file using curl:

curl -o index.wasm.gz "https://download.dfinity.systems/ic/b9a0f18dd5d6019e3241f205de797bca0d9cc3f8/canisters/ic-icp-index-canister.wasm.gz"

Candid URL: https://raw.githubusercontent.com/dfinity/ic/$IC_VERSION/rs/rosetta-api/icp_ledger/index/index.did

Command to download this Candid file using curl:

curl -o index.did "https://raw.githubusercontent.com/dfinity/ic/b9a0f18dd5d6019e3241f205de797bca0d9cc3f8/rs/rosetta-api/icp_ledger/index/index.did"

Setting up the index​

Step 1: Configure the dfx.json file.​

Open the dfx.json file in your project's directory. Add the icp_index_canister Candid and Wasm configuration, and if necessary, insert the canister data for your ICP ledger.

If you followed the guide on local ledger setup and used the same project folder for both the ICP ledger and ICP index, your dfx.json file should look like this:

{
  "canisters": {
    "icp_index_canister": {
      "type": "custom",
      "candid": "https://raw.githubusercontent.com/dfinity/ic/d87954601e4b22972899e9957e800406a0a6b929/rs/rosetta-api/icp_ledger/index/index.did",
      "wasm": "https://download.dfinity.systems/ic/d87954601e4b22972899e9957e800406a0a6b929/canisters/ic-icp-index-canister.wasm.gz",
            "remote": {
        "id": {
          "ic": "qhbym-qaaaa-aaaaa-aaafq-cai"
        }
      }
    },
    "icp_ledger_canister": {
      "type": "custom",
      "candid": "https://raw.githubusercontent.com/dfinity/ic/d87954601e4b22972899e9957e800406a0a6b929/rs/rosetta-api/icp_ledger/ledger.did",
      "wasm": "https://download.dfinity.systems/ic/d87954601e4b22972899e9957e800406a0a6b929/canisters/ledger-canister.wasm.gz",
      "remote": {
        "id": {
          "ic": "ryjl3-tyaaa-aaaaa-aaaba-cai"
        }
      }
    }
  },
  "output_env_file": ".env",
  "version": 1
}

In an existing project, you would need to add the icp_index_canister and icp_ledger_canister canisters to the canisters section.

Step 2: Start a local replica.​

dfx start --clean --background

Step 3: Deploy the index canister​

Here it is assumed that the canister ID of your local ICP ledger is ryjl3-tyaaa-aaaaa-aaaba-cai; otherwise, replace it with your ICP ledger canister ID.

Deploying to the playground is ICP's equivalent of deploying to a testnet network.

In this workflow, you cannot deploy this canister to the playground (using flag dfx deploy --playground) because it does not accept gzipped Wasm files.

dfx deploy icp_index_canister --specified-id qhbym-qaaaa-aaaaa-aaafq-cai --argument '(record {ledger_id = principal "ryjl3-tyaaa-aaaaa-aaaba-cai"})'

The ICP index canister will start synching right away and will automatically try to fetch new blocks from the ICP ledger every few seconds.

Step 4: Check the status of the ledger​

You can check that the correct ledger ID was set but running the following command.

dfx canister call qhbym-qaaaa-aaaaa-aaafq-cai ledger_id '()'

The result is the ledger canister ID that the index canister is using to sync.

(principal "ryjl3-tyaaa-aaaaa-aaaba-cai")

To check how many blocks have been synched call:

dfx canister call qhbym-qaaaa-aaaaa-aaafq-cai status '()'

It should return something like this:

(record { num_blocks_synced = 1 : nat64 })

Depending on how many mint operations you created while setting up your ICP ledger, the number of synced blocks here will be 0 if no initial balances were parsed, or X if X initial balances were parsed. In the case of this tutorial, the guide on setting up a local ledger was used and there only one initial balance was parsed as an initialization argument. Hence, the number of blocks synched at this stage is 1.

Using the index canister​

You can check that the synchronization of the index is working by creating a transaction on the ICP ledger and then checking the status of that transaction. If you followed the guide on setting up an ICP ledger locally your default identity should have some ICP to be send. Send some ICP to any principal with the command:

dfx canister call ryjl3-tyaaa-aaaaa-aaaba-cai icrc1_transfer '(record { to = record { owner = principal "npki3-wdfh4-siaeq-orwh4-bh5of-r7mxr-i35lm-6f2eh-rtmwp-dmzmn-tae";};  amount = 100000:nat;})'

Then, check the status with the command:

dfx canister call qhbym-qaaaa-aaaaa-aaafq-cai status '()'

It should now indicate that an additional block was synced compared to the last time you called the status endpoint. You may have to wait a couple of seconds for the index canister to sync.

(record { num_blocks_synced = 2 : nat64 })

Fetch blocks​

You can use the ICP index canister to fetch blocks like so. You have to specify the block at which you want to start fetching from (i.e. the lowest index you want to fetch). If you want to start from the beginning you have to set start to 0. Similarly, the length parameter indicates the number of blocks you would like to fetch. Since the last status call indicated that there are two blocks that were synced, you can set this to 2. Note that if you specify more than 2 blocks it will simply return the maximum number of blocks the index contains (the limit of blocks per call is usually set to 2000 blocks).

dfx canister call qhbym-qaaaa-aaaaa-aaafq-cai get_blocks '(record{start=0:nat;length=2:nat})'

Which will return a vector the encoded blocks:

(
  record {
    blocks = vec {
      blob "\12\0a\08\90\83\a6\c6\c7\b8\a6\c3\17\1a=\12-\12\22\0a \0amCu\816\9bTj\fd\efa<\a9\c0\81\a2R\ca,F\e7\ec)\e5\10\bc\10\b2\13\fa\27\1a\07\08\80\d0\db\c3\f4\02\22\002\0a\08\90\83\a6\c6\c7\b8\a6\c3\17";
      blob "\0a\22\0a \912w)\02\f3a\9e\bc+\eax\e8D\b9\c9\09\14\12\cc%ZNRJ\06\c7?\a8\d1\97/\12\0a\08\a8\cd\b5\fb\a6\ba\a6\c3\17\1aW\1aS\0a\22\0a \0amCu\816\9bTj\fd\efa<\a9\c0\81\a2R\ca,F\e7\ec)\e5\10\bc\10\b2\13\fa\27\12\22\0a \930\d6\d8\cd\8ar\a5\a9Z\b7\d6@P\18\c4\ca^\bd\0cN\c1o6\eb\91\dbu\14\bd\86#\1a\04\08\a0\8d\06\22\03\08\90N\22\00";
    };
    chain_length = 2 : nat64;
  },
)

To fetch a vector of all transactions your default account was involved in you can use the following commands: Find out the principal of your default account:

dfx identity get-principal

In the case of this tutorial this returns:

hdq6b-ncywm-yajd5-4inc6-hgpzp-55xnp-py7d5-uqt6o-cv5c6-rrhwa-zqe

Then you can query the transactions for this principal with the default subaccount set by calling:

dfx canister call qhbym-qaaaa-aaaaa-aaafq-cai get_account_transactions '(record{account=record {owner = principal "hdq6b-ncywm-yajd5-4inc6-hgpzp-55xnp-py7d5-uqt6o-cv5c6-rrhwa-zqe"}; max_results=2:nat})'

The result will include the initial mint operation as well as the transfer that you made.

(
  variant {
    Ok = record {
      balance = 99_999_890_000 : nat64;
      transactions = vec {
        record {
          id = 1 : nat64;
          transaction = record {
            memo = 0 : nat64;
            icrc1_memo = null;
            operation = variant {
              Transfer = record {
                to = "9330d6d8cd8a72a5a95ab7d6405018c4ca5ebd0c4ec16f36eb91db7514bd8623";
                fee = record { e8s = 10_000 : nat64 };
                from = "0a6d437581369b546afdef613ca9c081a252ca2c46e7ec29e510bc10b213fa27";
                amount = record { e8s = 100_000 : nat64 };
              }
            };
            created_at_time = null;
          };
        };
        record {
          id = 0 : nat64;
          transaction = record {
            memo = 0 : nat64;
            icrc1_memo = null;
            operation = variant {
              Mint = record {
                to = "0a6d437581369b546afdef613ca9c081a252ca2c46e7ec29e510bc10b213fa27";
                amount = record { e8s = 100_000_000_000 : nat64 };
              }
            };
            created_at_time = opt record {
              timestamp_nanos = 1_695_211_378_870_682_000 : nat64;
            };
          };
        };
      };
      oldest_tx_id = opt (0 : nat64);
    }
  },
)

Accounts​

The ICP ledger uses an AccountIdentifier, which is calculated by hashing a Principal and Subaccount. This also means that the returned transactions will show accounts as the hash in bytes rather than the actual Accounts.

For example, the default principal hdq6b-ncywm-yajd5-4inc6-hgpzp-55xnp-py7d5-uqt6o-cv5c6-rrhwa-zqe with no subaccount set results in the hash 0a6d437581369b546afdef613ca9c081a252ca2c46e7ec29e510bc10b213fa27.

You can check a principal's AccountIdentifier by running:

dfx ledger account-id --of-principal hdq6b-ncywm-yajd5-4inc6-hgpzp-55xnp-py7d5-uqt6o-cv5c6-rrhwa-zqe

It will return the AccountIdentifier.

0a6d437581369b546afdef613ca9c081a252ca2c46e7ec29e510bc10b213fa27

Alternatively, you can also add a subaccount. This will change the AccountIdentifier although the principal is the same:

dfx ledger account-id --of-principal hdq6b-ncywm-yajd5-4inc6-hgpzp-55xnp-py7d5-uqt6o-cv5c6-rrhwa-zqe --subaccount 0000000000000000000000000000000000000000000000000000000000000001
bd719f30834fe34f420904cde95a2cef6404ef7a8489cde57056b4daddab28b1

You can also always check the current balance of an account by calling:

dfx canister call qhbym-qaaaa-aaaaa-aaafq-cai icrc1_balance_of '(record{owner = principal "hdq6b-ncywm-yajd5-4inc6-hgpzp-55xnp-py7d5-uqt6o-cv5c6-rrhwa-zqe"})'
(99_999_890_000 : nat64)