In the blockchain space, dApps that want to operate must interact with data stored in the blockchain it is built on. In other words, a dApp needs to query data from the underlying blockchain.
Querying data is not simple, and is even becoming more and more complicated based on the huge amount of data stored scattered in each block. To help dApps easily query data in the Blockchain, Indexing is a useful tool.
In this article, we will learn about the concept of Indexing in blockchain, about the opportunities and challenges of this piece of the puzzle.
Indexing and Querying Blockchain Data
Indexing or, in full, “Blockchain data indexing” – blockchain data indexing is the process of creating a new database from existing data on the blockchain.
Why is Data indexing needed?
Blockchain is a distributed system that stores data, and accessing this data directly can take a lot of time and resources. Therefore, Indexing are services that provide a superior copy based on traditional databases, allowing data to be queried at high speed and easier.
Limitations in querying blockchain data:
- Hierarchical data: This is beneficial in terms of decentralization, but data is distributed across many different nodes instead of gathered in one place, making it difficult to query and filter data.
- TUnderstand efficient query language: Each type of database will be suitable for certain query languages. For example, centralized databases are suitable for SQL query language. As for distributed databases like on blockchain, no query language is suitable. Therefore, it makes it difficult to query data
- API Restrictions: Current APIs are still limited in providing the ability to query highly complex data.
To summarize, we can think of the role of data indexing as creating a table of contents for a book with thousands of pages. Without a table of contents, readers have to flip through each sheet of paper to find the content they are interested in, whereas if they number the pages and then create a table of contents for the book, finding information will become easy. more.
Operational process of a Data indexing platform:
- Step 1: Extract (collect) data from the underlying blockchain, then store it in a new database. The data collected may include transaction details, contract addresses, etc.
- Step 2: Index (index) the data. Common indexing methods include:
+ Index tokens and NFTs to track trading activities, trading volume, supply, etc.
+ Index wallet addresses, smart contract addresses to track balances, transaction history, etc.
+ Index transaction information including sender, recipient, timestamp, hashcode,…etc - Step 3: Data indexing platforms provide APIs for dApps.
- Step 4: Through this, dApps can access the secondary database created from step 2 easily, instead of directly querying data from the blockchain.
Although blockchain is highly decentralized, secure and transparent, there are still limitations in the lack of features that help retrieve data directly and easily. Therefore, dApp developers who want to query blockchain data will have to use services from secondary sources, coming from Indexing projects.
So what are the outstanding Indexing projects in the market today?
Some Industry Leading Data Indexing Projects
The Graph – Graph Protocol
dApps can take advantage of The Graph’s blockchain data indexing tool, Subgraph. At its core, Subgraph is an open API that allows dApp developers to retrieve data from blockchains like Ethereum and other EVM-chains using the data query language they developed, GraphQL.
It can be understood that the role of The Graph is like a Google of blockchain, users can search for any data on Ethereum through The Graph by just performing simple queries.
Subgraphs are APIs that allow users to access data on the blockchain. For each protocol, The Graph provides different subgraphs.
Graph Protocol works thanks to the following 3 pillars:
-
Indexers: Are node operators for Graph Protocol, and receive rewards in emissions token GRT to encourage node operation. These indexers will have to stake GRT tokens to participate in the process of providing indexing and query processing services. The GRT they stake will be confiscated if they provide inaccurate, malicious data.
-
Curators: These people have the role of searching for subgraphs that need to be indexed. From there receive GRT as an incentive reward.
- Delegators: These people indirectly participate in the operation of the Graph Protocol, but do not run the node but delegate GRT to the Indexers.
Anyone can participate in Graph Protocol’s model, choosing to become one of the 3 roles above. Finally, Graph users – dApps that need to use data from the blockchain will integrate the API that The Graph provides, then pay as a service.
Subquery Network
Subquery Network is also a project that provides similar data services, but serves mainly on the Polkadot ecosystem. Subquery provides the following main tools:
-
SubSQL’s SDK: Subquery SDK is a software toolkit (Software Development Kit) provided by Subquery Network, helping developers build and deploy their own data indexes on Subquery Network.
-
SubQuery Projects: A function that allows programmers to publish their Subquery projects to SubQuery Explorer to be stored and used privately or for the community to use.
-
SubSQL Explorer: This is a Market Place, where SubQuery Projects are listed and users who need to use data will go to Subquery Explorer, select the products provided and pay to use them.
Subquery Network is also operated in a similar way to The Graph, with a model that includes the participation of:
- RPC Provider: People running nodes for Subquery Network, these people will receive SQT as a reward. Need to stake SQT to participate.
- Data Indexers: Maintain quality SubSQL projects, then get SQT as a reward. Need to stake SQT to participate.
- Delegators: Authorize SQT for the above units.
Finally, the user – the consumer – will use the provided SubQuery Projects and pay in SQT for using the service.
Conclude
Above is basic information about Blockchain data indexing and outstanding projects in the industry. This is an indispensable piece of infrastructure for the overall development of dApps in the crypto market, and also opens up opportunities for small users to contribute to indexing protocols to earn fees. service.
