Update readmes

This commit is contained in:
pistomat
2023-12-22 11:33:12 +01:00
parent e9ebf6f0c6
commit 03686d0dff
7 changed files with 29 additions and 23 deletions

View File

@@ -22,7 +22,7 @@ While VM integration is certainly the quickest and probably most accessible one
### Indexing
For indexing purposes, it is required that you provide a [substreams](https://substreams.streamingfast.io/) package that emits a specified set of messages. If your protocol already has a [substreams package](https://github.com/messari/substreams) package for indexing implemented, you can adjust it to emit the required messages.
For indexing purposes, it is required that you provide a [substreams](https://substreams.streamingfast.io/) package that emits a specified set of messages. If your protocol already has a [substreams package](https://github.com/messari/substreams) for indexing implemented, you can adjust it to emit the required messages.
**VM Integration** Currently the only supported integration is for EVM protocols in order to complement the Solidity protocol logic. **[Read more here.](indexing/vm-integration/README.md)**
**Custom Entity Integration** Coming soon, this integration will complement the upcoming native Rust protocol logic.

View File

@@ -0,0 +1,3 @@
# Native Integration
Coming soon...

View File

@@ -1,4 +1,6 @@
# Substreams Integration
# VM Integration
Our indexing integrations use the Substreams library to transform raw blockchain data into higher level data streams. This is done by implementing a Rust module that is compiled into a SPKG file and then loaded by the Substreams server.
## Example
@@ -35,9 +37,9 @@ We have integrated the **Ambient** protocol as a reference, see `/substreams/eth
git clone https://github.com/propeller-heads/propeller-protocol-lib
```
## Understanding the Substreams integration
## Understanding the Substreams VM integration
Substreams is a new indexing technology, which uses Rust modules to compose raw blockchain data streams into higher level data streams, in out case specific to the protocol. These modules together with the protobuf definitions and manifest are then wrapped into SPKG packages (more info [here](https://substreams.streamingfast.io/quick-access/glossary#spkg-.spkg)) that is then run remotely on the Substreams server.
Substreams is a new indexing technology, which uses Rust modules to compose raw blockchain data streams into higher level data streams, in our case specific to the protocol. These modules together with the protobuf definitions and manifest are then wrapped into SPKG packages (more info [here](https://substreams.streamingfast.io/quick-access/glossary#spkg-.spkg)) that are then run remotely on the Substreams server.
For more information, read the [quick explanation of Substreams](https://thegraph.com/docs/en/substreams/) or jump into the [Substreams documentation](https://substreams.streamingfast.io/). It describes the functions that need to be implemented as well as the manifest file.
@@ -53,7 +55,7 @@ First get familiar with the raw ProtoBuf definitions provided by us:
You can also create your own intermediate ProtoBufs. These files should reside in your own substreams package, e.g. `./substreams/ethereum-template/proto/custom-messages.proto`. You have to link these files in the `substreams.yaml` file, see the [manifest docs](https://substreams.streamingfast.io/developers-guide/creating-your-manifest) for more information or you can look at the official substreams example integration of [UniswapV2](https://github.com/messari/substreams/blob/master/uniswap-v2/substreams.yaml#L20-L22).
*Note: Internally we are referring to the substreams integration as `Tycho`, which is why our protobuf files are under the `proto/tycho` directory.*
*Note: Internally we are referring to our indexing library as `Tycho`, which is why our protobuf files are under the `proto/tycho` directory.*
### Rust module

View File

@@ -48,14 +48,14 @@ message Attribute {
}
// A struct describing a part of the protocol.
// Note: For example this can be a UniswapV2 pair, that would track the two ERC20 tokens used by the pair,
// the contract would be only the single sontract. The attributes would be empty for the VM integration,
// because we track all the relevant info via storage slots and balance changes.
// Note: For example this can be a UniswapV2 pair, that tracks the two ERC20 tokens used by the pair,
// the component would represent a single contract. In case of VM integration, such component would
// not need any attributes, because all the relevant info would be tracked via storage slots and balance changes.
// It can also be a wrapping contract, like WETH, that has a constant price, but it allows swapping tokens.
// This is why the name ProtocolComponent is used instead of "Pool" or "Pair".
message ProtocolComponent {
// A unique identifier for the component within the protocol.
// Can be a stringified address or a string describing the trading pair.
// Can be e.g. a stringified address or a string describing the trading pair.
string id = 1;
// Addresses of the ERC20 tokens used by the component.
repeated bytes tokens = 2;
@@ -70,7 +70,7 @@ message ProtocolComponent {
}
// A struct for following the changes of Total Value Locked (TVL) of a protocol component.
// Note that if the ProtocolComponent contains multiple contracts, the TVL is tracked for the component as a whole.
// Note that if a ProtocolComponent contains multiple contracts, the TVL is tracked for the component as a whole.
// E.g. for UniswapV2 pair WETH/USDC, this tracks the USDC and WETH balance of the pair contract.
message BalanceChange {
// The address of the ERC20 token whose balance changed.

View File

@@ -18,7 +18,7 @@ message ContractSlot {
message ContractChange {
// The contract's address
bytes address = 1;
// The new balance of the contract, empty bytes indicates no change.
// The new native balance of the contract, empty bytes indicates no change.
bytes balance = 2;
// The new code of the contract, empty bytes indicates no change.
bytes code = 3;

View File

@@ -1,11 +1,10 @@
# Subtreams packages
This directory contains all substream packages that are used by the extractors to access certain data from diffrent
blockchains.
This directory contains all substream packages that are used to index integrated protocols across different blockchains.
## Adding a new package
To add a new package add folder. The naming convention is `[CHAIN]-[PROTOCOL_SYSTEM]`.
To add a new package add a folder. The naming convention is `[CHAIN]-[PROTOCOL_SYSTEM]`.
### Manifest
In this new folder add a manifest file `substreams.yaml`. You can use the template below to get started:
@@ -20,14 +19,11 @@ protobuf:
files:
- vm.proto
- common.proto
# You can specify any internal proto files here
importPaths:
# This is different compared to the substreams example,
# we need to share protobuf definitions with tycho you
# are invited to reuse existing definitions if they are
# useful to you.
- ../../proto/evm/v1
# any private message types only used in internal modules
# can remain local to the crate.
- ../../proto/tycho/evm/v1/
# Any private message types only used in internal modules
# can remain local to the folder.
- ./proto
binaries:
@@ -66,7 +62,11 @@ prost = "0.11"
```
There are already some generated rust files in the `src/pb` directory. These are generated from the protobuf files in the
There are already some generated rust files in the `src/pb` directory. These are generated
from the protobuf files in the `/proto/tycho/evm/v1` directory. They specify the output protobuf messages
we want to generate. The input Block is specified by the sustreams crate, specifically the [sf.ethereum.type.v2.Block](https://github.com/streamingfast/substreams-ethereum/blob/develop/core/src/pb/sf.ethereum.type.v2.rs) message.
You can define your own protobuf messages, make a new directory `/substreams/[CHAIN]-[PROTOCOL]/proto` for them.
Now we can generate the Rust protobuf code:

View File

@@ -82,7 +82,8 @@ fn map_changes(
.collect();
for block_tx in block.transactions() {
// Extract storage changes for all contracts relevant to this ProtocolComponent (i.e. Ambient)
// Extract storage changes for all contracts relevant to this protocol system.
// Ambient is a protocol system consisting of many ProtocolComponents (one for each pool), but they all share the same AMBIENT_CONTRACT contract.
let mut storage_changes = block_tx
.calls
.iter()