Liquidity.Party/tycho-protocol-sdk
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: remove READMEs and point to online docs

Browse Source
This commit is contained in:
Thales
2025-01-21 23:39:46 -03:00
committed by tvinagre
parent 1400fd413a
commit bd8cfa352f
15 changed files with 2 additions and 904 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -1,5 +1,5 @@
# Tycho Protocol SDK
Tycho Protocol SDK is a library used by solvers to integrate DEX liquidity.
Tycho Protocol SDK is a library to integrate DEXs and other onchain liquidity protocols into Tycho.
Please refer to the [README.md](docs/README.md) for more information.
Please refer to the [documentation](https://docs.propellerheads.xyz/tycho/for-dexs/protocol-integration-sdk) for more information.

BIN
docs/.gitbook/assets/image.png
View File

Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 51 KiB

0
docs/.gitkeep
View File

36
docs/README.md
View File

@@ -1,36 +0,0 @@
# Propeller Protocol Lib
Protocol lib is a library used by Propellerheads.xyz solvers to integrate decentralized protocols. Currently, only swap/exchange protocols are supported.
## Integration Process
To integrate with PropellerHeads solvers, two components need to be provided:
* **Protocol logic:** Provides simulations of the protocols logic.
* **Indexing**: Provides access to the protocol state used by the simulation. This component is optional if your protocol is stateless.
To propose an integration, create a pull request in this repository with the above components implemented.
### Protocol Logic
PropellerHeads currently exposes two integration modes to specify the protocols' underlying logic:
* **VM Integration:** This integration type requires implementing an adapter interface in any language that compiles to the respective vm byte code. This SDK provides the interface only in Solidity. [**Read more here.**](logic/vm-integration/)
* **Native Rust Integration:** Coming soon, this integration type requires implementing a Rust trait that describes the protocol logic.
While VM integration is certainly the quickest and probably most accessible one for protocol developers, native implementations are much faster and allow us to consider the protocol for more time-sensitive use cases - e.g. quoting.
### Indexing
For indexing purposes, it is required that you provide a [substreams](https://substreams.streamingfast.io/) package that emits a specified set of messages. If your protocol already has a [substreams package](https://github.com/messari/substreams) for indexing implemented, you can adjust it to emit the required messages.
**VM Integration** Currently the only supported integration is for EVM protocols in order to complement the Solidity protocol logic. [**Read more here.**](https://github.com/propeller-heads/propeller-venue-lib/blob/main/docs/indexing/vm-integration/README.md) **Custom Entity Integration** Coming soon, this integration will complement the upcoming native Rust protocol logic.
### Execution
For execution purposes, the implementation of the `SwapExecutor` and `SwapStructEncoder` interfaces is required. Without these components, trades cannot be executed on-chain, making them critical parts of the integration.
**SwapExecutor**: The SwapExecutor is responsible for performing swaps by interacting with the underlying liquidity pools, handling token approvals, managing input/output amounts, and ensuring gas-efficient and secure execution. Each protocol must implement its own `SwapExecutor` (Solidity contract), tailored to its specific logic and requirements.
**SwapStructEncoder**: The `SwapStructEncoder` encodes the necessary data structures required for the `SwapExecutor` to perform swaps. It ensures that the swap details, including input/output tokens, pool addresses, and other protocol-specific parameters, are correctly formatted and encoded before being passed to the `SwapExecutor`. Each protocol must implement its own `SwapStructEncoder` Python class and ensure compatibility with its `SwapExecutor`.

20
docs/SUMMARY.md
View File

@@ -1,20 +0,0 @@
# Table of contents
- [Propeller Protocol Lib](README.md)
## Logic
- [VM Integration](logic/vm-integration/README.md)
- [Ethereum: Solidity](logic/vm-integration/ethereum-solidity.md)
- [Native Integration](logic/native-integration.md)
## Indexing
- [Overview](indexing/overview.md)
- [Getting Started](indexing/getting-started.md)
- [VM Integration](indexing/vm-integration/README.md)
- [Native Integration](indexing/native-integration.md)
- [Reserved Attributes](indexing/reserved-attributes.md)
## Execution
- [Swap Executor](execution/swap-executor.md)

54
docs/execution/swap-encoder.md
View File

@@ -1,54 +0,0 @@
# Implementing a SwapExecutor for a Protocol
## Overview
The `SwapStructEncoder` interface is designed to encode the necessary data for a swap, which will be used by the `SwapExecutor` to interact with a liquidity pool. The encoder is responsible for structuring the swap details, including the input/output tokens, pool addresses, and any additional protocol-specific parameters.
Each protocol must implement its own `SwapStructEncoder` and ensure that the swap data is correctly encoded for the `SwapExecutor`.
### Dev environment
- Run `aws codeartifact login --tool pip --repository protosim --domain propeller` so you can access the propeller packages.
- Create the dev environment `conda env create -f propeller-swap-encoders/environment_dev.yaml`
- Activate it with `conda activate propeller-swap-encoders`
- Install dependencies with `pip install -r propeller-swap-encoders/requirements.txt`
- Should you get a pyyaml installation error execute the following command: `pip install "cython<3.0.0" && pip install --no-build-isolation pyyaml==5.4.1`
You can import the abstract class `SwapStructEncoder` from `propeller-solver-core` in your python code like:
```python
from core.encoding.interface import SwapStructEncoder
```
## Key Methods
This is the `SwapStructEncoder` interface can be found [here](https://github.com/propeller-heads/defibot/blob/7ea38b92e60e182471f513c2aeef0370c4b3766a/propeller-solver-core/core/encoding/interface.py#L31).
- **encode_swap_struct**
- **Purpose**: To encode the swap details into a bytes object that the SwapExecutor can use to execute the swap.
- **Parameters**:
- `swap`: A dictionary containing the swap details. These are the fields present in this dict:
- `pool_id: str`: The identifier for the liquidity pool where the swap will occur.
- `sell_token: EthereumToken`: The token that will be sold in the swap (e.g., DAI).
- `buy_token: EthereumToken`: The token that will be bought in the swap (e.g., WETH).
- `split: float`: Indicates how the swap should be split between pools (often set to 0).
- `sell_amount: int`: The amount of the sell token to be used in the swap.
- `buy_amount: int`: The amount of the buy token to be received from the swap.
- `token_approval_needed: bool`: A boolean indicating if token approval is needed before the swap.
- `pool_tokens`: An optional tuple containing additional token data specific to the pool.
- `pool_type: str`: The type of pool where the swap will be executed (e.g., "BalancerStablePoolState").
- `receiver`: The address that will receive the output tokens from the swap.
- `encoding_context`: Additional context information necessary for encoding (see more [here](https://github.com/propeller-heads/defibot/blob/7ea38b92e60e182471f513c2aeef0370c4b3766a/propeller-solver-core/core/encoding/interface.py#L9))
- `**kwargs`: Any additional protocol-specific parameters that need to be included in the encoding.
- **Returns**: A bytes object containing the encoded swap data.
## Implementation Steps
1. **Define Protocol-Specific Encoding Logic**: Implement the `encode_swap_struct` function to encode the swap details specific to the protocol. This may include encoding token addresses, pool addresses, and other necessary parameters into a bytes format.
2. **Compatibility with SwapExecutor**: Ensure that the encoded data is compatible with the `SwapExecutor` implementation for the protocol. The `SwapExecutor` will rely on this data to perform the swap accurately.
3. **Testing**: Thoroughly test the encoding process with various swap scenarios to ensure that the encoded data is correct and that the `SwapExecutor` can process it without errors.
## Example Implementation
See the example implementation of a `SwapExecutor` for Balancer [here](../../propeller-swap-encoders/propeller_swap_encoders/balancer.py) and test [here](../../propeller-swap-encoders/propeller_swap_encoders/tests/test_balancer.py).

44
docs/execution/swap-executor.md
View File

@@ -1,44 +0,0 @@
# Implementing a SwapExecutor for a Protocol
## Overview
The `ISwapExecutor` interface is designed to perform swaps on a liquidity pool.
It allows for flexible interaction by accepting either the amount of the input token or the amount of the output token
as parameters, returning the corresponding swapped amount.
This interface is essential for creating a `SwapExecutor` specific to a protocol.
The `SwapExecutor` works in conjunction with the `SwapStructEncoder`, which encodes the necessary data required for the swap.
This encoded data is passed to the `SwapExecutor`, enabling it to perform the swap according to the protocol's specific logic.
## Key Methods
- **swap(uint256 givenAmount, bytes calldata data)**
- **Purpose**: To perform a token swap, either specifying the input amount to get the output amount or vice versa.
- **Parameters**:
- `givenAmount`: The amount of the token (input or output) for the swap.
- `data`: Encoded information necessary for the swap (e.g., pool address, token addresses - depends on the protocol),
provided by the `SwapStructEncoder`.
- **Returns**: The amount of the token swapped.
## Implementation Steps
1. **Define Protocol-Specific Logic**: Implement the `swap` function to interact with the protocol's liquidity pool.
Use the `data` parameter to encode necessary information like pool and token addresses.
2. **Handling Input and Output**: Depending on the provided `givenAmount`, determine whether it's an input or output
swap. Calculate the corresponding swapped amount based on the pool's pricing logic.
3. **Error Handling**: Use `ISwapExecutorErrors` (`InvalidParameterLength` and `UnknownPoolType`) to manage potential
errors, such as invalid parameter lengths or unknown pool types in the swap logic.
4. **Token Approvals**: If the protocol requires token approvals (allowances) before swaps can occur,
manage these approvals within the implementation to ensure smooth execution of the swap.
5. **Token Transfer Support**: Ensure that the implementation supports transferring received tokens to a designated
receiver address, either within the swap function or through an additional transfer step.
6. **Gas Efficiency**: Ensure the implementation is gas-efficient. Strive for optimal performance in the swap logic.
The usage of assembly is not necessary.
7. **Security Considerations**: Follow common security best practices, such as validating inputs, ensuring proper
access control, and safeguarding against reentrancy attacks.
## Example Implementation
See the example implementation of a `SwapExecutor` for Balancer [here](../../evm/src/balancer-v2/BalancerSwapExecutor.sol).

204
docs/indexing/getting-started.md
View File

@@ -1,204 +0,0 @@
# Getting Started
## How to Integrate
Before starting, it is important to have a good understanding of the protocol we are aiming to integrate.
It is essential to understand:
- Which contracts are involved in the protocol and what functions do they serve. How do they affect the behaviour of the component being integrated?
- What conditions (e.g. oracle update) or what kind of method calls can lead to a relevant state change on the protocol, which ultimately changes the protocols behaviour if observed externally.
- Are there components added or removed, and how are they added. Most protocols use either a factory contract, which can be used to deploy new components, or they use a method call that provisiona a new component within the overall system.
Once the workings of the protocol are clear the implementation can start.
## Setup
PropellerHeads indexing integrations are provided as [substreams](https://substreams.streamingfast.io/) skpg files. If you do not know substreams yet, make sure to go check them out and set up their [cli](https://substreams.streamingfast.io/documentation/consume/installing-the-cli) before continuing.
Please start a new package for your integration, by copying the `ethereum-template` to a new name. The convention is: `[CHAIN]-[PROTOCOL_SYSTEM]` please make sure to also adjust: `cargo.toml` as well as `substreams.yaml` accordingly.
It should be possible now to generate the necessary protobuf code:
```bash
substreams protogen substreams.yaml --exclude-paths="sf/substreams,google"
```
Next please register the new package with the workspace by registering it as a workspace member. This is simply done by adding the package's name to the members list under `substreams/Cargo.toml`.
You are ready to start coding. The template you just copied already contains the `tycho-substreams` package as a dependency. This package contains all the necessary output types that tycho expects, as well as some interim helper types and functions for common tasks.
Before continuing though, it is important to understand the concept of map modules and store modules. You can read about those [here](https://substreams.streamingfast.io/documentation/develop/manifest-modules/types).
## Overview
In the following section, we outline the typical procedure for structuring an integration for the Virtual Machine (VM) implementation type:
Commonly, protocols employ factory contracts to deploy a contract for each swap component (also known as a pool or pair). We will explain how to efficiently index this system of contracts. It's important to note, however, that the techniques described below are broadly applicable, even if a protocol deviates from this specific pattern it should be possible to index it and emit the required messages.
{% hint style="info" %}
The following examples and code snippets have been taken from the ethereum-balancer substream. The complete code is available [here](https://github.com/propeller-heads/tycho-protocol-sdk/tree/main/substreams/ethereum-balancer-v2).
{% endhint %}
Usually an integration consists of the following modules:
- `map_components(block)`
- This map module extracts any newly created components by inspecting the block model (e.g factory contract logs). The recommended output model for this module is `BlockTransactionProtocolComponents.`
- `store_components(components, components_store)`
- This store module takes the detected components and stores any necessary information about the component for downstream modules. For vm integrations the address is most likely enough.
- `map_relative_balances(block, components_store)`:
- This map module is necessary for protocols that do not emit absolute balances changes (of ERC20 tokens and/or the native token). Since no absolute balances values are available the block model most likely will only provide deltas. Our sdk provides helpers to convert these into absolute balances as required by our data model. The responsibility of the module is to extract these relative changes and communicate them. The recommended output models for this module is `BlockBalanceDeltas`
- `store_balances(balance_deltas, balance_store)`:
- This module stores the relative balances deltas in an additive store, which essentially converts them into absolute balances.
- `map_protocol_changes(balance_deltas, balance_store, components_store, ...)`:
- The module that pulls everything together and build the final output model: `BlockChanges`.
The DAG formed by this structure can be seen below:
```mermaid
graph TD;
map_components[map: map_components];
sf.ethereum.type.v2.Block[source: sf.ethereum.type.v2.Block] --> map_components;
store_components[store: store_components];
map_components --> store_components;
map_relative_balances[map: map_relative_balances];
sf.ethereum.type.v2.Block[source: sf.ethereum.type.v2.Block] --> map_relative_balances;
store_components --> map_relative_balances;
store_balances[store: store_balances];
map_relative_balances --> store_balances;
map_changes[map: map_changes];
sf.ethereum.type.v2.Block[source: sf.ethereum.type.v2.Block] --> map_changes;
map_components --> map_changes;
map_relative_balances --> map_changes;
store_components --> map_changes;
store_balances -- deltas --> map_changes;
```
### Tracking Components
Usually the first step consists of detecting the creation of new components and storing their contract addresses in a store so they can be properly tracked further downstream.
Later we'll have to emit balance and state changes based on the set of currently tracked components.
{% hint style="info" %}
Emitting state changes of components that have not been previously announced is considered an error.
{% endhint %}
Usually you want to start by implementing a map module that emits any newly created components. These newly created components are detected by inspecting the `sf.ethereum.type.v2.Block` model.
The output message should then contain as much information about the component available at that time, as well as the transaction that created the protocol component.
The recommended action here is to implement a `factory.rs` module that will help with detecting newly deployed components. Then use that module within a map handler to detect any newly emitted protocol components. The recommended output model for this first handler is `BlockTransactionProtocolComponents`:
```protobuf
// A message containing protocol components that were created by a single tx.
message TransactionProtocolComponents {
Transaction tx = 1;
repeated ProtocolComponent components = 2;
}
// All protocol components that were created within a block with their corresponding tx.
message BlockTransactionProtocolComponents {
repeated TransactionProtocolComponents tx_components = 1;
}
```
Note that a single transaction may emit multiple newly created components. In this case it is expected that the `TransactionProtocolComponents.components` contains multiple `ProtocolComponents`.
Once emitted, the protocol components should be stored in a Store since we will later have to use this store to decide whether a contract is interesting to us or not.
### Tracking Absolute Balances
Tracking balances can be tricky since often balance information is only available in relative values.&#x20;
This means the relative values have to be aggregated by component and token to arrive at an absolute value. Additionally, throughout this aggregation we need to track the balance changes per transaction within a block.
Since this is challenging the following approach is recommended:
#### 1. Index relative balance changes
To accurately process a block and report the respective balance changes, implement a handler that utilises the `BlockBalanceDeltas` struct. It is crucial to ensure that each `BalanceDelta` within a component token pair is assigned a strictly increasing ordinal. This specificity is key to maintaining the integrity of aggregated values at the transaction level. Incorrect ordinal sequencing could lead to inaccurate balance reporting.
Below is an example of an interface for a handler. This handler interfaces with a store that employs an integer as an indicator to denote whether a specific address (identified by the keys) is a component.
```rust
#[substreams::handlers::map]
pub fn map_relative_balances(
block: eth::v2::Block,
components_store: StoreGetInt64,
) -> Result<BlockBalanceDeltas, anyhow::Error> {
todo!()
}
```
Our Substreams SDK provides the `tycho_substream::balances::extract_balance_deltas_from_tx` function that extracts all relevant `BalanceDelta` from ERC20 `Transfer` events for a given transaction (see Curve implementation).
#### 2. Aggregate balances with an additive store
To aggregate `BlockBalanceDeltas` messages into absolute values efficiently while maintaining transaction level granularity, we can leverage the additive `StoreAddBigInt` type with a store module.
The `tycho_substream::balances::store_balance_changes` helper function is available for this purpose, streamlining the implementation significantly.&#x20;
Thus, the typical use case can be addressed with the provided snippet:
```rust
#[substreams::handlers::store]
pub fn store_balances(deltas: BlockBalanceDeltas, store: StoreAddBigInt) {
tycho_substreams::balances::store_balance_changes(deltas, store);
}
```
#### 3. Combine absolute values with component and address
Last but not least, we have to associate the absolute balances with their respective transaction, component and token.
To simplify the final step of aggregating balance changes, utilise the `tycho_substream::balances::aggregate_balances_changes` helper function. Detailed instructions are available in the function's docstring. Essentially, it outputs aggregated `BalanceChange` structs per transaction. These aggregates can then be seamlessly integrated into `map_protocol_changes` for retrieving absolute balance changes associated with each transaction.
Below is a snip on how it is usually used:
```rust
#[substreams::handlers::map]
pub fn map_protocol_changes(
block: eth::v2::Block,
grouped_components: BlockTransactionProtocolComponents,
deltas: BlockBalanceDeltas,
components_store: StoreGetInt64,
balance_store: StoreDeltas,
) -> Result<BlockChanges> {
let mut transaction_contract_changes: HashMap<_, TransactionChanges> = HashMap::new();
aggregate_balances_changes(balance_store, deltas)
.into_iter()
.for_each(|(_, (tx, balances))| {
transaction_contract_changes
.entry(tx.index)
.or_insert_with(|| TransactionChanges::new(&tx))
.balance_changes
.extend(balances.into_values());
});
}
```
### Tracking State Changes
In vm implementations, it's crucial to accurately identify and extract all relevant contract changes. Typically, there's a one-to-one mapping between contracts and components which allows us to follow the convention of utilising the hex-encoded address as the component's ID.
To facilitate the extraction of pertinent changes from the expanded block model, we recommend using the `tycho_substreams::contract::extract_contract_changes` helper function. This function significantly simplifies the process.
Below, we illustrate how to leverage a component store to define a predicate. This predicate serves to pinpoint the contract addresses that are of particular interest:
```rust
use tycho_substreams::contract::extract_contract_changes;
let mut transaction_contract_changes: HashMap<_, TransactionChanges> = HashMap::new();
extract_contract_changes(
&block,
|addr| {
components_store
.get_last(format!("pool:{0}", hex::encode(addr)))
.is_some()
},
&mut transaction_contract_changes,
);
```

3
docs/indexing/native-integration.md
View File

@@ -1,3 +0,0 @@
# Native Integration
Coming soon...

53
docs/indexing/overview.md
View File

@@ -1,53 +0,0 @@
# Overview
This page gives an overview over the data model required to ingest protocol state into the PropellerHeads solver.&#x20;
To integrate a protocol PropellerHeads rely on either native or vm logic. Most integration will likely choose to use the VM, as this is usually less effort, the guide will focus mostly on providing state for vm integrations.&#x20;
Native integration should operate following exactly the same pattern, just that they should emit changed attributes instead of changes contract storage slots.&#x20;
### Understanding the Data Model
PropellerHeads ingest all data versioned by block and transaction. This helps maintain a low latency feed and deal correctly with chains that can experience reverts.
This means each state change that is communicated must be communicated with its respective transaction that caused the change.
Next, for each emitted transactions that carries state changes, the corresponding block must be provided as well.
So basically when processing a block we need to emit the block itself, all transactions that introduced protocol state changes and last but not least the state changes themselves, associated to their corresponding transaction.
**The data model that encodes changes, transaction and blocks in messages, can be found** [**here**](https://github.com/propeller-heads/tycho-protocol-sdk/tree/main/proto/tycho/evm/v1)**.**&#x20;
#### Models
The models below are used for communication between Substreams and Tycho indexer, as well as between Substreams modules.
Our indexer expects to receive a `BlockChanges` output from your Substreams package.
{% @github-files/github-code-block url="https://github.com/propeller-heads/tycho-protocol-sdk/blob/main/proto/tycho/evm/v1/common.proto" %}
Please be aware that changes need to be aggregated on the transaction level, it is considered an error to emit `BlockChanges` with duplicated transactions present in the `changes` attributes.
#### Integer Byte encoding
Many of the types above are variable length bytes. This allows for flexibility across blockchains but require agreeing on an informal interface, so later applications know how to interpret these bytes.
**Integers:** especially integers used to communicate balances, should always be encoded as unsigned big-endian integer. This is simply because balances serve multiple purposes within the system and need to be decoded in multiple location of the messages journey.
**Strings**: If you need to store strings, please use utf-8 encoding to store them as bytes.
**Attributes:** the value encoding for attributes is variable. It depends on the use case. Since the attributes are highly dynamic they are only used by the corresponding logic components, so the encoding can be tailored to the logic implementation: E.g. since Rust uses little endian one may choose to use little endian encoding for integers if the native logic module is written in Rust.
#### Special attribute names
Certain attribute names are reserved exclusively for specific purposes in our simulation process. Please use them only for their intended functions. See the [list of reserved attributes](./reserved-attributes.md)
### Changes of interest
PropellerHeads integrations should at least communicate the following changes:
- Any changes to the protocol state, for VM integrations that usually means contract storage changes of all contracts whose state may be accessed during a swap operation.
- Any newly added protocol component such as a pool, pair, market, etc. Basically anything that signifies that a new operation can be executed now using the protocol.
- ERC20 Balances, whenever the balances of one contracts involved with the protocol change, this change should be communicated in terms of absolute balances.
Please see the getting started page to see how to actually implement an integration.

166
docs/indexing/reserved-attributes.md
View File

@@ -1,166 +0,0 @@
# Reserved attribute names
Certain attribute names are reserved exclusively for specific purposes. Please use them only for their intended applications.
## Static Attributes
The following attributes names are reserved and must be given using `ProtocolComponent.static_att`. These attributes MUST be immutable. If it can ever change, it should be given as a state attribute (see below) for this component id.
- ### <u>**manual_updates**</u>
#### Description
The `manual_updates` static attribute determines whether the component update should be manually triggered using the `update_marker` state attribute. By default, updates occur automatically whenever there is a change in any of the required contracts. However, in scenarios where a contract undergoes frequent changes, automatic updates may not be desirable. For instance, a change in Balancer Vault storage should only trigger updates for the specific pools affected by the change, rather than for all pools indiscriminately. The `manual_updates` field helps to control and prevent unnecessary updates in such cases.
If it's enable, updates on this component are only triggered by emitting an `update_marker` state attribute (described below).
#### Type
This attribute must be set to [1u8] to enable manual updates.
#### Example Usage
```rust
Attribute {
name: "manual_updates".to_string(),
value: [1u8],
change: ChangeType::Creation.into(),
}
```
- ### <u>**pool_id**</u>
#### Description
The `pool_id` static attribute is used to specify the identifier of the pool when it differs from the `ProtocolComponent.id`. For example, Balancer pools have a component ID that corresponds to their contract address, and a separate pool ID used for registration on the Balancer Vault contract.
**Notice**: In most of the cases, using `ProtocolComponent.id` directly is preferred over `pool_id`.
#### Type
This attribute value must be provided as a UTF-8 encoded string in bytes.
#### Example Usage
```rust
Attribute {
name: "pool_id".to_string(),
value: format!("0x{}", hex::encode(pool_registered.pool_id)).as_bytes(),
change: ChangeType::Creation.into(),
}
```
## State Attributes
The following attributes names are reserved and must be given using `EntityChanges`. Unlike [Static Attributes](#static-attributes), state attributes are used for dynamic attributes and are allowed to change at anytime.
- ### <u>**stateless_contract_addr**</u>
#### Description
The `stateless_contract_addr_{index}` field is used to specify the address of a stateless contract required by the component. This field is essential for components that interact with stateless contracts, particularly in scenarios involving `DELEGATECALL`. If the bytecode of this stateless contract can be retreived in Substreams, it must be passed using the `stateless_contract_code` attribute (see below).
An index is used if multiple stateless contracts are needed. This index should start at 0 and increment by 1 for each additional `stateless_contract_addr`.
The value for `stateless_contract_addr_{index}` can be provided in two ways:
1. **Direct Contract Address**: A static contract address can be specified directly.
2. **Dynamic Address Resolution**: Alternatively, you can define a function or method that dynamically resolves and retrieves the stateless contract address at runtime. This can be particularly useful in complex contract architectures, such as those using a dynamic proxy pattern. It is important to note that the called contract must be indexed by the Substreams module.
#### Type
This attribute value must be provided as a UTF-8 encoded string in bytes.
#### Example Usage
##### 1. Direct Contract Address
To specify a direct contract address:
```rust
Attribute {
name: "stateless_contract_addr_0".into(),
value: format!("0x{}", hex::encode(address)).into_bytes(),
change: ChangeType::Creation.into(),
}
Attribute {
name: "stateless_contract_addr_1".into(),
value: format!("0x{}", hex::encode(other_address)).into_bytes(),
change: ChangeType::Creation.into(),
}
```
##### 2. Dynamic Address Resolution
To specify a function that dynamically resolves the address:
```rust
Attribute {
name: "stateless_contract_addr_0".into(),
// Call views_implementation() on TRICRYPTO_FACTORY
value: format!("call:0x{}:views_implementation()", hex::encode(TRICRYPTO_FACTORY)).into_bytes(),
change: ChangeType::Creation.into(),
}
```
- ### <u>**stateless_contract_code**</u>
#### Description
The `stateless_contract_code_{index}` field is used to specify the code for a given `stateless_contract_addr`.
An index is used if multiple stateless contracts are needed. This index must match with the related `stateless_contract_addr`.
#### Type
This attribute value must be provided as bytes.
#### Example Usage
```rust
Attribute {
name: "stateless_contract_code_0".to_string(),
value: code.to_vec(),
change: ChangeType::Creation.into(),
}
```
- ### <u>**balance_owner**</u>
#### Description
The `balance_owner` field is used to specify the address of the account that owns the protocol component tokens, in cases where the tokens are not owned by the protocol component itself or the component specifies multiple contract addresses. This is particularly useful for protocols that use a vault, for example Balancer.
#### Type
This attribute value must be provided as bytes.
#### Example Usage
```rust
Attribute {
name: "balance_owner".to_string(),
value: VAULT_ADDRESS.to_vec(),
change: ChangeType::Creation.into(),
}
```
- ### <u>**update_marker**</u>
#### Description
The `update_marker` field is used to indicate that a pool has changed, thereby triggering an update on the protocol component when `manual_update` is enabled.
#### Type
This attribute value must be provided as bytes.
#### Example Usage
```rust
Attribute {
name: "update_marker".to_string(),
value: vec![1u8],
change: ChangeType::Update.into(),
};
```

99
docs/indexing/vm-integration/README.md
View File

@@ -1,99 +0,0 @@
# VM Integration
Our indexing integrations use the Substreams library to transform raw blockchain data into higher level data streams. This is done by implementing a Rust module that is compiled into a SPKG file and then loaded by the Substreams server.
## Example
We have integrated the **Balancer** protocol as a reference, see `/substreams/ethereum-balancer` for more information.
## Step by step
1. Install [Rust](https://www.rust-lang.org/tools/install), you can do so with the following command:
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
1. Install [Substreams CLI](https://substreams.streamingfast.io/getting-started/installing-the-cli), you can either use brew:
```bash
brew install streamingfast/tap/substreams
```
use precompiled binaries
```bash
# Use correct binary for your platform
LINK=$(curl -s https://api.github.com/repos/streamingfast/substreams/releases/latest | awk '/download.url.*linux/ {print $2}' | sed 's/"//g')
curl -L $LINK | tar zxf -
```
or compile from source:
```bash
git clone https://github.com/streamingfast/substreams
cd substreams
go install -v ./cmd/substreams
```
1. Start by making a local copy of the Propeller Protocol Lib repository:
```bash
git clone https://github.com/propeller-heads/tycho-protocol-sdk
```
## Understanding the Substreams VM integration
Substreams is a new indexing technology, which uses Rust modules to compose raw blockchain data streams into higher level data streams, in our case specific to the protocol. These modules together with the protobuf definitions and manifest are then wrapped into SPKG packages (more info [here](https://substreams.streamingfast.io/quick-access/glossary#spkg-.spkg)) that are then run remotely on the Substreams server.
For more information, read the [quick explanation of Substreams](https://thegraph.com/docs/en/substreams/) or jump into the [Substreams documentation](https://substreams.streamingfast.io/). It describes the functions that need to be implemented as well as the manifest file.
### ProtoBuf files
Generally these describe the raw blockchain data that we get on the input stream and the output data that we want to produce using the Rust module.
If you are unfamiliar with ProtoBuf at all, you can start with the [official documentation](https://protobuf.dev/overview/).
First get familiar with the raw ProtoBuf definitions provided by us:
- [common.proto](../../../proto/tycho/evm/v1/common.proto) - Common types used by all integration types
You can also create your own intermediate ProtoBufs. These files should reside in your own substreams package, e.g. `./substreams/ethereum-template/proto/custom-messages.proto`. You have to link these files in the `substreams.yaml` file, see the [manifest docs](https://substreams.streamingfast.io/developers-guide/creating-your-manifest) for more information or you can look at the official substreams example integration of [UniswapV2](https://github.com/messari/substreams/blob/master/uniswap-v2/substreams.yaml#L20-L22).
*Note: Internally we are referring to our indexing library as `Tycho`, which is why our protobuf files are under the `proto/tycho` directory.*
### Rust module
The goal of the rust module is to implement the logic that will transform the raw blockchain data into the desired output data.
*This is the actual integration code that you will be writing!*
The module is a Rust library that is compiled into a SPKG (`.spkg`) file using the Substreams CLI and then loaded by the Substreams server. It is defined by the `lib.rs` file (see the [Balancer reference example](../../../substreams/ethereum-balancer/src/lib.rs)).
Read our [Substreams README.md](../../../substreams/README.md) for more information on how to write the Rust module.
### How to implement the integration
1. Create a new directory for your integration by cloning the template, rename all the references to `ethereum-template` to `[CHAIN]-[PROTOCOL_SYSTEM]`:
```bash
cp -r ./substreams/ethereum-template ./substreams/[CHAIN]-[PROTOCOL_SYSTEM]
```
1. Implement the logic in the Rust module `lib.rs`. The main function to implement is the `map_protocol_changes` function, which is called for every block.
```rust
#[substreams::handlers::map]
fn map_protocol_changes(
block: eth::v2::Block,
) -> Result<tycho::BlockChanges, substreams::errors::Error> {}
```
The `map_protocol_changes` function takes a raw block as input and returns a `BlockChanges` struct, which is derived from the `BlockChanges` protobuf message in [common.proto](../../../proto/tycho/evm/v1/common.proto).
1. The `BlockChanges` is a list of `TransactionChanges`, which includes these main fields:
- list of `ContractChange` - All storage slots that have changed in the transaction for every contract tracked by any ProtocolComponent
- list of `EntityChanges` - All the attribute changes in the transaction
- list of `ProtocolComponent` - All the protocol component changes in the transaction
- list of `BalanceChange` - All the token balances changes in the transaction
See the [Balancer reference example](../../../substreams/ethereum-balancer/src/lib.rs) for more information.
1. If you are more advanced with Substreams, you can define more steps than a single "map" step, including defining your own protobuf files. Add these protobuf files in your `pb` folder and update the manifest accordingly. This allows for better parallelization of the indexing process. See the official documentation of [modules](https://substreams.streamingfast.io/concepts-and-fundamentals/modules#modules-basics-overview).
### Testing
Read the [Substreams testing docs](https://github.com/propeller-heads/propeller-venue-lib/blob/main/testing/README.md) for more information on how to test your integration.

3
docs/logic/native-integration.md
View File

@@ -1,3 +0,0 @@
# Native Integration
Coming soon...

71
docs/logic/vm-integration/README.md
View File

@@ -1,71 +0,0 @@
# VM Integration
This page describes the interface required to implement protocol logic component.
To create a VM integration, it is required to provide a manifest file as well as an implementation of the corresponding adapter interface.
## Examples
Following exchanges have been integrated using VM approach:
* Uniswap V2 (see `/evm/src/uniswap-v2`)
* Balancer V2 (see `/evm/src/balancer-v2`)
## Step by step
### Prerequisites
1. Install [Foundry](https://book.getfoundry.sh/getting-started/installation#using-foundryup), start by downloading and installing the Foundry installer:
```bash
curl -L https://foundry.paradigm.xyz | bash
```
then start a new terminal session and run
```bash
foundryup
```
2. Start by making a local copy of the Propeller Protocol Lib repository:
```bash
git clone https://github.com/propeller-heads/tycho-protocol-sdk
```
3. Install forge dependencies:
```bash
cd ./tycho-protocol-sdk/evm/
forge install
```
### Understanding the ISwapAdapter
Read the documentation of the [Ethereum Solidity interface](ethereum-solidity.md). It describes the functions that need to be implemented as well as the manifest file. Additionally read through the docstring of the [ISwapAdapter.sol](https://github.com/propeller-heads/propeller-venue-lib/blob/main/evm/src/interfaces/ISwapAdapter.sol) interface and the [ISwapAdapterTypes.sol](https://github.com/propeller-heads/propeller-venue-lib/blob/main/evm/src/interfaces/ISwapAdapterTypes.sol) interface which defines the data types and errors used by the adapter interface. You can also generate the documentation locally and the look at the generated documentation in the `./docs` folder:
```bash
cd ./evm/
forge doc
```
### Implementing the ISwapAdapter interface
Your integration should be in a separate directory in the `evm/src` folder. Start by cloning the template directory:
```bash
cp ./evm/src/template ./evm/src/<your-adapter-name>
```
Implement the `ISwapAdapter` interface in the `./evm/src/<your-adapter-name>.sol` file. There are two reference implementations, one for Uniswap V2 and the other for Balancer V2.
### Testing your implementation
Clone the `evm/test/TemplateSwapAdapter.t.sol` file and rename it to `<your-adapter-name>.t.sol`. Implement the tests for your adapter, make sure all implemented functions are tested and working correctly. Look at the examples of `UniswapV2SwapAdapter.t.sol` and `BalancerV2SwapAdapter.t.sol` for reference. The [Foundry test guide](https://book.getfoundry.sh/forge/tests) is a good reference, especially the chapter for [Fuzz testing](https://book.getfoundry.sh/forge/fuzz-testing), which is used in both the Uniswap and Balancer tests.
We are using fork testing, i.e. we are running a local Ethereum node and fork the mainnet state. This allows us to test the integration against the real contracts and real data. To run the tests, you need to set the `ETH_RPC_URL` environment variable to the URL of an ethereum RPC. It can be your own node or a public one, like [Alchemy](https://www.alchemy.com/) or [Infura](https://infura.io/).
Finally, run the tests with:
```bash
cd ./evm
forge test
```

149
docs/logic/vm-integration/ethereum-solidity.md
View File

@@ -1,149 +0,0 @@
---
description: Provide protocol logic using the ethereum virtual machine
---
# Ethereum: Solidity
## Swap/exchange protocol
To integrate an EVM exchange protocol the [ISwapAdapter.sol ](https://github.com/propeller-heads/tycho-protocol-sdk/blob/main/evm/interfaces/ISwapAdapter.sol)should be implemented. Additionally, a manifest file is required that summarises some metadata about the protocol.
{% hint style="info" %}
Although the interface is specified for Solidity, you are not limited to writing the adapter contract in Solidity. We can use any compiled evm bytecode. So if you prefer e.g. Vyper, you are welcome to implement the interface using Vyper. Unfortunately we do not provide all the tooling for Vyper contracts yet, but you can certainly submit compiled Vyper byte code.
{% endhint %}
The manifest file contains information about the author, as well as additional static information about the protocol and how to test the current implementation. The file below lists all valid keys.
```yaml
# Information about the author helps us to reach out in case of issues.
author:
name: Propellerheads.xyz
email: alan@propellerheads.xyz
# Protocol Constants
constants:
# The minimum gas usage of a swap using the protocol, excluding any token transfers
protocol_gas: 30000
# Minimum capabilities we can expect, individual pools may extend these.
# To learn about Capabilities, see ISwapAdapter.sol
capabilities:
- SellSide
- BuySide
- PriceFunction
# The files containing the adapter contract (byte)code
contract:
# The contract runtime (i.e. deployed) bytecode (required if no source is provided)
runtime: UniswapV2SwapAdapter.bin
# If you submit the source our CI can generate the bytecode
source: UniswapV2SwapAdapter.sol
# Deployment instances used to generate chain-specific bytecode.
# Used by the runtime bytecode build script.
instances:
- chain:
name: mainnet
id: 1
# Arguments passed to the constructor when building the contract
arguments:
- "0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f"
# Specify some automatic test cases in case getPoolIds and
# getTokens are not implemented.
tests:
instances:
- pool_id: "0xB4e16d0168e52d35CaCD2c6185b44281Ec28C9Dc"
sell_token: "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2"
buy_token: "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48"
block: 17000000
chain:
name: mainnet
id: 1
```
#### Price (optional)
Calculates pool prices for specified amounts (optional).
The returned prices should be in `buyToken/sellToken` units.
The returned prices should include all protocol fees. In case the fee is dynamic, the returned price is expected to include the minimum fee.
Ideally this method should be implemented, although it is optional as the price function can be numerically estimated from the swap function. In case it is not available, it should be flagged accordingly via capabilities, and calling it should revert using the NotImplemented error.
The method needs to be implemented as view as this is usually more efficient and can be run in parallel.
```solidity
function price(
bytes32 poolId,
IERC20 sellToken,
IERC20 buyToken,
uint256[] memory sellAmounts
) external view returns (Fraction[] memory prices);
```
#### Swap
Simulates swapping tokens on a given pool.
This function should be state modifying, meaning it should actually execute the swap and change the state of the VM accordingly.
Please include a gas usage estimate for each amount. This can be achieved e.g. by using the `gasleft()` function.
The return type `Trade` has a price attribute which should contain the value of `price(specifiedAmount)`. As previously mentioned, the price function support is optional, it is valid to return a zero value for this price (in that case it will be estimated numerically). To return zero please use `Fraction(0, 1)`.
```solidity
function swap(
bytes32 poolId,
IERC20 sellToken,
IERC20 buyToken,
OrderSide side,
uint256 specifiedAmount
) external returns (Trade memory trade);
```
#### GetLimits
Retrieves the limits for each token.
This method returns the maximum amount of a token that can be traded. The limit is reached when the change in the received amounts is zero or close to zero. If in doubt, overestimate the limit. The swap function should not error with LimitExceeded if called with any amounts below the limit.
```solidity
function getLimits(bytes32 poolId, OrderSide side)
external
returns (uint256[] memory);
```
#### getCapabilities
Retrieves the capabilities of the selected pool.
```solidity
function getCapabilities(bytes32 poolId, IERC20 sellToken, IERC20 buyToken)
external
returns (Capability[] memory);
```
#### getTokens (optional)
Retrieves the tokens for the given pool.
_Mainly used for testing as this is redundant with the required substreams implementation._
```solidity
function getTokens(bytes32 poolId)
external
returns (IERC20[] memory tokens);
```
#### getPoolIds (optional)
Retrieves a range of pool IDs.
_Mainly used for testing. It is alright to not return all available pools here. Nevertheless, this is useful to test against the substreams implementation. If implemented, it saves time writing custom tests._
```solidity
function getPoolIds(uint256 offset, uint256 limit)
external
returns (bytes32[] memory ids);
```