* feat: add native balance in StorageChanges
This will be used by the next update of DCI to properly track native balance for all the contract that it's indexing. Also added a performance optimization, we now ignore slot updates if the value after the transaction is the same than before.
* docs: add docs on how to generate test assets
---------
Co-authored-by: zizou <111426680+flopell@users.noreply.github.com>
* fix: Misc improvements
Make module_name optional and default to map_protocol_changes
Add build instructions for the DB image inside of the docker-compose
Improve error message
Remove unnecessary module_name from yaml files
#time 39m
* fix: Bring back module_name in balancer v3 tests
#time 8m
* fix: Fix balance and allowance overwrites
For tokens with proxies we need to use the address that is returned by the detector
#time 13m
- There are only two post-cancun PancakeV3 pools.
- One was ORDER-WETH which I guess had a liquidity issue, it failed with `StateDecodingFailure pool="0xb2dc4d7627501338b578985c214208eb32283086" error=Missing attributes tick_liquidities`
- The second one is this USDT-USDf pool used for this test, though we fail to detect the balance slot of USDT: `WrongSlotError(“Slot override didn’t change balance.“)`
- For this reason, we skip execution.
- Also includes stop_block fix (better explained in Uniswap V3 test PR)
This fixes the error:
test-runner-1 | 2025-09-20T00:59:08.199125Z ERROR Substreams pack command failed: Error: validation failed: module "map_balance_changes" has invalid proto output type "proto:tycho.evm.uniswap.v3.BalanceDeltas": proto message type "tycho.evm.uniswap.v3.BalanceDeltas" not found in package proto definitions
And now it matches how it's done in UniswapV2
* refactor: Misc improvements to code
- Decouple validating logic from TychoRunner
- Move all data fetching and decoding the tycho message into the same method
- Split validate_state into validate_state, validate_token_balances and simulate_and_execute
- Make rpc_provider and runtime attributes of TestRunner
- Add references where possible to avoid clones
- Remove unnecessary code
- Make clippy happy
#time 2h 36m
#time 0m
#time 3m
* chore: Use tycho deps and foundry from tycho_simulation
This is to try to decrease the risk of using conflicting versions in the different repositories
#time 32m
#time 0m
* chore: Read RPC_URL in main.rs
#time 10m
* fix: Support eth trades (skip balance and allowance overwrites) and set balance overwrite to amount in
For tokens like USDC setting the balance super high was making us getting blacklisted
#time 1h 12m
* fix: Fix curve tests and filter components_by_id with the expected_component_ids
#time 1h 30m
#time 0m
* fix: Don't use all the possible executor addresses. Hardcode just one for the test
Refactor overwrites logic:
- renamed functions
- moved logic around that fits together
- don't use StateOverrides and then convert to alloy overrides. Use alloy's directly
#time 1h 21m
* fix: Assume that the executors mapping starts at storage value 1
Move setup_router_overwrites away from the rpc and into the execution file
Delete unnecessary get_storage_at
#time 33m
Before this commit we were using the latest `ContractSlot` for both previous and new value. This is not correct because if the value changed twice we would have the middle value emitted as "previous". For example if a value was changed like this in a single transaction 1 -> 10 -> 2 we would have `new_value=2` and `previous_values=10` while the previous value was actually 1.
Co-authored-by: zizou <111426680+flopell@users.noreply.github.com>
* feat: (WIP) Simulate txs against RPC
* fix: include test_executor_addresses.json
We should probably fetch this from execution instead. Will do later
* feat: (WIP) Attempt to use TokenProxyOverwriteFactory
* feat: (WIP) Attempt to use TokenProxyOverwriteFactory
* feat: Add simulating execution using debug_traceCall
Decoded amount out from execution and compared with the one from simulation
Allow for printing the execution traces
Moved all execution methods into execution.rs
Created traces.rs (heavily clauded) that prints the traces nicely and gets the function selectors frm 4byte (I decided not to use EtherscanIdentifier from foundry because adding foundry as a dependency is really heavy and I expect problems because of it)
Temporary: hardcoded storage slots for test_weighted_pool_v4 to pass
#time 1h 55m
* feat: Use BalanceSlotDetector and AllowanceSlotDetector from tycho_ethereum
#time 1h 14m
* fix: Improve traces.rs
#time 2m
* feat: Add skip_execution
Also get tycho router bytecode at compile time
#time 22m
* test: Skip tests that are before the Shanghai upgrade
Remove unnecessary TODOs
#time 35m
* feat: Point tycho dependencies to specific commits
#time 9m
* fix: Set historical_trade to true in TychoRouterEncoderBuilder
#time 41m
#time 0m
* feat: Improve traces.rs (use foundry instead of 4bytes)
Improved the format of the printed trace as well
#time 1h 7m
* fix: After merge fixes
#time 3m
* feat: Use new executors_addresses in tycho-execution and read it at startup
#time 1h 25m
* feat: Create a new flag to enable execution_traces
#time 9m
* fix: Small code, log and trace improvements
Add trying to decode method's calldatas in tracing
#time 1m
* fix: Simplify simulate_transactions_with_tracing by introducing a new method: bytes_to_fixed_32
#time 13m
* fix: After merge fixes
#time 1m
* fix: Include executor's bytecode at compile time
#time 11m
---------
Co-authored-by: TAMARA LIPOWSKI <data.lipowski@extaccount.com>
Co-authored-by: Diana Carvalho <diana@propellerheads.xyz>
It's not good to keep this in the test_runner because then we have multiple points that we need to edit the test config, which can be hard to find.
We need this because the module name for uniswap_v2 is so far different from the rest of the protocols.
- Used a random post-Shanghai-created USV2 pool state with enough liquidity for the test
Necessary fixes:
- Don't hardcode to EVMPoolState
- Don't expect adapter to always be set.
- UniswapV2 names its module differently for some reason... this seems like a special case so instead of updating all integration test yamls, I'm just hardcoding it in test_runner.rs. Perhaps not the best decision?
- Log the static attributes if not found.
- Get the substreams yaml path from config instead of hardcoding it.
- This was failing. Make this more lenient so the user can specify component id in the test file in a non-case-sensitive manner.
- Also switched around a wrong token order which was causing a failure (not sure we care about token order but we may in the future? Python didn't care...)
- These are just used for comparison purposes and will eventually be deleted.
- Also balancer v3 test fix: add protocol_system to yaml, and remove "excludePaths" from protobuf in substreams.yaml (this entry was not supported)
* feat: Add encoding to test suite
Use execution from simulation
Add protocol_system to all test files and pass it to run tycho
Add encoding_utils.rs
#time 5h 0m
#time 1m
#time 7m
* refactor: Move encoding swap to its own method to simplify main code
Rename encoding_utils.rs to encoding.rs
#time 20m
#time 0m
#time 0m
* feat: update balancer v2 spkg to make it return old slot values
This is needed for the new DCI version that uses old values to look into exactly what part of the slot change and compare with the retrigger offset
* refactor: use crate registry instead of github
---------
Co-authored-by: zizou <111426680+flopell@users.noreply.github.com>
* Add Ekubo TWAMM support
* Change order of words
* Account TWAMM order balances
* Fix tracking wrong component balance deltas
Swapped and PositionUpdated are the only events affecting pool TVL
* Fix fee addition
Fees are a .64 instead of a .128 since v2 & the result is rounded
* Consistent naming
* cargo fmt
* Add method for selecting store method from change type
* Only store the affected sale rate delta on OrderUpdated events
* Remove unnecessary parameterization
* Index Ekubo MEV-resist pools
* cargo clippy
* feat: add balancer swapAdapter and Substreams
* fix: undo tycho-substreams logs, ignore abi on rustmft
* ci: prevent warnings from failing CI
* ci: skip size check on CI
* chore: forge fmt
* feat: vault balance from storage
Vault contract tokenBalance message are set according to the vault
storage changes in the `_reserveOf` storage variable VaultStorage.sol
contract
This was the culprit that caused the failure in simulation since
balancer enforces the invariant that `token.balanceOf(vault_addr) == _reservesOf[token]`
* ci: warnings
* fix: avoid duplicated balance changes
* fix: order by ordinal
* chore: format
* feat: extract new contracts before extracting balance changes
* feat: skip unnecessary steps if no balance change is found
* refactor: filter out account balances for tokens that aren't part of any protocol components.
On the indexer side, when we receive an account balance, we need to know about the token. This commit ensure that the token was introduced before we emit any account balance with it.
* refactor: don't index liquidity buffers.
Liquidity buffers rely on rate providers. Therefore we need DCI (feature to be able to index previously created contract) to deal with them.
* refactor: cleanup tests and add docstrings
* chore: lock tycho-substreams version
* ci: set Foundry workflow to use stable foundry
* feat(DCI): Add DCI Entrypoints to BalancerV3 components (#218)
* refactor: fix typo in weighted_pool_factory_contract name
* feat: add rate_providers static attributes
* feat: add DCI entrypoints to BalancerV3 components
* fix: set default trade price to Fraction(0, 1)
* feat: remove buffers as components
Buffers are to be used internally by Boosted pools (stable/weighted pools that use ERC4626 tokens). They are not to be treated as a separate swap component.
* test: update test blocks
Extend tests some tests block range to ensure liquidity was added to the pool and can be simulated on
* feat: remove buffers as components
Remove balance updates for buffer components
* feat: listen for pool pause/unpause events
* chore: formating
* fix: encoding call data
* test: update Balancer V3 tests to use DCI
* test: set indexer log level to info
* docs: add comment on support of boosted pools
* feat: update balancer v3 package version
---------
Co-authored-by: Thales <thales@datarevenue.com>
Co-authored-by: zizou <111426680+flopell@users.noreply.github.com>
Co-authored-by: Louise Poole <louise@datarevenue.com>
Co-authored-by: Louise Poole <louisecarmenpoole@gmail.com>
* fix(sdk): properly handle deletion -> creation -> deletion sequence
This commit resolves a bug that occurred when an attribute underwent a deletion, followed by creation, and then another deletion. The system would incorrectly ignore the final deletion because the attribute was mistakenly marked as newly created during the process.
* docs: fix typo
Co-authored-by: Louise Poole <louisecarmenpoole@gmail.com>
---------
Co-authored-by: zizou <111426680+flopell@users.noreply.github.com>
Co-authored-by: Louise Poole <louisecarmenpoole@gmail.com>
* feat: Add DCI Entrypoints to BalancerV2 components
* fix: miscellaneous fixes before Balancer V2 resync
This commit fixes the entrypoints created by Balancer v2 packages, removes some disabled factories and remove support for BPT tokens (this is still to be investigated but so far we won't be able to support them)
* refactor: fix CI and bump version
* chore: update comments
---------
Co-authored-by: Thales <thales@datarevenue.com>
Co-authored-by: zizou <111426680+flopell@users.noreply.github.com>
Co-authored-by: Louise Poole <louise@datarevenue.com>
* chore: add repository URL to ethereum-maverick-v2.yaml
* chore: bump maverick-v2 version.
This bump is needed because we changed the version of `tycho-substreams` in a previous commit (to fix the bug with contract creation).
---------
Co-authored-by: zizou <111426680+flopell@users.noreply.github.com>
