dexorder/ai
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

backend redesign

Browse Source
This commit is contained in:
Tim Olson
2026-03-11 18:47:11 -04:00
parent 8ff277c8c6
commit e99ef5d2dd
210 changed files with 12147 additions and 155 deletions
Download Patch File Download Diff File Expand all files Collapse all files

179
backend.old/src/exchange_kernel/README.md Normal file
View File

@@ -0,0 +1,179 @@
# Exchange Kernel API
A Kubernetes-style declarative API for managing orders across different exchanges.
## Architecture Overview
The Exchange Kernel maintains two separate views of order state:
1. **Desired State (Intent)**: What the strategy kernel wants
2. **Actual State (Reality)**: What currently exists on the exchange
A reconciliation loop continuously works to bring actual state into alignment with desired state, handling errors, retries, and edge cases automatically.
## Core Components
### Models (`models.py`)
- **OrderIntent**: Desired order state from strategy kernel
- **OrderState**: Actual current order state on exchange
- **Position**: Current position (spot, margin, perp, futures, options)
- **Asset**: Asset holdings with metadata
- **AccountState**: Complete account snapshot (balances, positions, margin)
- **AssetMetadata**: Asset type descriptions and trading parameters
### Events (`events.py`)
Order lifecycle events:
- `OrderSubmitted`, `OrderAccepted`, `OrderRejected`
- `OrderPartiallyFilled`, `OrderFilled`, `OrderCanceled`
- `OrderModified`, `OrderExpired`
Position events:
- `PositionOpened`, `PositionModified`, `PositionClosed`
Account events:
- `AccountBalanceUpdated`, `MarginCallWarning`
### Base Interface (`base.py`)
Abstract `ExchangeKernel` class defining:
**Command API**:
- `place_order()`, `place_order_group()` - Create order intents
- `cancel_order()`, `modify_order()` - Update intents
- `cancel_all_orders()` - Bulk cancellation
**Query API**:
- `get_order_intent()`, `get_order_state()` - Query single order
- `get_all_intents()`, `get_all_orders()` - Query all orders
- `get_positions()`, `get_account_state()` - Query positions/balances
- `get_symbol_metadata()`, `get_asset_metadata()` - Query market info
**Event API**:
- `subscribe_events()`, `unsubscribe_events()` - Event notifications
**Lifecycle**:
- `start()`, `stop()` - Kernel lifecycle
- `health_check()` - Connection status
- `force_reconciliation()` - Manual reconciliation trigger
### State Management (`state.py`)
- **IntentStateStore**: Storage for desired state (durable, survives restarts)
- **ActualStateStore**: Storage for actual exchange state (ephemeral cache)
- **ReconciliationEngine**: Framework for intent→reality reconciliation
- **InMemory implementations**: For testing/prototyping
## Standard Order Model
Defined in `schema/order_spec.py`:
```python
StandardOrder(
symbol_id="BTC/USD",
side=Side.BUY,
amount=1.0,
amount_type=AmountType.BASE, # or QUOTE for exact-out
limit_price=50000.0, # None for market orders
time_in_force=TimeInForce.GTC,
conditional_trigger=ConditionalTrigger(...), # Optional stop-loss/take-profit
conditional_mode=ConditionalOrderMode.UNIFIED_ADJUSTING,
reduce_only=False,
post_only=False,
iceberg_qty=None,
)
```
## Symbol Metadata
Markets describe their capabilities via `SymbolMetadata`:
- **AmountConstraints**: Min/max order size, step size
- **PriceConstraints**: Tick size, tick spacing mode (fixed/dynamic/continuous)
- **MarketCapabilities**:
- Supported sides (BUY, SELL)
- Supported amount types (BASE, QUOTE, or both)
- Market vs limit order support
- Time-in-force options (GTC, IOC, FOK, DAY, GTD)
- Conditional order support (stop-loss, take-profit, trailing stops)
- Advanced features (post-only, reduce-only, iceberg)
## Asset Types
Comprehensive asset type system supporting:
- **SPOT**: Cash markets
- **MARGIN**: Margin trading
- **PERP**: Perpetual futures
- **FUTURE**: Dated futures
- **OPTION**: Options contracts
- **SYNTHETIC**: Derived instruments
Each asset has metadata describing contract specs, settlement, margin requirements, etc.
## Usage Pattern
```python
# Create exchange kernel for specific exchange
kernel = SomeExchangeKernel(exchange_id="binance_main")
# Subscribe to events
kernel.subscribe_events(my_event_handler)
# Start kernel
await kernel.start()
# Place order (creates intent, kernel handles execution)
intent_id = await kernel.place_order(
StandardOrder(
symbol_id="BTC/USD",
side=Side.BUY,
amount=1.0,
amount_type=AmountType.BASE,
limit_price=50000.0,
)
)
# Query desired state
intent = await kernel.get_order_intent(intent_id)
# Query actual state
state = await kernel.get_order_state(intent_id)
# Modify order (updates intent, kernel reconciles)
await kernel.modify_order(intent_id, new_order)
# Cancel order
await kernel.cancel_order(intent_id)
# Query positions
positions = await kernel.get_positions()
# Query account state
account = await kernel.get_account_state()
```
## Implementation Status
✅ **Complete**:
- Data models and type definitions
- Event definitions
- Abstract interface
- State store framework
- In-memory stores for testing
⏳ **TODO** (Exchange-specific implementations):
- Concrete ExchangeKernel implementations per exchange
- Reconciliation engine implementation
- Exchange API adapters
- Persistent state storage (database)
- Error handling and retry logic
- Monitoring and observability
## Next Steps
1. Create concrete implementations for specific exchanges (Binance, Uniswap, etc.)
2. Implement reconciliation engine with proper error handling
3. Add persistent storage backend for intents
4. Build integration tests
5. Add monitoring/metrics collection

75
backend.old/src/exchange_kernel/__init__.py Normal file
View File

@@ -0,0 +1,75 @@
"""
Exchange Kernel API
The exchange kernel provides a Kubernetes-style declarative API for managing orders
across different exchanges. It maintains both desired state (intent) and actual state
(current orders on exchange) and reconciles them continuously.
Key concepts:
- OrderIntent: What the strategy kernel wants
- OrderState: What actually exists on the exchange
- Reconciliation: Bringing actual state into alignment with desired state
"""
from .base import ExchangeKernel
from .events import (
OrderEvent,
OrderSubmitted,
OrderAccepted,
OrderRejected,
OrderPartiallyFilled,
OrderFilled,
OrderCanceled,
OrderModified,
OrderExpired,
PositionEvent,
PositionOpened,
PositionModified,
PositionClosed,
AccountEvent,
AccountBalanceUpdated,
MarginCallWarning,
)
from .models import (
OrderIntent,
OrderState,
Position,
Asset,
AssetMetadata,
AccountState,
Balance,
)
from .state import IntentStateStore, ActualStateStore
__all__ = [
# Core interface
"ExchangeKernel",
# Events
"OrderEvent",
"OrderSubmitted",
"OrderAccepted",
"OrderRejected",
"OrderPartiallyFilled",
"OrderFilled",
"OrderCanceled",
"OrderModified",
"OrderExpired",
"PositionEvent",
"PositionOpened",
"PositionModified",
"PositionClosed",
"AccountEvent",
"AccountBalanceUpdated",
"MarginCallWarning",
# Models
"OrderIntent",
"OrderState",
"Position",
"Asset",
"AssetMetadata",
"AccountState",
"Balance",
# State management
"IntentStateStore",
"ActualStateStore",
]

361
backend.old/src/exchange_kernel/base.py Normal file
View File

@@ -0,0 +1,361 @@
"""
Base interface for Exchange Kernels.
Defines the abstract API that all exchange kernel implementations must support.
Each exchange (or exchange type) will have its own kernel implementation.
"""
from abc import ABC, abstractmethod
from typing import Callable, Any
from .models import (
OrderIntent,
OrderState,
Position,
AccountState,
AssetMetadata,
)
from .events import BaseEvent
from ..schema.order_spec import (
StandardOrder,
StandardOrderGroup,
SymbolMetadata,
)
class ExchangeKernel(ABC):
"""
Abstract base class for exchange kernels.
An exchange kernel manages the lifecycle of orders on a specific exchange,
maintaining both desired state (intents from strategy kernel) and actual
state (current orders on exchange), and continuously reconciling them.
Think of it as a Kubernetes-style controller for trading orders.
"""
def __init__(self, exchange_id: str):
"""
Initialize the exchange kernel.
Args:
exchange_id: Unique identifier for this exchange instance
"""
self.exchange_id = exchange_id
# -------------------------------------------------------------------------
# Command API - Strategy kernel sends intents
# -------------------------------------------------------------------------
@abstractmethod
async def place_order(self, order: StandardOrder, metadata: dict[str, Any] | None = None) -> str:
"""
Place a single order on the exchange.
This creates an OrderIntent and begins the reconciliation process to
get the order onto the exchange.
Args:
order: The order specification
metadata: Optional strategy-specific metadata
Returns:
intent_id: Unique identifier for this order intent
Raises:
ValidationError: If order violates market constraints
ExchangeError: If exchange rejects the order
"""
pass
@abstractmethod
async def place_order_group(
self,
group: StandardOrderGroup,
metadata: dict[str, Any] | None = None
) -> list[str]:
"""
Place a group of orders with OCO (One-Cancels-Other) relationship.
Args:
group: Group of orders with OCO mode
metadata: Optional strategy-specific metadata
Returns:
intent_ids: List of intent IDs for each order in the group
Raises:
ValidationError: If any order violates market constraints
ExchangeError: If exchange rejects the group
"""
pass
@abstractmethod
async def cancel_order(self, intent_id: str) -> None:
"""
Cancel an order by intent ID.
Updates the intent to indicate cancellation is desired, and the
reconciliation loop will handle the actual exchange cancellation.
Args:
intent_id: Intent ID of the order to cancel
Raises:
NotFoundError: If intent_id doesn't exist
ExchangeError: If exchange rejects cancellation
"""
pass
@abstractmethod
async def modify_order(
self,
intent_id: str,
new_order: StandardOrder,
) -> None:
"""
Modify an existing order.
Updates the order intent, and the reconciliation loop will update
the exchange order (via modify API if available, or cancel+replace).
Args:
intent_id: Intent ID of the order to modify
new_order: New order specification
Raises:
NotFoundError: If intent_id doesn't exist
ValidationError: If new order violates market constraints
ExchangeError: If exchange rejects modification
"""
pass
@abstractmethod
async def cancel_all_orders(self, symbol_id: str | None = None) -> int:
"""
Cancel all orders, optionally filtered by symbol.
Args:
symbol_id: If provided, only cancel orders for this symbol
Returns:
count: Number of orders canceled
"""
pass
# -------------------------------------------------------------------------
# Query API - Read desired and actual state
# -------------------------------------------------------------------------
@abstractmethod
async def get_order_intent(self, intent_id: str) -> OrderIntent:
"""
Get the desired order state (what strategy kernel wants).
Args:
intent_id: Intent ID to query
Returns:
The order intent
Raises:
NotFoundError: If intent_id doesn't exist
"""
pass
@abstractmethod
async def get_order_state(self, intent_id: str) -> OrderState:
"""
Get the actual order state (what's currently on exchange).
Args:
intent_id: Intent ID to query
Returns:
The current order state
Raises:
NotFoundError: If intent_id doesn't exist
"""
pass
@abstractmethod
async def get_all_intents(self, symbol_id: str | None = None) -> list[OrderIntent]:
"""
Get all order intents, optionally filtered by symbol.
Args:
symbol_id: If provided, only return intents for this symbol
Returns:
List of order intents
"""
pass
@abstractmethod
async def get_all_orders(self, symbol_id: str | None = None) -> list[OrderState]:
"""
Get all actual order states, optionally filtered by symbol.
Args:
symbol_id: If provided, only return orders for this symbol
Returns:
List of order states
"""
pass
@abstractmethod
async def get_positions(self, symbol_id: str | None = None) -> list[Position]:
"""
Get current positions, optionally filtered by symbol.
Args:
symbol_id: If provided, only return positions for this symbol
Returns:
List of positions
"""
pass
@abstractmethod
async def get_account_state(self) -> AccountState:
"""
Get current account state (balances, margin, etc.).
Returns:
Current account state
"""
pass
@abstractmethod
async def get_symbol_metadata(self, symbol_id: str) -> SymbolMetadata:
"""
Get metadata for a symbol (constraints, capabilities, etc.).
Args:
symbol_id: Symbol to query
Returns:
Symbol metadata
Raises:
NotFoundError: If symbol doesn't exist on this exchange
"""
pass
@abstractmethod
async def get_asset_metadata(self, asset_id: str) -> AssetMetadata:
"""
Get metadata for an asset.
Args:
asset_id: Asset to query
Returns:
Asset metadata
Raises:
NotFoundError: If asset doesn't exist
"""
pass
@abstractmethod
async def list_symbols(self) -> list[str]:
"""
List all available symbols on this exchange.
Returns:
List of symbol IDs
"""
pass
# -------------------------------------------------------------------------
# Event Subscription API
# -------------------------------------------------------------------------
@abstractmethod
def subscribe_events(
self,
callback: Callable[[BaseEvent], None],
event_filter: dict[str, Any] | None = None,
) -> str:
"""
Subscribe to events from this exchange kernel.
Args:
callback: Function to call when events occur
event_filter: Optional filter criteria (event_type, symbol_id, etc.)
Returns:
subscription_id: Unique ID for this subscription (for unsubscribe)
"""
pass
@abstractmethod
def unsubscribe_events(self, subscription_id: str) -> None:
"""
Unsubscribe from events.
Args:
subscription_id: Subscription ID returned from subscribe_events
"""
pass
# -------------------------------------------------------------------------
# Lifecycle Management
# -------------------------------------------------------------------------
@abstractmethod
async def start(self) -> None:
"""
Start the exchange kernel.
Initializes connections, starts reconciliation loops, etc.
"""
pass
@abstractmethod
async def stop(self) -> None:
"""
Stop the exchange kernel.
Closes connections, stops reconciliation loops, etc.
Does NOT cancel open orders - call cancel_all_orders() first if desired.
"""
pass
@abstractmethod
async def health_check(self) -> dict[str, Any]:
"""
Check health status of the exchange kernel.
Returns:
Health status dict with connection state, latency, error counts, etc.
"""
pass
# -------------------------------------------------------------------------
# Reconciliation Control (advanced)
# -------------------------------------------------------------------------
@abstractmethod
async def force_reconciliation(self, intent_id: str | None = None) -> None:
"""
Force immediate reconciliation.
Args:
intent_id: If provided, only reconcile this specific intent.
If None, reconcile all intents.
"""
pass
@abstractmethod
def get_reconciliation_metrics(self) -> dict[str, Any]:
"""
Get metrics about the reconciliation process.
Returns:
Metrics dict with reconciliation lag, error rates, retry counts, etc.
"""
pass

250
backend.old/src/exchange_kernel/events.py Normal file
View File

@@ -0,0 +1,250 @@
"""
Event definitions for the Exchange Kernel.
All events that can occur during the order lifecycle, position management,
and account updates.
"""
from enum import StrEnum
from typing import Any
from pydantic import BaseModel, Field
from ..schema.order_spec import Float, Uint64
# ---------------------------------------------------------------------------
# Base Event Classes
# ---------------------------------------------------------------------------
class EventType(StrEnum):
"""Types of events emitted by the exchange kernel"""
# Order lifecycle
ORDER_SUBMITTED = "ORDER_SUBMITTED"
ORDER_ACCEPTED = "ORDER_ACCEPTED"
ORDER_REJECTED = "ORDER_REJECTED"
ORDER_PARTIALLY_FILLED = "ORDER_PARTIALLY_FILLED"
ORDER_FILLED = "ORDER_FILLED"
ORDER_CANCELED = "ORDER_CANCELED"
ORDER_MODIFIED = "ORDER_MODIFIED"
ORDER_EXPIRED = "ORDER_EXPIRED"
# Position events
POSITION_OPENED = "POSITION_OPENED"
POSITION_MODIFIED = "POSITION_MODIFIED"
POSITION_CLOSED = "POSITION_CLOSED"
# Account events
ACCOUNT_BALANCE_UPDATED = "ACCOUNT_BALANCE_UPDATED"
MARGIN_CALL_WARNING = "MARGIN_CALL_WARNING"
# System events
RECONCILIATION_FAILED = "RECONCILIATION_FAILED"
CONNECTION_LOST = "CONNECTION_LOST"
CONNECTION_RESTORED = "CONNECTION_RESTORED"
class BaseEvent(BaseModel):
"""Base class for all exchange kernel events"""
model_config = {"extra": "forbid"}
event_type: EventType = Field(description="Type of event")
timestamp: Uint64 = Field(description="Event timestamp (Unix milliseconds)")
exchange: str = Field(description="Exchange identifier")
metadata: dict[str, Any] = Field(default_factory=dict, description="Additional event data")
# ---------------------------------------------------------------------------
# Order Events
# ---------------------------------------------------------------------------
class OrderEvent(BaseEvent):
"""Base class for order-related events"""
intent_id: str = Field(description="Order intent ID")
order_id: str | None = Field(default=None, description="Exchange order ID (if assigned)")
symbol_id: str = Field(description="Symbol being traded")
class OrderSubmitted(OrderEvent):
"""Order has been submitted to the exchange"""
event_type: EventType = Field(default=EventType.ORDER_SUBMITTED)
client_order_id: str | None = Field(default=None, description="Client-assigned order ID")
class OrderAccepted(OrderEvent):
"""Order has been accepted by the exchange"""
event_type: EventType = Field(default=EventType.ORDER_ACCEPTED)
order_id: str = Field(description="Exchange-assigned order ID")
accepted_at: Uint64 = Field(description="Exchange acceptance timestamp")
class OrderRejected(OrderEvent):
"""Order was rejected by the exchange"""
event_type: EventType = Field(default=EventType.ORDER_REJECTED)
reason: str = Field(description="Rejection reason")
error_code: str | None = Field(default=None, description="Exchange error code")
class OrderPartiallyFilled(OrderEvent):
"""Order was partially filled"""
event_type: EventType = Field(default=EventType.ORDER_PARTIALLY_FILLED)
order_id: str = Field(description="Exchange order ID")
fill_price: Float = Field(description="Fill price for this execution")
fill_quantity: Float = Field(description="Quantity filled in this execution")
total_filled: Float = Field(description="Total quantity filled so far")
remaining_quantity: Float = Field(description="Remaining quantity to fill")
commission: Float = Field(default=0.0, description="Commission/fee for this fill")
commission_asset: str | None = Field(default=None, description="Asset used for commission")
trade_id: str | None = Field(default=None, description="Exchange trade ID")
class OrderFilled(OrderEvent):
"""Order was completely filled"""
event_type: EventType = Field(default=EventType.ORDER_FILLED)
order_id: str = Field(description="Exchange order ID")
average_fill_price: Float = Field(description="Average execution price")
total_quantity: Float = Field(description="Total quantity filled")
total_commission: Float = Field(default=0.0, description="Total commission/fees")
commission_asset: str | None = Field(default=None, description="Asset used for commission")
completed_at: Uint64 = Field(description="Completion timestamp")
class OrderCanceled(OrderEvent):
"""Order was canceled"""
event_type: EventType = Field(default=EventType.ORDER_CANCELED)
order_id: str = Field(description="Exchange order ID")
reason: str = Field(description="Cancellation reason")
filled_quantity: Float = Field(default=0.0, description="Quantity filled before cancellation")
canceled_at: Uint64 = Field(description="Cancellation timestamp")
class OrderModified(OrderEvent):
"""Order was modified (price, quantity, etc.)"""
event_type: EventType = Field(default=EventType.ORDER_MODIFIED)
order_id: str = Field(description="Exchange order ID")
old_price: Float | None = Field(default=None, description="Previous price")
new_price: Float | None = Field(default=None, description="New price")
old_quantity: Float | None = Field(default=None, description="Previous quantity")
new_quantity: Float | None = Field(default=None, description="New quantity")
modified_at: Uint64 = Field(description="Modification timestamp")
class OrderExpired(OrderEvent):
"""Order expired (GTD, DAY orders)"""
event_type: EventType = Field(default=EventType.ORDER_EXPIRED)
order_id: str = Field(description="Exchange order ID")
filled_quantity: Float = Field(default=0.0, description="Quantity filled before expiration")
expired_at: Uint64 = Field(description="Expiration timestamp")
# ---------------------------------------------------------------------------
# Position Events
# ---------------------------------------------------------------------------
class PositionEvent(BaseEvent):
"""Base class for position-related events"""
position_id: str = Field(description="Position identifier")
symbol_id: str = Field(description="Symbol identifier")
asset_id: str = Field(description="Asset identifier")
class PositionOpened(PositionEvent):
"""New position was opened"""
event_type: EventType = Field(default=EventType.POSITION_OPENED)
quantity: Float = Field(description="Position quantity")
entry_price: Float = Field(description="Entry price")
side: str = Field(description="LONG or SHORT")
leverage: Float | None = Field(default=None, description="Leverage")
class PositionModified(PositionEvent):
"""Existing position was modified (size change, etc.)"""
event_type: EventType = Field(default=EventType.POSITION_MODIFIED)
old_quantity: Float = Field(description="Previous quantity")
new_quantity: Float = Field(description="New quantity")
average_entry_price: Float = Field(description="Updated average entry price")
unrealized_pnl: Float | None = Field(default=None, description="Current unrealized P&L")
class PositionClosed(PositionEvent):
"""Position was closed"""
event_type: EventType = Field(default=EventType.POSITION_CLOSED)
exit_price: Float = Field(description="Exit price")
realized_pnl: Float = Field(description="Realized profit/loss")
closed_at: Uint64 = Field(description="Closure timestamp")
# ---------------------------------------------------------------------------
# Account Events
# ---------------------------------------------------------------------------
class AccountEvent(BaseEvent):
"""Base class for account-related events"""
account_id: str = Field(description="Account identifier")
class AccountBalanceUpdated(AccountEvent):
"""Account balance was updated"""
event_type: EventType = Field(default=EventType.ACCOUNT_BALANCE_UPDATED)
asset_id: str = Field(description="Asset that changed")
old_balance: Float = Field(description="Previous balance")
new_balance: Float = Field(description="New balance")
old_available: Float = Field(description="Previous available")
new_available: Float = Field(description="New available")
change_reason: str = Field(description="Why balance changed (TRADE, DEPOSIT, WITHDRAWAL, etc.)")
class MarginCallWarning(AccountEvent):
"""Margin level is approaching liquidation threshold"""
event_type: EventType = Field(default=EventType.MARGIN_CALL_WARNING)
margin_level: Float = Field(description="Current margin level")
liquidation_threshold: Float = Field(description="Liquidation threshold")
required_action: str = Field(description="Required action to avoid liquidation")
estimated_liquidation_price: Float | None = Field(
default=None,
description="Estimated liquidation price for positions"
)
# ---------------------------------------------------------------------------
# System Events
# ---------------------------------------------------------------------------
class ReconciliationFailed(BaseEvent):
"""Failed to reconcile intent with actual state"""
event_type: EventType = Field(default=EventType.RECONCILIATION_FAILED)
intent_id: str = Field(description="Order intent ID")
error_message: str = Field(description="Error details")
retry_count: int = Field(description="Number of retry attempts")
class ConnectionLost(BaseEvent):
"""Connection to exchange was lost"""
event_type: EventType = Field(default=EventType.CONNECTION_LOST)
reason: str = Field(description="Disconnection reason")
class ConnectionRestored(BaseEvent):
"""Connection to exchange was restored"""
event_type: EventType = Field(default=EventType.CONNECTION_RESTORED)
downtime_duration: int = Field(description="Duration of downtime in milliseconds")

194
backend.old/src/exchange_kernel/models.py Normal file
View File

@@ -0,0 +1,194 @@
"""
Data models for the Exchange Kernel.
Defines order intents, order state, positions, assets, and account state.
"""
from enum import StrEnum
from typing import Any
from pydantic import BaseModel, Field
from ..schema.order_spec import (
StandardOrder,
StandardOrderStatus,
AssetType,
Float,
Uint64,
)
# ---------------------------------------------------------------------------
# Order Intent and State
# ---------------------------------------------------------------------------
class OrderIntent(BaseModel):
"""
Desired order state from the strategy kernel.
This represents what the strategy wants, not what currently exists.
The exchange kernel will work to reconcile actual state with this intent.
"""
model_config = {"extra": "forbid"}
intent_id: str = Field(description="Unique identifier for this intent (client-assigned)")
order: StandardOrder = Field(description="The desired order specification")
group_id: str | None = Field(default=None, description="Group ID for OCO relationships")
created_at: Uint64 = Field(description="When this intent was created")
updated_at: Uint64 = Field(description="When this intent was last modified")
metadata: dict[str, Any] = Field(default_factory=dict, description="Strategy-specific metadata")
class ReconciliationStatus(StrEnum):
"""Status of reconciliation between intent and actual state"""
PENDING = "PENDING" # Not yet submitted to exchange
SUBMITTING = "SUBMITTING" # Currently being submitted
ACTIVE = "ACTIVE" # Successfully placed on exchange
RECONCILING = "RECONCILING" # Intent changed, updating exchange order
FAILED = "FAILED" # Failed to submit or reconcile
COMPLETED = "COMPLETED" # Order fully filled
CANCELED = "CANCELED" # Order canceled
class OrderState(BaseModel):
"""
Actual current state of an order on the exchange.
This represents reality - what the exchange reports about the order.
May differ from OrderIntent during reconciliation.
"""
model_config = {"extra": "forbid"}
intent_id: str = Field(description="Links back to the OrderIntent")
exchange_order_id: str = Field(description="Exchange-assigned order ID")
status: StandardOrderStatus = Field(description="Current order status from exchange")
reconciliation_status: ReconciliationStatus = Field(description="Reconciliation state")
last_sync_at: Uint64 = Field(description="Last time we synced with exchange")
error_message: str | None = Field(default=None, description="Error details if FAILED")
# ---------------------------------------------------------------------------
# Position and Asset Models
# ---------------------------------------------------------------------------
class AssetMetadata(BaseModel):
"""
Metadata describing an asset type.
Provides context for positions, balances, and trading.
"""
model_config = {"extra": "forbid"}
asset_id: str = Field(description="Unique asset identifier")
symbol: str = Field(description="Asset symbol (e.g., 'BTC', 'ETH', 'USD')")
asset_type: AssetType = Field(description="Type of asset")
name: str = Field(description="Full name")
# Contract specifications (for derivatives)
contract_size: Float | None = Field(default=None, description="Contract multiplier")
settlement_asset: str | None = Field(default=None, description="Settlement currency")
expiry_timestamp: Uint64 | None = Field(default=None, description="Expiration timestamp")
# Trading parameters
tick_size: Float | None = Field(default=None, description="Minimum price increment")
lot_size: Float | None = Field(default=None, description="Minimum quantity increment")
# Margin requirements (for leveraged products)
initial_margin_rate: Float | None = Field(default=None, description="Initial margin requirement")
maintenance_margin_rate: Float | None = Field(default=None, description="Maintenance margin requirement")
# Additional metadata
metadata: dict[str, Any] = Field(default_factory=dict, description="Exchange-specific metadata")
class Asset(BaseModel):
"""
An asset holding (spot, margin, derivative position, etc.)
"""
model_config = {"extra": "forbid"}
asset_id: str = Field(description="References AssetMetadata")
quantity: Float = Field(description="Amount held (positive or negative for short positions)")
available: Float = Field(description="Amount available for trading (not locked in orders)")
locked: Float = Field(description="Amount locked in open orders")
# For derivative positions
entry_price: Float | None = Field(default=None, description="Average entry price")
mark_price: Float | None = Field(default=None, description="Current mark price")
liquidation_price: Float | None = Field(default=None, description="Estimated liquidation price")
unrealized_pnl: Float | None = Field(default=None, description="Unrealized profit/loss")
realized_pnl: Float | None = Field(default=None, description="Realized profit/loss")
# Margin info
margin_used: Float | None = Field(default=None, description="Margin allocated to this position")
updated_at: Uint64 = Field(description="Last update timestamp")
class Position(BaseModel):
"""
A trading position (spot, margin, perpetual, futures, etc.)
Tracks both the asset holdings and associated metadata.
"""
model_config = {"extra": "forbid"}
position_id: str = Field(description="Unique position identifier")
symbol_id: str = Field(description="Trading symbol")
asset: Asset = Field(description="Asset holding details")
metadata: AssetMetadata = Field(description="Asset metadata")
# Position-level info
leverage: Float | None = Field(default=None, description="Current leverage")
side: str | None = Field(default=None, description="LONG or SHORT (for derivatives)")
updated_at: Uint64 = Field(description="Last update timestamp")
class Balance(BaseModel):
"""Account balance for a single currency/asset"""
model_config = {"extra": "forbid"}
asset_id: str = Field(description="Asset identifier")
total: Float = Field(description="Total balance")
available: Float = Field(description="Available for trading")
locked: Float = Field(description="Locked in orders/positions")
# For margin accounts
borrowed: Float = Field(default=0.0, description="Borrowed amount (margin)")
interest: Float = Field(default=0.0, description="Accrued interest")
updated_at: Uint64 = Field(description="Last update timestamp")
class AccountState(BaseModel):
"""
Complete account state including balances, positions, and margin info.
"""
model_config = {"extra": "forbid"}
account_id: str = Field(description="Account identifier")
exchange: str = Field(description="Exchange identifier")
balances: list[Balance] = Field(default_factory=list, description="All asset balances")
positions: list[Position] = Field(default_factory=list, description="All open positions")
# Margin account info
total_equity: Float | None = Field(default=None, description="Total account equity")
total_margin_used: Float | None = Field(default=None, description="Total margin in use")
total_available_margin: Float | None = Field(default=None, description="Available margin")
margin_level: Float | None = Field(default=None, description="Margin level (equity/margin_used)")
# Risk metrics
total_unrealized_pnl: Float | None = Field(default=None, description="Total unrealized P&L")
total_realized_pnl: Float | None = Field(default=None, description="Total realized P&L")
updated_at: Uint64 = Field(description="Last update timestamp")
metadata: dict[str, Any] = Field(default_factory=dict, description="Exchange-specific data")

472
backend.old/src/exchange_kernel/state.py Normal file
View File

@@ -0,0 +1,472 @@
"""
State management for the Exchange Kernel.
Implements the storage and reconciliation logic for desired vs actual state.
This is the "Kubernetes for orders" concept - maintaining intent and continuously
reconciling reality to match intent.
"""
from abc import ABC, abstractmethod
from typing import Any
from collections import defaultdict
from .models import OrderIntent, OrderState, ReconciliationStatus
from ..schema.order_spec import Uint64
# ---------------------------------------------------------------------------
# Intent State Store - Desired State
# ---------------------------------------------------------------------------
class IntentStateStore(ABC):
"""
Storage for order intents (desired state).
This represents what the strategy kernel wants. Intents are durable and
persist across restarts. The reconciliation loop continuously works to
make actual state match these intents.
"""
@abstractmethod
async def create_intent(self, intent: OrderIntent) -> None:
"""
Store a new order intent.
Args:
intent: The order intent to store
Raises:
AlreadyExistsError: If intent_id already exists
"""
pass
@abstractmethod
async def get_intent(self, intent_id: str) -> OrderIntent:
"""
Retrieve an order intent.
Args:
intent_id: Intent ID to retrieve
Returns:
The order intent
Raises:
NotFoundError: If intent_id doesn't exist
"""
pass
@abstractmethod
async def update_intent(self, intent: OrderIntent) -> None:
"""
Update an existing order intent.
Args:
intent: Updated intent (intent_id must match existing)
Raises:
NotFoundError: If intent_id doesn't exist
"""
pass
@abstractmethod
async def delete_intent(self, intent_id: str) -> None:
"""
Delete an order intent.
Args:
intent_id: Intent ID to delete
Raises:
NotFoundError: If intent_id doesn't exist
"""
pass
@abstractmethod
async def list_intents(
self,
symbol_id: str | None = None,
group_id: str | None = None,
) -> list[OrderIntent]:
"""
List all order intents, optionally filtered.
Args:
symbol_id: Filter by symbol
group_id: Filter by OCO group
Returns:
List of matching intents
"""
pass
@abstractmethod
async def get_intents_by_group(self, group_id: str) -> list[OrderIntent]:
"""
Get all intents in an OCO group.
Args:
group_id: Group ID to query
Returns:
List of intents in the group
"""
pass
# ---------------------------------------------------------------------------
# Actual State Store - Current Reality
# ---------------------------------------------------------------------------
class ActualStateStore(ABC):
"""
Storage for actual order state (reality on exchange).
This represents what actually exists on the exchange right now.
Updated frequently from exchange feeds and order status queries.
"""
@abstractmethod
async def create_order_state(self, state: OrderState) -> None:
"""
Store a new order state.
Args:
state: The order state to store
Raises:
AlreadyExistsError: If order state for this intent_id already exists
"""
pass
@abstractmethod
async def get_order_state(self, intent_id: str) -> OrderState:
"""
Retrieve order state for an intent.
Args:
intent_id: Intent ID to query
Returns:
The current order state
Raises:
NotFoundError: If no state exists for this intent
"""
pass
@abstractmethod
async def get_order_state_by_exchange_id(self, exchange_order_id: str) -> OrderState:
"""
Retrieve order state by exchange order ID.
Useful for processing exchange callbacks that only provide exchange_order_id.
Args:
exchange_order_id: Exchange's order ID
Returns:
The order state
Raises:
NotFoundError: If no state exists for this exchange order ID
"""
pass
@abstractmethod
async def update_order_state(self, state: OrderState) -> None:
"""
Update an existing order state.
Args:
state: Updated state (intent_id must match existing)
Raises:
NotFoundError: If state doesn't exist
"""
pass
@abstractmethod
async def delete_order_state(self, intent_id: str) -> None:
"""
Delete an order state.
Args:
intent_id: Intent ID whose state to delete
Raises:
NotFoundError: If state doesn't exist
"""
pass
@abstractmethod
async def list_order_states(
self,
symbol_id: str | None = None,
reconciliation_status: ReconciliationStatus | None = None,
) -> list[OrderState]:
"""
List all order states, optionally filtered.
Args:
symbol_id: Filter by symbol
reconciliation_status: Filter by reconciliation status
Returns:
List of matching order states
"""
pass
@abstractmethod
async def get_stale_orders(self, max_age_seconds: int) -> list[OrderState]:
"""
Find orders that haven't been synced recently.
Used to identify orders that need status updates from exchange.
Args:
max_age_seconds: Maximum age since last sync
Returns:
List of order states that need refresh
"""
pass
# ---------------------------------------------------------------------------
# In-Memory Implementations (for testing/prototyping)
# ---------------------------------------------------------------------------
class InMemoryIntentStore(IntentStateStore):
"""Simple in-memory implementation of IntentStateStore"""
def __init__(self):
self._intents: dict[str, OrderIntent] = {}
self._by_symbol: dict[str, set[str]] = defaultdict(set)
self._by_group: dict[str, set[str]] = defaultdict(set)
async def create_intent(self, intent: OrderIntent) -> None:
if intent.intent_id in self._intents:
raise ValueError(f"Intent {intent.intent_id} already exists")
self._intents[intent.intent_id] = intent
self._by_symbol[intent.order.symbol_id].add(intent.intent_id)
if intent.group_id:
self._by_group[intent.group_id].add(intent.intent_id)
async def get_intent(self, intent_id: str) -> OrderIntent:
if intent_id not in self._intents:
raise KeyError(f"Intent {intent_id} not found")
return self._intents[intent_id]
async def update_intent(self, intent: OrderIntent) -> None:
if intent.intent_id not in self._intents:
raise KeyError(f"Intent {intent.intent_id} not found")
old_intent = self._intents[intent.intent_id]
# Update indices if symbol or group changed
if old_intent.order.symbol_id != intent.order.symbol_id:
self._by_symbol[old_intent.order.symbol_id].discard(intent.intent_id)
self._by_symbol[intent.order.symbol_id].add(intent.intent_id)
if old_intent.group_id != intent.group_id:
if old_intent.group_id:
self._by_group[old_intent.group_id].discard(intent.intent_id)
if intent.group_id:
self._by_group[intent.group_id].add(intent.intent_id)
self._intents[intent.intent_id] = intent
async def delete_intent(self, intent_id: str) -> None:
if intent_id not in self._intents:
raise KeyError(f"Intent {intent_id} not found")
intent = self._intents[intent_id]
self._by_symbol[intent.order.symbol_id].discard(intent_id)
if intent.group_id:
self._by_group[intent.group_id].discard(intent_id)
del self._intents[intent_id]
async def list_intents(
self,
symbol_id: str | None = None,
group_id: str | None = None,
) -> list[OrderIntent]:
if symbol_id and group_id:
# Intersection of both filters
symbol_ids = self._by_symbol.get(symbol_id, set())
group_ids = self._by_group.get(group_id, set())
intent_ids = symbol_ids & group_ids
elif symbol_id:
intent_ids = self._by_symbol.get(symbol_id, set())
elif group_id:
intent_ids = self._by_group.get(group_id, set())
else:
intent_ids = self._intents.keys()
return [self._intents[iid] for iid in intent_ids]
async def get_intents_by_group(self, group_id: str) -> list[OrderIntent]:
intent_ids = self._by_group.get(group_id, set())
return [self._intents[iid] for iid in intent_ids]
class InMemoryActualStateStore(ActualStateStore):
"""Simple in-memory implementation of ActualStateStore"""
def __init__(self):
self._states: dict[str, OrderState] = {}
self._by_exchange_id: dict[str, str] = {} # exchange_order_id -> intent_id
self._by_symbol: dict[str, set[str]] = defaultdict(set)
async def create_order_state(self, state: OrderState) -> None:
if state.intent_id in self._states:
raise ValueError(f"Order state for intent {state.intent_id} already exists")
self._states[state.intent_id] = state
self._by_exchange_id[state.exchange_order_id] = state.intent_id
self._by_symbol[state.status.order.symbol_id].add(state.intent_id)
async def get_order_state(self, intent_id: str) -> OrderState:
if intent_id not in self._states:
raise KeyError(f"Order state for intent {intent_id} not found")
return self._states[intent_id]
async def get_order_state_by_exchange_id(self, exchange_order_id: str) -> OrderState:
if exchange_order_id not in self._by_exchange_id:
raise KeyError(f"Order state for exchange order {exchange_order_id} not found")
intent_id = self._by_exchange_id[exchange_order_id]
return self._states[intent_id]
async def update_order_state(self, state: OrderState) -> None:
if state.intent_id not in self._states:
raise KeyError(f"Order state for intent {state.intent_id} not found")
old_state = self._states[state.intent_id]
# Update exchange_id index if it changed
if old_state.exchange_order_id != state.exchange_order_id:
del self._by_exchange_id[old_state.exchange_order_id]
self._by_exchange_id[state.exchange_order_id] = state.intent_id
# Update symbol index if it changed
old_symbol = old_state.status.order.symbol_id
new_symbol = state.status.order.symbol_id
if old_symbol != new_symbol:
self._by_symbol[old_symbol].discard(state.intent_id)
self._by_symbol[new_symbol].add(state.intent_id)
self._states[state.intent_id] = state
async def delete_order_state(self, intent_id: str) -> None:
if intent_id not in self._states:
raise KeyError(f"Order state for intent {intent_id} not found")
state = self._states[intent_id]
del self._by_exchange_id[state.exchange_order_id]
self._by_symbol[state.status.order.symbol_id].discard(intent_id)
del self._states[intent_id]
async def list_order_states(
self,
symbol_id: str | None = None,
reconciliation_status: ReconciliationStatus | None = None,
) -> list[OrderState]:
if symbol_id:
intent_ids = self._by_symbol.get(symbol_id, set())
states = [self._states[iid] for iid in intent_ids]
else:
states = list(self._states.values())
if reconciliation_status:
states = [s for s in states if s.reconciliation_status == reconciliation_status]
return states
async def get_stale_orders(self, max_age_seconds: int) -> list[OrderState]:
import time
current_time = int(time.time())
threshold = current_time - max_age_seconds
return [
state
for state in self._states.values()
if state.last_sync_at < threshold
]
# ---------------------------------------------------------------------------
# Reconciliation Engine (framework only, no implementation)
# ---------------------------------------------------------------------------
class ReconciliationEngine:
"""
Reconciliation engine that continuously works to make actual state match intent.
This is the heart of the "Kubernetes for orders" concept. It:
1. Compares desired state (intents) with actual state (exchange orders)
2. Computes necessary actions (place, modify, cancel)
3. Executes those actions via the exchange API
4. Handles retries, errors, and edge cases
This is a framework class - concrete implementations will be exchange-specific.
"""
def __init__(
self,
intent_store: IntentStateStore,
actual_store: ActualStateStore,
):
"""
Initialize the reconciliation engine.
Args:
intent_store: Store for desired state
actual_store: Store for actual state
"""
self.intent_store = intent_store
self.actual_store = actual_store
self._running = False
async def start(self) -> None:
"""Start the reconciliation loop"""
self._running = True
# Implementation would start async reconciliation loop here
pass
async def stop(self) -> None:
"""Stop the reconciliation loop"""
self._running = False
# Implementation would stop reconciliation loop here
pass
async def reconcile_intent(self, intent_id: str) -> None:
"""
Reconcile a specific intent.
Compares the intent with actual state and takes necessary actions.
Args:
intent_id: Intent to reconcile
"""
# Framework only - concrete implementation needed
pass
async def reconcile_all(self) -> None:
"""
Reconcile all intents.
Full reconciliation pass over all orders.
"""
# Framework only - concrete implementation needed
pass
def get_metrics(self) -> dict[str, Any]:
"""
Get reconciliation metrics.
Returns:
Metrics about reconciliation performance, errors, etc.
"""
return {
"running": self._running,
"reconciliation_lag_ms": 0, # Framework only
"pending_reconciliations": 0, # Framework only
"error_count": 0, # Framework only
"retry_count": 0, # Framework only
}