major agent refactoring: wiki knowledge base, no RAG, no Qdrant, no Ollama
This commit is contained in:
@@ -1,71 +0,0 @@
|
||||
# Agent System Architecture
|
||||
|
||||
The Dexorder AI platform uses a sophisticated agent harness that orchestrates between user interactions, LLM models, and user-specific tools.
|
||||
|
||||
## Core Components
|
||||
|
||||
### Gateway
|
||||
Multi-channel gateway supporting:
|
||||
- WebSocket connections for web/mobile
|
||||
- Telegram integration
|
||||
- Real-time event streaming
|
||||
|
||||
### Agent Harness
|
||||
Stateless orchestrator that:
|
||||
1. Fetches context from user's MCP server
|
||||
2. Routes to appropriate LLM model based on license
|
||||
3. Calls LLM with embedded context
|
||||
4. Routes tool calls to user's MCP or platform tools
|
||||
5. Saves conversation history back to MCP
|
||||
|
||||
### Memory Architecture
|
||||
Three-tier storage system:
|
||||
- **Redis**: Hot state for active sessions and checkpoints
|
||||
- **Qdrant**: Vector search for RAG and semantic memory
|
||||
- **Iceberg**: Cold storage for durable conversations and analytics
|
||||
|
||||
### User Context
|
||||
Every interaction includes:
|
||||
- User ID and license information
|
||||
- Active channel (websocket, telegram, etc.)
|
||||
- Channel capabilities (markdown, images, buttons)
|
||||
- Conversation history
|
||||
- Relevant memories from RAG
|
||||
- Workspace state
|
||||
|
||||
## Skills vs Subagents
|
||||
|
||||
### Skills
|
||||
Self-contained capabilities for specific tasks:
|
||||
- Market analysis
|
||||
- Strategy validation
|
||||
- Indicator development
|
||||
- Defined in markdown + TypeScript
|
||||
- Use when task is well-defined and scoped
|
||||
|
||||
### Subagents
|
||||
Specialized agents with dedicated memory:
|
||||
- Code reviewer with review guidelines
|
||||
- Risk analyzer with risk models
|
||||
- Multi-file knowledge base
|
||||
- Custom system prompts
|
||||
- Use when domain expertise is needed
|
||||
|
||||
## Global vs User Memory
|
||||
|
||||
### Global Memory (user_id="0")
|
||||
Platform-wide knowledge available to all users:
|
||||
- Trading concepts and terminology
|
||||
- Platform capabilities
|
||||
- Indicator documentation
|
||||
- Strategy patterns
|
||||
- Best practices
|
||||
|
||||
### User Memory
|
||||
Personal context specific to each user:
|
||||
- Conversation history
|
||||
- Preferences and trading style
|
||||
- Custom indicators and strategies
|
||||
- Workspace state
|
||||
|
||||
All RAG queries automatically search both global and user-specific memories.
|
||||
@@ -1,88 +1,17 @@
|
||||
# Model Context Protocol (MCP) Integration
|
||||
# User Sandbox
|
||||
|
||||
Dexorder uses the Model Context Protocol for user-specific tool execution and state management.
|
||||
Each user has a dedicated sandbox environment that persists their data across sessions.
|
||||
|
||||
## Container Architecture
|
||||
## Persistent Storage
|
||||
|
||||
Each user has a dedicated Kubernetes pod running:
|
||||
- **Agent Container**: Python environment with conda packages
|
||||
- **Lifecycle Sidecar**: Manages container lifecycle and communication
|
||||
- **Persistent Storage**: User's git repository with indicators/strategies
|
||||
User scripts (indicators, strategies, research) are stored in a git repository inside the user's sandbox. They survive session disconnects and reconnections.
|
||||
|
||||
## Authentication Modes
|
||||
- Indicators are in the `indicator` category and can be listed with `PythonList(category="indicator")`
|
||||
- Strategies are in the `strategy` category and can be listed with `PythonList(category="strategy")`
|
||||
- Research scripts are in the `research` category and can be listed with `PythonList(category="research")`
|
||||
|
||||
Three MCP authentication modes:
|
||||
## Session Lifecycle
|
||||
|
||||
### 1. Public Mode (Free Tier)
|
||||
- No authentication required
|
||||
- Container creates anonymous session
|
||||
- Limited to read-only resources
|
||||
- Session expires after timeout
|
||||
|
||||
### 2. Gateway Auth Mode (Standard)
|
||||
- Gateway authenticates user
|
||||
- Passes verified user ID to container
|
||||
- Container trusts gateway's authentication
|
||||
- Full access to user's tools and data
|
||||
|
||||
### 3. Direct Auth Mode (Enterprise)
|
||||
- User authenticates directly with container
|
||||
- Gateway forwards encrypted credentials
|
||||
- Container validates credentials independently
|
||||
- Highest security for sensitive operations
|
||||
|
||||
## MCP Resources
|
||||
|
||||
The container exposes standard resources:
|
||||
|
||||
### context://user-profile
|
||||
User preferences and trading style
|
||||
|
||||
### context://conversation-summary
|
||||
Recent conversation context and history
|
||||
|
||||
### context://workspace-state
|
||||
Current chart, indicators, and analysis state
|
||||
|
||||
### context://system-prompt
|
||||
User's custom agent instructions
|
||||
|
||||
### indicators://list
|
||||
Available indicators with signatures
|
||||
|
||||
### strategies://list
|
||||
User's trading strategies
|
||||
|
||||
## Tool Execution Flow
|
||||
|
||||
1. User sends message to gateway
|
||||
2. Gateway queries user's MCP resources for context
|
||||
3. LLM generates response with tool calls
|
||||
4. Gateway routes tool calls:
|
||||
- Platform tools → handled by gateway
|
||||
- User tools → proxied to MCP container
|
||||
5. Tool results returned to LLM
|
||||
6. Final response sent to user
|
||||
7. Conversation saved to MCP container
|
||||
|
||||
## Container Lifecycle
|
||||
|
||||
### Startup
|
||||
1. Gateway receives user connection
|
||||
2. Checks if container exists
|
||||
3. Creates pod if needed (cold start ~5-10s)
|
||||
4. Waits for container ready
|
||||
5. Establishes MCP connection
|
||||
|
||||
### Active
|
||||
- Container stays alive during active session
|
||||
- Receives tool calls via MCP
|
||||
- Maintains workspace state
|
||||
- Saves files to persistent storage
|
||||
|
||||
### Shutdown
|
||||
- Free users: timeout after 15 minutes idle
|
||||
- Paid users: longer timeout based on license
|
||||
- Graceful shutdown saves state
|
||||
- Persistent storage retained
|
||||
- Fast restart on next connection
|
||||
- Sandbox starts automatically when the user connects
|
||||
- Cold start takes a few seconds if the sandbox was idle
|
||||
- All workspace state and scripts are preserved across reconnects
|
||||
|
||||
132
gateway/knowledge/platform/workspace.md
Normal file
132
gateway/knowledge/platform/workspace.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# Workspace
|
||||
|
||||
The Workspace is the user's current UI context — what they are looking at, what is selected, and what persistent state belongs to their session. It is a collection of named **stores** that are kept in sync between the web client, the gateway, and the user's sandbox container.
|
||||
|
||||
Use `WorkspaceRead(store_name)` to read any store and `WorkspacePatch(store_name, patch)` to update it. Patches use JSON Patch (RFC 6902) format.
|
||||
|
||||
---
|
||||
|
||||
## Stores
|
||||
|
||||
### `chartState` — Current chart view (persistent)
|
||||
|
||||
Tracks what the user is currently looking at on the TradingView chart.
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `symbol` | string | Active trading pair in `SYMBOL.EXCHANGE` format (e.g. `BTC/USDT.BINANCE`) |
|
||||
| `period` | number | OHLC bar period in seconds (e.g. `900` = 15 min, `3600` = 1 h) |
|
||||
| `start_time` | number \| null | Unix timestamp of left edge of visible range, or null for auto |
|
||||
| `end_time` | number \| null | Unix timestamp of right edge of visible range, or null for auto |
|
||||
| `selected_shapes` | string[] | IDs of currently selected drawing/annotation shapes |
|
||||
|
||||
When the user says "the current chart" or "what's selected", read `chartState` first.
|
||||
|
||||
---
|
||||
|
||||
### `indicators` — Active indicators on the chart (persistent)
|
||||
|
||||
A flat map of `indicator_id → IndicatorInstance`. Each entry represents one study currently plotted on the TradingView chart.
|
||||
|
||||
**`IndicatorInstance` fields:**
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `id` | string | Unique ID for this instance |
|
||||
| `pandas_ta_name` | string | Internal name used in strategy/indicator scripts (e.g. `RSI_14`, `custom_MyIndicator`) |
|
||||
| `instance_name` | string | Human-readable label shown on chart |
|
||||
| `parameters` | object | Key/value parameter map (e.g. `{ length: 14 }`) |
|
||||
| `tv_study_id` | string? | TradingView study ID (assigned by TV after the study is added) |
|
||||
| `tv_indicator_name` | string? | TradingView indicator name for built-in studies |
|
||||
| `tv_inputs` | object? | TradingView input overrides keyed by TV input name |
|
||||
| `visible` | boolean | Whether the study is visible on the chart |
|
||||
| `pane` | string | `"price"` to overlay on price pane, `"separate"` for its own panel |
|
||||
| `symbol` | string? | Override symbol if different from `chartState.symbol` |
|
||||
| `created_at` | number? | Unix timestamp when added |
|
||||
| `modified_at` | number? | Unix timestamp when last changed |
|
||||
| `custom_metadata` | object? | Present only for `custom_*` indicators; drives TradingView custom study construction (see below) |
|
||||
|
||||
**`custom_metadata` sub-fields** (for custom indicators only):
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `display_name` | string | Human-readable indicator title shown in TV |
|
||||
| `parameters` | object | Parameter schema: `{ name: { type, default, description, min?, max? } }` |
|
||||
| `input_series` | string[] | Input price series required (e.g. `["close"]`) |
|
||||
| `output_columns` | array | Each entry: `{ name, display_name?, description?, plot? }` where `plot` has `{ style, color?, linewidth?, visible? }` |
|
||||
| `pane` | `"price"` \| `"separate"` | Default pane placement |
|
||||
| `filled_areas` | array? | Shaded regions between two plots or hlines |
|
||||
| `bands` | array? | Horizontal reference lines (e.g. RSI 70/30) |
|
||||
|
||||
---
|
||||
|
||||
### `shapes` — Chart drawings and annotations (persistent)
|
||||
|
||||
```json
|
||||
{ "shapes": { "<shape_id>": Shape } }
|
||||
```
|
||||
|
||||
Each `Shape` has:
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `id` | string | Unique shape ID |
|
||||
| `type` | string | Drawing type (e.g. `"trend_line"`, `"horizontal_line"`, `"rectangle"`) |
|
||||
| `points` | array | Control points: `{ time: unix_ts, price: number, channel?: string }` |
|
||||
| `color` | string? | Hex color |
|
||||
| `line_width` | number? | Line thickness |
|
||||
| `line_style` | string? | `"solid"`, `"dotted"`, `"dashed"` |
|
||||
| `properties` | object? | Additional type-specific properties |
|
||||
| `symbol` | string? | Symbol the shape belongs to |
|
||||
| `created_at` | number? | Unix timestamp |
|
||||
| `modified_at` | number? | Unix timestamp |
|
||||
|
||||
---
|
||||
|
||||
### `indicator_types` — Custom indicator registry (persistent)
|
||||
|
||||
```json
|
||||
{ "types": { "<script_name>": CustomIndicatorMetadata } }
|
||||
```
|
||||
|
||||
Maps custom indicator script names to their `CustomIndicatorMetadata` (same structure as `custom_metadata` above). Populated when a custom indicator is created or updated by the indicator agent. The web client uses this to register custom TradingView studies.
|
||||
|
||||
---
|
||||
|
||||
### `strategy_types` — Strategy registry (persistent)
|
||||
|
||||
```json
|
||||
{ "types": { "<script_name>": StrategyMetadata } }
|
||||
```
|
||||
|
||||
Maps strategy script names to their metadata. Used by the web client to know which strategies are available.
|
||||
|
||||
---
|
||||
|
||||
### `research_types` — Research script registry (persistent)
|
||||
|
||||
```json
|
||||
{ "types": { "<script_name>": ResearchMetadata } }
|
||||
```
|
||||
|
||||
Maps research script names to their metadata.
|
||||
|
||||
---
|
||||
|
||||
### `channelState` — Connected channels (transient, gateway-only)
|
||||
|
||||
Tracks which communication channels (WebSocket, Telegram, etc.) are connected to the current session. **Not synced to web clients.**
|
||||
|
||||
```json
|
||||
{ "connected": { "<channel_id>": { type, connectedAt, capabilities } } }
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sync Protocol
|
||||
|
||||
Stores are kept in sync using JSON Patch (RFC 6902) messages:
|
||||
- **snapshot** — full state dump sent on connect or after missed patches
|
||||
- **patch** — incremental change with a monotonic sequence number
|
||||
|
||||
Stores marked `persistent` are saved to the user's container at `/data/workspace/{store_name}.json` and survive session reconnects.
|
||||
Reference in New Issue
Block a user