backend redesign

backend.old/src/trigger/PRIORITIES.md

@@ -0,0 +1,216 @@
+# Priority System
+
+Simple tuple-based priorities for deterministic execution ordering.
+
+## Basic Concept
+
+Priorities are just **Python tuples**. Python compares tuples element-by-element, left-to-right:
+
+```python
+(0, 1000, 5) < (0, 1001, 3)  # True: 0==0, but 1000 < 1001
+(0, 1000, 5) < (1, 500, 2)   # True: 0 < 1
+(0, 1000) < (0, 1000, 5)     # True: shorter wins if equal so far
+```
+
+**Lower values = higher priority** (processed first).
+
+## Priority Categories
+
+```python
+class Priority(IntEnum):
+    DATA_SOURCE = 0        # Market data, real-time feeds
+    TIMER = 1              # Scheduled tasks, cron jobs
+    USER_AGENT = 2         # User-agent interactions (chat)
+    USER_DATA_REQUEST = 3  # User data requests (charts)
+    SYSTEM = 4             # Background tasks, cleanup
+    LOW = 5                # Retries after conflicts
+```
+
+## Usage Examples
+
+### Simple Priority
+
+```python
+# Just use the Priority enum
+trigger = MyTrigger("task", priority=Priority.SYSTEM)
+await queue.enqueue(trigger)
+
+# Results in tuple: (4, queue_seq)
+```
+
+### Compound Priority (Tuple)
+
+```python
+# DataSource: sort by event time (older bars first)
+trigger = DataUpdateTrigger(
+    source_name="binance",
+    symbol="BTC/USDT",
+    resolution="1m",
+    bar_data={"time": 1678896000, "open": 50000, ...}
+)
+await queue.enqueue(trigger)
+
+# Results in tuple: (0, 1678896000, queue_seq)
+#                    ^  ^            ^
+#                    |  |            Queue insertion order (FIFO)
+#                    |  Event time (candle end time)
+#                    DATA_SOURCE priority
+```
+
+### Manual Override
+
+```python
+# Override at enqueue time
+await queue.enqueue(
+    trigger,
+    priority_override=(Priority.DATA_SOURCE, custom_time, custom_sort)
+)
+
+# Queue appends queue_seq: (0, custom_time, custom_sort, queue_seq)
+```
+
+## Common Patterns
+
+### Market Data (Process Chronologically)
+
+```python
+# Bar from 10:00 → (0, 10:00_timestamp, queue_seq)
+# Bar from 10:05 → (0, 10:05_timestamp, queue_seq)
+#
+# 10:00 bar processes first (earlier event_time)
+
+DataUpdateTrigger(
+    ...,
+    bar_data={"time": event_timestamp, ...}
+)
+```
+
+### User Messages (FIFO Order)
+
+```python
+# Message #1 → (2, msg1_timestamp, queue_seq)
+# Message #2 → (2, msg2_timestamp, queue_seq)
+#
+# Message #1 processes first (earlier timestamp)
+
+AgentTriggerHandler(
+    session_id="user1",
+    message_content="...",
+    message_timestamp=unix_timestamp  # Optional, defaults to now
+)
+```
+
+### Scheduled Tasks (By Schedule Time)
+
+```python
+# Job scheduled for 9 AM → (1, 9am_timestamp, queue_seq)
+# Job scheduled for 2 PM → (1, 2pm_timestamp, queue_seq)
+#
+# 9 AM job processes first
+
+CronTrigger(
+    name="morning_sync",
+    inner_trigger=...,
+    scheduled_time=scheduled_timestamp
+)
+```
+
+## Execution Order Example
+
+```
+Queue contains:
+1. DataSource (BTC @ 10:00) → (0, 10:00, 1)
+2. DataSource (BTC @ 10:05) → (0, 10:05, 2)
+3. Timer (scheduled 9 AM)   → (1, 09:00, 3)
+4. User message #1          → (2, 14:30, 4)
+5. User message #2          → (2, 14:35, 5)
+
+Dequeue order:
+1. DataSource (BTC @ 10:00)  ← 0 < all others
+2. DataSource (BTC @ 10:05)  ← 0 < all others, 10:05 > 10:00
+3. Timer (scheduled 9 AM)    ← 1 < remaining
+4. User message #1           ← 2 < remaining, 14:30 < 14:35
+5. User message #2           ← last
+```
+
+## Short Tuple Wins
+
+If tuples are equal up to the length of the shorter one, **shorter tuple has higher priority**:
+
+```python
+(0, 1000) < (0, 1000, 5)      # True: shorter wins
+(0,) < (0, 1000)              # True: shorter wins
+(Priority.DATA_SOURCE,) < (Priority.DATA_SOURCE, 1000)  # True
+```
+
+This is Python's default tuple comparison behavior. In practice, we always append `queue_seq`, so this rarely matters (all tuples end up same length).
+
+## Integration with Triggers
+
+### Trigger Sets Its Own Priority
+
+```python
+class MyTrigger(Trigger):
+    def __init__(self, event_time):
+        super().__init__(
+            name="my_trigger",
+            priority=Priority.DATA_SOURCE,
+            priority_tuple=(Priority.DATA_SOURCE.value, event_time)
+        )
+```
+
+Queue appends `queue_seq` automatically:
+```python
+# Trigger's tuple: (0, event_time)
+# After enqueue:   (0, event_time, queue_seq)
+```
+
+### Override at Enqueue
+
+```python
+# Ignore trigger's priority, use override
+await queue.enqueue(
+    trigger,
+    priority_override=(Priority.TIMER, scheduled_time)
+)
+```
+
+## Why Tuples?
+
+✅ **Simple**: No custom classes, just native Python tuples
+✅ **Flexible**: Add as many sort keys as needed
+✅ **Efficient**: Python's tuple comparison is highly optimized
+✅ **Readable**: `(0, 1000, 5)` is obvious what it means
+✅ **Debuggable**: Can print and inspect easily
+
+Example:
+```python
+# Old: CompoundPriority(primary=0, secondary=1000, tertiary=5)
+# New: (0, 1000, 5)
+
+# Same semantics, much simpler!
+```
+
+## Advanced: Custom Sorting
+
+Want to sort by multiple factors? Just add more elements:
+
+```python
+# Sort by: priority → symbol → event_time → queue_seq
+priority_tuple = (
+    Priority.DATA_SOURCE.value,
+    symbol_id,  # e.g., hash("BTC/USDT")
+    event_time,
+    # queue_seq appended by queue
+)
+```
+
+## Summary
+
+- **Priorities are tuples**: `(primary, secondary, ..., queue_seq)`
+- **Lower = higher priority**: Processed first
+- **Element-by-element comparison**: Left-to-right
+- **Shorter tuple wins**: If equal up to shorter length
+- **Queue appends queue_seq**: Always last element (FIFO within same priority)
+
+That's it! No complex classes, just tuples. 🎯