sandbox connected and streaming
This commit is contained in:
@@ -109,7 +109,7 @@ DexOrder is an AI-powered trading platform that combines real-time market data p
|
||||
|
||||
### 2. User Containers
|
||||
|
||||
**Location:** `client-py/`
|
||||
**Location:** `sandbox/`
|
||||
**Language:** Python
|
||||
**Purpose:** Per-user isolated workspace and data storage
|
||||
|
||||
@@ -415,12 +415,12 @@ User authenticates → Gateway checks if deployment exists
|
||||
### RBAC
|
||||
|
||||
**Gateway ServiceAccount:**
|
||||
- Create deployments/services/PVCs in `dexorder-agents` namespace
|
||||
- Create deployments/services/PVCs in `dexorder-sandboxes` namespace
|
||||
- Read pod status and logs
|
||||
- Cannot delete, exec, or access secrets
|
||||
|
||||
**Lifecycle Sidecar ServiceAccount:**
|
||||
- Delete deployments in `dexorder-agents` namespace
|
||||
- Delete deployments in `dexorder-sandboxes` namespace
|
||||
- Delete PVCs (conditional on user type)
|
||||
- Cannot access other resources
|
||||
|
||||
@@ -428,7 +428,7 @@ User authenticates → Gateway checks if deployment exists
|
||||
|
||||
### Admission Control
|
||||
|
||||
All pods in `dexorder-agents` namespace must:
|
||||
All pods in `dexorder-sandboxes` namespace must:
|
||||
- Use approved images only (allowlist)
|
||||
- Run as non-root
|
||||
- Drop all capabilities
|
||||
@@ -544,13 +544,13 @@ kubectl apply -k deploy/k8s/prod
|
||||
|
||||
# Push images to registry
|
||||
docker push ghcr.io/dexorder/gateway:latest
|
||||
docker push ghcr.io/dexorder/agent:latest
|
||||
docker push ghcr.io/dexorder/sandbox:latest
|
||||
docker push ghcr.io/dexorder/lifecycle-sidecar:latest
|
||||
```
|
||||
|
||||
**Namespaces:**
|
||||
- `dexorder-system` - Platform services (gateway, infrastructure)
|
||||
- `dexorder-agents` - User containers (isolated)
|
||||
- `dexorder-sandboxes` - User containers (isolated)
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ User agent containers self-manage their lifecycle to optimize resource usage. Co
|
||||
|
||||
### 1. Lifecycle Manager (Python)
|
||||
|
||||
**Location**: `client-py/dexorder/lifecycle_manager.py`
|
||||
**Location**: `sandbox/dexorder/lifecycle_manager.py`
|
||||
|
||||
Runs inside the agent container and tracks:
|
||||
- **Activity**: MCP tool/resource/prompt calls reset the idle timer
|
||||
@@ -85,7 +85,7 @@ Runs alongside the agent container with shared PID namespace. Monitors the main
|
||||
- `USER_TYPE`: License tier (`anonymous`, `free`, `paid`, `enterprise`)
|
||||
- `MAIN_CONTAINER_PID`: PID of main container (default: 1)
|
||||
|
||||
**RBAC**: Has permission to delete deployments and PVCs **only in dexorder-agents namespace**. Cannot delete other deployments due to:
|
||||
**RBAC**: Has permission to delete deployments and PVCs **only in dexorder-sandboxes namespace**. Cannot delete other deployments due to:
|
||||
1. Only knows its own deployment name (from env)
|
||||
2. RBAC scoped to namespace
|
||||
3. No cross-pod communication
|
||||
@@ -164,12 +164,12 @@ Configured via `USER_TYPE` env var in deployment.
|
||||
**Lifecycle Sidecar**:
|
||||
- Can delete its own deployment only
|
||||
- Cannot delete other deployments
|
||||
- Scoped to dexorder-agents namespace
|
||||
- Scoped to dexorder-sandboxes namespace
|
||||
- No exec, no secrets access
|
||||
|
||||
### Admission Control
|
||||
|
||||
All deployments in `dexorder-agents` namespace are subject to:
|
||||
All deployments in `dexorder-sandboxes` namespace are subject to:
|
||||
- Image allowlist (only approved images)
|
||||
- Security context enforcement (non-root, drop caps, read-only rootfs)
|
||||
- Resource limits required
|
||||
@@ -198,7 +198,7 @@ kubectl apply -k deploy/k8s/dev # or prod
|
||||
```
|
||||
|
||||
This creates:
|
||||
- Namespaces (`dexorder-system`, `dexorder-agents`)
|
||||
- Namespaces (`dexorder-system`, `dexorder-sandboxes`)
|
||||
- RBAC (gateway, lifecycle sidecar)
|
||||
- Admission policies
|
||||
- Network policies
|
||||
@@ -257,7 +257,7 @@ cd lifecycle-sidecar
|
||||
go build -o lifecycle-sidecar main.go
|
||||
|
||||
# Run (requires k8s config)
|
||||
export NAMESPACE=dexorder-agents
|
||||
export NAMESPACE=dexorder-sandboxes
|
||||
export DEPLOYMENT_NAME=agent-test
|
||||
export USER_TYPE=free
|
||||
./lifecycle-sidecar
|
||||
@@ -277,7 +277,7 @@ export USER_TYPE=free
|
||||
|
||||
Check logs:
|
||||
```bash
|
||||
kubectl logs -n dexorder-agents agent-user-abc123 -c agent
|
||||
kubectl logs -n dexorder-sandboxes sandbox-user-abc123 -c agent
|
||||
```
|
||||
|
||||
Verify:
|
||||
@@ -289,19 +289,19 @@ Verify:
|
||||
|
||||
Check sidecar logs:
|
||||
```bash
|
||||
kubectl logs -n dexorder-agents agent-user-abc123 -c lifecycle-sidecar
|
||||
kubectl logs -n dexorder-sandboxes sandbox-user-abc123 -c lifecycle-sidecar
|
||||
```
|
||||
|
||||
Verify:
|
||||
- Exit code file exists: `/var/run/agent/exit_code` contains `42`
|
||||
- RBAC permissions: `kubectl auth can-i delete deployments --as=system:serviceaccount:dexorder-agents:agent-lifecycle -n dexorder-agents`
|
||||
- RBAC permissions: `kubectl auth can-i delete deployments --as=system:serviceaccount:dexorder-sandboxes:sandbox-lifecycle -n dexorder-sandboxes`
|
||||
- Deployment name matches: Check `DEPLOYMENT_NAME` env var
|
||||
|
||||
### Gateway can't create deployments
|
||||
|
||||
Check gateway logs and verify:
|
||||
- ServiceAccount exists: `kubectl get sa gateway -n dexorder-system`
|
||||
- RoleBinding exists: `kubectl get rolebinding gateway-agent-creator -n dexorder-agents`
|
||||
- RoleBinding exists: `kubectl get rolebinding gateway-sandbox-creator -n dexorder-sandboxes`
|
||||
- Admission policy allows image: Check image name matches allowlist in `admission-policy.yaml`
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
@@ -60,10 +60,10 @@ All resources follow a consistent naming pattern based on `userId`:
|
||||
```typescript
|
||||
userId: "user-abc123"
|
||||
↓
|
||||
deploymentName: "agent-user-abc123"
|
||||
serviceName: "agent-user-abc123"
|
||||
pvcName: "agent-user-abc123-data"
|
||||
mcpEndpoint: "http://agent-user-abc123.dexorder-agents.svc.cluster.local:3000"
|
||||
deploymentName: "sandbox-user-abc123"
|
||||
serviceName: "sandbox-user-abc123"
|
||||
pvcName: "sandbox-user-abc123-data"
|
||||
mcpEndpoint: "http://sandbox-user-abc123.dexorder-sandboxes.svc.cluster.local:3000"
|
||||
```
|
||||
|
||||
User IDs are sanitized to be Kubernetes-compliant (lowercase alphanumeric + hyphens).
|
||||
@@ -82,7 +82,7 @@ Templates use simple string replacement:
|
||||
- `{{deploymentName}}` - Computed deployment name
|
||||
- `{{serviceName}}` - Computed service name
|
||||
- `{{pvcName}}` - Computed PVC name
|
||||
- `{{agentImage}}` - Agent container image (from env)
|
||||
- `{{sandboxImage}}` - Agent container image (from env)
|
||||
- `{{sidecarImage}}` - Lifecycle sidecar image (from env)
|
||||
- `{{storageClass}}` - Kubernetes storage class (from env)
|
||||
|
||||
@@ -145,16 +145,16 @@ Environment variables:
|
||||
|
||||
```bash
|
||||
# Kubernetes
|
||||
KUBERNETES_NAMESPACE=dexorder-agents
|
||||
KUBERNETES_NAMESPACE=dexorder-sandboxes
|
||||
KUBERNETES_IN_CLUSTER=true # false for local dev
|
||||
KUBERNETES_CONTEXT=minikube # for local dev only
|
||||
|
||||
# Container images
|
||||
AGENT_IMAGE=ghcr.io/dexorder/agent:latest
|
||||
SANDBOX_IMAGE=ghcr.io/dexorder/sandbox:latest
|
||||
SIDECAR_IMAGE=ghcr.io/dexorder/lifecycle-sidecar:latest
|
||||
|
||||
# Storage
|
||||
AGENT_STORAGE_CLASS=standard
|
||||
SANDBOX_STORAGE_CLASS=standard
|
||||
```
|
||||
|
||||
## Security
|
||||
@@ -162,9 +162,9 @@ AGENT_STORAGE_CLASS=standard
|
||||
The gateway uses a restricted ServiceAccount with RBAC:
|
||||
|
||||
**Can do:**
|
||||
- ✅ Create deployments in `dexorder-agents` namespace
|
||||
- ✅ Create services in `dexorder-agents` namespace
|
||||
- ✅ Create PVCs in `dexorder-agents` namespace
|
||||
- ✅ Create deployments in `dexorder-sandboxes` namespace
|
||||
- ✅ Create services in `dexorder-sandboxes` namespace
|
||||
- ✅ Create PVCs in `dexorder-sandboxes` namespace
|
||||
- ✅ Read pod status and logs (debugging)
|
||||
- ✅ Update deployments (future: resource scaling)
|
||||
|
||||
@@ -226,7 +226,7 @@ kubectl apply -k deploy/k8s/dev
|
||||
# .env
|
||||
KUBERNETES_IN_CLUSTER=false
|
||||
KUBERNETES_CONTEXT=minikube
|
||||
KUBERNETES_NAMESPACE=dexorder-agents
|
||||
KUBERNETES_NAMESPACE=dexorder-sandboxes
|
||||
```
|
||||
|
||||
4. Run gateway:
|
||||
@@ -242,9 +242,9 @@ wscat -c "ws://localhost:3000/ws/chat" -H "Authorization: Bearer your-jwt"
|
||||
|
||||
The gateway will create deployments in minikube. View with:
|
||||
```bash
|
||||
kubectl get deployments -n dexorder-agents
|
||||
kubectl get pods -n dexorder-agents
|
||||
kubectl logs -n dexorder-agents agent-user-abc123 -c agent
|
||||
kubectl get deployments -n dexorder-sandboxes
|
||||
kubectl get pods -n dexorder-sandboxes
|
||||
kubectl logs -n dexorder-sandboxes sandbox-user-abc123 -c agent
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
@@ -262,7 +262,7 @@ kubectl apply -k deploy/k8s/prod
|
||||
```
|
||||
|
||||
3. Gateway runs in `dexorder-system` namespace
|
||||
4. Creates agent containers in `dexorder-agents` namespace
|
||||
4. Creates agent containers in `dexorder-sandboxes` namespace
|
||||
5. Admission policies enforce image allowlist and security constraints
|
||||
|
||||
## Monitoring
|
||||
|
||||
@@ -55,7 +55,7 @@ Two ZMQ patterns handle different delivery requirements:
|
||||
### File Structure
|
||||
|
||||
```
|
||||
client-py/dexorder/
|
||||
sandbox/dexorder/
|
||||
├── events/
|
||||
│ ├── __init__.py
|
||||
│ ├── publisher.py # EventPublisher class
|
||||
@@ -66,7 +66,7 @@ client-py/dexorder/
|
||||
### Event Publisher Class
|
||||
|
||||
```python
|
||||
# client-py/dexorder/events/publisher.py
|
||||
# sandbox/dexorder/events/publisher.py
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
@@ -295,7 +295,7 @@ class EventPublisher:
|
||||
### Event Types
|
||||
|
||||
```python
|
||||
# client-py/dexorder/events/types.py
|
||||
# sandbox/dexorder/events/types.py
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from enum import IntEnum
|
||||
@@ -465,7 +465,7 @@ class EventAck:
|
||||
### Pending Event Persistence
|
||||
|
||||
```python
|
||||
# client-py/dexorder/events/pending_store.py
|
||||
# sandbox/dexorder/events/pending_store.py
|
||||
|
||||
import json
|
||||
import aiofiles
|
||||
@@ -1169,7 +1169,7 @@ apiVersion: networking.k8s.io/v1
|
||||
kind: NetworkPolicy
|
||||
metadata:
|
||||
name: agent-to-gateway-events
|
||||
namespace: dexorder-agents
|
||||
namespace: dexorder-sandboxes
|
||||
spec:
|
||||
podSelector:
|
||||
matchLabels:
|
||||
|
||||
Reference in New Issue
Block a user