# Kubernetes Deployment

This directory contains Kubernetes manifests using [Kustomize](https://kustomize.io/) for managing dev and production environments.

## Structure

```
deploy/k8s/
├── base/                    # Base manifests (shared)
│   ├── backend.yaml
│   ├── web.yaml
│   ├── ingress.yaml
│   ├── init.yaml
│   └── kustomization.yaml
├── dev/                     # Dev overlay (minikube)
│   ├── infrastructure.yaml  # Kafka, Postgres, MinIO, Flink, Relay, Ingestor
│   ├── ingress-dev.yaml     # Dev ingress (dexorder.local)
│   ├── patches.yaml         # Dev-specific patches
│   ├── kustomization.yaml
│   └── secrets/
│       ├── *.yaml           # Actual secrets (gitignored)
│       └── *.yaml.example   # Templates
├── prod/                    # Production overlay
│   ├── patches.yaml         # Prod patches (replicas, resources, gVisor)
│   ├── kustomization.yaml
│   └── secrets/
│       ├── *.yaml           # Actual secrets (gitignored)
│       └── *.yaml.example   # Templates
└── configmaps/              # Shared ConfigMaps
    ├── relay-config.yaml
    ├── ingestor-config.yaml
    └── flink-config.yaml
```

## Dev Environment (Minikube)

### Prerequisites

- [minikube](https://minikube.sigs.k8s.io/docs/start/)
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
- Docker

### Quick Start

```bash
# Start everything
bin/dev start

# Access the application
# Web UI: http://dexorder.local/
# Gateway: http://dexorder.local/api

# In another terminal, start tunnel for ingress
bin/dev tunnel
```

### Managing Dev Environment

```bash
# Rebuild images after code changes
bin/dev rebuild

# Redeploy services
bin/dev deploy

# Full restart (rebuild + redeploy)
bin/dev restart

# View status
bin/dev status

# View logs
bin/dev logs relay
bin/dev logs ingestor
bin/dev logs flink-jobmanager

# Open shell in pod
bin/dev shell relay

# Clean everything
bin/dev clean

# Stop minikube
bin/dev stop
```

### Setting Up Secrets (Dev)

```bash
# Copy example secrets
cd deploy/k8s/dev/secrets/
cp ai-secrets.yaml.example ai-secrets.yaml
cp postgres-secret.yaml.example postgres-secret.yaml
cp minio-secret.yaml.example minio-secret.yaml
cp ingestor-secrets.yaml.example ingestor-secrets.yaml

# Edit with actual values
vim ai-secrets.yaml  # Add your Anthropic API key

# Apply to cluster
bin/secret-update dev

# Or update a specific secret
bin/secret-update dev ai-secrets
```

### Updating Configs (Dev)

```bash
# Edit config files
vim deploy/configmaps/relay-config.yaml

# Apply changes
bin/config-update dev

# Or update specific config
bin/config-update dev relay-config
```

### Dev vs Docker Compose

The minikube dev environment mirrors production more closely than docker-compose:

| Feature | docker-compose | minikube |
|---------|---------------|----------|
| Environment parity | ❌ Different from prod | ✅ Same as prod |
| Secrets management | `.env` files | K8s Secrets |
| Configuration | Volume mounts | ConfigMaps |
| Service discovery | DNS by service name | K8s Services |
| Ingress/routing | Port mapping | nginx-ingress |
| Resource limits | Limited support | Full K8s resources |
| Init containers | No | Yes |
| Readiness probes | No | Yes |

## Production Environment

### Prerequisites

- Access to production Kubernetes cluster
- `kubectl` configured with production context
- Production secrets prepared

### Setting Up Secrets (Prod)

```bash
# Copy example secrets
cd deploy/k8s/prod/secrets/
cp ai-secrets.yaml.example ai-secrets.yaml
cp postgres-secret.yaml.example postgres-secret.yaml
# ... etc

# Edit with production values
vim ai-secrets.yaml

# Apply to cluster (will prompt for confirmation)
bin/secret-update prod

# Or update specific secret
bin/secret-update prod ai-secrets
```

### Updating Configs (Prod)

```bash
# Edit production configs if needed
vim deploy/configmaps/relay-config.yaml

# Apply changes (will prompt for confirmation)
bin/config-update prod
```

### Deploying to Production

```bash
# Verify kubectl context
kubectl config current-context

# Apply manifests
kubectl apply -k deploy/k8s/prod/

# Check rollout status
kubectl rollout status statefulset/ai-backend
kubectl rollout status deployment/ai-web

# View status
kubectl get pods,svc,ingress
```

## Kustomize Overlays

### Dev Overlay

- **imagePullPolicy: Never** - Uses locally built images
- **Infrastructure services** - Kafka, Postgres, MinIO, Flink, Relay, Ingestor
- **Local ingress** - `dexorder.local` (requires `/etc/hosts` entry)
- **No gVisor** - RuntimeClass removed (not available in minikube)
- **Single replicas** - Minimal resource usage

### Prod Overlay

- **imagePullPolicy: Always** - Pulls from registry
- **Multiple replicas** - HA configuration
- **Resource limits** - CPU/memory constraints
- **gVisor** - Security sandbox via RuntimeClass
- **Production ingress** - `dexorder.ai` with TLS

## Infrastructure Services (Dev Only)

These services are included in the dev environment but are expected to be managed separately in production:

- **Kafka** - KRaft mode (no Zookeeper), single broker
- **PostgreSQL** - Iceberg catalog metadata
- **MinIO** - S3-compatible object storage
- **Iceberg REST Catalog** - Table metadata
- **Flink** - JobManager + TaskManager
- **Relay** - ZMQ message router
- **Ingestor** - CCXT data fetcher

In production, you would typically use:
- Managed Kafka (Confluent Cloud, MSK, etc.)
- Managed PostgreSQL (RDS, Cloud SQL, etc.)
- Object storage (S3, GCS, Azure Blob)
- Flink Kubernetes Operator or managed Flink

## Troubleshooting

### Minikube not starting

```bash
minikube delete
minikube start --cpus=6 --memory=12g --driver=docker
```

### Images not found

Make sure you're using minikube's docker daemon:

```bash
eval $(minikube docker-env)
bin/dev rebuild
```

### Ingress not working

Start minikube tunnel in another terminal:

```bash
bin/dev tunnel
```

### Secrets not found

Create secrets from examples:

```bash
cd deploy/k8s/dev/secrets/
cp *.example *.yaml
vim ai-secrets.yaml  # Edit with actual values
bin/secret-update dev
```

### Pods not starting

Check events and logs:

```bash
kubectl get events --sort-by=.metadata.creationTimestamp
kubectl describe pod <pod-name>
kubectl logs <pod-name>
```

## CI/CD Integration

For automated deployments, you can use:

```bash
# Build and push images
docker build -t registry.example.com/dexorder/ai-web:$TAG .
docker push registry.example.com/dexorder/ai-web:$TAG

# Update kustomization with new tag
cd deploy/k8s/prod
kustomize edit set image dexorder/ai-web=registry.example.com/dexorder/ai-web:$TAG

# Deploy
kubectl apply -k deploy/k8s/prod/
```