ai/flink/README.md

Flink Deployment for K8s Cluster

Install Flink Kubernetes Operator

# Add the Flink Helm repository
helm repo add flink-operator-repo https://downloads.apache.org/flink/flink-kubernetes-operator-1.9.0/
helm repo update

# Install the operator
helm install flink-kubernetes-operator flink-operator-repo/flink-kubernetes-operator \
  -f values.yaml \
  --namespace flink --create-namespace

# Wait for operator to be ready
kubectl wait --for=condition=ready pod -l app.kubernetes.io/name=flink-kubernetes-operator -n flink --timeout=300s

Create Service Account

kubectl create serviceaccount flink -n default
kubectl create clusterrolebinding flink-role-binding-default \
  --clusterrole=edit \
  --serviceaccount=default:flink

Deploy Flink Cluster

# Apply the Flink cluster manifest
kubectl apply -f flink-cluster.yaml

# Check cluster status
kubectl get flinkdeployment -n default

# Check pods
kubectl get pods -n default | grep flink

Access Flink Web UI

# Port forward to access the UI locally
kubectl port-forward svc/trading-flink-rest 8081:8081 -n default

# Open browser to http://localhost:8081

Prometheus Metrics

Flink exposes metrics on port 9249. Prometheus will automatically discover and scrape these metrics via pod annotations:

• prometheus.io/scrape: "true"
• prometheus.io/port: "9249"
• prometheus.io/path: "/metrics"

To verify metrics are being exported:

kubectl exec -it <flink-jobmanager-pod> -n default -- curl localhost:9249/metrics

Submit a Job

# Example: Submit a jar file
kubectl exec -it <jobmanager-pod> -- flink run /path/to/job.jar

Uninstall

# Delete Flink cluster
kubectl delete flinkdeployment trading-flink -n default

# Delete operator
helm uninstall flink-kubernetes-operator -n flink