110 lines
2.5 KiB
Markdown
110 lines
2.5 KiB
Markdown
# Test Clients
|
|
|
|
Test clients for the DexOrder trading system.
|
|
|
|
## History Client
|
|
|
|
Tests the historical OHLC data request/response pattern between clients, Flink, and ingestors.
|
|
|
|
### Quick Start
|
|
|
|
```bash
|
|
cd history_client
|
|
./run-test.sh
|
|
```
|
|
|
|
This will:
|
|
1. Start all required services (Kafka, Flink, Ingestor)
|
|
2. Wait for services to initialize
|
|
3. Run the test client to query historical data
|
|
4. Display the results
|
|
|
|
### What it tests
|
|
|
|
- **ZMQ Communication**: Client → Flink REQ/REP pattern (port 5559)
|
|
- **Work Distribution**: Flink → Ingestor PUB/SUB with exchange prefix filtering (port 5555)
|
|
- **Response Channel**: Ingestor → Flink DEALER/ROUTER pattern (port 5556)
|
|
- **Data Flow**: Request → Ingestor fetches data → Response back to Flink → Response to client
|
|
|
|
### Expected Flow
|
|
|
|
1. **Client** sends OHLCRequest to Flink (REQ/REP)
|
|
- Ticker: `BINANCE:BTC/USDT`
|
|
- Period: 3600s (1 hour)
|
|
- Range: Jan 1-7, 2026
|
|
|
|
2. **Flink** publishes DataRequest to ingestor work queue (PUB/SUB)
|
|
- Topic prefix: `BINANCE:`
|
|
- Any ingestor subscribed to BINANCE can respond
|
|
|
|
3. **Ingestor** receives request, fetches data, sends back response
|
|
- Uses CCXT to fetch from exchange
|
|
- Sends DataResponse via DEALER socket
|
|
- Also writes to Kafka for Flink processing
|
|
|
|
4. **Flink** receives response, sends back to client
|
|
- Matches response by request_id
|
|
- Returns data or error to waiting client
|
|
|
|
### Manual Testing
|
|
|
|
Run the Python client directly:
|
|
|
|
```bash
|
|
cd history_client
|
|
pip install pyzmq
|
|
python client.py
|
|
```
|
|
|
|
Edit `client.py` to customize:
|
|
- Flink hostname and port
|
|
- Ticker symbol
|
|
- Time range
|
|
- Period (e.g., 3600 for 1h, 86400 for 1d)
|
|
|
|
## Docker Compose Profiles
|
|
|
|
The test client uses a Docker Compose profile to avoid starting automatically:
|
|
|
|
```bash
|
|
# Start all services
|
|
docker-compose up -d
|
|
|
|
# Run test client
|
|
docker-compose --profile test up history-test-client
|
|
|
|
# Or start everything including test
|
|
docker-compose --profile test up
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
### Service logs
|
|
|
|
```bash
|
|
docker-compose logs -f ingestor
|
|
docker-compose logs -f flink-jobmanager
|
|
```
|
|
|
|
### Check ZMQ ports
|
|
|
|
```bash
|
|
# From inside Flink container
|
|
netstat -tlnp | grep 555
|
|
```
|
|
|
|
### Verify ingestor subscriptions
|
|
|
|
Check ingestor logs for:
|
|
```
|
|
Subscribed to exchange prefix: BINANCE:
|
|
Subscribed to exchange prefix: COINBASE:
|
|
```
|
|
|
|
### Test without Docker
|
|
|
|
1. Start Kafka: `docker-compose up -d kafka`
|
|
2. Build and run Flink app locally
|
|
3. Run ingestor: `cd ingestor && npm start`
|
|
4. Run test: `cd test/history_client && python client.py`
|