redesign fully scaffolded and web login works

gateway/knowledge/README.md

@@ -0,0 +1,94 @@
+# Dexorder Knowledge Base
+
+This directory contains global knowledge documents that are automatically loaded into the RAG system as platform-wide knowledge (user_id="0").
+
+## Structure
+
+- **platform/**: Platform architecture and capabilities
+- **trading/**: Trading concepts and fundamentals
+- **indicators/**: Indicator development and usage
+- **strategies/**: Strategy development and patterns
+
+## Document Format
+
+Documents should be in Markdown format with:
+- Clear headings for chunking
+- Optional YAML frontmatter for tags
+- Code examples where relevant
+- Cross-references to other docs
+
+### Example with Frontmatter
+
+```markdown
+---
+tags: [trading, risk-management, position-sizing]
+---
+
+# Risk Management
+
+Content here...
+```
+
+## How It Works
+
+1. At gateway startup, the DocumentLoader scans this directory
+2. Each markdown file is chunked by headers (max ~1000 tokens per chunk)
+3. Chunks are embedded using the configured embedding service
+4. Embeddings are stored in Qdrant with user_id="0" (global namespace)
+5. Content hash tracking enables incremental updates
+
+## Updating Documents
+
+### During Development
+- Edit markdown files
+- Restart gateway or call reload endpoint: `POST /admin/reload-knowledge`
+
+### In Production
+- Update markdown files in git
+- Deploy new version
+- Gateway will detect changes and update vectors automatically
+
+## RAG Integration
+
+When users query the agent:
+1. Their query is embedded
+2. Qdrant searches both global (user_id="0") and user-specific vectors
+3. Relevant chunks from these docs are included in context
+4. LLM generates response with platform knowledge
+
+## Adding New Documents
+
+1. Create markdown file in appropriate subdirectory
+2. Use clear section headers (##, ###) for automatic chunking
+3. Include practical examples and code samples
+4. Add tags in frontmatter if using complex categorization
+5. Restart gateway or reload knowledge
+
+## Best Practices
+
+- **Keep chunks focused**: Each section should cover one topic
+- **Use examples**: Code samples and practical examples help
+- **Link concepts**: Reference other docs for deeper dives
+- **Update regularly**: Keep knowledge current with platform changes
+- **Test queries**: Verify RAG retrieves relevant chunks
+
+## Maintenance
+
+The DocumentLoader tracks:
+- Content hashes for change detection
+- Number of chunks per document
+- Last update timestamps
+
+Check logs for load statistics:
+```
+Knowledge documents loaded: { loaded: 5, updated: 2, skipped: 3 }
+```
+
+Monitor Qdrant collection stats:
+```
+GET /health
+{
+  "qdrantVectors": 1234,
+  "qdrantIndexed": 1234
+}
+```