backend redesign

doc.old/data.md

@@ -0,0 +1,21 @@
+# Data Sources and ETL
+
+We'll use Pandas and ArcticDB as our data foundation. This gives easy access to numpy and a wide spectrum of Python analysis libraries.
+
+ArcticDB
+[ArcticDB](https://arcticdb.com/)
+We'll use unstructured dynamic schemas for flexibility and ad-hoc adjustment by the assistant.
+
+## Canonical Formats
+* OHLCV
+  * columns
+  * open_time high_time etc optional columns
+* Tick Data (later not now)
+* Order Book Data (later not now)
+
+## Symbol Definition
+Symbols must follow a canonical formatting that provides a unique string identifier:
+
+type|...(type dependent)...
+ohlc|exchange:base/quote
+