Data Workflow

QTPyLib’s new workflow module includes some handy methods for working with external data sources when backtesting.

Working with External Data

Sometimes, you’d want to backtest your strategies using market data you already have from sources other than the Blotter. Before you can use market data from external data sources, you’ll need to convert it into a QTPyLib-compatible data format.

Once the data is converted, it can be read by your strategies as CSV files. You can also save the converted data in your Blotter’s MySQL database so it can be used just like any other data captured by your Blotter.

Code Example:

# load the workflow module
from qtpylib import workflow as wf

# Load some data from Quandl
import quandl
aapl = quandl.get("WIKI/AAPL", authtoken="your token here")

# convert the data into a QTPyLib-compatible
# data will be saved in ~/Desktop/AAPL.BAR.csv
df = wf.prepare_data("AAPL", data=aapl, output_path="~/Desktop/")

# store converted bar data in MySQL
# optional, requires a running Blotter
wf.store_data(df, kind="BAR")

Note

The first argument in prepare_data() must be a valid string as IB tuple (just like the those specified in your strategy’s instruments parameter). For a complete list of available methods and parameters for each method, please refer to the Workflow API for a full list of available parameters for each method.

Using CSV files when Backtesting

Once you have your CSV files in a QTPyLib-compatible format, you can backtest using this data using the --data flag when running your backtests, for example:

$ python strategy.py --backtest --start 2015-01-01 --end 2015-12-31 --data ~/mycsvdata/ --output ~/portfolio.pkl

Please refer to Back-Testing Using QTPyLib for more information about back-testing using QTPyLib.