How do I tell Zipline to backtest using my pandas dataframe?

kbs-code · November 11, 2023, 7:24pm

I am working using a local repository and this notebook within a remote repository which mirrors it:

kbs-code/algo_trader/blob/master/backtests/zipline_broad_market_ema_vortex.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NOT PRODUCTION READY"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from zipline.api import order_target, record\n",
    "from zipline import run_algorithm\n",
    "from zipline.data.bundles import register\n",

This file has been truncated. show original

In the notebook, I loaded some financial data and used it to create a pandas dataframe. I used Github Copilot to suggest an ingest_func to process the dataframe. I attempted to run my algorithm with this function:

result = run_algorithm(
  start=df.index[0],
  end=df.index[-1],
  initialize=initialize,
  handle_data=handle_data,
  capital_base=100000,
  bundle='my-bundle')

After I ran this cell, I got the following error:

ValueError: no data for bundle 'my-bundle' on or before 2023-11-11 19:00:08.261489+00:00
maybe you need to run: $ zipline ingest -b my-bundle

I then navigated to my local algo_trader environment and ran zipline ingest -b my-bundle, which gave me this error:

Error: No bundle registered with the name 'my-bundle'

How do I properly tell Zipline to utilize the dataframe I created?

gnzsnz · February 11, 2024, 6:13pm

you can use the frame loader

https://zipline.ml4trading.io/api-reference.html#zipline.pipeline.loaders.frame.DataFrameLoader

there is an example in the book

github.com

stefan-jansen/machine-learning-for-trading/blob/main/08_ml4t_workflow/04_ml4t_workflow_with_zipline/02_backtesting_with_zipline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Backtesting with zipline - Pipeline API with Custom Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Pipeline API](https://www.quantopian.com/docs/user-guide/tools/pipeline) facilitates the definition and computation of alpha factors for a cross-section of securities from historical data. The Pipeline significantly improves efficiency because it optimizes computations over the entire backtest period rather than tackling each event separately. In other words, it continues to follow an event-driven architecture but vectorizes the computation of factors where possible. \n",
    "\n",
    "A Pipeline uses Factors, Filters, and Classifiers classes to define computations that produce columns in a table with PIT values for a set of securities. Factors take one or more input arrays of historical bar data and produce one or more outputs for each security. There are numerous built-in factors, and you can also design your own `CustomFactor` computations.\n",
    "\n",
    "The following figure depicts how loading the data using the `DataFrameLoader`, computing the predictive `MLSignal` using the Pipeline API, and various scheduled activities integrate with the overall trading algorithm executed via the `run_algorithm()` function. We go over the details and the corresponding code in this section.\n",
    "\n",
    "![The Pipeline Workflow](../../assets/zip_pipe_flow.png)\n",

This file has been truncated. show original

and you have zipline test code to use as a reference as well.