Trade on close and intra-bar order execution

Strategy Backtesting
#1

Hi Guys,

I have a few strategies developed in Python using some other backtesting libraries and I would like to port them into Zipline. I am working only with daily OHLCV data and I would like to know whether it is possible (using standard features or a workaround):

  • to submit orders at the close of the current bar, so that they are active for the next bar,
  • to get stop or limit orders filled with the price that is between the high and low of a bar when the order condition is met,
  • to open or close positions at the close price of a bar?
#2

I see no reason why this shouldn’t work. You are pretty much describing zipline standard capabilities. As far as I know, everything that you described shouldn’t require any advanced tweaking. Although, I am not sure if it’s possible to force a filled order price.

#3

Consider this example:

from zipline.api import order_percent, record, symbol, get_datetime, cancel_order, get_open_orders, get_order
from zipline.assets import Asset
from zipline import run_algorithm


def initialize(context):
        context.bfx = {}
        context.order_id = ''
        context.asset = symbol('AAPL')

def handle_data(context, data):
        if context.order_id == '':
                context.order_id = order_percent(context.asset, 0.1, limit_price=35.00)
        if len(context.portfolio.positions) > 0 and len(get_open_orders()) == 0:
                context.order_id = order_target(context.asset, 0, limit_price=38.00)
        prices = data.current(context.asset, ['open', 'high', 'low', 'close'])
        print(get_datetime(), prices.open, prices.high, prices.low, prices.close) 
        print(get_datetime(), get_order(context.order_id)) 
        print(get_datetime(), context.portfolio.positions[context.asset])
        print('-'*80)


result = run_algorithm(
        start   = pd.Timestamp('2019-01-01').tz_localize('UTC'),
        end     = pd.Timestamp('2019-02-01').tz_localize('UTC'),
        initialize      = initialize,
        handle_data     = handle_data,
        capital_base    = 10000,
        bundle  = 'nyse',
        data_frequency  = 'daily'
)

This hastly written strategy is supposed to place a limit entry buy order on bar 1 and place a limit exit order once the entry order is filled and there is an open position.

It seems like the entry order is indeed submitted on the first bar and filled on the second bar. However the filled price is the close price (+commission), not the limit price even though open > limit > low:

2019-01-02 21:00:00+00:00 37.7 38.664 37.539 38.438
2019-01-02 21:00:00+00:00 Event({'id': 'ba2a023ae876436b874ded648ef8278b', 'dt': Timestamp('2019-01-02 21:00:00+0000', tz='UTC'), 'reason': None, 'created': Timestamp('2019-01-02 21:00:00+0000', tz='UTC'), 'amount': 26, 'filled': 0, 'commission': 0, 'stop': None, 'limit': 35.0, 'stop_reached': False, 'limit_reached': False, 'sid': Equity(3 [AAPL]), 'status': <ORDER_STATUS.OPEN: 0>})
2019-01-02 21:00:00+00:00 Position({'asset': Equity(3 [AAPL]), 'amount': 0, 'cost_basis': 0.0, 'last_sale_price': 0.0, 'last_sale_date': None})
--------------------------------------------------------------------------------
2019-01-03 21:00:00+00:00 35.045 35.468 34.563 34.609
2019-01-03 21:00:00+00:00 Event({'id': 'facd0aaac9d64f788c1abb7f316f618f', 'dt': Timestamp('2019-01-03 21:00:00+0000', tz='UTC'), 'reason': None, 'created': Timestamp('2019-01-03 21:00:00+0000', tz='UTC'), 'amount': -26, 'filled': 0, 'commission': 0, 'stop': None, 'limit': 38.0, 'stop_reached': False, 'limit_reached': False, 'sid': Equity(3 [AAPL]), 'status': <ORDER_STATUS.OPEN: 0>})
2019-01-03 21:00:00+00:00 Position({'asset': Equity(3 [AAPL]), 'amount': 26, 'cost_basis': 34.6273045, 'last_sale_price': 34.609, 'last_sale_date': Timestamp('2019-01-03 21:00:00+0000', tz='UTC')})
--------------------------------------------------------------------------------

Going farther the limit price of the exit order ($38) is reached on 2019-01-17 when the high reaches $38.37, but the order gets filled on the following bar:

--------------------------------------------------------------------------------
2019-01-17 21:00:00+00:00 37.532000000000004 38.374 37.303 37.936
2019-01-17 21:00:00+00:00 Event({'id': 'facd0aaac9d64f788c1abb7f316f618f', 'dt': Timestamp('2019-01-03 21:00:00+0000', tz='UTC'), 'reason': None, 'created': Timestamp('2019-01-03 21:00:00+0000', tz='UTC'), 'amount': -26, 'filled': 0, 'commission': 0, 'stop': None, 'limit': 38.0, 'stop_reached': False, 'limit_reached': False, 'sid': Equity(3 [AAPL]), 'status': <ORDER_STATUS.OPEN: 0>})
2019-01-17 21:00:00+00:00 Position({'asset': Equity(3 [AAPL]), 'amount': 26, 'cost_basis': 34.6273045, 'last_sale_price': 37.936, 'last_sale_date': Timestamp('2019-01-17 21:00:00+0000', tz='UTC')})
--------------------------------------------------------------------------------
2019-01-18 21:00:00+00:00 38.335 38.428 37.965 38.17
2019-01-18 21:00:00+00:00 Event({'id': 'facd0aaac9d64f788c1abb7f316f618f', 'dt': Timestamp('2019-01-18 21:00:00+0000', tz='UTC'), 'reason': None, 'created': Timestamp('2019-01-03 21:00:00+0000', tz='UTC'), 'amount': -26, 'filled': -26, 'commission': 0.026000000000000002, 'stop': None, 'limit': 38.0, 'stop_reached': False, 'limit_reached': True, 'sid': Equity(3 [AAPL]), 'status': <ORDER_STATUS.FILLED: 1>})
2019-01-18 21:00:00+00:00 Position({'asset': Equity(3 [AAPL]), 'amount': 0, 'cost_basis': 0.0, 'last_sale_price': 0.0, 'last_sale_date': None})
--------------------------------------------------------------------------------
#4

Hmm, I think I found the explanation:

I need to experiment with that InstantSlippage they were talking about.

#5

I do not think this InstantSlippage model does the trick, because still:

To use this slippage model, you would need to include set_slippage(InstantSlippage()) in your algorithm’s initialize function. Keep in mind that orders would still begin filling on the next bar.

like they said in https://github.com/quantopian/zipline/issues/1844 , so ie. it is not possible to have a stop loss order for a short position filled on the same bar as a stop entry order to go long at a level that falls between the low and high of the bar. Or at least I do not know how to achieve that.

#6

Notice that all the timestamps are daily…and your first order has a timestamp of 01-03, with the second being 01-19…handle_data will only happen once…order events only happen at the end of the day…make sure that in simulation, your rescheduling events happen at least 3 days apart…yes, it is weird, so you might have to rethink your idea of a ‘bar’…easiest solution to your problem might be to use minutely data…I use daily data for long-term backtests, as it is cheaper and I’ve hard-learned some of the ins and outs…there is reason Quantopian went to minutely data in it’s last few years…the semantics of the daily data plus no-look-ahead-bias makes it weird.