How can I append to an existing bundle?

Hi,

I have reasonably deeply researched how could I append new update data to an existing zipline bundle (i.e. make an initial ingest, and on a daily basis append new data), and have come to the conclussion that bcolz would need to be reconsidered as a database framework for storing bundle data. The reasong that drives to obtain that conclusion is that the attrs file inside the daily_equities.bcolz folder holds row numbers for the starting position for the different securities, and apparently you may only use a single starting position, hence enabling only 1 time data ingestion. So basically I believe all the persistence approach would need to be reformulated.

Is there any ongoing effort or zipline fork trying to attain that. I find zipline a marvelous tool (also true that haven’t researched others). But this is an important drawback. If handling a relative small universe, it’s fine, but if managing universe sizes in the order of thousands of securities, this work procedure becomes unfeasible.

Any hints or comments are greatly appreciated.

Regards,

Carlos.