I have build a docker image with a jupyter quant environment. it has everything required to run machine learning for trading code. you might need to install some additional packages, but the image can handle that fine.
I had to find the sweet spot between being usefull, fully features and have reasonable size. so most machine learnings utisl and time series packages are in.