We describe how deep learning methods can be applied to forecast stock returns from high frequency order book states. We review the literature in this area and describe a study where we evaluate return forecasts for several deep learning models for a large subset of symbols traded on the Nasdaq exchange. We investigate whether transformations of the order book states are necessary and relate the...