In this talk, we will present a framework for analyzing the exact dynamics of iterative methods such as stochastic gradient descent, randomized Kaczmarz algorithms, iterative coordinate descent in the high-dimensional limit.