We consider the distributed estimation problem when compressed data is exchanged between computers. Our study reveals precise trade-offs between compressibility and rates of statistical estimation.