What Happens To BERT Embeddings During Fine-tuning?

While much recent work has examined how linguistic information is encoded in pretrained sentence representations, comparatively little is understood about how these models change when adapted to solve downstream tasks.