Federer, Syllable by Syllable

We used an unsupervised algorithm borrowed from neuroscience (keypoint-MoSeq) to split and cluster Federer's movements into short, repeated pieces called behavioral "syllables". This was done from his joints alone, with zero knowledge of tennis. Some of these syllables correspond to conventional tennis strokes, like the serve or discrete parts of forehands and backhands, but others, like "Stroll, head down", are perhaps less obvious and more interesting.

Each dot on the map below is one instance of a syllable, and instances are clustered together by similarity. Click a dot and drag the circle to see what lives in different parts of the map. You can drag its edge and expand to include more instances, or contract it to narrow in. Colour by year to see which movements belong to which era.

Colour by

Scroll to zoom · drag to pan · click a dot to circle it · drag the circle to move it, its edge to size it

We were also curious whether joints can be used to find meaningful differences in his strokes across different epochs of his career. We first labelled his strokes into serves, forehands and backhands, and then used a lot of Opus to perform joint analysis. Some of these changes were not obvious to us before measuring, such as his forehand stance closing by about 15°. Some other examples are shown below.

With the same data we can measure footwork speed and court coverage, see how his patterns shift under pressure, or ask which stroke sequences come before a won or lost point, and do this across different players and compare styles. This is just the tip.

Forehand: more closed stance

Feet angle was 49° in 2003–07, 65° in 2017–19.

Forehand: ball contact further from body

Contact point was 54 cm from his hip center in 2003–07 and grew to 62 cm in 2017–19.

Forehand: higher finish

Height of his hitting wrist was 18 cm below his shoulders in 2003–07 and 4 cm above in 2017–19.

Serve: straighter body at contact

His shoulders were 32 cm to the side of his feet at contact in 2003–07 and 21 cm in 2017–19.

Data

TV recordings of 45 of his US Open matches, 2003–2019, some only highlights. We only include live play from the main camera angle, in which Federer is standing on the near end.
His joints (head, shoulders, elbows, wrists, hips, knees, ankles and feet) were tracked in every frame.
Every hit was labelled automatically as a serve, forehand or backhand.
Distances are in centimetres, from the court's known size.

Syllables

His movement was split into short, repeated movements ("syllables") with keypoint-MoSeq, from his joints alone. It found 30; a typical one lasts 0.5 s.
The serve was over-segmented into several syllables in a row, so they were merged into one, Serve.
On the map, each dot is one occurrence of a syllable, and similar movements sit close together (UMAP).
The names of each syllable were given by hand after watching each syllable's typical clips.

Then vs Now

His 274 forehands and 375 serves, typical strokes only: none hit on the run or far from the baseline. His 166 backhands were tested the same way and nothing changed reliably.
Camera zoom, picture quality, where he stood and how he moved were regressed out before comparing.
Each match gives one number, his average. 2017–19 is compared with 2003–07 by Welch's t-test, corrected (Holm) for the number of measures. A change is shown when the corrected p is below 0.05 and it is at least 0.4 of the spread between his strokes.
Charts: each dot is an era's average with its 95% interval; stars mark the corrected p (* below 0.05, ** 0.01, *** 0.001). The grey line is his opponents in the same matches.