How outdated screenshots show up in real docs

A team of researchers went looking for outdated screenshots in VS Code's documentation and found 210 issue reports about them. Their most useful result is a negative one: how old a screenshot is tells you almost nothing about whether it is wrong.

The study

"Detecting Outdated Screenshot from GUI Document", by Ye Tang, Aoyang Yan, Hui Liu, Na Meng and Hao Zhong, was published in ACM Transactions on Software Engineering and Methodology in April 2026 (volume 35, issue 5, article 133). It is peer-reviewed. The authors studied VS Code's docs first, then built a tool, DOSUD, that spots outdated screenshots automatically.

Every number below was checked against the published paper. If you can't get past ACM's paywall, the authors' preprint has the same figures.

How common they are

VS Code is a good place to look, because its docs lean on pictures. Of its 590 documentation pages, 455 contain screenshots.

What they counted	Number
VS Code docs pages with screenshots	455 of 590
Issue reports found by searching "update screenshot" and "outdated screenshot"	210
Reports sampled for a closer look (20%)	44, from 27 reporters, fixed by 18 programmers
Fixes that changed the text as well as the picture	24 of 44

That last row matters. In more than half the sampled fixes, the screenshot was wrong because the feature had changed, so the words around it were wrong too.

Who they confuse

Every one of the 44 sampled reports came from a real user or programmer, which the authors take as evidence that the screenshot confused someone. It is not only beginners: 25.9% of the reporters had more than ten years of programming experience. One reporter's complaint was simply that a tab had become a dropdown and might confuse people.

Age is not the test

The obvious shortcut is to flag any screenshot that hasn't been updated in a while. The authors tried it. They measured how long fixed screenshots had gone between updates, took the median, 297 days, and used it as a cut-off.

The 297-day rule flagged 70.3% of VS Code's screenshots as outdated.
They checked 154 of the flagged screenshots by hand. Only 8 were actually outdated.
One false alarm had not been touched in 1,281 days and was still accurate.

An old screenshot can be right and a new one can be wrong. To know which, compare the screenshot with the product as it is now.

That holds for any docs, not only VS Code's. A "last updated" date on an image is a reminder, not a check.

What the detector found

DOSUD pulls the screenshots out of a document and uses a trained model to judge whether each one is outdated, looking at the small areas where versions of a screen differ. Run on live documentation, it found 20 outdated screenshots across 50 Android app websites and 17 in VS Code's docs. VS Code's developers confirmed and fixed every report.

We don't point at other companies' docs on this site. VS Code is the exception because the study is published and the VS Code team fixed what it found.

A wider survey, with a caveat

GitHub's 2017 Open Source Survey asked about 5,500 randomly sampled open-source users about documentation. 93% had seen incomplete or outdated documentation, and 60% of contributors said they rarely or never contribute to docs. Two caveats. It measures what people noticed, not what was wrong. And it puts incomplete and outdated in one answer, so it tells you nothing about screenshots on their own.

What to do about it

If age doesn't tell you, the product has to. Check screenshots against the app every time the app changes.

Take screenshots from the build. If your end-to-end tests capture the screens your docs show, each build makes new ones from the current app. That replaces guessing from age with a fresh picture of the product. Screenshot automation shows the test.
Embed a URL that follows the build. Screenery publishes each screenshot at an @latest URL that a push on the default branch moves. Docs that embed it show the newest capture without anyone re-uploading. See How @latest moves.
Review changed screenshots with the code. On a pull request, Screenery compares each screenshot with @latest and comments with what changed, so a reviewer sees which docs pictures a change touches. See Pull-request comments.
Find the screenshots no test takes any more. By default, a default-branch push removes any name it didn't push, so a deleted test's @latest URL answers 404. In a project set to additive mode, or one that pushes with partial, nothing removes a name, and a deleted test's URL keeps serving the last picture. Screenery's wilting alerts flag a screenshot no build has pushed while other builds kept pushing. That is not an age check: a test that still runs and still produces the same picture never wilts.

Pairings Up worked this way through a full rebrand and retook no screenshots by hand. See We renamed our app and never retook a screenshot.

Sources

Every page below was opened on 13 November 2026, and each number on this page was checked against it.