Screenery

Screenery / Guides

How to audit your docs and marketing for outdated screenshots

This is a one-time sweep. You list every screenshot you publish and every place it appears, decide which to check first, then hold each one up against the product as it is today. The date on the file won't tell you which ones are wrong. The product will.

Why sorting a documentation audit by date fails

Most content audits start by sorting pages by "last edited" and reviewing the oldest. For screenshots that sends you to the wrong place. One study tested exactly this. Researchers studying VS Code's documentation took the median time between updates for screenshots that had been fixed, 297 days, and flagged every screenshot older than that.

An old screenshot of a screen nobody has touched is still right. A screenshot taken last month is wrong if the screen changed last week. There is more on why page age is a weak signal in our write-up of that study, and a broader summary of what the research says about product screenshots.

Wikimedia's documentation toolkit reaches the same conclusion for whole pages. Its freshness check asks reviewers to score content from 1 (fresh) to 5 (completely outdated), and warns that "last revision date doesn't correlate directly with freshness."

1. Scope the audit

Screenshots live in more places than the docs site, and different people own each place. Write the list of places before you look at a single image.

The last four are marketing screenshots, and they are the ones a docs-only audit misses. Name an owner for each place now. You will need them in step 5.

2. Build the inventory

Make a spreadsheet with one row per embed, not one row per image file. A screenshot used on three pages is three rows, because each page can need a different fix, or none.

Column What goes in it
Image File path or URL
Embedded in Page URL, slide number, or store listing and slot
Product screen Which screen and state it shows, such as "Billing settings, admin, dark mode"
Owner Who can change that page or file
Traffic Monthly views of the page it sits on
Image date Record it, but don't rank by it
Verdict Filled in during step 4

The "Product screen" column is the one that makes the rest of the audit work. It lets you join the inventory to a list of UI changes in step 3. Wikimedia's toolkit also advises against a catch-all notes column, because you can't sort or filter on it. Give each judgment its own column.

To fill the sheet:

3. Rank what to check first

A large product can have hundreds of rows. Rank them with signals that point at real changes, and use the image date only to break ties.

  1. Screens touched by recent UI changes. List every screen that changed since the last time anyone checked. Release notes, merged pull requests, design tickets and the changelog will name them. Every inventory row showing one of those screens goes to the top. A UI change is what makes a screenshot stale, so this is the strongest signal you have.
  2. High-traffic pages. Your analytics will name them: usually the homepage, pricing, and the first few getting-started pages. Wikimedia's toolkit ranks pages by page views and by how many other pages link to them, and the same two numbers work for screenshots.
  3. Pages reached from support tickets. Search tickets for links to your docs. When support agents keep sending people to a page, those people are trying to follow it step by step. That is where a wrong screenshot does the most harm.
  4. Reports from readers. Search your issue tracker and feedback forms for "screenshot", "outdated", "doesn't match" and "can't find". In the VS Code study, the researchers found their 210 reports by searching for "update screenshot" and "outdated screenshot". Of the 44 they sampled, 36 came from people outside the docs team.

Plone's documentation shows how long this can sit. In January 2024 a docs maintainer opened issue #1611 asking for current screenshots of both of Plone's interfaces, and for a check that flags when a new capture differs from the previous one. The issue notes that Plone 5's docs once had such a feature, which has since become unsupported. When we checked on 15 November 2026 it was still open, and the proposed first trial was a single page with four screenshots.

4. Compare each one with the product

Open the product on the same screen, in the same state the screenshot shows: the same role, plan, sample data, theme and window size. Then go through the image and ask:

That last question matters more than it looks. In the VS Code study, 24 of the 44 sampled fixes changed the text as well as the picture.

Give each row one verdict:

Text has an advantage here that pictures don't. Wen Siang Tan, Markus Wagner and Christoph Treude built a tool that finds references in documentation to code that no longer exists. In the 1,000 most popular GitHub projects, 28.9% of the projects they analysed (265 of 918) had at least one document with an outdated code reference. Their follow-up packages the check as a GitHub Action that scans docs on each pull request. It works because the tool can match a name in the text against the code. A screenshot has nothing to match, so you still have to compare it with the product.

5. Record and fix

After the sweep

An audit tells you what is wrong today. The next release starts the drift again. To stop them going stale again, the screenshots have to come from the product whenever it changes, not from a person remembering to retake them.

That is what Screenery does for the screens your end-to-end tests can reach. Your tests take the screenshots in CI. Screenery publishes each one at an @latest URL that a push on the default branch moves, so a docs page or README that embeds the URL shows the newest capture. On a pull request it comments with which screenshots changed. It doesn't reach a pasted image in a sales deck or an app store listing. Those rows stay in your sheet and still need an owner.

Sources

We opened every page below on 15 November 2026 and checked each number on this page against it.

  1. Tang, Yan, Liu, Meng and Zhong, "Detecting Outdated Screenshot from GUI Document", ACM TOSEM 35(5), article 133, April 2026
  2. The same paper, authors' preprint (PDF)
  3. Tan, Wagner and Treude, "Detecting outdated code element references in software repository documentation", Empirical Software Engineering 29, article 5, 2024 (open access)
  4. Tan, Wagner and Treude, "Wait, wasn't that code here before? Detecting Outdated Software Documentation", arXiv preprint 2307.04291, 2023
  5. Wikimedia Documentation Toolkit, Collection audit, including its Survey, Prioritize and Assess steps
  6. Plone documentation, issue #1611, "Automate screenshots and videos"

Related guides