Screenery

Screenery / Guides

Outdated documentation statistics: what the studies measured

Everyone agrees docs go out of date. Few numbers say how often, and the ones that do count very different things: survey answers, forum threads, code references, flagged pages. This page puts each number we could trace to its source in one table, with who was asked and what was counted. Then it says what the numbers can and can't support.

Every number in one table

We checked each row against the primary source on 15 November 2026. The last column says what kind of evidence it is, because a survey answer and a count of broken references are not the same kind of fact.

Source Year Who or what Sample What was counted Result Kind
Uddin and Robillard, IEEE Software 2015 IBM software professionals 323, in two surveys Ten API documentation problems, including "obsoleteness" Obsoleteness reported by 6 of 69 in the first survey; called a blocker at least once in the second Survey, one company
GitHub Open Source Survey 2017 Open-source users sampled at random on GitHub.com 5,500, from over 3,800 repositories Respondents who had run into incomplete or outdated documentation 93% Survey, perception
Aghajani et al., ICSE 2019 Emails, Stack Overflow threads, issues and pull requests about docs 878 artifacts read by hand Share of documentation content issues that were up-to-dateness problems 39% (190 of 485) Classified reports
Tan, Wagner and Treude, Empirical Software Engineering 2024 The 1,000 most-starred GitHub projects 918 projects now, 800 project histories Projects whose docs name a code element that no longer exists in the code 28.9% now (265 of 918); 82.3% at some point (658 of 800) Measured by tool
SlashData, Developer Nation Q1 2025 2025 Professional developers Survey of over 10,500 developers in 127 countries Developers naming "insufficient or outdated documentation" as a challenge 31%; 46% in large teams, 28% in small Survey, perception
GitBook (vendor) 2026 GitBook docs sites with 1,000 or more AI assistant questions a month Not stated Sites with at least one page flagged as possibly incorrect or outdated, September 2026 83%; 51% at 100 to 999 questions Vendor product data
Tang et al., ACM TOSEM 2026 VS Code's documentation 455 of 590 pages with screenshots Issue reports about outdated screenshots 210 Counted reports

The last row is about pictures, not text. It has its own page: the VS Code screenshot study covers how the authors found those 210 reports and why a screenshot's age turned out to be a poor test of whether it is wrong.

GitHub's Open Source Survey, 2017

This is the number most people quote. GitHub sampled 5,500 respondents at random from over 3,800 open-source repositories on GitHub.com, and the survey's published findings cover that random sample only. GitHub's summary says incomplete or outdated documentation was "observed by 93% of respondents". The same summary adds that 60% of contributors rarely or never contribute to documentation.

Two things limit it. It asks what people noticed, so it measures exposure, not the state of any document. A respondent who hit one stale page in five years counts the same as one who hits them daily. And the answer joins incomplete and outdated, so it can't tell you how many of the 93% met outdated docs in particular. It is still the best evidence that nearly everyone in open source has met bad docs.

Uddin and Robillard at IBM, 2015

"How API Documentation Fails", in IEEE Software, ran two surveys inside IBM with 323 software professionals in all. The first asked 69 people for real examples of good and bad API docs. The authors sorted 79 bad examples into ten problems. One of them, obsoleteness, means the documentation "referred to a previous version of the API". Six of the 69 respondents gave an example of it, against 20 for incompleteness.

The second survey asked 254 other IBM developers and architects how often they met each problem and how badly it hurt. The three most severe were ambiguity, incompleteness and incorrectness. Obsoleteness was one of six problems that respondents rated a "blocker" at least once, meaning they could not get past it and picked another API.

So outdated content was real but not the top complaint here. Missing and unclear content hurt more. The sample is one company's staff. The paper also reports frequency and severity as ratios on a chart, not as a headline share for each problem, so there is no single "X% of developers" figure to quote.

Aghajani et al., ICSE 2019

"Software Documentation Issues Unveiled" skipped surveys on purpose. The authors argue that asking developers gives a biased view. Instead they used keyword search to find 805,939 candidate artifacts in mailing lists, Stack Overflow, issue trackers and pull requests, then read 878 of them by hand. That gave a taxonomy of 162 types of documentation issue in four groups: what the docs say, how they say it, the process around them, and the tools.

Up-to-dateness is a category in the first group. The paper defines an outdated document as one "not in sync with other parts of a system", and separates it from incorrect or incomplete docs. In an outdated doc, the information was right and complete before something changed. The paper's figure puts 190 of the 485 artifacts about documentation content under up-to-dateness. The text puts it as 39% "of issues related to documentation content".

Be careful with that 39%. It is a share of content issues, not of all documentation issues, and one artifact can raise more than one issue. It also counts things people bothered to write about. A stale page nobody complained about never enters the data. One of the 162 issue types is "Screenshot does not reflect current GUI", which is the problem this site exists for.

Tan, Wagner and Treude, 2024

"Detecting outdated code element references in software repository documentation", in Empirical Software Engineering, is the only source here that measured text documentation directly instead of asking people or reading complaints. The authors' tool looks for names in a project's docs, such as functions or classes, that existed in the code when the doc was last updated but have since been deleted from it.

They ran it on the 1,000 most-starred GitHub projects and on all of Google's GitHub projects. Among the popular projects, 28.9% (265 of 918 analysed) had at least one outdated reference at the time. Across the full history of 800 of them, 82.3% (658) had been outdated at least once. The paper's introduction adds that these references "were typically outdated for years" before maintainers noticed and fixed them.

The authors also reported 19 instances of outdated documentation to 15 actively maintained Google projects. Maintainers had fixed 5 of them by the time the authors wrote the paper. Some of the others were false positives, a cost that comes with any automated detector.

The method is narrow by design. It catches a renamed function that the README still mentions. It can't catch a sentence that describes old behaviour, a wrong screenshot, or a step that no longer exists in the UI. So 28.9% is a floor for one kind of staleness in popular open-source projects, not a rate for documentation in general.

SlashData's Developer Nation, Q1 2025

SlashData's summary of its Q1 2025 State of the Developer Nation report lists "insufficient or outdated documentation" at 31% among the challenges professional developers face. It ties for first with code that is hard to read, maintain or extend, also 31%. The report is based on a survey of over 10,500 developers in 127 countries.

The split by team size says more. In large teams, 46% of developers reported documentation issues, against 28% in small teams (1 to 10 developers). In organisations over 30 years old, 39% named documentation. Like GitHub's survey, it joins insufficient with outdated, and it measures what developers report. SlashData sells market research, and the public post doesn't give the question wording or how many people answered this question.

GitBook's product data, 2026

This is vendor data, and GitBook sells a docs platform with an AI assistant. Its post of 16 October 2026 says that among GitBook sites getting at least 1,000 assistant questions a month, 83% had at least one page flagged as possibly incorrect or outdated in September 2026. For sites with 100 to 999 questions the share was 51%, and lower for quieter sites.

GitBook's own caveats are good, so we repeat them. A flag "identifies something to investigate, rather than confirming an error". Busier sites have more chances to be flagged, so the pattern doesn't mean popular docs are worse. The post doesn't say how many sites or pages are in each group, or exactly what raises a flag. Read it as evidence that reader questions expose stale pages, not as a rate of outdated documentation.

What the numbers show

What they don't show

What nobody has measured

These are the gaps we looked for and couldn't fill. Each would make a useful study.

Until someone does, the honest summary is short. Outdated docs are common, they last a long time, and nobody knows the rate for a typical docs site. For how this fits the wider research on screenshots and docs, see what the research says about product screenshots.

Numbers we left out

Getting your own number

The only rate that matters to you is your own. Pick a sample of docs pages, open the product next to each one, and mark what no longer matches. How to audit your docs for outdated screenshots walks through it for images. For screenshots, you can also remove the question. Screenery publishes screenshots from your tests at an @latest URL that each default-branch build updates, so the picture in the docs is the one the last build took. See how to keep documentation screenshots up to date.

Sources

We opened every page below on 15 November 2026 and checked each number on this page against it.

  1. GitHub, Open Source Survey, 2017
  2. Uddin and Robillard, "How API Documentation Fails", IEEE Software 32(4), 2015, and the authors' preprint (PDF)
  3. Aghajani, Nagy, Vega-Márquez, Linares-Vásquez, Moreno, Bavota and Lanza, "Software Documentation Issues Unveiled", ICSE 2019, pp. 1199-1210, and the authors' preprint (PDF)
  4. Tan, Wagner and Treude, "Detecting outdated code element references in software repository documentation", Empirical Software Engineering 29, article 5, 2024 (open access)
  5. SlashData, "Why software development challenges hurt the business", 19 June 2025, on the State of the Developer Nation Q1 2025 report
  6. GitBook, "By the numbers: Where documentation quality breaks down in 2026", 16 October 2026 (vendor data)
  7. Tang, Yan, Liu, Meng and Zhong, "Detecting Outdated Screenshot from GUI Document", ACM TOSEM 35(5), article 133, 2026

Related guides