Below are my notes and highlights from this session at Write The Docs Europe 2016 in Prague. This is part of a series I wrote during the conference. This is not meant to be transcriptions and may have missed points made during they talk. They solely reflect my interpretations of the talk.
API Documentation: Exploring the Information Needs of Software Developers
by Michael Meng
Michael is spent 12 years as a technical writer and has been a university professor for the last 4 years. He is preparing the next generation of technical writers. Michael presented a talk about deriving recommendations for writing practitioners.
APIs are central to many modern software architectures and developers are often faced with learning them, but:
- a lack adequate documentation resources is the most severe obstacle faced when learning a new API
- developers prefer other information sources even if the documentation is available
- a lot of information contained in API references tends to be of little value
In his study they used the JDK and .NET API references.
Research Questions
- Which strategies do developers use to get into a new API?
- What questions come first? Second? Later?
- How do Developers use API documentation, if it is used at all?
- What do they look for?
- What quality criteria do developers apply to API docs?
- What is regarded as good or bad?
He and his partners conducted an interview and questionnaire study to set the context. They then performed an observation study before drawing conclusions.
Interview and Questionnaire
This qualitative feedback mechanism explored the research space. Interviews were semi-structured in order to start a conversation. They developed 45 questions on 11 themes to elicit information. They interviewed 23 developers from various backgrounds and experience.
A follow up questionnaire to validate this data was administered to 113 participants online.
Results indicated that developers approached an API and first asked, “What can I do with the API? What are the features?” After this there tended to be a mix of interest in concepts and overall ideas while others just wanted to go straight to action and programming with handling special use cases.
This is best expressed as two groups, “I need a global understanding of he API” and “I am writing code and want to see code.”
Follow up questions tended to be along the lines of “I work though sample code of individual API functions” and “I want to read a getting started guide.”
The top challenges cited at this point were “wrong or incomplete docs” and “incomprehensible docs.”
When asked what they expect from good API docs, developers typically answered:
- up to date
- complete
- includes code samples
Observation study
After conducting the qualitative study and observational study was conducted with the goal of confirming the other findings. This practice of paired studies is based on the idea that what people say they do is not necessarily what they do.
The observational study had the developers use an unfamiliar API to accomplish an understood task. Through the use of eye tracking the research team was able to observe where time was spent and how the docs were used or not used.
The observational study revealed that:
- Developers don’t read, they scan. They tend to stop at code blocks, headings and links.
- Clear transparent, consistent navigation is important.
- Navigation needs to use content type, not document type as the classification. Developer skip categories they aren’t clear on.
- Participants always used sample code to begin developing their solutions.
- Powerful search is critical.
