A survey was sent to software developers and other software professionals to ask what was important in SDK documentation and where they would like to see improvement. The results indicated that the current state of typical SDK documentation is "Fair", which was the middle choice of five ("Excellent" to "Unusable"). The answers to multiple choice questions indicated that overviews, API references, sample code, and tutorials were all considered high importance, whereas blogs and forums were considered less important. When asked to write what they considered important, sample code was mentioned in 61% of the responses and overviews were mentioned in 30% of the responses. Also mentioned as important were help getting started, explanations of why something should be used, accuracy of information, and the ability to find information easily.
KB102002
Introduction
When a well-run technology company creates a new software application, it will spend a great deal of time and attention on the user interface so that the application is easy and enjoyable to use. When a technology company creates a software platform and releases an SDK, it should put a similar level of attention on the SDK's "user interface", which consists of the application programming interface (API ) and its documentation. Michi Henning wrote recently about the importance of APIs, and he stated, "A big problem with API documentation is that it is usually written after the API is implemented and often written by the implementer."[1] We cannot always control when documentation is written or who writes it, but this paper sheds some light on what software professionals are looking for in effective SDK documentation.
SDK Bridge sent out a survey to software developers, architects, managers, and technical writers to see what they felt was important in SDK documentation.The responses indicated a number of clear trends on what people considered important.
The Survey
The survey was relatively small, with 23 respondents, but it provided enough data to see commonality. Requests for the surveys were sent to personal contacts in the industry and some general mailing lists. When asked for their software role, the respondents replied as shown in Figure 1 below. The largest group was developers, which is to be expected given that they are the main users of SDKs. A disproportionately high number of technical writers responded given the number who were contacted, most likely due to the fact that this subject is personally important to writers.
Figure 1. Survey respondents, broken down by role.
The State of SDK Documentation
The survey asked, "How would you rate the average quality of documentation for SDKs that you have used or have considered using?" and provided the choices of Excellent, Good, Bad, Fair, or Unusable. The average value for all roles was Fair with a standard deviation of half a point, and no one chose either Excellent or Unusable.
What is Important
The survey asked, "Where would you most like to see improvements in SDK documentation?" Respondents were asked to put the following in order from highest priority to lowest: overviews and conceptual material, API references, sample code, tutorials, blogs, and forums.The results of the ranking are shown below in Figure 2, grouped into roles.
Figure 2. Ranking types of content
The standard deviation for all of these values was approximately 1 point, meaning that blogs and forums are clearly not considered high priority, but that the other variations are not statistically significant.
The survey then stated, "Rate how helpful each of these are for you in using an SDK," and listed the types of content as in the previous section. The choice of answers were: "very useful", "somewhat useful", "not very useful", and "not useful at all". When converted into a scale of 1 to 4, where 1 is not useful at all, and 4 is very useful, the data is shown in Figure 3.
Figure 3. Rating types of content by usefulness
Again the standard deviation was approximately 1 point. This suggests that blogs are not considered very useful at all, and forums are only considered slightly useful, but that all other types of content are generally considered useful by all roles.
Comments about SDK Documentation
The survey asked, "What makes SDK documentation helpful or not helpful?" and asked for general comments as well. There were a variety of answers, with many common themes.
Sample code. Although sample code was not rated significantly higher than other types of content in the multiple-choice part of the survey, it was mentioned by 61% of the respondents in their written comments. It was referred to as "the most helpful thing". Respondents wanted runnable examples that illustrate something, not just "Hello World" and talked about the desire for real world examples. There was a desire for very basic samples that would lead to more complex ones.
Help with getting started. Several respondents mentioned the difficulty of getting started with a new platform. One said, "People who make SDK documentation often don't think like a beginner who is just getting started with their SDK." Another requested "help with learning curve," and another suggested "going all the way down to the bottom (installation and setup of tools)". Another commented, "Once you've got the basics, it's easy to begin poking around on your own.But if you can't get started, that's frustrating."
Explanation of why. Eight of the respondents mentioned that it was important to understand why the software platform or individual APIs should be used. One summarized this as "Why, why, why, not what, what, what." Another cautioned that this should be from a programmer's point of view, and not the marketing department. Case studies and real-world examples were desired.
Accuracy is critical. Respondents wanted "absolute mathematical precision" and "no ambiguity". One respondent said that "the most unhelpful thing in SDK documentation is incorrect information," and another said, "Sometimes misinformation is worse than no information at all." This suggests how valuable technical reviews are; unfortunately, technical writers often find that getting developers' time for reviews is very difficult.
Ability to find information easily. This theme came up in several ways. Heavy-cross referencing and indexing was mentioned, as well as improving search capabilities. The ability to come at the information in different ways (by topic, alphabetically, etc.) was also mentioned.
Good overviews. 30% of the respondents mentioned overviews. Respondents desired "a high level overview of the concepts that bridges into the detailed API documentation" and said that it was "missing from many API documentations."
Content should be written by someone other than the developer.Two developers brought up this issue. One said it was not useful when documentation "is written by the same person that wrote the API. To them, everything makes perfect sense and requires little description." Another said, "Developers should only be the Subject Matter Experts, not the authors."
Conclusions
Good SDK documentation is extremely important for the adoption of software platforms. Given that companies have limited resources, where should they focus their efforts? Our survey suggests that sample code, overviews, and help getting people started should be given the highest priority. Content needs to explain why something should be used in addition to how it is used. Also, because accuracy is considered critical for documentation users, managers need to be sure that the developers who created the APIs are available for discussions with the writers and for technical reviews.
In the end, the best formula for effective SDK documentation is the combination of good writers who know how to write for a developer audience and developers who are willing to take the time to work with them. We hope our survey has provided some ideas on how best to make this collaboration happen.
References
[1] "API Design Matters", Communications of the ACM, Vol. 52, No. 5, p. 46, Association for Computing Machinery, 2009.
Tech Writer
Print This
