Collaborative API-documentation with iDocIt!

Since we launched iDocIt! at Google Code in April this year we used it in many of our projects. In April we knew that the API-documentation created with iDocIt! would become more complete than classical documentations created with e.g. Javadoc. But we had just very simple and rough thoughts about how to organize the work with iDocIt!. This changed over the last months and this blog posting is aboutt the way it changed.

The Javadoc usage guidelines define that API-documentation is owned by programmers [1]:

[...]

Who Owns and Edits the Doc Comments

The doc comments for the Java platform API specification is owned by [added by the author of this posting] programmers. However, they are edited by both programmers and writers. It is a basic premise that writers and programmers honor each other's capabilities and both contribute to the best doc comments possible. 

[...]

But does this imply that the whole API-documentation is written by them? We believe not. Clements et al. described that there are many stakeholders of API-documentation (e.g. managers, architects, testers, developers and some more)[2, page 237 and following]. When designing iDocIt! we had the strong intuition that due to their interest in the documented content, they could contribute something to the documentation. But who could contribute what?

We realized that the (especially on the level of interfaces) the semantics of an operation are defined mainly with business or domain specific terms. Many programmers are not familiar with these terms and this is no surprise. They are responsible for writing stable, maintainable and performant code (which all results in minimized overall development and maintenance costs). So why should they describe the business meaning of an operation if someone else could do this in less time? From our experience interfaces within a software systems are defined and specified by the architect. The person having this role has to specify the semantics in any case somewhere. But the architect does not care about illegal arguments for the operation or other constraint violations. The developer is responsible for describing these aspects. And finally the tester specifies test cases as invocation examples. This principle is shown by the figure above. We propose to contribute this directly to the API-documentation in the code. 

How to to it with iDocIt!? We are going to show this to you over the next three postings in this blog. The first posting will show the parts documented by the architect. The second deals with the developer's part and the third covers everything contributed by the tester. We are looking forward to your feedback :).

[1]: Sun Microsystems, Inc: How to Write Doc Comments for the Javadoc Tool, 2004

[2]: Paul Clements, Felix Bachmann, Len Bass, David Garlan, James Ivers, Reed Little, Robert Nord and Judith Stafford: Documenting Software Architectures, Addison Wesley, 2004

How to describe the relation between spec and code?

In nearly all of my projects I had to answer two categories of questions for many times:

1. Which changed lines of code belong to a specific ticket in the issue tracker? Example: after deploying the version of our application including feature #42 the application crashes often with a memory leak. We need to review all changed lines of code for e.g. unexpected side effects.
2. Why has a specific piece of code been implemented in the way it is? Example: the business expert believes that there is an error in the computation of discount rates for our customers. We need to read the corresponding code, understand the computation of discounts and explain it to the business expert. After that the expert states that this is a bug we are responsible for. Now we have to proof that the implemented behavior has been specified by showing when and were (e.g. a specification document).

All in both examples we need a mapping between code and spec-documents (e.g. issues in the issue tracker like Atlassian JIRA):

1. We know the issue and need the changed lines of code.
2. We know the changed lines of code and need the issue.

How could this relation be realized in the code? A simple approach could be to mention the issue id in the commit comments of the version control, e.g. "Fix issue #42: fix memory leak by clearing the cache regularly". With this approach were are able to identify all changed files for a given issue as well as the issue for a changed file. 

But in case of modified existing code this approach might be too crude. We need the issue more granular per lines. It would be possible to extract this information from the file's version-history of the version control. But this  could be very time consuming. This is why I started in my current project to mark changed lines with an inline comment:

// Changes due to issue #42

…

doSomeStuff();

…

// End changes due to issue #42

This has the big advantage to identify changed lines per issue with a simple full text search. But there is also a big disadvantage: after some changes the readability of the code decreases due to the marker-comments:

// Changes due to issue #42

…

doSomeStuff();

…

// Changes due to issue #07

…

doSomeMoreStuff();

…

// End changes due to issue #42, #07

Yet I do not know a sophisticating solution to this disadvantage and would be pleased about comments by the readers of blog :). Feel free to post your experiences and opinions.

Extension of my thoughts concerning the "multi verb in identifiers "-dilemma

Today I discussed my preceding blogpost and the two introduced strategies to solve it with some colleagues. During this discussion I realized that there is an important precondition for both strategies. In my last post I discussed the identifier "generate and send invoice" and I described two ways to refactor this identifier:

• Generalize (and merge) the verbs: "charge invoice"
• Specialize the adjective: "send ready-to-print invoice"

Both strategies rely strongly on the semantic of the noun "invoice" (the accusative object if you read the identifier as an imperative sentence). This noun is a business term and it implies a business meaning: a message which lists the ordered articles or services and the amount of money the recipient has to pay for them.

I asked myself if the strategies would work in case of a noun without such a business semantic, e.g. "generate and send mail". The verb "to charge" implies to provide an invoice, but it won't make sense for any message. So the identifier "charge mail" would lead into serious trouble because one could understand it in a way that the recipient has to pay for the mail (because the mail is charged). Applying the second strategy would cause a similar case: "send ready-to-print mail" does no tell the whole story since the content of the mail is not characterized. A mail could also just be a simple covering letter.

This means that both described strategies have a strong precondition: an accusative object with a specific meaning in the operation's domain. This leads to the following refactoring algorithm for operation identifiers with two predicates:

1) Specialize the accusative object as precise as possible.
2a) Generalize the two verbs.
2b) Specialize the accusative with an adjective.

I would like to thank my colleagues Gerwin Abbing (Blue Carat AG) and Stefan Jockenhövel (Paragon Systemhaus GmbH) for the interesting and inspiring discussions on this topic.

The "multi verb in identifiers "-dilemma and strategies how to solve it

The identifier of an operation plays an important role for the understanding of the code using the operation as well as for understanding the operation itself. Operations usually have the meaning of  a bundle of instructions or commands which should be executed by the computer. I believe that this is the reason why programmers name them with an imperative sentence (e.g. "get the customer by id", "generate invoice" or "compute price") or a substantified verbal phrase ("customer id filter", "invoice generation" or "price computation"). This style in general seems to me to be widely established and recommended (see Robert C. Martin's "Clean Code", page 25 and following).

Sometimes I observe operation-identifiers containing two verbs in the grammatical role of a predicate and linked by a conjunction, like "generate and send invoice". I remember that I intuitively also tried to name operations this way for three or four times. Consider this piece of code as an example (in Java-syntax):

public static void generateAndSendInvoice(Order order) throws IOException{

Document invoice = generateInvoice(order);

sendInvoice(invoice, order.getCustomer());

}

Since operations often contain many commands it seems strange to me to mention two of them explicitly in these few cases for different reasons:

• I do not really understand the set of criteria to select the actions mentioned in the identifier from the set of instructions in the operation. If these criteria could not be described, they could not be added to naming conventions in a project. This could lead to different naming styles in one project and decrease the readability of the code.
• I think that identifiers could become very confusing if they reflect the internal control structure of an operation. Consider the above example with a condition. The invoice should only be send, if its recipient is verified: "generate and send invoice if recipient is ok". Again, I think programs containing these kind of identifiers become hard to read.
• The identifier represents what the operation is responsible for. From my point of view, the responsibility of generating the invoice remains in the operation "generateInvoice", but the identifier "generateAndSendInvoice" implies that this operation is also responsible for the generation, but this is not the case.

I asked myself why I wanted to label operations with two predicates.  Actually I didn't find a sophisticating answer to that question. I have the impression that programmers usually choose identifiers to express the intention or the final result of an operation. This is in line with observations made by Robert C. Martin (see page 18 of his book "Clean Code").  For example, the operation "computePrice" may compute every summand included in the final price, but its identifier does not express this. This rule would be violated by identifiers with two predicates since they describe not only the intended action, but also its intermediate steps (e.g. the computation of summands or the generation of an invoice).  One reason for this kind of extended identifiers might be that the identifier "send invoice" does not tell the whole story. Imagine that this operation does not only submit a invoice as mail via SMTP, but also creates a printed invoice from a PDF document. Since the verb "to send" has a strong meaning in the context of software engineering, the programmer might think that this identifier confuses the other team members. As solution to this dilemma it has been extended by mentioning the generation of the invoice document: "generate and send invoice".

How could this dilemma be compensated? From my point of view there are other strategies to solve this problem than extending the identifier with more than one predicate (this list may be incomplete):

• The programmer could generalize the verb in a way covering both actions: the invoice generation and submission. For the above example such a verb could be "to charge", e.g. "charge Order".
• A second option could be to specialize the object e.g. with an adjective. The submission of the invoice to the customer is the final intention of the operation, so it could be named "send ready-to-print invoice". Since the invoice is not part of the operation's signature, the identifier implies that the operation receives the invoice in another way.

I conclude, that it is hard work for programmers to find short and concise names for operations. I described two strategies to refactor names with two verbs. All in all I think that there is a huge need for a set of naming conventions going beyond technical terms such as delete, find, create and so on. Such a verb-lexicon should define per verb what an operation labeled with the verb is allowed to do and vice versa. The 19 thematic grids included in the current iDocIt! release could be a starting point for projects to define their lexicons.

This post was inspired massively by discussions with my colleagues Timm Schwemann, Julian Sirapanji (both Hamburger Berater Team GmbH) and Stefan Jockenhövel (Paragon Systemhaus GmbH). Thanks a lot ;).