Essays/CalTrops

Why Manuals Are So Uninformative

Caltrops

Caltrops (or calthrops) are things strewn in the text which do nothing but slow the reader down. Maybe even bring her to a complete halt.

Typical caltrops:

• Using a jargon word, without explaining it, especially one which resembles a common word.
• Treating two words as interchangeable that mean different things to most people.
• Conversely, making a pedantic distinction between two words that most people see as synonyms.
• Calling the same thing by different names in the same paragraph.
• Referring to some identifier or reserved word in a code sample in a non-code font, or in a way which dissolves into the narrative.
• Using metalanguage in code examples without carefully explaining it (or better still: avoiding it).
• Dropping into mathematical (legal, financial or other) jargon when it's all about programming.
• Overusing passives, gerundives, subjunctives, big words, subordinate clauses and other pompous constructs.
• Wisecracking when there's no guarantee the reader will recognize and applaud your devastating sense of humor.

J documentation, alas, is full of caltrops. Often these come about by the very efforts of its originators to make the J language more approachable, e.g. by using grammatical terms from spoken languages, like adverb and gerund to describe its concepts. Concepts indeed which are novel ones to people who can code in several other programming languages.

If the purpose of the documentation is to help a beginner to understand J, then caltrops are a bad thing. It's worth going to a lot of effort to remove them.

Why do professional people strew their write-ups with caltrops?

Caltrops are good things to use in expository text which is designed to show off your knowledge. If the reader can't understand you, then it follows (...doesn't it?) that you're cleverer than she is.

Some social psychologists genuinely believe this is why technical wizards strew their reports with caltrops. They are asserting their superiority and irreplaceability in your world.

There is a less insulting explanation. Perhaps professional people write in pompous English because they think that's how it ought to look. They've been brought up on old textbooks, from way back when the written word and the spoken word were two different languages, often the preserves of different classes of people. They're used to foggy user guides in their daily work, and can no longer tell good documentation from bad. Really clear documentation is a big effort to write, and they're often in a hurry. Baudelaire once apologized to a friend for the long letter because he "didn't have time to write a short one".

In the old Soviet Union, if the political commissar could understand your proposal, you wouldn't get your funding. But the Soviet Union had no monopoly on obfuscation. If your explanation is so simple a four-year-old can understand it, then some technically-naive manager might conclude a four-year-old can write it. So he asks himself: what exactly are you being paid for?

In daily life we're surrounded by public notices which give us all a bad example from our earliest days.

Passengers must refrain from engaging in the consumption of intoxicating beverages.

(Can you just see the signwriter taking a friend into a bar with the words "Let's engage in the consumption of intoxicating beverages"?)

All too seldom do you see:

Don't drink on the bus.

Why not?

The reasoning behind it is never voiced, because it would straightaway be seen for what it is: totally absurd. Here's how it goes. When you're talking to people who are so ignorant they need to be told such obvious things, you must assert your superiority by talking down to them. Speaking to them in their own language is to descend to their level. You risk not being taken seriously. Yet the signwriter will see no contradiction in translating signs into Spanish and Vietnamese in California, say, or Welsh in the UK, or French in Canada. Admittedly there are laws in all these states compelling it.

Why is Information Technology stuffed with bad documentation?

This would puzzle the Man from Mars. An industry whose sole purpose is to impart information efficiently can't communicate know-how about its own products in an easily understood form.

Everyone feels qualified to understand the human mind. Because everyone owns one, having done so from birth, and generally got to know it quite well. When lay people finally grasp what a psychology paper is saying, they typically respond with "Well, I knew that all along!" So university departments of Psychology have good reason to tolerate prolixity in their publications. Perhaps even to encourage it.

Yet that's never the lay person's response to a paper about computers, which are far, far simpler information processing machines. Nor do the IT industry's bosses have any reason to tolerate opaque manuals. Any impediment to uptake of a firm's product harms profits directly. And yet obscurity in IT manuals flourishes like couch grass in your lawn and seems to be just as hard to remove.

Here's a typical product of the IT industry, taken from the Debian Handbook:

As for the ar program, it allows handling files of the same name: ar t archive displays the list of files contained in such an archive, ar x archive extracts the files from the archive into the current working directory, ar d archive file deletes a file from the archive, etc. Its man page (ar(1)) documents its many other operations. ar is a very rudimentary tool that a Unix administrator would only use on rare occasions, but admins routinely use tar, a more evolved archive and file management program. This is why it is easy to restore dpkg in the event of an erroneous deletion.

You can find instances of most of the caltrops listed at the head of this article.

There are several reasons peculiar to the IT industry for handbooks getting written in this vein.

• The manual has been written by the programmer.
  • • He talks mostly to people like himself.
    • He has talked a lot to his fellow programmers over the course of the project. During that time design concepts have become clarified, i.e. their meaning has shifted. But he's still using the same old words.
    • He never gets to see a real live user, much less a beginner.
    • He is apt to assume that all beginners know nothing and start with an empty slate. They have not been weaned off competing products, whose lawyers have bent the ears of the project manager to use different words for essentially the same thing, to avoid a court battle over look-and-feel.

• The manual has been written by a professional documenter.
  • • Documenters work on another floor, in another building, maybe in another country. They only communicate with the programmer by email.
    • Documenters typically have a degree in English or some arts subject like Philosophy, or even (hell's bells!) Psychology.

• Unlike the program, the manual has never been debugged.
  • • Neither programmer nor documenter has ever sat behind one-way glass watching a sample user struggling to make sense of it.
    • There is no mechanism to elicit feedback from users, much less to use it to revise the manual in an orderly way.

• Neither programmer nor documenter have been set measurable objectives for the manual.
  • • Nobody even knows what measurable objectives looks like, let alone how to perform the necessary measurements.
    • Someone has been given the task of "documenting" the system, because (like your tax return) it's something that needs to be done.
    • At product release, the writer can point to the mere existence of a manual as evidence for complete success.
    • What might measurable objectives look like?
• Can 50% of readers find the answer to their question within 5 minutes?
• Can 80% of readers find the answer to their question at all?
• Do less than 10% of readers give up in disgust?
• What basic knowledge and skills are necessary for a reader to understand the manual? Do 90% of readers possess these basics?

There's a popular myth, fostered by the sales force, that if a product is usable, it is usable per-se, i.e. as an inherent characteristic. Nobody asks: usable by whom? Likewise nobody asks of a manual: readable by whom? With what background and training? With how much time on their hands?

Readability of a manual is the counterpart of usability of a software product. The two are not only analogous, but contribute materially to each other. Is it such a surprise if an easy-to-use product is easy to explain to someone? And vice-versa?

So what's the answer?

There's no simple one. It's like asking: what's the answer to bugs in program code? But maybe a good way to start is to reconsider one's attitudes to product development and manual writing. To give them comparable status, for a start, if not comparable resources.

It's a loose analogy to talk of a manual as being "code" for the "programming" of a new user. Apart from a rich library of resources to call upon, a program generally has the computer to itself. Or rather today's operating systems are smart enough to let the program think it has. Not so the poor put-upon user, whose mind is a jumble of associated snippets of knowledge and procedures, with no way at the outset of telling what's relevant and what's simply a distraction. That's something that only comes with expertise.

But the human learner is a goal-directed information processing machine, and as such obeys the laws which must be followed by all such machines. There's plenty you can do to farm bugs inside a program you're developing. Recognising this, and desisting from it, takes you a long way down the road to bug-free code.

It follows that failures in comprehension, or misunderstandings arising out of reading a manual, can be thought of as somehow like bugs in a program.

But just as a programmer can blame platform shortcomings for code bugs, so a manual writer can blame the reader, usually her intelligence, for failures in his ability to communicate. There's more justification for it too, since the reader is a highly variable natural product, not engineered to the same high standards of reproducibility as today's computer platforms.

In Berthold Brecht's time, the East German government proclaimed that the people had forfeited the confidence of the government. Brecht pointed out that if that was the case, the government's proper course of action was to dissolve the people and elect a new one. (Something the Syrian regime seems to be taking literally.)

Product developers hoping to enlist the discretionary user (i.e. the user who can turn her back on the product at her own discretion) don't have this kind of privilege or power. If the user doesn't understand the manual, or doesn't find it useful, then it's the manual writer's fault. Period.

So here's a rough-and-ready prescription for more effective manuals, or instructional material in general:

• Recognise that the manual is part of the product just as much as the code. Specifically it is part of the UI (user interface). The more seamlessly you can make it a part, and a co-equal part in terms of priorities and resources, the more the manual will work well with the product and vice-versa. Put like this, you wonder why anyone needs to be told such an obvious thing.

• Ask if you really need a manual, at least as it's traditionally conceived. A hardcopy manual goes out-of-date very quickly, which doesn't help to make it a seamless extension of the UI. Far from the seams beginning to show, wide cracks can appear. There's no button you can press to update, or even destroy, every copy in existence. Downloadable PDFs aren't the answer either. You can hardly warn the customer to re-read every part of the latest copy of the PDF she's ever read, because there's been some changes.

• Maybe an on-line help (OLH) is a better vehicle? But even that is old-fashioned, stemming from the days when the UI was a collection of a dozen or so screens. It was written first, then the OLH bolted on when the UI had more or less stabilized, typically two weeks before product release. Today there's nothing in the way of the UI being the size of Wikipedia, instructing the user in its own use in a way directed at the task the user wants to perform, not the functionality the product manager chooses to provide. Nothing, that is, except the knowledge of how to design or maintain such a beast for integrity plus responsiveness.

J happens to have a community and a wiki, which serves as a knowledge-base. Rather than expend effort in writing and updating user guides and reference manuals of a sort we've grown used to over the last 30 years, might we instead seek to integrate J's IDE with this wiki in as seamless a way as possible? Apple seems to be going this way with Xcode, and what they've achieved is worth a look. Both to appreciate what can be done nowadays, and as a rich source of ideas on how not to go about it.

-- Ian Clark <<DateTime(2014-04-14T15:24:25Z)>>