Vocabulary/Unreadability
The alleged unreadability of J - and what to do about it
This essay is written to be read by complete newcomers to the world of J. It assumes little or no knowledge of the J language.
The other day I asked an expert in C to take a look at J, merely to show him in what environment I hoped to use his incredibly useful utility for the Raspberry Pi. His reply was caustic: "Looks worse that PERL!"
I was stung by his casual dismissal of a language dear to my heart, but on reflection I had to admit that we J-ers do bring it on ourselves. The Wikipedia article on J is like J documentation in general: clean, terse, unassailably logical. Mr Spock might say of it "It's a programming language, Jim, but not as we know it." But most programmers would empathize with the Russian acrobat interviewed on film in his first year at the Cirque du Soleil — "I have been here since six months - and already I do not understand everything!!!"
It's a common criticism that J code is "unreadable". But to have any meaning, this assertion needs qualifying:
"Unreadable" by whom, and for what purpose?
J is unreadable by a 17th century scholar
In Fermat's day, mathematicians would state a theorem formally like this (usually in Latin):
For a right triangle, the square on the hypotenuse is equal to the sum of the squares on the other two sides.
It might well baffle Fermat to see the same theorem expressed in J like this:
isRight=: ([: *: {:) = ([: +/ [: *: }:)
isRight 3 4 5
1
isRight 3 4 6
0
But if asked to consider what purpose the formalism serves, and how well it serves it, Fermat might come to agree that the J definition better serves the purpose of driving an engine to test whether a given triangle is indeed right (i.e. right-angled).
Yes, Fermat (1601-1665) would have known about engines to automate complex tasks ("mills" in middle English, "moulins" in French), whether grinding corn, working novelty clocks or making music. Talking heads were legendary. And Fermat would have known about the Pascaline (1642) - essentially the calculating machine of the 20th century.
BASIC is unreadable by a 17th century scholar
Compare this with BASIC. This too is unreadable by a 17th century scholar — or indeed by anyone else who lacks essential specialized knowledge.
The original Dartmouth BASIC (Kemeny and Kurtz) was introduced in 1964 at Dartmouth College, USA, to provide computer access to non-science students. It was a language designed at the outset to be readable by non-specialists in IT.
But is it really that "readable"?
Consider this statement:
10 PRINT CHR$(205.5+RND(1)); : GOTO 10
To understand it properly, i.e. to predict how it behaves when executed, you need specialized knowledge, particularly of the PRINT statement. What does that colon do? What about the semicolon? These things cannot be guessed by looking up PRINT in an English dictionary — much less CHR$, RND or GOTO, which won't even be found.
Is the fabled readability of BASIC just that: a fable? An illusion brought on by seeing a familiar word or two?
Why J, like BASIC, needs specialized knowledge to read it
J has many powerful features to allow readable code to be written. But these features don't stop you writing obscure code. That can be said of any computer language, even BASIC, as we've shown above.
However you do need specialized knowledge to read most published J, because it hasn't often been the coder's intention to show a friendly face. Rather to exhibit J's advanced features, above all its terseness.
The benefits claimed for terseness in J are broadly those claimed by modern mathematical notation for scientific work, viz. that a formula like:
h² = a² + b²
has operational advantages over a verbose formula like The square on the hypotenuse is equal to the sum of the squares on the other two sides.
Redefining our triangle test to enhance its readability
Let's take the trouble to define isRight in a more open way, which Fermat himself (had he spoken English) might have readily grasped:
VERB=: 13
sum=: +/
of=: ]
on=: ]
the=: ]
square=: *:
squares=: *:
hypotenuse=: {:
otherTwoSides=: }:
LHS=: VERB : 'the square on the hypotenuse of y'
RHS=: VERB : 'the sum of the squares on the otherTwoSides of y'
isRight=: LHS=RHS
Load the above script into J and test it with:
isRight 3 4 5 1 isRight 3 4 6 0
Note that the verbs: of on the merely return the value of their y-argument. They are nothing but noise-words, contributing nothing that matters to the definition, with the possible exception of readability.
Or possibly not.
So let us omit them and redefine isRight as follows...
LHS=: VERB : 'square hypotenuse y' RHS=: VERB : 'sum squares otherTwoSides y' isRight=: LHS=RHS
Now let's use the J primitive Fix (f.) to "compile" the definition of isRight ...
isRight f.
([: *: {:) = [: +/ [: *: }:
The phrase generated by f. turns out to be our original definition of isRight . The only difference is the omission of a redundant pair of brackets surrounding the phrase on the right hand side (RHS) of the = sign.
Lessons for beginners in writing readable J
The lessons for beginners are:
- use extra parentheses to improve clarity, without overdoing them.
- define bigger verbs from smaller verbs, rather than the final verb all on one line.
- choose verb names which express the intended usage in-context, not the full generality of the underlying primitive(s).
The first two points almost go without saying. But the last point bears some discussion.
J words in the standard library which assist code readability
J has a number of highly generalized primitives, like Rank ("), Power (^:), Explicit (:), Level At (L:), encapsulating principles way beyond most coders' training or experience. The primitive: Dot Product (.) embodies Matrix Theory. The primitive: Sequential Machine (;:) embodies Turing Machine Theory. This need not make such primitives hard to use in simple special cases, provided sensitive simplification is carried out to the coder's level of knowledge.
Such simplification has been achieved in the standard library script stdlib.ijs, which defines the core of the _z_ locale. This locale is visible from every other, including _base_, which is the beginner's natural home locale. So, for a J word like toupper_z_, it looks just as if toupper has been defined in the coder's namespace, without the coder having to do so explicitly.
The upshot is to supply a collection of language secondaries, i.e. collections of words which are not part of the core J language (which has no reserved identifiers), but can be used as if they were.
For J, the language primaries are of course the J primitives themselves.
Example 1: inv (read: invert)
inv=: ^:_1
This is a special case of Power (^:). But the beginner needs to know no more than that it inverts a large class of primitive verbs, providing handy solutions to subtle problems. Like so:
NB. Routine use of Copy (#) ... NB. Remove the space between 'b' and 'c' in 'ab c' 1 1 0 1 # 'ab c' abc NB. Cunning use of Copy (#) together with inv ... NB. Insert a space between 'b' and 'c' in 'abc' 1 1 0 1 #inv 'abc' ab c
Example 2: do
do=: ". do '1+1' 2
This treats its (string) y-argument as a phrase in J syntax for execution. But the primitive: Do (".) happens to have other uses, e.g. to convert numerals in string-form to numbers.
It does no harm to define a separate verb for the latter task to emphasize how it's being used in the current context:
number=: ". x=: number '123.56' x+1 124.56
Example 3: each
each=: &.>
The primitive: Under (&.) and its use in combination with Open (>) is not easily explained to a beginner. But what the resulting verb each does is not hard to grasp.
There is a stdlib verb toupper for use like this:
toupper 'alpha' ALPHA
All items of a list must be of the same type. But J has a type: boxed which effectively lets you combine different types in the same list. The primitive: Words (;:) can split an English sentence into a boxed list:
;: 'alpha beta gamma' +-----+----+-----+ |alpha|beta|gamma| +-----+----+-----+
But as it stands, toupper doesn't appear to work with boxed data...
Of course it doesn't! Verb toupper converts ASCII lowercase alpha characters, nothing else — and that includes boxes.
toupper ;: 'alpha beta gamma' +-----+----+-----+ |alpha|beta|gamma| +-----+----+-----+
However the stdlib word each lets us apply toupper separately to each of the strings inside the boxes:
toupper each ;: 'alpha beta gamma' +-----+----+-----+ |ALPHA|BETA|GAMMA| +-----+----+-----+
J makes it easier than most languages to define meaningful aliases for all its primitives. Likewise simple expressions in these primitives.
It lets you derive a verb taking one argument (a monad) from a verb taking two arguments (a dyad).
It also lets you derive adverbs from conjunctions, and verbs from adverbs.
The two verbs square and squares in our isRight example are a case in point. Both are different names for the same verb, called: Square (*:).
This trick defers having to explain to the reader that Square (*:) will square not only a single number but also each atom of a list of numbers.
Thus a potential source of perplexity to the beginner is turned into a vehicle for imparting a powerful J principle about lists in general. But not until the beginner is ready for it.
Reasons for writing terse J code
Terse J code is hard to maintain on a casual basis. It is hard to follow if you have not (recently) written it yourself. So why do experienced J coders do it?
To re-use the definition of a tried-and-tested utility that won't need reading ever again
Example: deb (Read: "delete extra blanks")
deb=: #~ (+. (1: |. (> </\)))@(' '&~:)
The verb deb is defined in stdlib.ijs and loaded by default into the 'z-'locale. Its behavior is easily investigated and understood without having to read its definition. If you need a copy in your own script (eg for a standalone app) it can be copy-pasted as a single short line.
For idioms that are well-known to experienced J programmers
Example: mean
mean=: +/ % #
To understand how the phrase +/ % # works, you need to know quite a lot of J. But statisticians see it often and can spot its occurrence in the densest code.
There is no need to give it a J name like mean . If forced to choose a name for the phrase, different coders might opt for avg, mean, or even E ("expectation") e.g. as used in Expected value.
Having to choose a name introduces scope for confusion, whereas the idiom itself is unambiguous.
To "fix" or "compile" a word definition
J's terseness makes it easy to publish code samples for use by others, altered or unaltered. Sophisticated scripts can be sent as short email attachments. Sometimes the sender wishes to "set-in-stone" certain J words to signal that these need not (or should not) be altered.
To refine an algorithm
Physicists and engineers appreciate the terseness of mathematical expressions for advanced analysis. The same goes for J-ers who study efficient algorithms. They may wish to avoid names for verbs proliferating in their working namespace, preferring to work with the phrases themselves. This can result in code which terrifies a non-specialist, consisting as it does solely of primitives — the notorious "one-liners".
Eugene McDonnell (At Play With J) gives several examples of one-liners. But he builds them up gradually with careful explanation. This is ideal for explaining an advanced topic to non-specialists (e.g. finding the 10,000,000,000th prime number).
Eugene's book provides a masterclass in the effective use of J in number theory and games design.
To eliminate redundant parentheses
This invites the question: redundant to whom, and for what purpose? To coders familiar with J's parsing rules, all parentheses are redundant unless the J interpreter itself needs them to resolve ambiguity. What's more, parentheses can not only be redundant, they can be wrong. Omitting them avoids the possibility.
That said, omitted parentheses is one of the biggest reasons for beginners finding J code hard to read. A common situation is when a pair of sub-phrases express a symmetry in the problem to be solved, but the right-hand pair has been omitted because J doesn't need them. Putting them back greatly improves clarity.
For instance, let's revisit Pythagoras's Theorem, our original example. This is how it looks when J has "fixed" (f.) the code, or when an expert coder omits redundant parentheses:
isRight=: ([: *: {:) = [: +/ [: *: }:
But we know the Theorem basically states that a given pair of expressions are equal. So it would greatly assist clarity to see the phrase laid out like this:
() = ()
Then we might guess that one of the () represents the square on the hypotenuse and the other the sum of the squares on the other two sides. Replacing the omitted parentheses (and maybe with a bit more knowledge of J) we see our guess is correct:
([: *: {:) = ([: +/ [: *: }:)
J is rich in tools for analyzing phrases, e.g. the 5!: range of Foreigns (system services), plus the library scripts lint.ijs and trace.ijs. They will fully-parenthesize a given phrase, show the order it's broken down for execution, and print out intermediate values.
Nobody criticizes Python for being hard to read by candlelight when written on a gravestone. So why criticize J for being hard to read by a desk-lamp when written down on paper? Most programmers nowadays read computer code on-screen. They let the computer help them read it. In this environment, J code is easy to copy/paste from a PDF document into a J interpreter, where it can be executed, traced and analyzed in many different ways.
Some J code still gets published on paper. But for all the flexibility that confers, it might as well be written on a gravestone in a darkened vault. But even in such an unsuitable environment J has the edge on Python — being generally terser, it is quicker to transcribe by hand onto a screen.
Don't confuse nouns and verbs with variables and functions
We come now to perhaps the hardest thing about J for programmers trained in other languages: its English-grammar metaphor. According to this metaphor, a J script consists of phrases (some of which are called sentences) consisting of words (usually names or primitives) each of which can be one of the four primary parts of speech — noun, verb, adverb, conjunction. It's a surprisingly difficult concept for existing programmers, some of whom misunderstand it for years. Although people who learn J as their first language typically don't see any problem.
We've loosely referred to nouns and verbs, allowing the J newcomer to imagine they are just J's cute names for what other languages call variables and functions. This is not the case. J has good cause to introduce new terms, from which it benefits in economy, flexibility and generality. But the newcomer soon flounders in perplexity if these terms are misunderstood at the outset.
It's not as if they are neologisms. Nor are they novel uses of everyday terms. Noun and verb are familiar concepts to non-programmers and are seldom misunderstood. However to someone who has learned to program in a "Fortran" language, meaning one of the languages descended from Fortran, e.g. Java, C, C++, BASIC, they do seem to get misunderstood. This has a serious impact on learning and reading J.
APL, J's forerunner, has variables and functions just like a "Fortran" language. A variable in a "Fortran" language possesses both a value and an identifier (a name). The value of a variable can change, as it does by executing a statement such as: Z=Z+1 .
Not so, J.
A noun (whether atom or list) is a value, pure and simple. Here are examples of nouns:
'alpha' 1.5 (1 2 3 4 5 6 7)
A noun never changes: though it may be discarded and replaced in use by a different noun. Thus by evaluating the superficially similar J phrase: Z=:Z+1 the name Z is made to mean a new value: viz. the noun obtained from evaluating Z+1 .
Strictly speaking, Z is a pronoun. Executing: Z=: Z+1 has not "altered the value of some variable called Z — it has changed the meaning of the pronoun: Z to mean the new noun produced by evaluating Z+1 . This point cannot be overstressed. Z is not the noun (though J-ers loosely talk of "the noun Z", meaning: the noun referred-to by pronoun Z).
A close analogy to this principle is to be found in Apple® HyperTalk, which has a reserved identifier: it.
The word it is a pronoun in the J sense. And also indeed in the grammar of standard English.
The word it serves as the receptacle for the last value computed. It is used like this:
ask "What is the value?" put it into the card field "display"
Why does this distinction between nouns and pronouns matter?
In several other languages, such as J's cousin APL, identifiers do not change their meaning. Instead values get assigned to variables, which are created if they don't already exist (by a process known as implicit declaration).
Once we have executed any statement using the identifier Z to the left of ←, e.g. as in Z←0 , then Z has thereby been (implicitly) declared as a variable and we cannot then make Z mean a function (and vice-versa) unless we erase the variable first by use of ⎕EX ...
⎕EX 'Z'
∇r←x Z y
...
∇
In J, by contrast, we are free at any time to make the name Z mean any value we like. It needn't be a noun: it can be a verb, an adverb or a conjunction. Note that the terms noun, verb, adverb and conjunction are only ever used to mean a part of speech — they are not properties of "variables" or "functions" (which are not J concepts anyway).
So there is nothing to stop us making Z mean first a noun then a verb in quick succession, like this:
Z=: 99.9 Z=: +/
Strictly speaking we should now call Z a proverb not a pronoun, but this entails no new principle. So we prefer to call Z a name, meaning it to cover pronoun, proverb, pro-adverb and pro-conjunction.
Progressive elimination of intermediate names from a verb definition
What else follows from this principle?
The name Z, once it takes on a "meaning", i.e. a value, can be replaced by that value in any phrase that contains Z, yielding an equivalent phrase. As already said, that value can be a noun, verb, adverb or conjunction.
Example: isRight
In our triangle tester, the 3 names isRight, LHS and RHS have values. We can see these values by typing the names into the J session. A bare name is a valid J phrase, and any phrase so treated is "evaluated" and the value printed out. As already said, it doesn't matter if this value is a noun, verb, adverb or conjunction.
isRight LHS=RHS
Let's replace LHS in the definition of isRight by its value (a verb) like this:
isRight=: ([: square hypotenuse) = RHS
Note: the primitive: Cap ([:) is needed here to tell J not to look for the x-argument to square. (J generated Cap ([:) for us when we first defined LHS and RHS.)
Now do the same for RHS:
isRight=: ([: square hypotenuse) = ([: sum [: squares otherTwoSides)
This shows that the "readability" of a given J phrase is a matter of judiciously choosing names for selected sub-phrases, to suit the knowledge of the person meant to read it. If the said person can read J fluently, then she will have no need for proverbs referring to sub-phrases. A sentence written entirely in primitives will be quite clear to her.
In this way we can eliminate all proverbs, leaving just trains of primitives. Once redundant parentheses are removed, and maybe with some x- and y-arguments swapped around certain verbs, provided they are symmetric, we will end up with the original expression we pictured Fermat finding so baffling:
isRight=: ([: *: {:) = ([: +/ [: *: }:)
And Fix (f.) will compute this for us from the original definition.
Conclusion
J is no less readable than BASIC, say, provided the reader possesses the necessary knowledge about important language features.
This applies equally to BASIC or J. By itself a knowledge of the English language is not enough to understand code in either programming language.
It follows that the apparent "readability" of BASIC is an illusion caused by the reassuring presence of familiar (English) words. Such "readability" does not translate into true comprehension.
It cannot possibly do so, because not all statements (e.g. PRINT) can be understood without specialized knowledge of BASIC which cannot be guessed from an ordinary English dictionary.
However a lot of published J code tends to be less readable than it might be, due to the elimination of verb names and parentheses. But this is not an inherent feature of the language. Indeed J offers many tools and techniques to write easily comprehensible code, if you know your target audience.
- Putting back some "redundant" parentheses can help the eye break down lines of J code into meaningful phrases.
- Replacing selected phrases with judiciously-chosen names can enormously increase readability of code.
- A special case of this is defining verbs as synonyms for the more generalized J primitives, maybe several names for the same primitive.
- The standard library (stdlib) defines many such "cover verbs". These are automatically made available to users in the z-locale.