NYCJUG/2013-02-12

beautiful code, Clojure, Jem, JavaScript, timing C code, physics engine, interactive web, WebGL, literacy, recursion, emacs, ranking levels of J learners, interrupted learning heightens attention

Location:: Heartland

Meeting Agenda for NYCJUG 20130212

1. Beginner's regatta: an outside opinion of beautiful code and some ways
this plays to J's strengths: see "The Exceptional Beauty of the Doom 3D code
with some J addendums.pdf".

The final example in this leads us directly to a comparison of some code in
Clojure and its equivalent in J: see "Drop Nth in Clojure and J + new J8.pdf".


2. Show-and-tell: learning a little about JavaScript: see "Why JavaScript is
a Jem.pdf".

See "Timing C-Code Externally.pdf".

Powerful, client-side graphics and physics engines - see "WebGL Demo with
Canonjs and Threejs-some Code.pdf"


3. Advanced topics: problems with tools - see "Programming Literacy Done Right
is about the Tools.pdf"

See "Recursive programming and partially-applied conjunction discussion on
J-Forum.pdf".


4. Learning, teaching and promoting J, et al.: Should we name the levels of
J learners?  See "Introductory Example - Emacs Learner Rankings -
concentrating on the ranking levels.pdf".

Better learning - see "Interrupted Learning - Zeigarnik Effect.pdf".

Beginner's regatta

What Makes Code Beautiful?

We read an article by Shawn McGrath, creator of the Dyad game engine, about the beauty of the (C) source code for the game Doom. He started by talking how ugly his own code had gotten under deadline pressure.

. In the final six weeks of Dyad development I added over 13k lines . of code. MainMenu.cc ballooned to 24,501 lines. The once-beautiful . source code was a mess riddled with #ifdefs, gratuitous function . pointers, ugly inline SIMD and asm code—I learned a new term: . "code entropy."

Upon searching the net to learn how other large projects organized their code, he discovered that his code "wasn't actually that bad compared to everything else out there!" An exception to this generally sorry state of affairs was the source code for Doom 3.

This inspired him to define what constitutes beauty in code. Some of the general answers he got back were these:

. • Code should be locally coherent and single-functioned: One function should do exactly one thing. It should be clear about what it's doing. . • Local code should explain, or at least hint at the overall system design. . • Code should be self-documenting. Comments should be avoided whenever possible. Comments duplicate work when both writing and reading code: if you need to comment something to make it understandable it should probably be rewritten.

He then went into some detail, with examples from the Doom 3 code, of what makes code beautiful, some of which speak directly to issues J coders would appreciate, other things, not so much. I've re-formulated some of his points to highlight their pertinence to J in particular.

Structure in Data

McGrath admires the lexical parser of Doom, how numerous nouns auxilliary to the code - such as animation and config files - are kept out of the mainstream of the code logic and dealt with by a single, unified parser and lexer. This is analgous to how, in J, we might keep the bodies of large data objects in files or in-line "0 : 0" nouns.

Knowing What Data Might Change

His comments on the use of the "const" keyword are less directly relevant to a J programmer but are insofar as they aim to isolate mutable from immutable data, much the way we hide local information in a verb and attempt to get and return explicit arguments as much as possible.

Appropriate Level of Commenting

His preference for minimal comments should resonate with the natural tendency of J coders toward terseness, though I've often seen this taken too far in the J world. Would it really kill you to use more than one letter for a name once in a while?

Efficient Use of Space

From a J perspective, I was a little amused at his examples of efficient use of vertical space but it was a step in the right direction. He shows this piece of code as an example of not wasting vertical space (from a section of the code called t_stencilShadow::R_ChopWinding()):

for ( i = 0 ; i < in->numVerts ; i++) {
    dot = plan.Distance( in->verts[i] );
    dists[i] = dot;
    if ( dot < -LIGHT_CLIP_EPSILON ) {
        sides[i] = SIDE_BACK;
    } else if ( dot > LIGHT_CLIP_EPSILON ) {
        sides[i] = SIDE_FRONT;
    } else {
        sides[i] = SIDE_ON;
    }
    counts[sides[i]]++;
}

He likes this because it leaves space for him to see the code surrounding this so he can better understand the context of this fragment, an argument familiar to those of us who understand the value of terseness. However, I wonder, even in the context of a vacuous language like C, if that same paragraph would suffer from being rendered even more succinctly, like this:

for( i=0; i<in->numVerts; i++)
{   dists[i]=dot=plane.Distance(in->verts[i]);
    if(      dot < -LIGHT_CLIP_EPSILON){ sides[i]=SIDE_BACK;}
    else if( dot >  LIGHT_CLIP_EPSILON){ sides[i]=SIDE_FRONT;}
    else                               { sides[i]=SIDE_ON;}
    counts[ sides[i]]++;
}

In fact, pushing together and aligning the logic like this gives a clue to how we might make code like this much more properly succinct in a language like J. Hence, we will digress to explore this.

.. A J Digression ...

How might we write something like the afore-mentioned t_stencilShadow::R_ChopWinding() in J?

We could start by noticing there are three cases:

1. “dot” is lower than the range (-LIGHT_CLIP_EPSILON, LIGHT_CLIP_EPSILON), or 2. “dot” is higher than this range, or 3. “dot” is the same as one of the endpoints of this range.

This might lead us to the following three expressions:

   dot=. _10 [ LIGHT_CLIP_EPSILON=: 9
   /:dot,LIGHT_CLIP_EPSILON*_1 1      NB. SIDE_BACK
0 1 2
   /:dot,LIGHT_CLIP_EPSILON*_1 1      NB. SIDE_FRONT
1 2 0
   dot=. LIGHT_CLIP_EPSILON           NB. SIDE_ON
   /:dot,LIGHT_CLIP_EPSILON*_1 1
1 0 2

Combining the three cases by treating each combination as digits of a base-3 number allows us to compress each result into a distinct number:

   3#.0 1 2, 1 2 0,:1 0 2
5 15 11

But these numbers don’t appear particularly meaningful. Could we do better?

First, let’s assign the results at which we’re aiming into single digit numbers since that’s probably what they really are and this is easier to deal with.

Now, consider the two expressions dot < -LIGHT_CLIP_EPSILON and dot > LIGHT_CLIP_EPSILON. As we’ve already hinted, these reduce to the simpler concept of : is dot outside the range plus-or-minus LIGHT_CLIP_EPSILON?

Notice that we can put the first conditional into the same form as the second if we negate both sides of the inequality and flip it around, giving us

. (-dot) > --LIGHT_CLIP_EPSILON or . (-dot) > LIGHT_CLIP_EPSILON

So we could write this part in J like this (shown here with dot assigned to test each of our three cases):

   (dot*_1 1)>/LIGHT_CLIP_EPSILON [ dot=. _10
1 0
   (dot*_1 1)>/LIGHT_CLIP_EPSILON [ dot=. 10
0 1
   (dot*_1 1)>/LIGHT_CLIP_EPSILON [ dot=. LIGHT_CLIP_EPSILON
0 0

This gives us Boolean results that map nicely into the binary representations of two, one, and zero. Not only that, since J lives and breathes arrays, we don’t have to compare each scalar in a loop. This allows us to build a truly succinct statement of the logic for the loop so proudly reduced from 18 to 11 lines in the original essay.

   2 1 0{'SIDE_BACK';'SIDE_FRONT';'SIDE_ON'  NB. Put these in proper order.
+-------+----------+---------+
|SIDE_ON|SIDE_FRONT|SIDE_BACK|
+-------+----------+---------+

Set verts to be some test case values and make sure these select what's expected.

   verts=. _10 10 9 _9
   (verts*/_1 1)>/LIGHT_CLIP_EPSILON
1 0
0 1
0 0
0 0
   2#.(verts*/_1 1)>/LIGHT_CLIP_EPSILON
2 1 0 0

Finally, combine this into a single expression:

   ('SIDE_BACK';'SIDE_FRONT';'SIDE_ON'){~2#.(verts*/_1 1)>/LIGHT_CLIP_EPSILON
+-------+----------+---------+---------+
|SIDE_ON|SIDE_FRONT|SIDE_BACK|SIDE_BACK|
+-------+----------+---------+---------+

Drop-nth in Clojure

In which we compare a Clojure implementation of a simple concept to one in J.

[From http://batsov.com/articles/2013/01/20/drop-nth-in-clojure/]

For some reason the standard Clojure library doesn’t have a drop-nth function (although it has take-nth). Luckily implementing it is trivial:

 1 (defn drop-nth			
 2   [n coll]				
 3   (->> coll				
 4        (map vector (iterate inc 1))	
 5        (remove #(zero? (mod (first %) n)))
 6        (map second)))

Let’s try it out:

 1 (drop-nth 3 (range 1 10))		
 2 ;; => (1 2 4 5 7 8)			
 3 (drop-nth 5 (range 1 10))		
 4 ;; => (1 2 3 4 6 7 8 9)			
 5 (drop-nth 5 (range 1 20))		
 6 ;; => (1 2 3 4 6 7 8 9 11 12 13 14 16 17 18 19)

Looks good to me. Hopefully it will be of some use to someone.

Drop-nth in J

   5 (13 : 'y#~0~:x|y') i.20
1 2 3 4 6 7 8 9 11 12 13 14 16 17 18 19
   5 (]#~0:~:[|]) i.20
1 2 3 4 6 7 8 9 11 12 13 14 16 17 18 19
   3 (]#~0:~:[|]) i.10
1 2 4 5 7 8

How This Works

The tacit J expression ]#~0:~:[|] works in much the same fashion as does the Clojure example: it compresses out (#) elements whose modulus (|)base x ([) value is zero (0:); rather, it keeps the elements whose base x modulus is not zero (0: ~:).

A little reflection may lead us to notice that this is not as general as it might be: it mimics the Clojure example's implicit assumption that our (y) argument is always (a range of) integers. However, in the absence of a reason to make it more general, it's better to leave it as it is.

J801 beta Available for All Platforms

[From JSoftware, announced on the J Forums]

The main reason for the beta is to simplify development of the Qt desktop.

This replaces the earlier J7 GTK desktop, which will not be supported in J8. At the same time, the desktop wd interface is updated to match Qt layouts.

For those not making essential use of either J6 wd or J7 GTK, the new beta will be a straightforward upgrade. In particular, the J engine, J console and JHS are the same in both J7 and J8.

This beta also brings a new development focus. J7 was such a major change from J6 that we expected many users would stay with J6, and we supported both versions in updates to the base system and addons. In future, we will focus development efforts only on J8, and not try to keep J6 or J7 up to date.

Show-and-tell

In Praise of JavaScript

We looked at some reasons why "JavaScript is a much under-rated language" - see Why JavaScript is a Jem.pdf. In spite of swiping the term "Jem" that we Jedi already use for J pearls of code, the article makes some good points.

Of particular interest to the J community is the language's dynamic nature, that JavaScript objects are completely plastic, implying a rather weak notion of type. This is also reflected in the language's minority approach to object orientation that more closely allies it with a language like J: JavaScript allows you to create an object without first defining a class to act as a template.

One interesting point is the author's assertion that JavaScript is a functional language (in addition to being an object-oriented one). He states that "a function is an object that carries a special method that is executed by the invoke operator, i.e. a pair of round brackets ()" and that functions in JavaScript are first-class objects. However, he also concedes that JavaScript lacks a lot of very basic functional programming features. He also mentions the importance and usefulness of closures in JavaScript but without going into any detail about what these are other than something that "most beginners don't understand".

Another article, part of the same document, elaborates on JavaScript arrays. The language's support for arrays appears to be a little better than that of other non-APL languages, which is to say, somewhat pitiful. However, this opens the possibility for J programmers to leverage our array-handling expertise by, say, generating JavaScript code to take advantage of that language's ubiquity and finesse on the web browser side of the world.

Using J as a Timer

File:Timing C-Code Externally.pdf shows how we can use J's "shell" facility (from the "task" script) to time an external, compiled module. The motivation for this was a question about how the performance of repeated subtraction compares to that of straight-forward division in C code. Since C is so well-optimized for simple, looping solutions, even the overhead of very many subtractions may be competitive with division (in the integer realm) as division is known to be relatively expensive.

Of course, there are ways to build timing into C code. However, since it's been quite a while since I've done this - and it was on a different architecture than the one now available to me - I figured it would be faster to write stand-alone C modules and time them externally using J. This seems to work quite well and showed that repeated subtraction appears to take the same amount of time as division for the examples chosen.

Graphics and Physics Engines

We looked at an amazing demo of some powerful, client-side graphics and physics engines. Take a look at the real-time rendering in this web-based game - http://granular.cs.umu.se/cannon.js/examples/threejs_fps.html - to get an idea of what's possible with a relatively small amount (about four pages) of code.

Advanced topics

We discussed implications of the tools we use and the implications of programming as literacy - see http://www.xconomy.com/san-francisco/2013/01/08/programming-literacy-done-right-its-about-the-tools/ or read File:Programming Literacy Done Right is about the Tools.pdf

Recursive Programming in J

Personally, this is something I try to avoid because of stack limitations, but it is a handy technique sometimes. There was a discussion of it on the J Forum, summarized File:Recursive programming and partially-applied conjunction discussion on J-Forum.pdf.

Learning, teaching and promoting J

We looked at one idea for names for the levels of Emacs programmers File:Introductory Example - Emacs Learner Rankings - concentrating on the ranking levels.pdf. I don't know how constructive this is but it might be fun to come up with a similar hierarchy for J aspirants.

Zeigarnik Effect

The Zeigarnik Effect is the phenomenon that people better remember unfinished tasks than completed ones - see File:Interrupted Learning - Zeigarnik Effect.pdf.

Attachments

File:WebGL Demo with Canonjs and Threejs-some Code.pdf