NYCJUG/2009-03-10

NYCJUG meeting, code simplification, J-like, "point and figure" charting, candlestick charting, built-in documentation, code management, JOD, statistics, probability, inverse cumulative normal distribution, APL film

NYC JUG Meeting of 20090310

We looked an an example of code, based on code we looked at last month, that was made more J-like but still reflected more of a conventional approach to coding. Then we investigated some ideas for the holy grail of "self-documenting" code by leaving "development trails", or remnants of how code was put together. We also answered some questions about database design and how to work with tables of data in a clear and self-documented manner.

We talked about some "coding in the large" issues of project management - Ken had high praise for John Baker's "JOD" - J Object Dictionary. We also learned about the recent project to assemble a J statistical library and examined some code to calculated the inverse of the cumulative normal distribution with an eye to its suitability to being expanded to a general inverse adverb.

Finally, we talked about Catherine Lathwell's recent announcement of work on an APL film and what we might contribute to this effort.

Meeting Agenda for NYC JUG 20090310

1. Beginner's regatta: simplifying non-J-like code to be more J-like: see
"pfchart.ijs" and "Understanding Point & Figure Charting.doc"


2. Show-and-tell: does leaving your development work with the code aid
understanding?  See "Documentation by Showing Development.doc".


3. Advanced topics: stats lib project - see "Normal Distributions in J.doc".


4. Learning, teaching and promoting J: APL film?  See "A Programming
Language Film.doc".

Proceedings

Beginner's regatta

We examined a re-working of code for "candlestick" charting that we had looked at last month. We had noted numerous characteristics of this beginner's attempt that had various characteristics of code that wasn't very J-like such as a scalar orientation, unnecessary looping, and repeated blocks of code, all of which led to a program which looked to be far larger (about 200 lines) than necessary. In the meantime, a member of the J-forum made a stab at improving the code.

The improved code retained much of the non-J-like gross structure of the original code, including its major loop, but weighed in at about half the size by replacing repeated code blocks with a parameterized version of them. The resulting code is a step in the right direction but, at about 100 lines, still seems far too long and loopy for what it accomplishes. However, it's hard to get motivated to improve code when you're skeptical of its purpose. The sort of analysis to which this method belongs - technical analysis - is not given much weight by most serious researchers in finance as it has little compelling theoretical basis and has not proven effective according to any reputable study I've seen.

However, if someone else is interested in further improving this code as pedagogical exercise, information on this kind of charting can be found here.

However, like much technical analysis, it does produce interesting-looking pictures as we can see here in this example of the program output (in blue) of the Dow-Jones Index closing prices for 2007 and 2008. The simple time-series chart of this same series is inset for comparison.

Show-and-tell

We looked at an example of documenting code by "leaving traces" - showing parts of the session in which the code was developed to include examples of what a sample data structure looks like and what problems led to certain approaches or instances of data-cleaning.

This got us onto the topic of cleaning user input and how this seems to consume an inordinate amount of effort, especially when coding a module for use by others. Someone claimed that something like 40% of the time spent coding is geared toward cleaning and checking the input. This is especially true when processing input from sources outside a system's control, such as spreadsheets filled in by various people.

Advanced topics

Ken L. has been using John Baker's "JOD" - J Object Dictionary - as a way to manage code and has been very happy with how it works. There's a print-on-demand book available as documentation as well as the usual, on-line kind right here on the J-Wiki.

Learning and Teaching J

Catherine Lathwell has undertaken to document some of the history of APL on film. She plans to give a talk to APL-BUG on May 11 in San Francisco at the Computer History Museum there: http://aprogramminglanguage.wordpress.com/.

Scan of Meeting Notes