NYCJUG/2008-05-13
Error messages, common beginner questions, statistics and data visualization for schools, ideas from other languages
Location:: Offices of BEST in Hoboken, NJ
Meeting Summary
We discussed the difficulties and advantages of J error messages - most people had objections. On this topic and others, it may be profitable to begin compiling beginners' questions; those attached here represent a start of this compilation.
On the topic of displaying and saving numeric matrixes, we saw a partial solution that came up when putting together notes on the problem.
We reviewed a proposal on writing a statistics and data visualization package for (middle and high) schools in both J and APL. This proposal comes from a friend in Australia who has gotten some enthusiasm for the idea from teachers at his son's school.
Finally, we talked about the necessity of being open to good ideas from other languages. Some of us need to be reminded that J is not the be-all and end-all of programming languages.
Agenda
Meeting Agenda for NYC JUG 20080513
-----------------------------------
1. Beginner's regatta: are J error messages suitable for beginners? See
"Error Messages For Beginners".
Some other beginner questions: see "Questions from a Beginner".
2. Show-and-tell: displaying numeric matrixes: saving pictures automatically -
see "Save Viewmats.doc".
3. Advanced topics: project idea: Statistics and Data Visualization package
for schools - see "Statistics Project Proposal.doc".
4. Learning and teaching J: an outreach effort: haunting software discussion
boards - see "Reply to Criticism about J code for Pascal Triangle.doc".
Proceedings
Beginner's Regatta
Error Messages for Beginners
Thomas suggested that the short and simple answer to "Are J error messages suitable for beginners?" is "No".
This discussion was prompted by a request on the J forum for help with a short expression that was returning a domain error. It was evident that the poster was looking at parts of the phrase that were not in error because he didn't understand a few basic underpinnings of J error messages. Specifically, he didn't understand that the error location is indicated by three extra spaces in the display of the offending line.
However, the more puzzling aspect of these messages are the general characterization of different errors as, e.g. "domain" or "syntax" error. This is somewhat puzzling to experienced J users as well, though more as a matter of a more abstract dispute about exactly what general type of error pertains in a given situation. For instance, sometimes a syntax error obscures what is more properly a value error. [Does anyone have a simple example of this to illustrate the problem?]
In practice, though, I think each of us is prone to our own characteristic errors and we learn what to look at for our most common ones.
FAQ for Beginners
There seems to be a few very common areas in which J novices have trouble. Some of these are task-oriented, like "How do I read a .csv file into a variable?" whereas others are more general, like "Why is the order of execution so weird?" I propose to start accumulating these questions with an eye to putting together a FAQ for beginners.
Here's a [[[Guides/BeginnerQuestionsCompilation]]: start on this].
Show-and-Tell
We discussed issues with generating and saving images based on numeric matrixes. There's a problem with displaying an image using the handy "viewmat" utility: a sequence of commands to display and save an image will work fine when entered in a J session but will fail to save the image when run from within a script. Apparently the image is not actually displayed until the J session returns to user-input mode so attempting to save it within a script gets only a blank image.
Here's a handout where I explored this problem.
Attempting to Save Viewmats
We can easily, interactively, generate, view, and save a picture displayed with "viewmat": 
load 'playTileMat080402.ijs' require 'viewmat media/platimg' coinsert 'jgl2' glqall=: (|. $ [:glqpixels 0 0&,)@glqwh TD=: 'C:\amisc\J\NYCJUG\200805\' sm0=. (2-/\1 o. i:3j999)+/2 o. i:5j999 viewmat sm0 (glqall'') writeimg TD,'sinSqr.png' 516960
But there's a problem if we attempt to run a sequence of phrases like this in order to save a sequence of pictures programmatically: 
(glqall'') writeimg TD,'sinSqr2.png' [ viewmat sm2=. sm1 +/ . * |:sm1 [ wd 'reset' [ (glqall'') writeimg TD,'sinSqr1.png' [ viewmat sm1=. sm0 +/ . * |:sm0
Yes, nothing shows up in the file.
viewmat sm1=. sm0 +/ . * |:sm0 (glqall'') writeimg TD,'sinSqr1.png' 518400
Dan Bron had run into this same problem as we can see on his J wiki page DanBron/Snippets/PrettyPictures:
Oleg provided a mechanism to save viewmat images to PNG files, so I feel that the following should generate all the image files for this table. However, it does not; I just get a bunch of empty white images.
Debugging indicates that it is probably a repaint/selected window issue. I do not know how to fix it. -- Dan Bron 2007-03-16 13:42:32
Well this steps into the uncharted treacherous territory of sync programmatic execution and async GUI. So I guess to do it like that, the use of timer might help. But that's an overkill.
I recommend using something that is proven and supported like the animate addon. You can designate each picture as a step in the animation. And all will be saved nicely in a series of files. I guess you could take it further by making code that will convert your boxed list of picture verbs into a script for animate automatically. Or have just one new animate script that will accept a list of picture verbs in some form and derive number of steps, etc. Maybe an Options dialog. -- Oleg Kobchenko 2007-03-16 14:35:59
One Solution
One way to overcome this problem is to avoid using "viewmat":
load 'logo' NB. Show the palette: ADJPAL viewmat i. 16 16 (glqall'') writeimg TD,'ADJPAL.png' 810000
$mm0=. +/~1 o. (o. 0.5)+i:3j999 1000 1000 ADJPAL viewmat mm0 setsize_jviewmat_ 1000 1000 (glqall'') writeimg TD,'mm0usual.png' 4000000
Now use this palette with functions from the "images" library: 
NB. Need to scale matrix to integers 0 to 255:
(<./,>./),mm0
_1.979985 1.999991
sc00FF
[: <. 0.5 + 0 255 scaleNums ]
$ADJPAL{~sc00FF mm0
1000 1000 3
(ADJPAL{~sc00FF mm0) write_image TD,'mm0wi.jpg'
47094
A Better Solution
David Mitchell came up with a fix for this problem: redefine "vmrun" in the "jviewmat" namespace.
from David Mitchell <davidmitchell@att.net> reply-to Programming forum <programming@jsoftware.com> to Programming forum <programming@jsoftware.com> date Tue, Dec 16, 2008 at 9:12 AM subject Re: [Jprogramming] savemat_jviewmat_ mailing list programming.jsoftware.com Filter messages from this mailing list Perhaps this is too simple or has side effects I haven't encountered, but it seems to work for me open'viewmat' change NB. ========================================================= vmrun=: 4 : 0 ... wd 'pmovex ',": 0 >. fx,fy,dif + fw,fh wd 'pshow' to NB. ========================================================= vmrun=: 4 : 0 ... wd 'pmovex ',": 0 >. fx,fy,dif + fw,fh jvm_show '' NB. DM 2008 12 16 wd 'pshow' Save and load viewmat.ijs . Run test_viewmat'' . On Vista 32, for me, Viewmat now displays correctly and saves a good BMP. -- Regards, David Mitchell
This change can be made to the file "viewmat.ijs" in the ~system/classes/view directory. This change has not been thoroughly tested for absence of side-effects.
Advanced Topics
Learning and Teaching J
Lately, as part of my SIGAPL volunteer outreach effort, I've been haunting software discussion boards with the idea of raising the profile of array-processing languages, J in particular. It's interesting to see how much overhead people are used to even when doing very simple things, so I take the opportunity to point out how incredibly simple it would be in J.
In order to avoid being only obnoxious, I also try to help them with their problem in their language of choice (though it may be the language foisted on them by programming class requirements). Recently, I got a rather vehement response to my nudging and I attempted to reply to it as nicely as I could. The topic was how to generate rows of Pascal's triangle in order to have the coefficients of a polynomial expansion; the original attempt was in C.
I present here a summary of the discussion this provoked because it illustrates how thinking about the problem in J can help with the formulation of a solution even in another language. John raised the point here that we should keep in mind how J is not the Ultimate Solution and that good programmers come up with amazing solutions using all sorts of tools; he also had some critical remarks about the piece of Haskell code presented by the other party in this exchange.
Reply to J Criticism about Pascal’s Triangle Code
In a discussion forum about writing code to generate Pascal’s triangle, I used the opportunity to promote J. The following response was posted:
That's because J is deliberately cryptic and hides all the hard work behind pre-written combinators and functions, in this case ! being "choose". Given the equivalent of these two defined in Haskell:
comb n k = foldl' (\x y -> ((n - x + 1) * y) `div` x) 1 [1..k] table f xs ys = foldr ((:) . (`map` ys) . f) [] xs[/code]
Then producing Pascal's triangle is as simple as join (table comb) [0..]
But overall a much simpler solution is
iterate (\xs -> zipWith (+) (0:xs) (xs++[0])) [1]
My reply to this follows: I suppose anything is cryptic if you make no effort to understand it.
Even so, your statement is very odd. Apparently you are under the mis-apprehension that your failure to understand is the fault of a succinct, powerful notation and this means that the language was "deliberately" designed to be hard to understand.
Why would you even think something like this? I mean, why would someone deliberately make something difficult to understand? Most people I know strive for clarity. Do you think that "1+1" is cryptic because it uses a symbol "+" instead of the word "plus"? I suppose it is if you don't know math.
The other odd thing you seem to imply is that avoiding hard, unnecessary work is a bad thing. So, you're against the idea of carefully designed, well-tested, powerful, pre-written functions that you can combine in a clear, logically consistent manner?
This is all the more puzzling as your initial answer
join (table comb) [0..]
apparently very closely resembles the J solution of
i. !/ i.
because each "i." generates a vector of consecutive integers and "/" - similarly to (table comb) - combines these vectors in a table using "!". The "!" function, with a right and left arguments x and y returns the number of ways x things can be chosen from y possibilities - pretty much a direct statement of what you are doing when determining the coefficient of a polynomial expansion.
There are, of course other ways to do this more algorithmically and less mathematically as you appear to show in your final example. I say "appear" because that example is rather cryptic insofar as it uses the pre-written function "zipWith".
In J, I could write something I presume is similar to this:
pascalNewRow=: 3 : '1,(2+/\y),1'
based on the following rule for generating a new row of the triangle from the previous one: start with 1; add together adjacent pairs for the middle section, and end with 1. The above expression "2+/\" adds together each adjacent 2 numbers across the first dimension of an array; the "3 :" defines a function. Once you know that, the code is a straightforward expression of the rule.
Similarly to the final expression in Haskell, we iterate the function "pascalNewRow" by using an iterator expressed by some symbols, "^:" instead of the word "iterate". So, the expression
pascalNewRow^:10 ] 1
returns the first ten rows of Pascal's triangle (starting with a "seed" of 1).
I find this much less cryptic than the Haskell expression but, of course, I've made the effort to understand it.
In any case, this has wandered far off topic. Why don't I conclude by giving a solution to the original problem? Taking my J code as a template, start by defining a function to add adjacent pairs of a vector:
int *addAdjacentPairs( int vlen, int *vec)
{ int vc, win2[2];
win2[0]=0;
win2[1]=vec[0];
for(vc=0;vc<=vlen;vc++)
{ vec[vc]=win2[0]+win2[1];
win2[0]=win2[1];
win2[1]=vec[vc+1];
}
// Danger: returned vector is 1 longer than input: allocate accordingly.
return 0;
}
and simply use it in a loop:
// PascalTriangle.c: print some rows of Pascal's triangle.
#include<stdio.h>
int main(void)
{ int vec[21];
int ii,vc,nn;
printf("How many lines (up to 20) do you want? ");
scanf("%d",&nn);
// Should verify result and handle errors but this is why I don't like these
// languages that don't handle arrays natively so I won't bother.
printf("\n");
for(ii=0;ii<nn;ii++)
{ vec[ii]=1;
for(vc=0;vc<=ii;vc++) printf("%i ",vec[vc]);
printf("\n");
addAdjacentPairs( ii, vec);
}
return 0;
}
This provides a nice example of how solving the problem with a high-level notation leads to a more elegant solution even in a lower-level language.
Scan of Meeting Notes
