NYCJUG/2005-03-08
public domain licensing, persistence of information, "Save Work" project, APL/J Rosetta Stone, date conversions, design considerations, learning APL and similar languages
Agenda
0. Group project: legalities - draft agreement
1. Group project: report on assignments
a. Harteg - initial screen
b. Dan - persistence of information
c. Jim - library report
d. John - thoughts on desired features
e. Devon - design considerations - see topic 2 below.
2. Assignments: going forward - what next?
a. Extreme programming -> unit tests _before_ code.
Can we combine screen persistence with automating user input for
testing?
b. Integration - use J projects? something else?
3. Learning and creating learning materials: a general framework
How to structure it? As a FAQ? Hypertext?
Suggested directory structure: what are main categories?
e.g.
Main Project
Utilities
Design documents (data dictionary?)
Documentation (Chris Lett)
Feedback to J Software
Web site, public release?
4. Howard Peele's talk - date and format?
"Best times for me seem to be: end of April, end of May, or early June.
Also:
Who is your audience? How many would you expect?"
Bonus Facts:
NB. The significance of the number 21 in a 21-gun salute is derived
NB. from adding the digits of 1776.
+/1 7 7 6
21
NB. A quick check I use to test the validity of a prime-number generator:
-/p: i.499
1776
Proceedings
[Everyone at the meeting received a letter from me a few days before outlining goals and establishing a framework for our group project. I'll point to it here as a separate document as it is rather lengthy: [1]
All present agreed that the group project should be released as a public domain work. We looked at some example agreements such as the "GNU General Public License" (File:Gpl.html) and a draft for us based on licenses with similar objectives (File:GroupAgreement.txt). We also considered the open-source licenses (File:OpenSource cpl1.0.php.html) but are wary of any restriction that would prevent any of us from commercializing this work if any of us want to. Here is an overview of open source licensing: File:OverviewOfOpenSourceLicenses opensource.html.
We agreed that we wanted to protect our use of any group code so that no one could license it and forbid us to use our own code.
Dan has been working on the "persistence of information" problem. Since the last meeting, I've spent some time perusing his "lexical closure" code - I still don't know what this means - but it allows you to handle name-value pairs specified in a file. The values look to be arbitrarily complex J data structures though maybe better suited to generating data than storing arbitrary data? I'm not sure about this.
We talked a little about being able to have config files with partial data if necessary. Dan's code (which he may have distributed the meeting before) has a nice default value mechanism which would be useful here.
"Save Work" Project
Where we want to go with this: we want software that will naturally conform itself to how each particular user usually uses it. Using the "save work" example, a user might have one or two or more typical, customized backup scenarios that are run at different frequencies. So, a quick, every-day backup should perhaps exclude certain large database files whereas a weekly, fuller backup would include these.
We had discussed an average user to whom the bulk of our interface would be targeted. I'm basing this on a composite of some friends of mine who use computers but are not computer experts. Our primary user is called "Vicki".
There was some speculation that the only things an average user, our "Vicki" perhaps, would ever back up are in her "My Documents" directory. This also raised the question of whether people know, or should have to know, where their files are.
Reports from Members
Jim reported on finding many more utility libraries than he knew existed before. We still lack principles on which to organize this. There has been discussion on the J Forum over the past few days on this topic - see the note from Neville Holmes "J for Beginners" on 4/4 and Raul D. Miller's reply on 4/5. Jim has suggested putting in a mention of our work here - maybe a little later.
Jim also provided a draft of a "Rosetta Stone": in this case, a rough translation between APL and J: File:APLJ Rosetta.rtf.
Also, let's not forget Chris Lett who, while not able to attend meetings in person, has graciously offered to document whatever we come up with.
John spent some time looking at the "parseDir" code I'd provided and had some good suggestions for organizing the hodge-podge of date conversion functions I've evolved over time. He'll look at something using regular expressions to parse dates in different formats.
Project Design
I spoke a little about design considerations - I'd like to incorporate the "Extreme Programming" idea of unit tests before code - well, maybe alongside code. Dan's lexical closure code provides an example of something of what I'd be looking for in his example section at the end of the code. I'm more interested in working examples than comprehensive code coverage. Dan mentioned how Roger Hui writes test scripts for J itself using assertions.
We looked at a couple of essays from Paul Graham on what good design means: [2] and [3]
We didn't agree on whether to use the J projects facility or not - does anyone have any experience with this?
When we talked about a general framework for learning and creating learning materials, Dan mentioned YAML - Yet Another Meta-Language - as a possibility for specifying complex data structures. We probably also need to explore the various options of keyfiles vs. APL workspaces vs. 5!:5 (linear representation) or 3!:5 (internal J representation) for storing state and work in progress. We may have also looked at something called a "Wiki" as a way to pool our work: File:WikiTool.ps.
I showed the group a type of file-directory linking that the OS (at least XP and NT) knows about: if you copy the file, the associated directory automatically gets copied as well. I don't know if this is really useful or just something that breaks cross-OS compatibility.
Learning APL and Similar Languages
There was an interesting exchange on the J Forum recently talking about understanding code and the relation between human language and computer language comprehension.
It looks like the thread was started by this response to an earlier message:
> From Alain Miville de Chêne <amd@INFODEV.CA> > Date 02/22/2005 07:54 AM > To JFORUM@PEACH.EASE.LSOFT.COM > Subject Re: [Jforum] inter J compiler : formal and informal names > > Alan Stebbens wrote: > ... >> So, which is easier to parse, read and comprehend? >> >> a=.,.<;._1 ' ',"$123.45 $345.67 ($99.99)" >> >> or >> >> ravel box cut ' ','$123.45 $345.67 ($99.99)'; >> > or > > a=. ,. < ;._1 ' ', '$123.45 $345.67 ($99.99)' > > make it stand out visually. The ;._1 is explicitely shown as a unit.
This prompted other responses and provoked some reminisces such as the following.
In reply to this message from Don Guinn:
Don Guinn <donguinn@PVTNETWORKS.NET>02/22/2005 05:11 PM To JFORUM@PEACH.EASE.LSOFT.COM Subject Re: [Jforum] inter J compiler : formal and informal names ... What I meant was that at first the compactness of a J definition is a little overwhelming until I get into it. Where the Basic code is so verbose that I can't see but a little piece at a time so it feels like I am accomplishing something even though I am actually moving very slowly. It's kind of like going 70 miles an hour in a sports car where you are close to the ground seems fast, where the same speed in a motorhome seems slow because you are farther from the road and can see so much more. By the way, in a motorhome freeways look like country roads! Could this be a parallel that in J or APL we think of solving some problem as almost trivial where solving that same problem in Basic or C as a major undertaking? ...
Mr. Gauthier wrote:
Date: Tue, 22 Feb 2005 18:09:33 -0600 From: PGauthier@CHECKFREE.COM Add to Address BookAdd to Address Book Subject: Re: [Jforum] inter J compiler : formal and informal names To: JFORUM@PEACH.EASE.LSOFT.COM I started with APL in 1974. I was always stunned, every time I touched upon other languages, to see people loving the contortions they had to go through to do anything more than simple "demo style" programming. So to me it's the other way around, I kept asking myself the question: "But what is it they enjoy in XXX language ?" It would be so simple to do it in APL/J. I did find good reasons to take very specific pieces of extensively used code and shovel it in an AP for obvious performance/scalability reasons. Thus programming in "C" or machine language from time to time (after having prototyped it in APL/J). Now, when I hear/read about the difficulties to read/understand APL/J I usually can classify it in two broad categories: A- The person is not a native speaker and it's not his/her first exposure to a programming language but has a genuine need for APL/J. B- The person is not looking at understanding/resolving problems by finding their primitives and would rather have a "point and click" RAD. Those in the A category need to "bite the bullet" and start developing their "native speaking" with the books/examples available at www.jsoftware.com. Those in the B category will probably not find a lot of motivation using APL/J. In other walks of life I am the person A and sometimes the person B. Interest/motivation vary considerably from one person to the next. It maybe so that VB or some other variations is the tools you need. My suggestion is to try to focus on some basic questions: - Do I need to understand and resolved problems in a complete and general way ? - Do I long for a clear, not ambiguous tool of though to help me resolve problems ? - Are the problems I want to resolve already resolved ? - Do I want to invest a sizeable chunk of my time in the acquisition of a mathematical nomenclature that will help me resolve/understand problems ? - What will I gain from learning a general mathematical notation which is executable on a computer ? etc. To conclude, there is one thing I am sure of for myself. APL/J are very easy/simple to read and developing with them is a breeze and has been for 30 years. When I have a need to understand a "new concept" I usually try to find some "top gun" code and "bite the bullet". BTW, J Phrases are like good exercises to master (after you have the basics covered). Whatever it is you are looking for, I wish you find it. Paul
Raul Miller replied on the question of intrinsic ease of understanding code:
Date: Tue, 22 Feb 2005 18:29:19 -0500 From: "Miller, Raul D" <rdmiller@USATODAY.COM> Subject: Re: [Jforum] inter J compiler : formal and informal names To: JFORUM@PEACH.EASE.LSOFT.COM I think the issue of how "easy" or "hard" it is to read someone else's code depends on a lot of factors. One factor is: how well was the code written? Another factor is: how well do you understand the problem being solved? Another factor has to do with your expectations: what do you expect to get out of reading the code. Just for fun, compare j's dot product with am analogous expression written in sql. J: dot=: +/ .* SQL: select t1.i, t2.k, sum(t1.v*t2.v) as v from t1 inner join t2 on t1.j=t2.j group by t1.i, t2.k The J expression is more general, but for some people the sql expression is "simpler to understand". Why? Does the sql expression give greater insight into the problem? In my opinion: not really. But the sql expression does nail down some specifics. You know, looking at the sql expression that you're dealing with two specific two-dimensional tables. And, you know some of the column names. You don't really get a better sense of what's going on, but you know a bunch of details, and that makes some people more comfortable. Of course, you could have used a different J expression: t1 +/ .* t2 This is closer to the sql expression, since it's now specific to two tables. But in J t1 and t2 could be anything -- at some point, t1 could be a scalar, later, it could be a vector, then a rank 2 array, then rank 3, ... SQL also has some flexibility, but not it's not flexible like J. Or, you could write something like this in VB. I'm not comfortable with VB, so I won't attempt it, but I'm pretty sure the code would be fairly concrete (like the sql expression) and fairly verbose (like the SQL expression). And, again, there would be lots of little details to latch on to, that lets a person glibly say "I understand that", even if they don't really know the architecture or the design of the code very well. Alternatively, when someone is talking about "easy" vs. "hard", they can mean many different things. -- Raul
Bill Harris had this to say, comparing the effort of understanding common language to the effort of understanding code.
Date: Tue, 22 Feb 2005 15:08:06 -0800 From: "Bill Harris" <bill_harris@FACILITATEDSYSTEMS.COM> Subject: Re: [Jforum] inter J compiler : formal and informal names To: JFORUM@PEACH.EASE.LSOFT.COM Stuart Bruff <stuartbruff@YAHOO.CO.UK> writes: > I didn't see this as expressing a 'preference', just a comment on the > degree of mental concentration required to interpret a J statement. J > is akin to translating from Russian or Japanese to English and then > trying to understand the content. VB is akin to translating from a > local dialect, much less mental effort involved and it's tantamount to > a single mental task. Once you're fluent in a language, the > "translation" effort drops off and it becomes easier to read and > understand, but it's pretty wearing if you're a novice or not familiar > with the specialist vocabularies. Stuart, I think you nailed it. When I moved to Germany to work (years back), I had studied German in high school and college, but my process for reading, writing, hearing, or speaking involved translating from one language to another. The first few months were really tough mentally -- I'd go home at night with headaches from working actively and only in German (i.e., translating) for 8+ hours, but I kept to two principles: only deal in German (i.e., refuse to deal in English unless I had to) and look up new words and grammatical constructions on a regular basis. The incentive? Besides my internal drive, speaking German was a necessity for work and for basic life tasks (eating, etc.); the second language where I lived was French, not English). After no more than 6 months, I discovered I was dreaming in German. While I was always reluctant to call myself fluent (reserving that for those who could pass as a native speaker) and my grammar didn't have a similar step-function improvement, I found I now was suddenly able to express and assimilate thoughts directly in German without any translation. I could keep up with others in speaking and listening. When I later moved back to the USA, I found that, for years, I'd naturally think about subjects I engaged in deeply in Germany in German first and then translate them to English. I suspect (without proof) I would have learned German earlier had I been exposed to it earlier in what I always knew of as the Berlitz total immersion method: no grammar, just talking and listening in German all day long. I think that's what the standard advice for learning J is all about: grab your computer, and use the Primer and other tools to immerse yourself in a dialog in J with your computer. It's hard, it's frustrating (by comparision, there were times I really wanted to speak English for a bit, but I held out), and it's eminently rewarding when your mind does that flip and J becomes natural. I think that's what the Morse code Web page was trying to convey. I think that's what my experience learning German was all about: reading more grammar books, memorizing rules, and learning more vocabulary would never have gotten me to where I needed to be; only forcing myself to think in German by prohibiting me from thinking in English got me there. I think that's how I learned the Dvorak keyboard: switching it over and then only typing with the new layout (for the first 2 weeks, I prayed for emails I could answer with "Yes" or "No" so I could type my responses within 5 minutes! :-). For me, the hard thing about J was finding time _in front of my computer_ to have that dialog. Lacking a handheld computer, I was tethered to a laptop or worse, and I was used to learning technical information from the printed page. I'm confident that my problems with J today are a result of not having been consistent enough in only using J to learn it. (My singular focus on immersion stops after internalizing something; I think it's very beneficial to be able to use multiple approaches, be it Dvorak/Qwerty, J/C, English/German, ....) As for the suggestions for syntax sugar for J (spaces around tokens, for example), what about ;:? Bill - -- Bill Harris 3217 102nd Place SE Facilitated Systems Everett, WA 98208 USA http://facilitatedsystems.com/ phone: +1 425 337-5541
Even though this repeats some of a messages included above, for completeness, I'll include the whole of Don Guinn's response to Stuart Bruff, where he raised the issue of the virtues of a compact notation.
Date: Tue, 22 Feb 2005 16:11:04 -0700 From: "Don Guinn" <donguinn@PVTNETWORKS.NET> Subject: Re: [Jforum] inter J compiler : formal and informal names To: JFORUM@PEACH.EASE.LSOFT.COM Actually about my remark on reading Basic vs reading J or APL. I find it much easier to read J or APL than Basic. Working with Basic I feel that I have one hand tied behind my back. So many things are almost there that it is frustrating. And Microsoft manuals repeat themselves so much from page to page that I find myself recognizing and skipping sections after I have seen them twenty or so times. I can go through a two inch Basic manual in an afternoon. Because I have skipped so much, what have I missed? What I meant was that at first the compactness of a J definition is a little overwhelming until I get into it. Where the Basic code is so verbose that I can't see but a little piece at a time so it feels like I am accomplishing something even though I am actually moving very slowly. It's kind of like going 70 miles an hour in a sports car where you are close to the ground seems fast, where the same speed in a motorhome seems slow because you are farther from the road and can see so much more. By the way, in a motorhome freeways look like country roads! Could this be a parallel that in J or APL we think of solving some problem as almost trivial where solving that same problem in Basic or C as a major undertaking? Stuart Bruff wrote: >> BTW if you prefer looking at 5 pages of VB rather than a single J >> line, it > > looks to me as if the power and terseness of J is eluding you, maybe the > initial barrier is higher but so rewarding ;O)< > =Funny thing. I found it not unreasonable spending an hour or so reading > several pages of Basic code to understand what it did, but I have > trouble spending five minutes reading a J statement which does the same > thing.= > > Not wishing to speak for Don, but merely as my interpretation of what > he wrote .... > > I didn't see this as expressing a 'preference', just a comment on the > degree of mental concentration required to interpret a J statement. J > is akin to translating from Russian or Japanese to English and then > trying to understand the content. VB is akin to translating from a > local dialect, much less mental effort involved and it's tantamount to > a single mental task. Once you're fluent in a language, the > "translation" effort drops off and it becomes easier to read and > understand, but it's pretty wearing if you're a novice or not familiar > with the specialist vocabularies. > > As an aside, I believe that Kanji has some similarities to J, in that > a symbol can have different meanings depending on whether it's stand > alone or adjacent to other symbols (Of course, not being a Japanese > speaker, I may be completely wrong :-\ ). I would be interested to > hear the opinions of the Japanese J users on how easy J is to pick up > or specific learning problems. > > Stuart > > ----- Original Message ----- From: <PGauthier@CHECKFREE.COM> > To: <JFORUM@PEACH.EASE.LSOFT.COM> > Sent: Tuesday, February 22, 2005 6:28 PM > Subject: Re: [Jforum] inter J compiler : formal and informal names > > > Personally, my favorite is: > > b=.(<;._1)"_ 1(L: 0) > v=.' ', '$123.45 $345.67 ($99.99)' > m=.>3#<v > c=.2 3$<m > > $b v > 3 > $b m > 3 3 > $b c > 2 3 > > NB: I had to read other people code in both APL and J and never found it > difficult to the degree you described. > > What I did notice is that some people are better designers than others > and their code is not only easier to understand but also better > modularized(put into verbs/adverbs/conjunctions) by more useful > modules in the context. > > Syntax coloring could help but we usually deal with so few at any one time > that inserting spacing in a piece of code is not an incommensurable task. > > BTW if you prefer looking at 5 pages of VB rather than a single J > line, it looks to me as if the power and terseness of J is eluding > you, maybe the initial barrier is higher but so rewarding ;O) > > Paul
Scan of Meeting Notes
[Category:design]]