NYCJUG/2005-02-08

From J Wiki
Jump to navigation Jump to search

Group project, Save Work, persistent data, raising interest in J, language as culture, picture albums, date arithmetic, catalog obverse, CSV files, beginner questions


Meeting Agenda for NYC JUG 20050208

1. Group project: focus on good design - see Cooper*, et al.

    a. What will it be?  Suggestions so far:

        i) brief tutorial on my backup utility; perhaps also alarmMsg ->
        sub-project of persistent window data -> user preference tracking.

        ii) sketch of an OpenGL project: boolean solids operation;
        (show de Casteljau algorithm);

        iii) further thoughts on what we might do with MIDI or other
        music-based idea;

        iv) collections utilities: mass rename, tracking of MP3s, photos;

        v) backtrack and code collection utility;

        vi) any other ideas?  General-purpose utilities like CSV read/write?

    b. How should we structure it?

    c. What are small sub-projects into which we can break it?

        i) what are general classes of utility functions? e.g.
        date-time (see "DateAdd" attached), file-handling.

    d. Use version control?
        See http://www.ericsink.com/scm/source_control.html
        Or use J projects?  SubEthaEdit?

    e. Coding standards?  Aim for multi-OS version?


2. Learning and creating learning materials: how to start this?  See point
   1.c. above - a general framework will give a broad target at which to aim.

   How to structure it?  As a FAQ?  Hypertext?

   What's in my "J/primer" directory?  Show DHMIdioms.txt - too idiosyncratic?

   Feedback to J Software.


3. Preliminary assignments: who can commit to how much by next month?


4. Show and tell: "DateAdd" -> dates.ijs & sfl.ijs: illustrating
   the difficulty of re-use.
  • The Inmates are Running the Asylum by Alan Cooper; see

[1]

Proceedings

Save Work Project - Possible Core: parseDir

I told a little bit about how "parseDir.ijs" works: File:ParseDir.ijs.

This is the backup facility I've been using for years - it could form the core of a "Save Work" project.

One feature of the script file is that it outputs the most commonly-used commands to the J session upon loading:

   load '~user/code/parseDir.ijs'
6!:2 '''FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP''=: getDirFlInfo_parseDir_ ''C:\'''
(<'\amisc\') fileVar_WS_&.>'FLNMS';'FLDTS';'FLSZS';'FLPARENT';'DIRNMS';'DIRDEP'
'batfl cmds'=. buildBatFl_parseDir_ 700e6;'C:\Temp\Recent\'
winexec ('cmd /c ',batfl);1

This makes it simple to avoid typing long commands by grabbing the lines from above that are close to what I want and modifying them if necessary.

First, build the global variables with information about all the files and directories on my C: drive (timing it):

   6!:2 '''FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP''=: getDirFlInfo_parseDir_ ''C:\'''
785.195
   0 60#:785.195
13 5.195

This was run on my laptop - 1.5GHz Thinkpad, 512M RAM, 31GB disk with 3.2GB free. As you can see above, it took about 13 minutes.

Now build the .BAT file to create target directories and copy 100M (=1e8) of the most recently-changed files to my thumb (K:) drive:

   'batfl cmds'=. buildBatFl_parseDir_ 1e8;'K:\'

Run the .BAT file (asynchronously):

   winexec ('cmd /c ',batfl);1
+--+------------------------------------------------------+-+
|33|cmd /c C:\DOCUME~1\DMCCOR~1\LOCALS~1\Temp\CDMDCopy.bat|1|
+--+------------------------------------------------------+-+

Build another command file to dump more files to a network drive:

   6!:2 '''batfl cmds''=. buildBatFl_parseDir_ 5e8;''U:\'''
3.55641

   winexec ('cmd /c ',batfl);1
+--+------------------------------------------------------+-+
|33|cmd /c C:\DOCUME~1\DMCCOR~1\LOCALS~1\Temp\CDMDCopy.bat|1|
+--+------------------------------------------------------+-+

AlarmMsg: GUI Example

"AlarmMsg.ijs" (File:AlarmMsg.ijs) is a simple GUI application I wrote that allows me to have a window pop up after a specified time to remind me of something.

SetalarmScreenShot.jpg

It's interesting in relation to the "Save Work" example only in that it exemplifies how one might save information between invocations of code. This code uses the file "AlarmParms.ini" to retain the settings entered by the user between invocations. The simple text file consists simply of "Name=Value" pairs corresponding to the three input fields on the dialog box:

ALRMSG=NYCJUG at Heartland on 34th at 18:30
SECS=0
MINS=525

So, each user update is written to this file to be available the next time the code is called. This persistence can be very useful as we often specify very similar parameters on subsequent invocations of code. It's a nice touch if the code remembers what we usually do and makes it available.

How To Get People Interested in J: Games?

Date:	Wed, 26 Jan 2005 12:36:26 -0500
From:	"Miller, Raul D" <rdmiller@USATODAY.COM>
Subject:	Re: [Jforum] Programming language (fwd)
To:	JFORUM@PEACH.EASE.LSOFT.COM

Fraser Jackson wrote:
> When I asked one small group of 12 year old boys what they wanted
> to program their answer was simple - games.  If Raul can write up
> and share some of his explorations of OpenGL it would be great.
> If we could have a 'Beginning OpenGl Game Programming'(Astle and
> Hawkins) but using J and its marvellous capability of interactive
> function development it would be a vast step forward in creating
> an environment for wider use of J.

I don't think I'm really ready to tackle these kinds of issues
yet.  My knowledge of OpenGL at the moment is extremely
sketchy and well below the level needed for a game more
advanced than "pong".

However, it's my impression that if J were to serve as a game
engine (as opposed to a game design tool), you would need
to incorporate some bits of C into the application, to get
performance to acceptable levels.

Of course, if you're using opengl 2.0 (shader language), you're
also using C (or something extremely close), but the compilation
and loading of that code is handled by the opengl drivers.  I'm
talking about the bits of code which run on the J side of the
opengl interface.

For example (and this example might be more a demonstration of
my ignorance of opengl than anything else), I'm under the impression
that the fastest way to pump vertex information into an opengl
driver is using glVertexPointer -- which has some implications
for the use of display lists (I'm presuming that a display
list would persist beyond individual calls to glVertexPointer).

Anyways, I'm likely to put together some small physics simulations
well before I'm likely to put together something which I think
fits in the "game" classification.  [And keep in mind that
my explorations of OpenGL are, at most, only a few hours a week.]

But I will try to take a look at the Astle and Hawkins book.

Thanks,

--
Raul

Language as Culture

[from http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=273&page=4] [interview with Alan Kay in ACM's Queue magazine]

[Alan Kay] In a history of Smalltalk I wrote for ACM, I characterized one way of looking at languages in this way: a lot of them are either the agglutination of features or they’re a crystallization of style. Languages such as APL, Lisp, and Smalltalk are what you might call style languages, where there’s a real center and imputed style to how you’re supposed to do everything. Other languages such as PL/I and, indeed, languages that try to be additive without consolidation have often been more successful. I think the style languages appeal to people who have a certain mathematical laziness to them. Laziness actually pays off later on, because if you wind up spending a little extra time seeing that “oh, yes, this language is going to allow me to do this really, really nicely, and in a more general way than I could do it over here,” usually that comes back to help you when you’ve had a new idea a year down the road. The agglutinative languages, on the other hand, tend to produce agglutinations and they are very, very difficult to untangle when you’ve had that new idea.

Also, I think the style languages tend to be late-binding languages. The agglutinative languages are usually early-binding. That makes a huge difference in the whole approach. The kinds of bugs you have to deal with, and when you have to deal with them, is completely different.

Some people are completely religious about type systems and as a mathematician I love the idea of type systems, but nobody has ever come up with one that has enough scope. If you combine Simula and Lisp—Lisp didn’t have data structures, it had instances of objects—you would have a dynamic type system that would give you the range of expression you need.

It would allow you to think the kinds of thoughts you need to think without worrying about what type something is, because you have a much, much wider range of things. What you’re paying for is some of the checks that can be done at runtime, and, especially in the old days, you paid for it in some efficiencies. Now we get around the efficiency stuff the same way Barton did on the B5000: by just saying, “Screw it, we’re going to execute this important stuff as directly as we possibly can.” We’re not going to worry about whether we can compile it into a von Neumann computer or not, and we will make the microcode do whatever we need to get around these inefficiencies because a lot of the inefficiencies are just putting stuff on obsolete hardware architectures.

I just think that’s a two-culture divide. I’ve seen many meetings where people are unable to communicate just because of the stylistic differences in approaches.

Building Picture Albums

Date:	Tue, 25 Jan 2005 09:24:37 -0800
From:	"Joey K Tuttle" <jkt@QUED.COM>  Add to Address BookAdd to Address Book
Subject:	Re: [Jforum] Programming language (fwd)
To:	JFORUM@PEACH.EASE.LSOFT.COM

On Fri, 21 Jan 2005, Brian M. Schott wrote:

>  to produce a dynamic webhosted application. Being a
>  (retired) kid at heart, myself, I know that I have had to
>  seek out perl for two of my own apps -- to get them on the
>  web -- because J cannot do so.
>
At 21:17 -0500 2005/01/24, Brian M. Schott wrote:
>afaik: J only supports html (right?);

At 08:40 -0500 2005/01/25, Brian M. Schott wrote:
>	I mean that the only web-related feature provided by
>j is html encoding. My terminology may be wrong (worse yet,
>my last statement may be wrong), but I mean that there are
no facilities for cgi interfacing, though.

Brian,

I meant (tried) to post this last friday when it was more
timely, but I failed twice (my own problems and distractions).

I'm not sure what you did with the PERL, but I happily use
j as CGI calls on my Apache web server - that is, a user
somewhere on a browser can create dynamic pages that are
built by j - this requires no knowledge of what is in the
background, and I like writing CGI in j much more than in
PERL -- although PERL can be a "module" in Apache, j could
be too with a little porting effort. This would make things
even faster and more efficient - but they work very well
already. As an added thought from one of the other posters,
the most used such application I have is building picture
albums dynamically. I gave a talk on this subject to the
APLbug (SF Bayarea User Group) a couple of years ago.

- joey

Date Arithmetic

A common thing we want to do is to find the difference between dates or add a certain number of days, weeks, or months to a given date. Visual Basic has a nice function "DateAdd" which does this.

DateAdd=: 3 : 0
NB.* DateAdd: limited mimic of VB DateAdd - DateAdd <unit to add>;date;num.
NB. Date in form YYYYMMDD.
   ts=. i. 0 [ units=. 'YMDW'      NB. Units are Year, Month, Day, Week.
   'unit dt addnum'=. y.
   assert (unit=. {.toupper unit) e. units
   dpu=. (372 31 1 7){~units i. unit    NB. 372=12*31: provisional "year"
   mobase=. 0 12 31
   assert. dt>18000101   NB. todayno only goes back to this date
   dt=. 0 100 100#:dt
   select. unit
   case. 'D';'W' do.
       dt=. 100#.todate (dpu*addnum)+todayno dt
   case. 'M';'Y' do.
       fakedayno=. mobase#.0 _1 0+dt
       dt=. 100#.todate todayno 0 1 0+mobase#:fakedayno+dpu*addnum
   end.
   assert. dt>18000101   NB. todayno only goes back to this date
NB. dtdifs=. 2-/\todayno&>(<0 100 100)#:&.> dtsa=. DateAdd&>(<'D';20050101),&.><&.>i:_10000
NB. 1 *./ . = dtdifs
NB. dtdifs=. 2-/\todayno&>(<0 100 100)#:&.> dtsa=. DateAdd&>(<'W';20050101),&.><&.>i:_10000
NB. 7 *./ . = dtdifs
)

This uses following J-supplied code:

NB. From ~system\main\dates.ijs
NB. =========================================================
NB.*todayno v converts dates to day numbers
NB. converts dates to day numbers, converse <todate>
NB.
NB. y. = dates
NB. x. = optional:
NB.   0 - dates in form <yyyy mm dd> (default)
NB.   1 - dates in form <yyyymmdd>
NB. 0 = todayno 1800 1 1, or earlier
NB.
NB. example:
NB.    todayno 1998 5 23
NB. 72460

todayno=: 3 : 0
0 todayno y.
:
a=. y.
if. x. do. a=. 0 100 100 #: a end.
a=. ((*/r=. }: $a) , {:$a) $,a
'y m d'=. <"_1 |: a
y=. 0 100 #: y - m <: 2
n=. +/ |: <. 36524.25 365.25 *"1 y
n=. n + <. 0.41 + 0 30.6 #. (12 | m-3),"0 d
0 >. r $ n - 657378
)

Did not find something to use exactly from Oleg's "sfl" addon (see his website [2]). This could form the basis of a more thorough version of "DateAdd".

NB. see http://www.imatix.com/html/sfl/index.htm for detailed description
NB. J Script  template
NB. Wednesday, May 17, 2000
NB. SFL dates routines

require jpath '~addons\sfl\sfl.ijs'

coclass'jsfl'

date_now=: '   date_now + i' cdm NB.             (void);
time_now=: '   time_now + i' cdm NB.	      (void);
leap_year=: '   leap_year + i i' cdm NB.	      (int year);
julian_date=: '   julian_date + i i' cdm NB.         (long date);
day_of_week=: '   day_of_week + i i' cdm NB.         (long date);
week_of_year=: '   week_of_year + i i' cdm NB.        (long date);
year_quarter=: '   year_quarter  + i i' cdm NB.       (long date);
next_weekday=: '   next_weekday  + i i' cdm NB.       (long date);
prev_weekday=: '   prev_weekday  + i i' cdm NB.       (long date);
pack_date=: '   pack_date     + i i' cdm NB.       (long date);
pack_time=: '  pack_time   + i i' cdm NB.         (long time);
unpack_date=: ' unpack_date   + i i' cdm NB.       (word packdate);
unpack_time=: ' unpack_time    + i i' cdm NB.      (word packtime);
default_century=: 'default_century  + i i'cdm NB.    (long *date);
date_to_days=: 'date_to_days   + i i' cdm NB.      (long date);
days_to_date=: 'days_to_date   + i i' cdm NB.      (long days);
date_to_timer=: ' date_to_timer   + i i i'cdm NB. time_t     (long date, long time);
timer_to_date=: ' timer_to_date  + i i' cdm NB.      (time_t time_secs);
timer_to_time=: ' timer_to_time  + i i' cdm NB.     (time_t time_secs);
timer_to_gmdate=: 'timer_to_gmdate + i i' cdm NB.     (time_t time_secs);
timer_to_gmtime=: 'timer_to_gmtime + i i' cdm NB.    (time_t time_secs);
time_to_csecs=: ' time_to_csecs   + i i' cdm NB.    (long time);
csecs_to_time=: ' csecs_to_time   + i i' cdm NB.    (long csecs);
future_date=: ' future_date  + i *i *i i i'cdm NB.       (long *date, long *time, long days, long csecs);
past_date=: '   past_date + i *i *i i i'cdm NB.          (long *date, long *time, long days, long csecs);
date_diff=: '  date_diff +i i i i i *i *i'cdm NB.           (long date1, long time1, long date2, long time2, long *days, long *csecs);
valid_date=: '   valid_date + i i'cdm NB.         (long date);
valid_time=: '   valid_time + i i'cdm NB.         (long time);
date_is_future=: ' date_is_future  + i i i'cdm NB.    (long date, long time);
date_is_past=: '  date_is_past  + i i i'cdm NB.      (long date, long time);
timezone_string=: ' timezone_string + *c'cdm NB.    (void);
local_to_gmt=: '   local_to_gmt  + i i i *i *i'cdm NB.      (long date, long time, long *gmdate, long *gmtime);
gmt_to_local=: '   gmt_to_local  + i i i *i *i'cdm NB.      (long gmdate, long gmtime, long *date, long *time);

NB. =========================================================

datenow=: first@ date_now
timenow=: first@ time_now
leapyear=: first@ leap_year
juliandate=: first@ julian_date

dayofweek=: first@ day_of_week
weekofyear=: first@ week_of_year
yearquarter=: first@ year_quarter
nextweekday=: first@ next_weekday
prevweekday=: first@ prev_weekday
packdate=: first@ pack_date
packtime=: first@ pack_time
unpackdate=: first@ unpack_date
unpacktime=: first@ unpack_time
datetodays=: first@ date_to_days
daystodate=: first@ days_to_date
timetocsecs=: first@ time_to_csecs
csecstotime=: first@ csecs_to_time
validdate=: first@ valid_date
validtime=: first@ valid_time
dateisfuture=: first@ date_is_future
dateispast=: first@ date_is_past
timezonestring=: memr0@first@ timezone_string

Catalog Obverse

Some interesting forum traffic on the catalog verb and its obverse.

Date:	Mon, 14 Feb 2005 13:09:03 -0500
From:	"Roger Hui" <rhui000@SHAW.CA>
Subject:	Re: [Jforum] Inverse of Catalogue
To:	JFORUM@PEACH.EASE.LSOFT.COM

x=: (2 3$'abcdef');'ghij'
   y=: catalogue x
   uncatalogue y
|index error: uncatalogue
|       uncatalogue y


On Sun, 13 Feb 2005 18:46:21 -0500, Mark D. Niemiec <mniemiec@INTERSERV.COM>
wrote:

>J does not provide an inverse of { Catalogue, but it is an easy operation
to perform:
>
>   uncatalogue =: (>`(i.@#@$<@([{&>({:@$$,)@|:)"0 _])@.(*@#@$))
>   catalogue =: { :. uncatalogue
>
>   [ y =: 'ht';'ao';'gtw'
>+--+--+---+
>|ht|ao|gtw|
>+--+--+---+
. . .

An idea for the inverse of catalog.

Date:	Sun, 13 Feb 2005 18:46:21 -0500
From:	"Mark D. Niemiec" <mniemiec@INTERSERV.COM>
Subject:	[Jforum] Inverse of Catalogue
To:	JFORUM@PEACH.EASE.LSOFT.COM

J does not provide an inverse of { Catalogue, but it is an easy operation to
perform:

   uncatalogue =: (>`(i.@#@$<@([{&>({:@$$,)@|:)"0 _])@.(*@#@$))
   catalogue =: { :. uncatalogue

   [ y =: 'ht';'ao';'gtw'
+--+--+---+
|ht|ao|gtw|
+--+--+---+
   [ caty =: { y
+---+---+---+
|hag|hat|haw|
+---+---+---+
|hog|hot|how|
+---+---+---+

+---+---+---+
|tag|tat|taw|
+---+---+---+
|tog|tot|tow|
+---+---+---+
   uncatalogue caty
+--+--+---+
|ht|ao|gtw|
+--+--+---+
   |.&.catalogue y
+--+--+---+
|th|ao|gtw|
+--+--+---+
   |."_1&.catalogue y
+--+--+---+
|ht|oa|gtw|
+--+--+---+
   |."_2&.catalogue y
+--+--+---+
|ht|ao|wtg|
+--+--+---+

-- Mark D. Niemiec <mniemiec@interserv.com>

Beginner's Session

We had a computer available so we showed how to answer some questions people had.

NB. [Session started: 2005 2 8 20 29 33.137]

NB. Jim has a matrix of boxed items he wants to convert to plain
NB. ASCII (including the boxing display characters).
   nn=. 3 3$'';'x';'y';'min';'3';'2';'max';'9';'10'
   nn
+---+-+--+
|   |x|y |
+---+-+--+
|min|3|2 |
+---+-+--+
|max|9|10|
+---+-+--+

NB. He has only to do this:
   ":nn
+---+-+--+
|   |x|y |
+---+-+--+
|min|3|2 |
+---+-+--+
|max|9|10|
+---+-+--+

NB. Looks the same but the shape tells us it isn't:
  $ ":nn
7 10

NB. I would probably put it a more common format (e.g. .CSV):
   matrix2csv nn
"","x","y"
"min","3","2"
"max","9","10"


NB. Example of standard function to use to split, e.g. headers from data:
   split 1 2 3 4
+-+-----+
|1|2 3 4|
+-+-----+
   2 split 1 2 3 4
+---+---+
|1 2|3 4|
+---+---+
NB. It works on items, so:
   split nn
+------+----------+
|++-+-+|+---+-+--+|
|||x|y|||min|3|2 ||
|++-+-+|+---+-+--+|
|      ||max|9|10||
|      |+---+-+--+|
+------+----------+


NB. In discussing the difficulties of handling .CSV files efficiently,
NB. particularly the overhead of checking whether a cell is numeric or
NB. not, Dan suggests grouping entire columns thusly:
   d=.5$,:4;'hello';'devon';9j9
   d                               NB. We have mixed cells but columns
+-+-----+-----+---+                NB. have homogenous types.
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+
|4|hello|devon|9j9|
+-+-----+-----+---+

NB. So, group elements of columns together.
   tocol=.<@:>"1@:|:
   tocol d
+---------+-----+-----+-------------------+
|4 4 4 4 4|hello|devon|9j9 9j9 9j9 9j9 9j9|
|         |hello|devon|                   |
|         |hello|devon|                   |
|         |hello|devon|                   |
|         |hello|devon|                   |
+---------+-----+-----+-------------------+

NB. My own, much-used, matrix-to-CSV-file function.
   m2c
4 : 0
NB.* m2c: Matrix x. to .CSV file y.
   dlim=. '' [ writeInPieces=. 0   NB. Default to write as single piece.
   if. 1=L. y. do. 'writeInPieces y.'=. y. end.
   if. writeInPieces do.           NB. Underlying function handles writing
       (1;',') matrix2csv x.;<y.   NB.  in pieces.
   else.                           NB. A "file number error" here
       (matrix2csv x.) 1!:2 <y.    NB.  may indicate a held file.
   end.
NB.EG data m2c filename
NB.EG data m2c 1;<filename     NB. Write file in pieces as we go to save space.
)

NB. and its inverse:
   c2m
3 : 0
   ',' csv2matrix 1!:1 <y.
:
   x. csv2matrix 1!:1 <y.
)

   csv2matrix
3 : 0
NB.* csv2matrix: change contents of .CSV file -> matrix of items from
NB. Comma-Separated-Values file y.  Left arg is delimiter if not comma.
NB. Handles quoted strings.
   ',' csv2matrix y.
:
   if. x.-:'' do. delim=. ',' else. delim=. x. end.
   if. 0=L. y. do. vec=. l2v y.   NB. Box lines if simple vec ->
   else. vec=. y. end.            NB.  vector of text lines.
   mat=. matMakeMaskedPtn vec;delim;<'"'
   mat=. mat-.&.>'"'              NB. Remove quotes.
)

NB. Example of using to read in .CSV file->J boxed matrix:
   $fl=. c2m '\gtaap\gaa\data\rawdatfl.csv'
365 101
   5{.fl
+----------+-----------------------------------+----------------------...
|Name      |US $ TO UK £ (GTIS) - EXCHANGE RATE|US $ TO JAPANESE YEN (...
+----------+-----------------------------------+----------------------...
|Code      |BRITPUS                            |JAPYNUS               ...
+----------+-----------------------------------+----------------------...
|Currency  |£                                  |Y                     ...
+----------+-----------------------------------+----------------------...
|12/31/1974|#n/a                               |#n/a                  ...
+----------+-----------------------------------+----------------------...
|1/31/1975 |#n/a                               |#n/a                  ...
+----------+-----------------------------------+----------------------...

NB. Split off the header from the data:
   'hdr fld'=. 3 split fl
   hdr
+--------+-----------------------------------+------------------------...
|Name    |US $ TO UK £ (GTIS) - EXCHANGE RATE|US $ TO JAPANESE YEN (GT...
+--------+-----------------------------------+------------------------...
|Code    |BRITPUS                            |JAPYNUS                 ...
+--------+-----------------------------------+------------------------...
|Currency|£                                  |Y                       ...
+--------+-----------------------------------+------------------------...
   3{.fld
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
|12/31/1974|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|68....
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
|1/31/1975 |#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|76....
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
|2/28/1975 |#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|#n/a|81....
+----------+----+----+----+----+----+----+----+----+----+----+----+---...
   'hdr fld'=. tocol each 3 split fl
   hdr
+--------+-----------------------------------+------------------------...
|Name    |US $ TO UK £ (GTIS) - EXCHANGE RATE|US $ TO JAPANESE YEN (GT...
|Code    |BRITPUS                            |JAPYNUS                 ...
|Currency|£                                  |Y                       ...
+--------+-----------------------------------+------------------------...
   fld
...
|9/30/1986 |1.44800|0.00647999|0.49350|0.60730|0.150700|0.43670|0.6284...
|10/31/1986|1.40650|0.00612600|0.48480|0.58430|0.148700|0.42940|0.6417...
|11/28/1986|1.43650|0.00617899|0.50710|0.60940|0.154900|0.44880|0.6508...
|12/31/1986|1.48300|0.00632900|0.52030|0.62230|0.157000|0.46020|0.6655...
|1/30/1987 |1.51350|0.00651499|0.54660|0.64810|0.163900|0.48600|0.6622...
|2/27/1987 |1.55500|0.00652500|0.54670|0.64850|0.164100|0.48480|0.6805...
|3/31/1987 |1.60750|0.00685899|0.55430|0.66310|0.166500|0.49040|0.7062...
|4/30/1987 |1.66150|0.00710699|0.55800|0.68100|0.167100|0.49320|0.7020...
|5/29/1987 |1.62950|0.00695200|0.54840|0.66230|0.164300|0.48690|0.7123...
|6/30/1987 |1.61750|0.00681200|0.54800|0.66050|0.164200|0.48700|0.7211
...

NB. Example of converting text to numbers with an error flag of _99:
   _99 ". '1 2 3 4'
1 2 3 4
   _99 ". '1 2 3 4'
1 2 3 4
    _99 ". '1 2 3 4 crap'
1 2 3 4 _99

NB. My own (external format) Number to J number function:
   n2j
_&".@(-.&'+')

NB. and its inverse:
   j2n
'S'&J2StdCvtNums
   J2StdCvtNums
3 : 0
NB.* J2StdCvtNums: convert char rep of num from J to "Standard", or
NB. vice-versa if left arg is 'J' or '2J'; optional 2x2 left argument allows
NB. 2 arbitrary conversions: col 0 is "from", col 1 is "to" char.
NB. Monadic case changes Standard numeric representation to J representation.
   (2 2$'-_Ee') J2StdCvtNums y.
:
   pw16=. 0j16                          NB. Precision width: 16 digits>.
   diffChars=. 2 2$'-_Ee'               NB. Convert '-'->'_' & 'E'->'e'
   toStd=. -.'J'-:''$'2'-.~,x.          NB. Flag conversion J->Standard
   if. 2 2-:$x. do. diffChars=x.        NB. if explicit conversion.
   elseif. toStd do. diffChars=. |."1 diffChars end.   NB. Convert other way
NB.   if. 0=1{.0$y. do. y.=. pw16":y. end. NB. Numeric to character
   if. 0=1{.0$y. do.                    NB. Numeric to character
       fmts=. (8=>(3!:0)&.>y.){0,pw16   NB. Full-precision floats only
NB.       whvn=. isValNum &.> y.
NB.       tty=. fmts":y.                   NB. If this is too slow, go back
       y.=. fmts":y.                   NB. If this is too slow, go back
NB.       y.=. whvn}tty,:y.                NB. If this is too slow, go back
NB.       y.=. pw16":y.                 NB.  to this.
   end.

   y.=. y.-.'+'                         NB. EG 1.23e+11 is ill-formed & the
   wh=. y.=0{0{diffChars                NB.  '+' is unnecessary.
   cn=. (wh#1{0{diffChars) (wh#i. $y.)}y.    NB. Translate chars that need it
   wh=. y.=0{1{diffChars                     NB.  but leave others alone.
   cn=. (wh#1{1{diffChars) (wh#i. $cn)}cn
   if. -.toStd do.                      NB. Special handling -> J nums
       if. '%'e. cn do.                 NB. Convert nn% -> 0.nn
           cn=. pw16":0.01*".cn-. '%'
       end.
       cn=. cn-.','                     NB. No ',' in J numbers
   end.
   cn
NB.EG 'S' J2StdCvtNums _3.14 6.02e_23   NB. Convert J numbers to std rep
)

   matrix2csv
3 : 0
NB.* matrix2csv: create .CSV file from matrix of items y.
NB. Inverse of csv2matrix but doesn't write to a file; does quote literals.
NB. Optional left arg is (scalar) field separator character.
NB. Do large mats in pieces to avoid using up memory.
NB. Optional flag 0{x. is 1 if we are to write file instead of just
NB. creating result to write to file.
   ',' matrix2csv y.
:
   ctr=. i. 0 [ csvfl=. '' [ doWrite=. 0
   if. isEnclosed x. do.
       'doWrite x.'=. x.
       'y. flnm'=. y.
   end.
   y.=. (_2{.1 1,$y.)$,y.     NB. Ensure is a matrix.
   if. 0>4!:0 <'flnm' do. flnm=. 'TEMP.TMP',~getTempDir '' end.
   flnm=. boxopen flnm
NB. Estimate Number of Rows Per Iteration: was a global.
   NRPI=. 100>.1000*<.0.5+0.001*(#y.)%(7!:5 <'y.')%15e6 NB. 15e6 for 256MB mem.
   numrows=. NRPI<.#y.
   if. doWrite do.
       '' 1!: 2 flnm          NB. Initialize file because we'll append later.
   end.
   while. 0 < # y. do.        NB. Do in pieces if too large.
      mat=. numrows{.y.
      whchar=. ' '=&.>1{.&.>0$&.>mat    NB. Where non-num (=char) fields?
      if. 1 e. ,whchar=. ;,whchar do.
          quotes=. ($mat)$whchar#&.>'"' NB. Put quotes around char fields.
          if. +./,>'"'e.&.>mat do.           NB. Duplicate any quotes that
              mat=. (>:&.>mat=&.>'"')#&.>mat NB.  are part of data.
          end.
          mat=. quotes,&.>mat,&.>quotes
      end.
      if. 1 e. whnum=. ,-.whchar do.    NB. Convert numerics to char so
          shape=. $mat                  NB. can append field separator.
          mat=. ]&.>,mat                NB. Enclose if simple array.
          mat=. (":&.>whnum#mat) (b2i whnum)}mat
          mat=. shape$mat
      end.
      mat=. mat,&.>x.
      mat=. ;(_1}."1 mat),.(_1}.&.>_1{."1 mat),&.><CR,LF
      if. doWrite do.
          mat 1!:3 flnm
      else.
          csvfl=. csvfl,mat
      end.
      y.=. numrows}.y.
      numrows=. numrows <. # y.
   end.
   csvfl
NB.EG Either
NB.EG    (matrix2csv datamat) 1!:2 <flnm
NB.EG or
NB.EG    (1;',') matrix2csv datamat;<flnm
)


NB. Dan shows us the tree display form he finds handy.
NB. Start with a long train of constant functions:
   g =: 1: 2: 3: 4: 5: 6: 7: 8: 9:

NB. Display it (first word in the namelist consisting only of 'g')
NB. with "5!:4":
   5!:4{.;:'g'
  +- 1:
  +- 2:
--+    +- 3:
  |    +- 4:
  +----+    +- 5:
       |    +- 6:
       +----+    +- 7:
            +----+- 8:
                 +- 9:
   g
1: 2: 3: 4: 5: 6: 7: 8: 9:

NB. Try it with a more familiar train:
   (+/%#)
+/ % #
   g=. (+/%#)
   5!:4{.;:'g'
  +- / --- +
--+- %
  +- #
   5!:4{.;:'fetch'
-- {::


NB. Jim wants to know how to activate or execute a J expression given
NB. as a text string; specifically, how to assign a variable indirectly.

c=: dyad define
   ". x.,5!:5{.;:'y.'
)
NB. Function "c" applies J evaluation of x. to values y. :
   'A=:' c 3 4 5 6
3 4 5 6
NB. Now "A" is 3 4 5 6.
   A
3 4 5 6
   ;/ 3 4 5
+-+-+-+
|3|4|5|
+-+-+-+
   A
3 4 5 6
NB. Left-hand side doesn't have to be assignment, can be any expression.
   B =: '+/'
   B c 3 4 5 6 7
25

   3 ; 4; 5 ; 6
+-+-+-+-+
|3|4|5|6|
+-+-+-+-+
   C=:3 ; 4; 5 ; 6
   C
+-+-+-+-+
|3|4|5|6|
+-+-+-+-+
   5!:5{.;:'C'
3;4;5;6
3;4;5;6
+-+-+-+-+
|3|4|5|6|
+-+-+-+-+

Scan of Meeting Notes

NYCJUGMeetingNotes20050208.jpg