NYCJUG/2015-01-13
Set operations, user input prompt, one-line statements, forward-fill, punchcards, educational software, Java BigDecimal, forensic DNA analysis software
Meeting Agenda for NYCJUG 20150113
1. Beginner's regatta: See "Coding Challenge: Sets". Also, see "Die - Prompt - Die!" and "The Power of One-line Statements". 2. Show-and-tell: David Lambert on bilinear interpolation: see "Bilinear Interpolation", and "David Lambert Interpolation". 3. Advanced topics: See "Forward Filling without Looping or Conditionals". 4. Learning and teaching J: See "Better Learning Through Expensive Software", "Why are we still programming like it's the punchcard era?", and "Drowning but Unaware of it", and "DNA Everywhere".
Beginner's regatta
We looked at a J solution (as well as its Haskell equivalent) to a "dailyprogrammer" problem to implement basic set operations.
Coding Challenge: Sets
[2015-01-05 Challenge #196 [Practical Exercise] Ready...set...set!] (self.dailyprogrammer)
submitted 2 days ago * by Elite68091 1
(Practical Exercise): Ready... set... Set!
The last practical exercise was well-received so I'm going to make another one. This one is less complicated and, if you're still finding your feet with object-oriented programming, should be great practice for you. This should be doable in functional languages too.
The idea of a Set can be very math-y when you delve deeper but this post only skims the surface, so it shouldn't pose any issue!
Background
A set is a mathematical concept that represents a collection of other objects. Those other objects can be numbers, words, operations or even sets themselves; for the (non-extension) purposes of the challenge they are integers only. A finite set is a set with only a finite number of items (unlike, for example, the set of all real numbers Rwhich has uncountably infinite members.)
A set is generally represented with curly brackets with the items separated by commas. So, for example, the set containing -3, 6 and 11 could be written as {-3, 6, 11}. This notation is called an extensional definition.
There are some distinctions between a set and the list/array data structure:
· Repeated items are ignored, so {-3, 6, 11} is exactly the same as {-3, -3, 6, 11}. To understand why this is so, think less of a set being a container of items, but rather the items are members of a set - much like how you can't be a subscriber on /r/DailyProgrammer twice.
· Order doesn't matter - {-3, 6, 11} is the same as {6, 11, -3} and so on.
· Sets are generally seen as immutable, which means that rather than adding an item A to a set S, you normally create a new set with all the members of S, and A. Immutable data structures are quite a common concept so this will serve as an intro to them if you've not came across them already.
· A set can be empty - {}. This is called the empty set, weirdly enough.
Sets have 3 main operations.
· Union, with the symbol ∪. An item is a member of set S, where S=A∪B, if it's a member of set A or set B.
. For example, let A={1, 4, 7} and let B={-4, 7, 10}. Then, A∪B={-4, 1, 4, 7, 10}.
· Intersection, with the symbol ∩. An item is a member of set S, where S=A∩B, if it is a member of set Aand set B.
. For example, let A={1, 4, 7} and let B={-4, 7, 10}. Then, A∩B={7}, as only 7 is a member of both sets.
· Complement, with the symbol '. An item is a member of set S, where S=A', if it's not a member of A.
. For example, let A={1, 4, 7}. Then, A' is every integer except 1, 4 and 7.
Specification
You are to implement a class representing a set of integers.
· To hold its content, you can use an array, list, sequence or whatever fits the language best. Consider encapsulating this (making it private) if your language supports it.
· The class should expose a method Contains, which accepts an integer and returns whether the set contains that integer or not.
· The constructor of the class should accept a list/array of integers which are to be the content of the set. Remember to ignore duplicates and order. Consider making it a variadic constructor (variable number of arguments) if your language supports it.
· The class should have static methods for Union and Intersection, which both accept two sets and return another set containing the union or intersection of those two sets respectively. Remember, our sets areimmutable, so create a new set rather tham modifying an existing one. Consider making these as binary operators (eg. + for union and * for intersection) if your language supports it.
· The class should have another static method for Equals or equals, which accepts two sets and returns a boolean value. This determines if the two sets contain the same items and nothing else.
Finally, the set should be convertible to a string in some way (eg. ToString, toString, to_s depending on the language) which shows all items in the set. It should show them in increasing order for readability.
If your language already has a class for sets, ignore it. The purpose of this exercise is to learn from implementing the class, not use the pre-existing class (although in most cases you would use the existing one.)
Making Use of your Language
The main challenge of this exercise is knowing your language and its features, and adapting your solution to them. For example, in Ruby, you would not write a ToString method - you would write a to_s method, as that is the standard in Ruby. In C++ and C#, you would not necessarily write static Union, Intersection methods - you have the ability to overload operators, and you should do so if it produces idiomatic code. The research for this is part of the task. You should also be writing clean, legible code. Follow the style guide for your language, and use the correct naming/capitalisation conventions, which differ from language to language.
Extension 1 (Intermediate)
If you are feeling up to it, change your class for a set of integers and create a generic set class (or, if your language has dynamic typing, a set of any comparable type.) Depending on your language you might need to specify that the objects can be equated - I know in C# this is by IEquatable but other language will differ. Some languages like Ruby don't even need to.
Extension 2 (Hard)
This will require some thinking on your end. Add a Complement static method to your class, which accepts a set and returns another set containing everything except what's in the accepted set.
. Of course, you can't have an array of every integer ever. You'll need to use another method to solve this extension, and adapt the rest of the class accordingly. Similarly, for the string conversion, you can't print an infinite number of items. For this reason, a set containing everything containing everything except 3 and 5 should print something like {3, 5}' (note the '). You could similarly use an overloaded operator for this - I've picked ! in my solution.
Addendum
Happy new year! I know /u/Coder_d00d has already wished you so, but now I do too. Have fun doing the challenge, help each other out and good luck for the new year.
---
Haskell
134671 1 7 points 2 days ago
Haskell! I smiled when I got to the fifth bullet point and realized the language had already solved it for me when I wrote deriving (Eq).
data Set a = Set [a] | ComplementSet [a] deriving (Eq, Show)
-- List difference.
(\\) :: Eq a => [a] -> [a] -> [a]
xs \\ ys = filter (`notElem` ys) xs
complement :: Set a -> Set a
complement (Set xs) = ComplementSet xs
complement (ComplementSet xs) = Set xs
setElem :: Eq a => a -> Set a -> Bool
setElem x (Set xs) = x `elem` xs
setElem x (ComplementSet xs) = x `notElem` xs
union :: Eq a => Set a -> Set a -> Set a
union (Set xs) (Set ys) = Set (xs ++ (ys \\ xs))
union (Set xs) (ComplementSet ys) = ComplementSet (ys \\ xs)
union (ComplementSet xs) (Set ys) = ComplementSet (xs \\ ys)
union (ComplementSet xs) (ComplementSet ys) =
complement $ intersection (Set xs) (Set ys)
intersection :: Eq a => Set a -> Set a -> Set a
intersection (Set xs) (Set ys) = Set (xs \\ (xs \\ ys))
intersection (Set xs) (ComplementSet ys) = Set (xs \\ ys)
intersection (ComplementSet xs) (Set ys) = Set (ys \\ xs)
intersection (ComplementSet xs) (ComplementSet ys) =
complement $ union (Set xs) (Set ys)
difference :: Eq a => Set a -> Set a -> Set a
difference a b = a `intersection` complement b
Elite68091 1[S] 2 points 2 days ago
Haskell's type system is pretty damn cool! Well done, great solution.
---
J
Godspiral 2 points 2 days ago*
In J, (the way I use sets, perhaps less than formal specs that have been built in J or other languages). If you have the choice, you would always prefer to not be stuck with an OOP implementation, just because there is an obvious tolist representation that you would always prefer to avoid the toset and tolist castings, as these just get in the way.
toset =: /:~@:~.
Converts any list or set into a sorted set. Works also for list/sets of records or even tables. (strings or numbers)
union =: toset@:,&toset NB. left and right args can be lists or sets. 2 2 3 union 1 1 4 3 2 1 2 3 4 intersect =: ([ #~ e.)&toset 2 5 3 intersect 1 1 4 3 2 2 3 ismember =: e. NB. same whether list or set. 2 e. 2 5 3 intersect 1 1 4 3 2 1 4 e. 2 5 3 intersect 1 1 4 3 2 0 setequal =: -:&toset 2 5 3 setequal 1 1 4 3 2 0 1 2 4 3 setequal 1 1 4 3 2 1
Adding to a set: (with lists or sets as input)
1 2 3 toset@:, 3 4 2 5 1 2 3 4 5
. permalink
Godspiral 1 point 2 days ago*
Complements are tricky or simple. They are simple if you want to know membership of the complement of a single set. The more tricky use is you want to know membership of the union of a list of sets together with the union of complements of other sets.
For these calcs, keep all sets separate
toset each <"1 i.3 4 ┌───────┬───────┬─────────┐ │0 1 2 3│4 5 6 7│8 9 10 11│ └───────┴───────┴─────────┘
For this use an adverb that sets complement mask for unions
ComplMask =: 1 : '[: +./ m = e. &>'
To test membership in the union of the first 2 unioned with the complement of the third:
4 (1 1 0 ComplMask) toset each <"1 i.3 4 1 9 (1 1 0 ComplMask) toset each <"1 i.3 4 0 NB. not member of full union due to member of 3rd rather than complement of it. 19 (1 1 0 ComplMask) toset each <"1 i.3 4 1 NB. complement of 3rd set.
For intersections of sets that may include complement masks
ComplMaskI =: 1 : '[: *./ m = e. &>' toset each 4<\ i.6 ┌───────┬───────┬───────┐ │0 1 2 3│1 2 3 4│2 3 4 5│ └───────┴───────┴───────┘ 1 (1 1 0 ComplMaskI) toset each 4<\ i.6 1 2 (1 1 0 ComplMaskI) toset each 4<\ i.6 0
Another function is:
issubset =: *./@:e. 0 2 *./@:e. 1 1 4 3 2 0 0 1 3 *./@:e. 1 1 4 3 2 1 3 3 2 *./@:e. 1 1 4 3 2 1
Elite68091 1[S] 2 points 2 days ago
A Response to the J solutions
Every time I see your J submissions
---
Godspiral 1 point 2 days ago*
Though the masks for working with multiple sets is semi-cool, for working with sets in general, J is really doing all of the work here. I would normally not bother using any of the assigned names, because many operations can skip the sorting step.
~. is called nub, and gives you the unique values in a list in the order they are found, and is generally more useful than the set (sorted) representation, which only ever matters when comparing equality. An issue when working with sets is that you often do not want to destroy the original list, and rather want to query the current unique values of the list (in many cases).
DevonMcC 1 point just now
Another useful primitive in J is -. (called "without"), which is at least parallel to the idea of complement: it removes one set from the other. So,
1 2 3 4 -. 2 4 1 3 'ABCDEFG' -. 'BDFG' ACE
Die - Prompt - Die!
Looking over a repeated question the J community gets from those starting to learn the language, we'd like to drive the first nail into the coffin of this bad paradigm for user interaction: the programmatically-generated user input prompt.
from: Amelia Stein <blindingyoureyes@gmail.com> to: programming@jsoftware.com date: Thu, Apr 12, 2007 at 12:01 PM subject:[Jprogramming] J script files
I am new to J programming, and I don't quite understand script files. How do you write the equivalent of a main function from C programming? I don't want to define a verb. I simply want to create a main program where I can query the user and except user input. …
from: Amelia Stein <blindingyoureyes@gmail.com> date: Thu, Apr 12, 2007 at 1:02 PM
I have read the manuals, and most of jsoftware.com without success. From what I have found the manuals only explain the primitives, and verb definitions. I don't want to write a verb that I have to pass arguments to. I don't want to write a one line definition. I want to write a driver function essentially. The function should have user prompts that explain what type of data to enter etc. I want to write a script the equivalent of this C code in J:
//Written in C
# include <stdio.h>
int main (void)
{
int i,x;
printf ("Enter a number:");
scanf("%d", &i);
x=call_on_function_that_I_still_have_to_write(i);
printf("%d", x);
return 0;
---
from: Amelia Stein <blindingyoureyes@gmail.com> date: Fri, Apr 13, 2007 at 3:10 PM
I have read a bunch of the J for C programmers book as well as the primer, dictionary, etc. I truly don't intend to write major programs in J. I am doing this for a class assignment. I am comparing J to C because I have to teach it to a class of people who only know C. I wanted to be able to show analogies. I know that J is strong on array handling, and I am slowly learning that it isn't meant for user input. For my assignment I need to Accept up to 30 values from the keyboard for each of two variables and then compute: number of entries, minimum value, max, range, mean, variance, standard deviation. I know how to do those tasks on the command line in J, but my professor is big on user prompts. He doesn't want to see a program written for programmers. He wants a user friendly program for people who don't know the language. ---
from: ramacd <ramacd@nbnet.nb.ca> date: Sun, Apr 15, 2007 at 12:05 PM
... On the original idea of this thread, having a REPL facility, which doesn't come in primitive-only J, doesn't strike me as a stretch of its capabilities. I do wonder at the (if you want a responsive computer, don't let a user near J) attitude, and where it comes from. Amelia's note that J "isn't meant for user input." crystallizes a big misgiving I have about J, even though I have been pro-J since J has been around. ---
from: Devon McCormick <devonmcc@gmail.com> date: Sun, Apr 15, 2007 at 1:38 PM
Randy –
I think it's a mis-characterization to say that J "isn't meant for user input". In fact, due to its interpretive nature, J handles user input beautifully compared to any conventional language, if you use the language itself. Of course, as an avid APLer, I'm sure this isn't news to you. But this prompt-based notion won't die, no matter how crippled it is compared to simply using native J. Take, for instance, the outline of the program that Amelia is writing: allow someone to enter some numbers, then return some descriptive statistics about them. Using J, I would naturally do something like
(<./,>./,mean,stddev) 97.1 4.7 33 1.3 22 60 27.6 96.4 87.1 67.5 1.3 97.1 49.67 36.744585
However, as Amelia's professor has constrained the problem, the "correct" session might look more like this:
Enter some numbers: 97.1 4.7 33 1.3 22 60 27.6 96.4 87.1 67.5 ** Error: can only accept 5 numbers. Enter some numbers: 97.1 4.7 33 1.3 22 The minimum is 1.3. The maximum is 97.1. The mean is 31.62. The standard deviation is 38.813876.
or some such limited, simplistic implementation. Notice how thoroughly inadequate this prompt-based paradigm is for getting any serious work done: it's probably constrained to some fixed number of entries or necessitates added complexity to accommodate a variable number of entries. Furthermore, an input mistake can only be corrected by re-typing the entire set of entries and the result can only be used by manually transferring it. (Of course, using a spreadsheet may also be a good way to handle a problem like this.)
This prompt+manual input notion is a lousy paradigm that is encumbered by widely-accepted, unnecessary limitations of the past 50 years. Today, we have scalpels and laser beams but most people still insist on using sticks and rocks.
However true all this all is, the problem of user input seems to be a recurring one in J. Those of us who use the language a lot have solved it for ourselves but it recurs regularly for beginners. We've talked about this a bit in the NYCJUG and the consensus seems to be 1) use file-based input, or 2) use a spreadsheet front-end.
One other idea is to use the GUI-building capabilities of some other language, like Java or VB, and interface the resulting GUI with J. Until get something like this fully fleshed out on the Wiki, we may have to make do by referring people to the character-based solutions for Amelie that came out of this thread.
The Power of One-line Statements
We explored the implications of this essay outlining the power of one-line statements. It's such a commonplace notion in a language like J that we might fail to appreciate how important it is, especially in the context of the REPL (read-evaluate-print-loop) under which J natively operates. The benefit of immediate feedback inherent in this way of working is amplified by being able to express complete, complex expressions in a single line.
Show-and-tell
David Lambert presented some work he's done in J on bilinear interpolation (linear interpolation along each of two dimensions). For a general introduction to the topic, take a look at this Wikipedia article on the subject. David mentioned that his code should extend easily to higher-dimensional interpolation with small modifications, e.g. changing the i.2 in the definition of n244 below to something like i.{:$].
David Lambert on Interpolation
Goal: given f at 4 corners determine f(x; y) within that square.
f2 f3 (1,1) f23 = f2+(f3-f2)*(x-x0)%(x1-x0) +---------------+ +--x------------+ | f(x,y) | | f(x,y) | | (x,y) | | (x,y) | | * (0,0) | | * | | * | | | | | | | | | | | |(_1,_1) | | f01 | +---------------+ +--x------------+ f0 f1
Algebraically this is a bilinear interpolant with terms constant, x, y, xy.
As an alternate route to the same formula and positional weights associated with each corner then 
. In finite element language, the weights are the values of the shape functions n,,i,, . We need a shape function for each node. Since f at a node should be the known value, n,,i,, must be 1 at its node,,i,,
and 0 at all other nodes, giving us equations to solve. Given the depicted nodal order, 4{.N2 are the coordinates of the corners of the reference element.
NB. 336330 (-: A.) 8 2 6 0 5 7 1 3 4 [<"1 N2 =: 336330 A.-.3x#.inv i.*:3 +-----+----+----+---+----+----+---+---+---+ |_1 _1|1 _1|_1 1|1 1|0 _1|_1 0|1 0|0 1|0 0| +-----+----+----+---+----+----+---+---+---+
Expecting y as an (x, y) vector, our 2-dimensional quadrilateral shape functions with 4 terms are
n244 =: ([: , [: *// ^/&(i.2)) :[: NB. all linear combinations
with which we determine the coefficients for each term.
identity =: =@:i. :[:
C244 =: x:inv@:(x:@:identity@:# %. n244"1)4{.N2
Yielding a dyadic interpolation hook involving a bunch of matrix products. Respectively, x and y are the known function values at the nodes f,,i,, and the local coordinate vector where to evaluate f.
mp =: ($: |:) : (+/ .*) i244 =: (mp (C244 mp~ n244)) ([: :)
Thus we have a general method to find interpolants of various orders and values at known coordinates, as seen interpolation#J here.
I'm interested in mesh generation for the free calculix finite-element program. Along an edge only the nodes along that edge determine the interpolation. Thus alignment of side-by-side shapes. By grading a mesh on my basis square using higher order interpolants (reposition the non-corner nodes), then interpolating onto new positions I can create a mesh (element connectivity indices into nodes ; nodes), and given other verbs to merge and draw. Or create tie-die shirt patterns, GRID =: |.,~"0/~(%~i:)100.
Advanced topics
Here we compare an "obvious" loopy, conditional expression of a simple algorithm to more elegant array-oriented solutions, and we find that these latter solutions are as easily understandable as the initial one.
Forward Filling without Looping or Conditionals
We discussed a problem I had posted to the forum - I'd been working in R and was having trouble getting my viewpoint back into a fully array-oriented way of thinking, so I illustrated what I wanted to do with some loopy, conditional code though I knew this wasn't a good way to do this in J. I got a number of good responses that may prove useful for showing the different perspective taken by array-oriented solutions.
from: Devon McCormick <devonmcc@gmail.com> to: J-programming forum <programming@jsoftware.com> date: Mon, Jan 5, 2015 at 11:06 AM subject:Forward fill w/o looping
I just threw together a short J solution that does what I want but it uses a loop and a conditional. I'm thinking there's got to be some kind of loopless prefix sort of solution but am at a loss to come up with it.
My code looks like this:
fillEmptyFwd=: 3 : 0
for_ix. i.<:#y do.
if. 0=#>y{~>:ix do. y=. (ix{y) (>:ix)}y end.
end.
y
)
An example usage would be this:
fillEmptyFwd '';'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';'' ++--+--+--+--+--+---+---+---+---+---+---+--+--+ ||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|Yo| ++--+--+--+--+--+---+---+---+---+---+---+--+--+
Any ideas on something less loopy/conditional?
---
from: R.E. Boss <r.e.boss@outlook.com> date: Mon, Jan 5, 2015 at 11:21 AM
Just from the hip.
;(<@(##{.);.2~ 1,~2>/\a:e.~]) '';'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';''
++--+--+--+--+--+---+---+---+---+---+---+--+--+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|Yo|
++--+--+--+--+--+---+---+---+---+---+---+--+--+
R.E. Boss
---
from: 'Pascal Jasmin' via Programming <programming@jsoftware.com> date: Mon, Jan 5, 2015 at 11:26 AM
dyadic ;. is your usual friend in these cases. Though Raul has often found even more elegant approaches.
a: (-.@= <;.1 ] )'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';''
┌──────┬─────┬───────┬───────┬─────┐
│┌──┬┬┐│┌──┬┐│┌───┬┬┐│┌───┬┬┐│┌──┬┐│
││Hi│││││Ho││││hee│││││haw│││││Yo│││
│└──┴┴┘│└──┴┘│└───┴┴┘│└───┴┴┘│└──┴┘│
└──────┴─────┴───────┴───────┴─────┘
; (# # {. )each a: (-.@= <;.1 ] )'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';''
┌──┬──┬──┬──┬──┬───┬───┬───┬───┬───┬───┬──┬──┐
│Hi│Hi│Hi│Ho│Ho│hee│hee│hee│haw│haw│haw│Yo│Yo│
└──┴──┴──┴──┴──┴───┴───┴───┴───┴───┴───┴──┴──┘
its a bit more complicated with possible leading blank: (this still works without leading blank)
; (# # {. ) each a: (] <;.1~ 1 (0}) -.@= )'';'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';''
┌┬──┬──┬──┬──┬──┬───┬───┬───┬───┬───┬───┬──┬──┐
││Hi│Hi│Hi│Ho│Ho│hee│hee│hee│haw│haw│haw│Yo│Yo│
└┴──┴──┴──┴──┴──┴───┴───┴───┴───┴───┴───┴──┴──┘
---
from: 'Pascal Jasmin' via Programming <programming@jsoftware.com> date: Mon, Jan 5, 2015 at 11:36 AM
as a single more general function
a: ;@:(] <@(# # {. );.1~ 1 (0}) -.@= ) '';'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';''
0 ;@:(] <@(# # {. );.1~ 1 (0}) -.@= ) 0 33 0 0 22 0 0 11 0 0 0
0 33 33 33 22 22 22 11 11 11 11
---
from: Roger Hui <rogerhui.canada@gmail.com>
date: Mon, Jan 5, 2015 at 11:59 AM
{{{ t=: '';'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';''
fillEmptyFwd t
┌┬──┬──┬──┬──┬──┬───┬───┬───┬───┬───┬───┬──┬──┐
││Hi│Hi│Hi│Ho│Ho│hee│hee│hee│haw│haw│haw│Yo│Yo│
└┴──┴──┴──┴──┴──┴───┴───┴───┴───┴───┴───┴──┴──┘
(({.t),b#t) {~ +/\b=: *#&>t
┌┬──┬──┬──┬──┬──┬───┬───┬───┬───┬───┬───┬──┬──┐
││Hi│Hi│Hi│Ho│Ho│hee│hee│hee│haw│haw│haw│Yo│Yo│
└┴──┴──┴──┴──┴──┴───┴───┴───┴───┴───┴───┴──┴──┘
---
from: 'Pascal Jasmin' via Programming <programming@jsoftware.com> date: Mon, Jan 5, 2015 at 1:46 PM
a verbed variation of Roger's
(] (({.@:[ , #~) {~ +/\@:] ) [: * #&>) '';'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';''
---
from: Roger Hui <rogerhui.canada@gmail.com>
date: Mon, Jan 5, 2015 at 3:08 PM
fef=: a:&~: (+/\@[ { {.@] , # ) ]
fef t
---
from: Jose Mario Quintana <jose.mario.quintana@gmail.com> date: Mon, Jan 5, 2015 at 3:15 PM
> I'm thinking there's got to be some kind of loopless prefix sort of solution but am at a loss to come up with it.
Maybe it was a suffix, rather than a prefix, solution what you really had in mind?
]`[@.('' -: ])&.>~/\.&.|.
---
from: Marshall Lochbaum <mwlochbaum@gmail.com> date: Mon, Jan 5, 2015 at 4:13 PM
Beat me to it by a little. You can tighten it up by not unboxing, and speed up the performance a tiny bit using selection rather than agenda.
(, {~ a:=[)/\.&.|.
Roger's solution is significantly faster due to the global select.
Summary of Solutions
Ran submissions on a slightly different input (having a new, non-empty character vector in the last position).
tnew=. '';'Hi';'';'';'Ho';'';'hee';'';'';'haw';'';'';'Yo';'You'
NB. From Pepe:
]`[@.('' -: ])&.>~/\.&.|. tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
NB. From Roger (2nd):
(a:&~: (+/\@[ { {.@] , # ) ]) tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
NB. (verbed version of Roger(1)) from Pascal:
(] (({.@:[ , #~) {~ +/\@:] ) [: * #&>) tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
NB. (verbed version of Roger(1)) from Pascal:
(] (({.@:[ , #~) {~ +/\@:] ) [: * #&>) tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
NB. Roger(1):
(({.t),b#t) {~ +/\b=: *#&>tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
NB. From R.E. Boss:
;(<@(##{.);.2~ 1,~2>/\a:e.~]) tnew
++--+--+--+--+--+---+---+---+---+---+---+--+--+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|Yo|
++--+--+--+--+--+---+---+---+---+---+---+--+--+
NB. * defect *
NB. Pascal(1):
; (# # {. ) each a: (] <;.1~ 1 (0}) -.@= ) tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
NB. Pascal(2) (more general):
a: ;@:(] <@(# # {. );.1~ 1 (0}) -.@= ) tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
This solution is more general because it works with simple, numeric vectors as well (forward filling zeros):
0 ;@:(] <@(# # {. );.1~ 1 (0}) -.@= ) 0 33 0 0 22 0 0 11 0 0 0
0 33 33 33 22 22 22 11 11 11 11
NB. Marshall:
(, {~ a:=[)/\.&.|. tnew
++--+--+--+--+--+---+---+---+---+---+---+--+---+
||Hi|Hi|Hi|Ho|Ho|hee|hee|hee|haw|haw|haw|Yo|You|
++--+--+--+--+--+---+---+---+---+---+---+--+---+
Learning and Teaching J
In this section we considered three essays that illustrate the ongoing, backward state of contemporary software and one that offers some hope.
Education Software
This is illustrated by its failure to improve education - see the article "Better Learning Through Expensive Software". If we took this title as a question, we could apply Betteridge's Law of Headlines and answer conclusively "no". A few quotes from the article indicate its general tone.
We live in a digital age, and most people in education have heard the claims of our students being "digital natives", having grown up submerged and surrounded in technology. The claim is that because today's children have access to technology they learn differently. I have yet to find any scientific evidence proving this, and I've read enough to throw cold water on the claim. The term "digital native" was coined in 2001 by an Ivy League (Oberlin, Yale, and Harvard Business School) writer, who has a self-interest in promoting the claims in his books.
...
I talked to a literacy specialist I've worked with who has been responsible for implementing three types of software across dozens of classrooms. She has experimented with many other types of software and applications, as have most teachers I know. I asked her if she has found anything that is worth the time and money. Without hesitating, she replied "No." At a whole-school level the time it takes to implement, that is, to force teachers to use a product they didn't ask for, they didn't choose, and they don't want, is a winless game.
...
The software companies trying to sell us expensive wares, all while our schools are being underfunded, will try to convince us their product will help teachers adopt project-based learning, but they won't. The software are simply shiny and new versions of a textbook.
The irony is that much of this push is being sold as "personalized learning." ... The ideas of personalized and computerized learning as a reform model both go back 100 years.
Programming with Punchcards
The next essay (also below as Why are we still programming like it is the punchcard era.pdf) asks the question "Why are we still programming like it is the punchcard era?" Maybe the dominant paradigms are a little beyond this stage, but not much - see our earlier section "Die - Prompt - Die!". From a J programmer's perspective, the essay diverges from from my own experience pretty rapidly with the opening sentence.
Here’s a scenario that’s familiar to most programmers: after making a seemingly minor program change in your text editor / IDE of choice, the compiler spews back at you tens or even hundreds of baffling compile errors.
Though, I have of course experienced enough of this with conventional languages to sympathize. Unfortunately, the proposed solution seems to start from a point at which things are already badly broken and attempts to fix it from there. The author gives examples of how he envisions a "semantic editor" working:
For example, if I would like to call the function squareRoot (which accepts numbers), and I am filling in the argument, a semantic editor would somehow prevent me from providing the string"hello!"....
• If the user enters something that is nonsense (like passing “hello!” to squareRoot), the editor should provide some feedback–perhaps the autocomplete box has the goal typeNumber at the top, and the string "hello!" is greyed out and unselectable, with its type (of String displayed).
• Plain text program editing gives users a feeling of directness and control. It’s easy to temporarily “break” the structure of the code and repair it later. In a semantic editor, we don’t want the user to get into a state where they are stuck and unable to make progress, because the UI is too constrained!
Maybe this idea can be fleshed out in a more useful fashion, but I prefer a concise powerful notation in an environment that gives me immediate feedback - both with error messages and by allowing me to easily look at intermediate results. Again, refer to what we've already covered above.
This idea is embedded in the "worse is better" syndrome from the start, as is evident from the assertion that "[a] semantic editor can be usable by children and newcomers to programming." As someone not-so-famously stated "Neophyte errors are irrelevant to language design".
Drowning but Unaware of it
Finally, to end our depressing litany of the way things are, we looked at an essay about using the Java language's "BigDecimal" class [The text of this article appears to be no longer available but you can look at the attached PDF in the Materials section below to find a copy of it.] This article - provided here as "Drowning but Unaware of it" - hopes to overcome the difficulty of using this set of libraries by providing examples and noting common errors - somehow not attributed to fundamentally flawed design. J's method of extended precision - putting "x" at the end of an integer or "x: " before a number to make it extended precision - seems pitifully simple by comparison to the multiple classes Java has for different flavors of extended precision.
Let's compare the complexity of this Java class to J's implementation. From the essay:
To use it, you create a Decimal object using one of the contructors identical to those of the BigDecimal class (see below for an example). Then, you can simply call any of the modifier methods without worry of reassignment. In fact, methods such as add, subtract, multiply, and divide purposely don't return anything just to reinforce this point, but you can modify this wrapper class to return the value if you desire. Here's an example:
1 Decimal total = new Decimal("106838.81"); 2 Decimal a = new Decimal("263970.96"); 3 BigDecimal b = new BigDecimal("879.35"); 4 Decimal c = new Decimal("366790.80"); 5 total.add(a); 6 total.subtract(b); 7 total.subtract(c); 8 System.out.println(total.toString()); which, by the way, returns the result "3139.62".
This corresponds to the J expression
]total=. (106838.81+263970.96-x: 879.35)-366790.80 3139.62
The article goes on to say this:
Other methods work the same way. For instance: 1 2 total.negate(); System.out.println(total.toString()); Returns the result "-3139.62".
Again, in J, this is simply:
]total=. -total _3139.62
Another Approach
Someone in the group mentioned that Mathematica uses the Fast Fourier Transform to perform multiplications of very large numbers.
One Bright Light
We attempt to end on an "up" note by referring to this summary of a talk (also below and in Materials section below as DNA Everywhere) given by Charles Brenner at APLBUG - the APL Bay Area User Group - this month. In it, he attributes his success with solving difficult problems in forensic DNA to his use of APL, both as a notation for ideas he had while assessing the problem, and for the insight its array-orientation provided him on efficient data manipulation. He tells of competing in a national competition, against a hundred or more competitors, many of whom were teams of many people funded with millions of dollars, and winning with his APL-based code written by a team of one with far less than millions of dollars of funding.
Charles Brenner - There's DNA Everywhere
I've made recent and rapid progress in my long-delayed venture to convert my forensic DNA interpretation work into the modern world of Dyalog APL.
The impetus is solving a particular new problem (as opposed to merely converting legacy code that implements old solutions). The particular problem at issue is to calculate whether and how strongly a suspect can be inculpated as a likely contributor to a DNA mixture, meaning a combination of several people's DNA. For example, rape evidence typically includes the victim and assailant DNA comingled. Or, DNA detection technology has becomes so sensitive that DNA can be detected on the grip of a gun from mere touch, but as a consequent complication it is common that several other people overlaid their DNA as well.
In my experience computer folks enjoy the scientific aspects of the story, and I would enjoy relating how in my view this project exemplifies the thesis that APL is a tool of thought. I credit APL with leading me to elegant and therefore very fast and flexible solutions to a problem for which the competing solutions are lumbered by complicated statistical and Monte Carlo methods. Elegance and simplicity lead to several concrete benefits. The first is conceptual development. From January through September of 2013, I made notes in APL as one after another solution to various aspects of the problem occurred to me. The brief APL notes ensured that the ideas really make sense, that they work together, and that I would not forget them. The brevity also revealed a simple but important point that others had overlooked: nesting the computation loops in the right order saves orders of magnitude in computation time.
That the program runs fast makes it much easier to see the forest in many ways -- testing, developing, designing. The leading competing programs take minutes or hours to find the answer for the "maximum likelihood (ML)" contributor proportions. (The meaning of those technical words doesn't matter for the story.) Having worked that hard, both of those programs then succumb to a natural tendency to call it a day.
The APL "DNA-VIEW Mixture Solution" program finds the ML answer in a fraction of a second which makes it much easier to think through to the fact that there's a lot more work to do, dozens or thousandsfold more computation before the result is logically defensible. Another example: With DNA forensic calculations, it's usual, if a background population is necessary, to compute assuming one particular racial population, and in a multi-racial country like the US to repeat the calculation for each race. After crafting a careful APL-flavor implementation -- element-wise inner products and suchlike -- it became obvious from the code itself that with little cost, essentially by replacing a scalar with a vector, I could not only calculate several races simultaneously but moreover can cater to the possibility of different races for two different contributors as for example with a gang rape. The idea to account for the possibility of one white and one black rapist thus is a bonus suggested by the concise APL formulation.
Materials
- File:Bilinear Interpolation.pdf Wikipedia article on bilinear interpolation
- File:FEM.pdf Bilinear Interpolation by David Lambert
-- Devon McCormick <<DateTime(2015-01-20T16:03:39-0200)>>