NYCJUG/2017-04-11

Meeting Agenda for NYCJUG 20170411

1. Beginner's regatta: see "Beyond 'Hidden Figures': Nurturing New Black and Latino Math Whizzes".

2. Show-and-tell: see "Randomly Checking Files on a Disk". 3. In development: see "Towards a Progressive Differencer".

4. Advanced topics: see "If-Else Simplification?"

+.--------------------+.

Grammar, it's the difference between knowing your shit, and knowing you're shit.

 - Anonymous Coward

Beginner's regatta

We discussed the following article and looked at some J solutions to some of the problems.

Beyond ‘Hidden Figures’: Nurturing New Black and Latino Math Whizzes

By AMY HARMONFEB. 17, 2017

One afternoon last summer at BEAM 6, an experimental program in downtown Manhattan for youths with a high aptitude for math, a swarm of 11- and 12-year-olds jockeyed for a better view of a poster labeled “Week One Challenge Problem.”

Is there a 10-digit number where the first digit is equal to how many 0’s are in the number, the second digit is equal to how many 1’s are in the number, the third digit is equal to how many 2’s are in the number, all the way up to the last digit, which is equal to how many 9’s are in the number?

Within the scrum was a trio of friends-in-formation: “Can we work on this during Open Math Time?” one asked. The second, wearing red-and-black glasses and dogged by the fear that he did not belong — “I’m really not that good at math,” he had told me earlier — lingered at the snack cart. “Leave some for the rest of us, J. J.,’’ demanded the third, gently elbowing him aside.

To Mira Bernstein, a BEAM instructor and a leading figure in the extracurricular math ecosystem that incubates many of the nation’s scientists and engineers, the scene was unremarkable, except for one striking feature: None of the children were wealthy, and few were white or Asian.

Are You Ready for Math Whiz Camp?

Try your hand at five math problems given to students at BEAM. The questions require no knowledge beyond addition and multiplication — but that doesn’t mean they’re easy.

Here are five math problems given to students at BEAM, a program for students with high math aptitude from New York City public schools with low-income populations. The questions require no knowledge beyond addition and multiplication. But that doesn't mean they're easy.

1 of 5

Lianna makes two 4-digit numbers using each of the digits 1, 2, 3, 4, 5, 6, 7 and 8 exactly once. If Lianna makes the numbers so that adding them gives the smallest possible total, what is that total?

A J solution:

   !8
40320
   $all8=. (i.!8) A. >:i.8
40320 8
   1357+2468
3825
   $pairs=. 1 0 0 0 1 0 0 0<;.1"1 all8
40320 2
   3{.pairs
+-------+-------+
|1 2 3 4|5 6 7 8|
+-------+-------+
|1 2 3 4|5 6 8 7|
+-------+-------+
|1 2 3 4|5 7 6 8|
+-------+-------+
   +&>/"1]3{.pairs
6 8 10 12
6 8 11 11
6 9  9 12
   10#:&>3{.pairs
1 2 3 4
5 6 7 8

1 2 3 4
5 6 8 7

1 2 3 4
5 7 6 8
   10#.&>3{.pairs
1234 5678
1234 5687
1234 5768
   $pairs=. 10#.&>pairs
40320 2
   (]i.<./) +/"1 pairs
1032
   1032{pairs
1357 2468
   +/3825=+/"1 pairs
16

2 of 5

Place each of the numbers 1, 2, 3, 4, 5, 6, 7, and 8 in a circle in the diagram below so that no two numbers that are 1 apart are in connected circles. (For example, 2 and 3 cannot be in circles that are connected by a line.) By TOM WRIGHT-PIERSANTI and AMY HARMON FEB. 17, 2017

 
  3 5
7 1 8 2   NB. Example solution found by trial & error
  4 6

  1 4
0 2 5 7   NB. Build this graph...
  3 6

ftg=. ".&><;._2 ] 0 : 0       NB. From-To Graph
0 0 0 1 1 1 1 2 2 2 2 2 2 3 3 3 3 4 4 4 4 5 5 5 5 5 5 6 6 6 6 7 7 7
1 2 3 0 2 4 5 0 1 3 4 5 6 0 2 5 6 1 2 5 7 1 2 3 4 6 7 2 3 5 7 4 5 6
)
   $all8        NB. All combos of 8 integers, from above: all8=. (i.!8) A. >:i.8
40320 8
   $-/&>(<ftg){&.><"1 all8
40320 34
   +/-.+./"1]1 e.~-/&>(<ftg){&.><"1 all8
4
   I.-.+./"1]1 e.~-/&>(<ftg){&.><"1 all8
7878 8604 31715 32441
   ixs=. I.-.+./"1]1 e.~-/&>(<ftg){&.><"1 all8

   >ixs { (<ftg){&.><"1 all8
2 2 2 5 5 5 5 8 8 8 8 8 8 6 6 6 6 3 3 3 3 1 1 1 1 1 1 4 4 4 4 7 7 7
5 8 6 2 8 3 1 2 5 6 3 1 4 2 8 1 4 5 8 1 7 5 8 6 3 4 7 8 6 1 7 3 1 4

2 2 2 6 6 6 6 8 8 8 8 8 8 5 5 5 5 4 4 4 4 1 1 1 1 1 1 3 3 3 3 7 7 7
6 8 5 2 8 4 1 2 6 5 4 1 3 2 8 1 3 6 8 1 7 6 8 5 4 3 7 8 5 1 7 4 1 3

7 7 7 3 3 3 3 1 1 1 1 1 1 4 4 4 4 5 5 5 5 8 8 8 8 8 8 6 6 6 6 2 2 2
3 1 4 7 1 5 8 7 3 4 5 8 6 7 1 8 6 3 1 8 2 3 1 4 5 6 2 1 4 8 2 5 8 6

7 7 7 4 4 4 4 1 1 1 1 1 1 3 3 3 3 6 6 6 6 8 8 8 8 8 8 5 5 5 5 2 2 2
4 1 3 7 1 6 8 7 4 3 6 8 5 7 1 8 5 4 1 8 2 4 1 3 6 5 2 1 3 8 2 6 8 5
   $solns=. >ixs { (<ftg){&.><"1 all8
4 2 34
   ~.0{0{solns                NB. values for 0-7
2 5 8 6 3 1 4 7
   ":&.>~.0{0{solns
+-+-+-+-+-+-+-+-+
|2|5|8|6|3|1|4|7|
+-+-+-+-+-+-+-+-+
   to=. ":&.>~.0{0{solns      NB. map to this from 0-7
   from=. ":&.>i.8
   graphPic rplc ,from,.to
  5 3
2 8 1 7   
  6 4

   ]graphDisp=. 3 4$'';1;4;'';0;2;5;7;'';3;6;''
+-+-+-+-+
| |1|4| |
+-+-+-+-+
|0|2|5|7|
+-+-+-+-+
| |3|6| |
+-+-+-+-+

   graphDisp{&.><~.0{0{solns
+-+-+-+-+
| |5|3| |
+-+-+-+-+
|2|8|1|7|
+-+-+-+-+
| |6|4| |
+-+-+-+-+
   
   2 0 1|:graphDisp{&.>/<"1 ~."1]0{"2 solns
+-+-+-+-+
| |5|3| |
+-+-+-+-+
|2|8|1|7|
+-+-+-+-+
| |6|4| |
+-+-+-+-+

+-+-+-+-+
| |6|4| |
+-+-+-+-+
|2|8|1|7|
+-+-+-+-+
| |5|3| |
+-+-+-+-+

+-+-+-+-+
| |3|5| |
+-+-+-+-+
|7|1|8|2|
+-+-+-+-+
| |4|6| |
+-+-+-+-+

+-+-+-+-+
| |4|6| |
+-+-+-+-+
|7|1|8|2|
+-+-+-+-+
| |3|5| |
+-+-+-+-+

   <"2]2 0 1|:graphDisp{&.>/<"1 ~."1]0{"2 solns
+---------+---------+---------+---------+
|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|
|| |5|3| ||| |6|4| ||| |3|5| ||| |4|6| ||
|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|
||2|8|1|7|||2|8|1|7|||7|1|8|2|||7|1|8|2||
|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|
|| |6|4| ||| |5|3| ||| |4|6| ||| |3|5| ||
|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|+-+-+-+-+|
+---------+---------+---------+---------+
   NB. Show reflections
   |:2 2$<"2]2 0 1|:graphDisp{&.>/<"1 ~."1]0{"2 solns
+---------+---------+
|+-+-+-+-+|+-+-+-+-+|
|| |5|3| ||| |3|5| ||
|+-+-+-+-+|+-+-+-+-+|
||2|8|1|7|||7|1|8|2||
|+-+-+-+-+|+-+-+-+-+|
|| |6|4| ||| |4|6| ||
|+-+-+-+-+|+-+-+-+-+|
+---------+---------+
|+-+-+-+-+|+-+-+-+-+|
|| |6|4| ||| |4|6| ||
|+-+-+-+-+|+-+-+-+-+|
||2|8|1|7|||7|1|8|2||
|+-+-+-+-+|+-+-+-+-+|
|| |5|3| ||| |3|5| ||
|+-+-+-+-+|+-+-+-+-+|
+---------+---------+ 

 

3 of 5

Is there a 10-digit number where the first digit is equal to how many 0’s are in the number, the second digit is equal to how many 1’s are in the number, the third digit is equal to how many 2’s are in the number, all the way up to the last digit, which digit is equal to how many 9’s are in the number?

 
   sums=. +/"1 (i.!10) A. i.10
   10+/ . = sums
0

Oops:

    6 2 1 0 0 0 1 0 0 0

As we mention in the story that inspired this quiz, this question was the first week’s challenge problem at BEAM camp last summer.

Here's an observation: There can't be more than one digit that's 4 or bigger. Why not? Well, the sum of the digits must be 10 (since each tells us how many spaces there are total). If there are 2 digits that are 4 (or bigger), then that's already at least 8. And only 1 of the 4’s can be in the 0 place, so the other 4 tells us there is a non-zero digit of which there are 4, which adds at least 4 to the sum of the digits — which is too much. (Similarly, we'll end up with too much for anything greater than 4.)

The above tells us that in the 4, 5, 6, 7, 8, and 9 places, there can be at most one 1, and all the rest must be 0’s. So there are at least five 0’s. Can there be more? If there were eight or nine 0’s, there would be a 1 in the 8 (and, respectively, 9) place. But then there must be at least a 1 in the 1 place, and this brings us to a number with 11 digits, which is too many. If there were seven 0’s, then there is a 7 in the 0 place, which means there has to be a 1 in the 7 place. This means there can't be a 0 in the 1 place, and there must be 0’s everywhere else. This looks like this:

7 _ 0 0 0 0 0 1 0 0

Now what belongs in the 1 place? If it's a 1, there should be a 2 in the number, which is a contradiction. But if it's a 2, that tells us there are two 1’s in the number, which we can see is not the case. So there's nothing we can put in the 1 place to make it work.

If there are six 0’s, then we have to put a 1 in the 6 place. That means the 1 place can't be a 0. It can't be a 1, either, because then there are two 1’s. If we put in a 2, then the 2 place can't be a 0. That means the rest of the spots must be 0’s.

6 2 _ 0 0 0 1 0 0 0

Thus, the 2 place must have a 1, since there is one 2. This number, 6 2 1 00 0 1 0 0 0, works. For good measure, let's show we can't have five 0’s. As before, we need to have a 1 in the 5’s spot, which tells us the number is:

5 _ _ _ 0 1 0 0 0 0

We can't put a 3 or larger number in the 1st place, because then we'd have to put a 1 in both the remaining spots, and in the 2’s place, it would tell us that there is a 2 somewhere. If we put a 2 in the 1’s place, then we need to place both a 1 in one of the blanks, and something non-zero in the other, but we don't have space to add another 2 or another 3. (Try this out yourself.)

   NB. (10^10) = #{10$<i.10

The 76 students, drawn from New York City public schools with low-income populations, were embarking on a curriculum that they would have to continue on their own during the school year to be eligible to apply for a second, even more intense math summer program, BEAM 7. That application is due at the end of this month, and with it comes a verdict of sorts on their membership in a math-geek subculture where it can be astonishingly difficult to find others who look like them.

“This is probably more math-y black and Hispanic kids than I’ve seen in my whole career,” said Dr. Bernstein, who received a Ph.D. in algebraic geometry from Harvard in the 1990s. “That’s why I’m here.” The extreme racial homogeneity in the rarefied realm of young math wizards has drawn little attention in a nation where racial equality in the basic institutions of civic life — schools, housing, health care, policing — remains elusive. But it has become an increasing source of consternation for some mathematicians, educators and business leaders, who see it directly linked to the striking underrepresentation of blacks and Latinos in high-paying, high-status jobs in finance, science and technology. As those occupations increasingly propel our society, they fear that enrichment programs for mathematically gifted children, while rooted in meritocratic ideals, have become a particularly potent means of reinforcing privilege.

Even as movie audiences celebrate “Hidden Figures,” the story of black women who overcame legally sanctioned discrimination to perform critical calculations in the race to put a man on the moon, educators say that new, subtler obstacles to higher-level math education have arisen. These have had an outsize influence on racial prejudice, they contend, because math prowess factors so heavily in the popular conception of intelligence — a concern that recently provoked the creation of “Mathematically Gifted and Black” and “Latin@s and Hispanics in Mathematical Sciences,” websites featuring math professionals from underrepresented backgrounds.

“Fundamentally, this is a question about power in society,” said Daniel Zaharopol, BEAM’s director. “Not just financial power, but who is respected, whose views are listened to, who is assumed to be what kind of person.” BEAM is short for Bridge to Enter Advanced Mathematics, and this, the program’s first year of BEAM 6, for students who had just completed the sixth grade, is what many within elite math circles see as the most promising effort yet to diversify their ranks. The four weeks, spent in a school near City Hall, would be intense: four hours of math a day taught by 10 experienced math teachers, several of them Ph.D.’s. There would be no prepping for standardized tests or effort to cover school material at a faster pace. Instead, as in the elite summer programs that Mr. Zaharopol had himself attended, BEAM focused on the kind of creative problem-solving that mathematicians say lie at the heart of the discipline.

And because one summer would not be anywhere near enough to equip the BEAM 6-ers with the same kind of math preparation as their more affluent peers, the real goal of the founders — a mix of hedge-fund millionaires and professional mathematicians — was to hook them enough to want to keep at it.

A graduate of the Massachusetts Institute of Technology who suspended his graduate studies in algebraic topology to launch BEAM, Mr. Zaharopol had some idea of what his students were up against. By age 12, many attendees of the highly selective programs on which BEAM was modeled had already won regional math contests and completed online math courses in subjects like number theory. Some had been introduced to advanced math in school honors programs or by math teams. Many were the children of scientists and engineers, or of well-off parents who turned to math as another way to give their children a competitive edge in the battle for admission to elite colleges.

“When I was 6, my dad taught me to use the abacus and do mental math,” one student at a selective program called MathPath wrote on the application form. By contrast, BEAM students come from environments where “math is not in the air,” Dr. Bernstein said.

The parents of the boy at the snack cart, Jonathan Jackson, do not have degrees in math or science. His school, Kipp STAR, is a well-regarded charter school in Harlem, but does not have a math team. And like many of the other students at BEAM, Jonathan, who is African-American, seemed to have already internalized the racial stereotypes about math that, studies have shown, shape self-image among young Americans of all backgrounds.

“They did a study that shows white kids have more chances,” he said once, remembering a fragment of something he had been told. Then he shrugged, “I don’t know if it’s really true.”

Show-and-tell

Randomly Checking Files on a Disk

Here is some code we developed recently to check how well the files on old CDs (circa 1999) have remained readable.

Initial Development

   qts''
2017 4 9 11 35 52.522
   6!:2 '''FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP''=. PllDirInfoEG ''H:\'''
1521.61
   $FLNMS
47081
   100%~#FLNMS
470.81
   13 : 'y?~<.0.5+x%~y'
] ?~ [: <. 0.5 + %~
   ixs=. 100 (] ?~ [: <. 0.5 + %~) #FLNMS
   ixs
27195 34652 44845 13589 32469 36436 1274 21003 33605 20029 1500 42222 9439 22064 42584...
   #ixs
471
   #~.ixs
471
   paths=. (DIRNMS{~ixs{FLPARENT),&.>'/',&.>ixs{FLNMS
   >2{.paths
H:/amisc/pix/Photos/2016Q4/20161030-1106/P1570241.JPG                                     
H:/amisc/pix/Photos/2016Q4/20161025-29/P1540431.JPG                                       
   ;LF,~&.>_2{.paths
H:/amisc/pix/Photos/2016Q3/20160916-19/P1400657.JPG
H:/amisc/pix/Photos/2016Q4/20161030-1106/P1560059.JPG

   readEnds=: 13 : '(fread y;0,x<.fsize y),fread y;(x<.fsize y),~(fsize y)-x<.fsize y'
   readEnds
([: fread ] ; 0 , [ <. [: fsize]) , [: fread] ; ([ <. [: fsize]) ,~ ([: fsize]) - [<.[: fsize ]
   fsize 'e:/amisc/J/J8asus170408.log'
8286
   20 readEnds 'e:/amisc/J/J8asus170408.log'
e:\amisc\ofldinfo>Re8asus170408.log'
8286
   6!:2 'dd=. 10 readEnds&.>paths'  NB. check readability of 1% of randomly-chosen files
qts''
209.125
   2017 4 9 12 12 25.971

Code

NB.* checkDisk.ijs: randomly check readability of
NB. randomly-chosen files on a disk. Use to spot-check
NB. integrity of backup disks like DVDs or BluRays.

NB. coclass CONM=: 'chkDsk'
require 'bkp'

checkSome=: 4 : 0
   tms=. 0 [ portion=. x [ disknm=. y
   if. -.nameExists 'FLNMS' do.
       tms=. 6!:2 '''FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP''=. PllDirInfoEG ''',disknm,'\'''
   end.
   ixs=. portion (] ?~ [: <. 0.5 + *~) #FLNMS     NB. Choose files randomly
   paths=. (DIRNMS{~ixs{FLPARENT),&.>'/',&.>ixs{FLNMS
   tms=. tms,6!:2 'dd=. 10 readEnds&.>paths'
)

readEnds=: 13 : '(fread y;0,x<.fsize y),fread y;(x<.fsize y),~(fsize y)-x<.fsize y'

The following does not work properly. The idea is to save the results of the time-consuming step in its own locale so that multiple checks can be run against it.

NB.* checkSaving: check disk's files but save directory & file info for re-use
NB. in user-specified class-object (namespace) name.
checkSaving=: 3 : 0
   'portion dsknm conm'=. y   NB. Portion to check, disk name, class-object name
   coclass conm [ svStartLocale=. coname ''
   if. -.nameExists FLNMS do.
       'FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP'=. PllDirInfoEG disknm,'\'
   end.
   tms=. portion checkSome dsknm
   coname svStartLocale
)

Some Examples with Timings

   qts''
2017 4 9 12 14 46.64
   0.02 checkSome 'H:'
54.162 363.278         NB. Timings for the two sections: gathering data and checking 2% of files.
   qts''
2017 4 9 12 21 44.889

   6!:2 '''FLNMS FLDTS FLSZS FLPARENT DIRNMS DIRDEP''=: getDirFlInfo ''H:\'''
0.799529
   $FLNMS
23415
   1!:44 'E:\amisc\J\NYCJUG\201704\'

   load 'checkDisk.ijs'
   portion=. 0.02
   ixs=. portion (] ?~ [: <. 0.5 + *~) #FLNMS     NB. Choose files randomly
   $paths=. (DIRNMS{~ixs{FLPARENT),&.>'/',&.>ixs{FLNMS
468
   6!:2 'dd=. 10 readEnds&.>paths'
190.764

Advanced topics

If-Else Simplification?

Not really a J topic per se, but which of these two equivalent variants seems simpler?

        if (nrow(rm_df)==numRows)
        {   success_fail<-"Success"
        } else
        {   success_fail <-"Fail"
        }

versus this equivalent:

        success_fail <-"Fail"
        if (nrow(rm_df)==numRows){ success_fail<-"Success" }

To me, the second version is clearly simpler and more straightforward. However, there is no lack of examples like the first, as clumsy as it is by comparison.