NYCJUG/2016-04-12
Meeting Agenda for NYCJUG 20160412
Beginner's regatta
We look at some Jedi ways to find things in J according to some J users and talk about shortcomings of the current ways of searching for particular J functionality.
J Addons Should be Easier for Users to Discover
from: Alex Shroyer <ashroyer@gmail.com> to: programming@jsoftware.com date: Fri, Dec 4, 2015 at 3:20 PM subject: [Jprogramming] Addons should be easier for users to discover
As Steve Yegge has <https://plus.google.com/+RipRowan/posts/eVeouesvaVX> pointed out <https://sites.google.com/site/steveyegge2/bambi-meets-godzilla>, accessibility and marketing are what separate ideas that survive from ones that pass into obscurity. One thing I think J could be doing better is making the standard libraries easier to discover.
Example:
The 4th puzzle on Advent of Code <http://adventofcode.com/> requires generating MD5 hashes of a string. I've been using J for several months, but the first thing I found when looking for MD5 was this page <http://code.jsoftware.com/wiki/User:Pascal_Jasmin/SHA_1(2c20)2_and_MD5_for_windows>. It wasn't until much later that I discovered I already possessed on my computer a J addon with this functionality, in the form of *addons/convert/misc/md5.ijs*!
This is an accessibility problem. In my ideal world, the top result for googling the phrase "J md5" would be a page from jsoftware.com containing documentation for the above addon, with pretty formatting, syntax highlighting, example uses, and links to related addons.
My realistic expectation is that typing queries such as "md5" or "ssl" into the search bar on jsoftware.com (or the code.jsoftware.com wiki) would at minimum return links to addons containing those words. Currently, the closest hit (other than wiki user Pascal Jasmin's page) is the j802 release notes page <http://code.jsoftware.com/wiki/System/ReleaseNotes/J802>, which mentions cryptographic functions exist, but provides no link to further documentation. If the J scripts library is supposed to be like the .NET Framework or CPAN, it needs to be more discoverable. Indexing addons for keywords would go a long way toward this, I think. I'm posting this complaint because J is my preferred language, and it makes me sad to see it losing mindshare to products like Matlab. Sincerely,
Alex
---
from: Joe Bogner <joebogner@gmail.com> date: Fri, Dec 4, 2015 at 4:04 PM
In general, I'd say that J has excellent documentation. It's well above average for non-main stream languages. That being said, there's certainly always room for improvement in documentation – especially when things are continuing to improve and change.
I had the same experience as Alex but eventually found my way to an answer, so I'll share my steps below.
1. Search for md5 on code.jsoftware.com - first hit pascal's. Try it, but ran into issues with some missing definitions (e.g. listhash) http://code.jsoftware.com/wiki/User:Pascal_Jasmin/SHA_1(2c20)2_and_MD5_for_windows
2. grep my addons folder for md5 in *.ijs... find out that it produces the wrong result on j64
3. Search for md5 on http://www.jsoftware.com/forumsearch, see issues with md5 on j64
4. Move on to next result of md5 on code.jsoftware.com http://code.jsoftware.com/wiki/System/ReleaseNotes/J802
I typically try the following resources when looking for an answer:
1. Search http://code.jsoftware.com/
2. Search http://jsoftware.com/
3. Search forums http://www.jsoftware.com/forumsearch
4. Search my local J installation *.ijs
5. Email the mailing list
This recipe has worked pretty well for me. I would say the searchability is quite high - the only downside is a few places to look. I'd rather have a few places than not enough places though.
---
from: Alex Shroyer <ashroyer@gmail.com> date: Fri, Dec 4, 2015 at 5:16 PM
Raul, I hope the search boxes on jsoftware.com and its wiki aren't also prevented from searching it!
Pascal, that's an interesting point. If I hadn't kept searching I probably would have only ever used the version you wrote up in the wiki.
Joe, I agree that the quality of the documentation is excellent. But it can be hard to find what you're looking for. The search method you use is pretty close to what I use as well, but I think we're somewhat unusual compared to an "average computer language user" who might give up more easily.
Here's an example of (in my opinion, very nicely done) documentation for community-generated addons for a programming language: http://package.elm-lang.org/
In that search bar, you can type a phrase and watch the results narrow before your eyes. Example: type 'sig' and see that not only does it find results where 'sig' is in the text (but not the title) but it also shows a warning that some packages are only compatible with a particular version of the runtime.
One can imagine other nice-to-have documentation-discovery features, such as user ratings or showing dependencies and such.
To satisfy my own itch, I started working on an analog to Matlab's interactive "help" function, which currently just returns names and their locales: https://github.com/hoosierEE/jhelp It's not much yet, but hopefully can be extended to include some of those "nice to have" features.
---
from: chris burke <cburke@jsoftware.com> date: Fri, Dec 4, 2015 at 5:29 PM
> http://jsoftware.com/robots.txt currently disallows search engines from scanning the site. [Raul Miller]
Thanks for the reminder. This was done for the old website + wiki, which was overloaded. I think it can be relaxed for the new wiki, as this has much better performance.
Show-and-tell
We look at a video that shows how to build a GUI in J.
Building a GUI in J
We looked at this script of a video demonstrating building a GUI-based application in J.
Script for Video on GUI Programming in J
This video is a brief introduction to GUI programming in J. We’re going to look at an example program I made that lets you browse the Mandelbrot set. If you want to follow along yourself, you can grab it from my github page: https://github.com/tangentstorm/mandelbrowse/
I’m jqt right now. I’m going to load the code:
load 'c:/ver/mandelbrowse/mbrowse.ijs'
And run it:
m =: mbrowse''
So, here’s the Mandelbrot set.
You can click anywhere on the image to zoom in on that point [ zoom in around (-0.5, 0.5) ] or right-click to zoom out. To keep this interaction fast, it starts out in a low resolution mode and the detail is turned down pretty low.
You can press the plus sign to increase the amount of detail after you zoom in. When you find an image you like, you can press space to switch to higher resolution. As you can see, it takes a while to redraw.
You can also press ‘r’ to reset everything back to the initial values.
That’s pretty much all this application does. Let’s take a look at the code and see how it works.
The Mandelbrot Set Itself
The most important line as far as the Mandelbrot set goes is line 46 here.
mbrot =: [: +/ 2 <&| ((+ *:) :: _:)"0^:(<ITERS) & 0
When you look at these images, the Mandelbrot set is the black part in the middle.
It’s a set of numbers on the of the complex plane from about negative two on the left to about 0.5 on the right and about 1.2 to -1.2 on the imaginary axis.
What you do is you take a number, let’s call it n and then starting from zero, you repeatedly square the value and then add n.
So for example, is n = 2 then 0 squared + 2 is 2 then 2 squared + 2 is 6 and 6 squared + 2 is 38 and it very quickly goes off to infinity, so we say it “diverges”.
But if you start with n = -1, then 0 squared + -1 is -1 and -1 squared + -1 is 0 and it just repeats in a cycle forever. The ones that go in a cycle are part of the Mandelbrot set, and the ones that go off to infinity are outside the set.
The trick is that for numbers close to the edge of the set, it can take quite a few steps before you know for sure whether or not it diverges.
So what we do here is use the power conjunction to perform a certain number of iterations and we just test whether or not the magnitude is less than two.
As soon as you move more than two units away from the origin, it always diverges. Then, since we’re doing this comparison multiple times, we end up with a whole series of bits for each point. What it’s doing is summing up the number of times it was true.
The smaller that number, the longer it took before we could tell it diverged. That doesn’t actually make any difference to whether or not it’s in the set, but what it does do is let us pick different colors and make these cool looking images.
Anyway, this mbrot verb figures out the color for each individual point.
Up here in the `reset` verb, we have a bunch of configuration settings.
Then `c2w` uses those settings to move the camera around and zoom in and out.
So then this ‘plane’ verb generates a 2d array of complex numbers for the area we’re looking at.
Then `render` applies the mbrot verb.
And then `repaint` just passes it to viewmat along with our custom color palette.
It’s actually `viewmatcc`, which is just the normal viewmat function, except it draws to a specific control in our window rather than making a window of its own.
So that’s a quick overview of how the actual Mandelbrot stuff works.
I really want to focus on the actual gui stuff, so I’m not going to explain that in any more detail.
If you have questions, feel free to ask in the comments or post to the mailing list or the IRC channel.
Or just go get the code and experiment.
The GUI Stuff
Right. So now onto the GUI stuff.
I’m going to skip over all this event handler stuff for a minute, and come down to this `create` verb.
Other than calling `reset` and `repaint` which we already looked at you can see it’s just a bunch of calls to this ‘wd’ verb. So, ‘wd’ is short for ‘window driver’, and it’s basically a little domain specific language for creating GUIs.
This first line says create a parent control (‘pc’) named ‘w’ (for window) you should be able to close it. and the minimum shape comes in from the configuration. The parent name is mandelbrowse that’s going to show up in the title bar.
It has a child control named ‘g’ for graphics and it’s an isidraw control. isi is iverson software inc. Which was the old company name for jsoftware.
So basically that’s where viewmatcc is going to draw. And what the cc stands for - child control.
There’s another child control called sb and it’s a status bar. And in QT you can have multiple sections in a status bar. So in our case, we’re adding a label called text.
And then we’re going to center the window and show it. That’s actually all you need to create a window in J. In fact, jqt is all event-driven, and it’s already running inside a gui mainloop, so you can just run wd manually to create a new window.
wd 'pc w closeok; pshow'
But really if you’re going to be writing GUI stuff you’re better off making a script.
Objects
So the reason this is called ‘create’ is because it’s a constructor. The verb we used to launch the program was mbrowse, and it’s actually defined down here in the z locale so it’s visible from everywhere.
This entire file is executing in a locale called ‘mbrowse’. That’s because the very first line… … is `coclass mbrowse`.
So locales are just namespaces, except they can be chained so that if the word you’re looking for isn’t defined, you can acquire it from the next link in the chain.
In fact, that’s what the ‘z’ locale is all about. It’s called ‘z’ because it’s always the last locale in the chain. Putting something in z means you can use it from anywhere as long as some other link in the chain didn’t override it.
Anyway, all ‘coclass’ does is change the current locale. It’s actually exactly the same as ‘cocurrent’ but for readability it indicates that you intend to treat the new locale as a class.
So then when you call ‘conew’ and pass it the name of a locale, it’ll actually create a new locale whose name is actually a number it’s going to insert the given locale into the chain by calling coinsert and then if you pass it a left argument it’s going to look for a verb called ‘create’ and call that as the constructor.
My personal preference coming from python is to hide the call to conew and just create one of these global verbs corresponding to the class name.
So that’s why we launch this thing with mbrowse” and we capture the result in a variable.
You actually don’t need conew at all to make a GUI, but if you don’t use it then if you open two windows at once, they wind up sharing the same variables and they interfere with each other. So adding those couple lines gives you a fresh copy of all your data.
So now if you want to inspect that data interactively you can say
cocurrent m
and then when you type
names''
you’ll see all your object’s private variables as well as these variables that wd sets up.
Events
Okay now that we’ve got our window and our class set up, the next step is to tell it how to behave.
That’s what these event handlers are all about.
The way this works is a naming convention. When we set up this window, we called it ‘w’ and we called the graphics control ‘g’.
So when you call ‘wd’ and create a named control like that it’s going to look in the current locale for verbs that use those names as a prefix and also correspond to the events associated with that kind of control. So for example, that ‘isidraw’ control has some events for mouse and keyboard interaction. Of it’s j, so the names are abbreviated, but here’s mouse button left up and mouse button right up mouse move and char which tells you when a keyboard character got pressed.
When these events come in, the data doesn’t come in as an argument instead, the information gets stored in those variables that wd creates in your locale. So for the isidraw control, when an event is running, all information about the event. the sysdata variable.
So this mw verb grabs the x and y coordinates of the mouse out sysdata and then convert to world coordinates based on the camera settings.
This little verb here (<.@%)~ is actually taking the floor of a division and it’s dividing by this GRAIN variable.
That’s because we actually have more pixels on the screen than data points in our matrix. GRAIN says how many real pixels correspond to each data point. And this division here is making the mouse snap to one of the actual points.
Anyway, that’s used to get the new center point when you left click. It pushes the old center onto a stack and then when you right click, it pops the old value off the stack. Unless it’s the only one left, in which case it just leaves it alone. Once the center changes, we have to repaint.
In theory, you should just be able to call viewmatcc, and pass it the name of the isidraw control. (‘g’)
But for some reason, in j804 that’s not working quite right, so I had to redefine it slightly to force it to repaint. Or you can just manually call this ‘glpaint’ verb.
Anyway, we only repaint when the image changes, and when that happens, we also want to update the status bar.
That’s really just a bunch of string formatting followed by a wd command to set the text. And of course we also update the status bar every time the user moves the mouse just so you can kind of see where you’re pointing. And then for the keyboard handler, we just do this char function.
There’s really not much to it. Sysdata is going to contain an array Because characters literals are only byte but with arrows and modifier keys you wind up needing more than one byte. I’m not messing with that stuff here though so I just take the head and then it’s a straight character comparison.
And that’s pretty much all there is to this whole application.
Hopefully that’ll get you started with making your own GUIs.
If you want to learn more, the best docs for the window driver are actually on the wiki http://code.jsoftware.com/wiki/Guides/Window_Driver
The ‘Overview’ page covers a lot of the basics I showed here but in more detail.
The ‘wd command reference’ explains the little mini language.
There’s also a whole set of drawing commands for isidraw in this gl2 locale.
And then if you look under Window Controls, this page is mostly about general things that all apply to all the controls, but at the very bottom here there’s a link to Wind Driver ChildClasses http://code.jsoftware.com/wiki/Guides/Window_Driver/ChildClasses
And that actually has details on all the controls you can use including their properties and events.
And then also if you go to.. Help / Studio / Qt Demos There’s all kinds of examples you can run and you can also view the source code.
There’s also some advanced demos and small applications under Help / Studio / Showcase.
If for some reason you don’t have this, you can probably install it under tools/package manager. I usually just install everything. It doesn’t take up much space at all. And then you have plenty of example code to look at.
Anyway, that’s it. Thanks for watching. If you liked this please subscribe and check out some of my other videos. Whether you liked it or not please feel free to leave a comment and let me know what you think.
See you next time!
GUI Code for Mandelbrot Browser
NB.* mbrowse.ijs: a Mandelbrot set browser from https://github.com/tangentstorm/mandelbrowse/
NB. -----------------------------------------------
NB. Mandelbrowse
NB.
NB. A visual Mandelbrot set explorer written in J.
NB. See README.md for instructions.
NB. -----------------------------------------------
NB. Copyright © 2016 Michal J Wallace
NB. Available for public use under the MIT license.
NB. -----------------------------------------------
coclass 'mbrowse'
NB. == dependencies ===============================
load 'viewmat'
NB. !! this is a workaround for a j8 bug (as of 01/02/2016)
NB. (without this fix, it won't redraw the viewmat control)
viewmatcc =: glpaint_jgl2_@viewmatcc_jviewmat_
NB. verb to convert space-separated hex strings into rgb triples:
rgb =: (3$256) #: [: dfh;._1 ' ',]
NB. == configuration ==============================
reset =: 3 : 0
iters 24 NB. # of iterations of the formula.
CENTER =: _1j0 NB. center of the view
SHAPE =: 160 120 NB. width, height of the data array
GRAIN =: 4 NB. pixels per array entry
STEP =: {. 4 3 % SHAPE NB. initial step between array units
SCALE =: 1 NB. user zoom factor
CENTERS =: ,CENTER NB. stack of center values
HIRES =: 0 NB. toggle with space key for high resolution
)
NB. color scheme: black, white, golds and blues
pal =: rgb '0 ffffff ffd21d b28f00 400fe8 1d2799 000055 000033'
NB. == mandelbrot verbs ===========================
NB. mbrot y → 0 if bounded
NB. else number of iterations where value was > 2
iters =: 3 : 0
NB. calling 'iters y' resets ITERS and rebuilds the 'mbrot' verb.
ITERS =: y
mbrot =: [: +/ 2 <&| ((+ *:) :: _:)"0^:(<ITERS) & 0
EMPTY NB. b/c a verb can't be the return value, or you get a syntax error.
)
c2w =: 3 : 0 NB. camera [0j0 in upper left] to world (centered,zoomed)
CENTER + SCALE * STEP * (-: j./<:SHAPE) -~ y
)
NB. draw our zoomed in section of the complex plane
plane =: 3 : 0
c2w (j.~/&i.~/ SHAPE)
)
NB. == graphics ====================================
render =: 3 : 0
img =: mbrot plane''
)
repaint =: 3 : 0
pal viewmatcc (render'');'g'
update_status''
)
NB. === event handlers =============================
mw =: 3 : 0 NB. mouse position in world cordinates
mxy =. 2 {. _".sysdata
c2w j./ GRAIN (<.@%)~ mxy
)
update_status =: 3 : 0
NB. the '+' in '+. +' negates the imaginary part, to flip
NB. the y coordinate. (the image is actually upside down,
NB. but it doesn't matter since the mandelbrot set is
NB. symmetric across the x axis.)
mp =. ' pos: ', ": 8j5 ": +. + mw''
sc =. ' scale: ', ": SCALE
cn =. ' center: ', ": +. + CENTER
hl =. ' [', ' res] ',~ > HIRES { 'lo';'hi'
it =. ' iterations: ', ": ITERS
wd 'set sb setlabel text "', hl, it, sc, cn, mp, '";'
)
w_g_mblup =: 3 : 0 NB. left mouse button
CENTERS =: CENTER, CENTERS
CENTER =: mw''
repaint [ SCALE =: -: SCALE
)
w_g_mbrup =: 3 : 0 NB. right mouse button
CENTER =: {. CENTERS
if. (# CENTERS) > 1 do. CENTERS =: }. CENTERS end.
repaint [ SCALE =: +: SCALE
)
w_g_mmove =: 3 : 0
update_status''
)
w_g_char =: 3 : 0 NB. keypress handler
select. {. sysdata NB. head so we can compare against individual characters
NB. space toggles low resolution for faster drawing
case. ' ' do.
HIRES =: -. HIRES
if. HIRES do. SCALE =: -: SCALE [ GRAIN =: -: GRAIN [ SHAPE =: +: SHAPE
else. SCALE =: +: SCALE [ GRAIN =: +: GRAIN [ SHAPE =: -: SHAPE end.
NB. +/- key change number of iterations, to change level of detail
case. '+' do. iters ITERS + 8
case. '-' do. iters 1 >. ITERS - 8
case. 'r' do. reset''
end.
repaint''
)
NB. == launch window and draw =====================
create =: 3 : 0
reset''
wd 'pc w closeok; minwh ', (": SHAPE * GRAIN), ';'
wd 'pn mandelbrowse;'
wd 'cc g isidraw;'
wd 'cc sb statusbar; set sb addlabel text;'
wd 'pcenter; pshow;'
repaint''
)
NB. debug verb to see just corners of a big 2d array:
corners =: [: ({.,:{:) ({.,{:)"1
NB. Register a global constructor, so you can launch like this:
NB.
NB. m =: mbrowse''
NB.
NB. You can then type `cocurrent m` to enter the application's
NB. locale in the J terminal.
NB.
NB. Once you've closed the window, you can type `cocurrent 'base'`
NB. (to re-enter the default locale), and then `codestroy__m''` to
NB. destroy the object.
NB.
mbrowse_z_ =: conew & 'mbrowse'
Advanced topics
Pepé explains his idea for an adverbial code gym, or "jym" as he likes to style it. Thomas tells about his "unbox" project.
Adverbial Jym
Pepé claims that everything can be tacit but it's not clear if any adverb can be made tacit. However, in 2009, he and Dan Bron figured out that you could use ^: and @ to give dynamic adverbs, ones that are not fixed at definition.
This gives us the flexibility to manipulate verbs the same way we can now manipulate nouns.
Tacit adverbs avoid name conflicts. Danyel mentioned that we already have something like this as there is an adverb to change a dyadic verb into a monadic verb, e.g. using "cap" ([:).
Unbox Project
Thomas told us about his idea for a community-supported version of J made available under the GPL (Gnu Product License). [This is now a moot point since J is now freely available under an open-source license.]
Some of the reasons he raised are that D. - for automatic differentiation - does not work very well and some facilities, like those for reading .CSV files, are not very efficient.
His ideas included allowing J addons to be coded in C (for efficiency) and how this could help with multi-threading. He also echoed some of Pepe's complaints about the behavior of gerunds in J and advocated for embedded functions. Additionally, we could handle run-time bindings by using function pointers.
Another thing he mentioned is the idea of "peephole optimizations" which implies being able to optimize a very small piece of code. To ease this and allow community-based enhancements, we could strive toward a shared library of DLLs to replace the monolithic J DLL.
Learning and Teaching J
We looked at two articles from the New Scientist magazine: one on how language shapes our thoughts and personality and another on how we might improve how we reason logically by using a kind of logic incorporating causal relations.
Happy talk: How your language shapes your brain and personality
The language you speak not only affects your brain's structure, it influences how you see the world and who you are. And as for bilingual brains...
Can learning a language rewire your brain?
| As our species evolved parts of our brain expanded, resulting in more computing power for language. It’s what makes us hard-wired for communication. What is perhaps more surprising is how language can shape our brains throughout our lives.
Most of the evidence for this comes from studies of people who are bilingual. Brain scan studies show that switching between two languages triggers different patterns of brain activity compared with speaking in one language, particularly in the prefrontal cortex. That part of the brain, at the very front of our skulls, is involved in organising and acting on information, including using working memory, reasoning and planning. Other studies show that bilinguals are faster at getting to grips with a new language. Quadrilinguist Arturo Hernandez, director of the Laboratory for the Neural Bases of Bilingualism at the University of Houston in Texas, says these differences could reflect differences in the architecture of bilingual brains. In other words, learning another language could change how your brain is wired. “It would make sense, if you have had this very different linguistic experience, to see some sort of stable, long-lasting effect,” Hernandez says. It may also make the brain more resilient. Ellen Bialystok at York University in Toronto, Canada, has found that lifelong bilinguals tend to be diagnosed with dementia on average 4.5 years later than monolinguals, and have more white matter, including in their prefrontal cortex. White matter is made of nerve fibres that connect different brain regions, shuttling information back and forth between them. So boosting language skills appears to build more connected brains – although Bialystok cautions that this still needs to be confirmed. |
|
More evidence for the benefits of second languages came last year from a study of 608 people who had had a stroke. Thomas Bak of the University of Edinburgh, UK, found that of the bilinguals among them, 40 per cent recovered full function, compared with only 20 per cent of monolinguals. Bak speculates that the mental gymnastics involved in speaking several languages could build extra connections that improve function and help cope with damage. “The idea is that if you have a lot of mental exercise, your brain is trained and can compensate better,” says Bak.
Can language influence how you see the world?
Time flows from back to front for English-speakers: we “cast our minds back” to the 1990s, and “hope for good times ahead”. It’s an example of a cultural concept encoded in language, but can language in turn influence how we think?
Maria Sera is a native Spanish-speaker who grew up believing all squirrels were female. The Spanish word for squirrel, ardilla, is feminine. As a linguist at the University of Minnesota, she has found some substance for her childhood belief. Studies of French and Spanish speakers, whose languages attribute genders to objects, suggest they associate those objects with masculine or feminine properties.
The idea that the language you speak could influence how you think dates back to 1940, when linguist Benjamin Lee Whorf proposed that people whose languages lack words for a concept would not understand it. It was relegated to fringe science until the early 2000s, when a few people began probing a related but more nuanced idea: that language can influence perception.
Greek, for instance, has two words for blue — ghalazio for light blue and ble for a darker shade. A study found that Greek speakers could discriminate shades of blue faster and better than native English speakers.
Language even seems to affect our sense of space and time. Some peoples, like the Guugu Yimithirr in Australia, don’t have words for relative space, like left and right, but do have terms for north, south, east and west. Studies have shown that they tend to be unusually skilled at keeping track of where they are in unfamiliar places. There is also some evidence that the direction in which your first language is written can influence your sense of time, with speakers of Mandarin more likely to think of time running from top to bottom than English speakers. And the language you speak may affect how you perceive others (see “Does your language shape your personality?”, below).
More generally, language helps us understand the world by allowing us to categorise things. Children are better at grouping objects if they have already learned the names of the categories they belong to. Conversely, after a stroke, people who have lost language skills can have trouble grouping objects. “It’s not that language just affects some high-level reasoning part of the brain,” says Gary Lupyan of the University of Wisconsin-Madison. “It’s changing our basic perceptual representations.”
Does your language shape your personality?
“To have another language is to possess a second soul,” Charlemagne is rumoured to have said. He may have been on to something. In the 1960s, sociolinguist Susan Ervin-Tripp of the University of California at Berkeley asked English-Japanese bilinguals to describe what was going on in ambiguous pictures. One person, for example, in Bilingualism.pdf told a different tale depending on their storytelling language. A picture of a woman leaning against a couch elicited a story in Japanese about a woman contemplating suicide after the loss of her fiancé. The same person, asked to respond at a separate session in English, said the woman was completing a sewing project for a class. “In general, there was more emotion in the Japanese stories,” Ervin-Tripp wrote in a description of the experiment. “The switch in language draws with it the cultural baggage associated with that language.”
Nairán Ramírez-Esparza at the University of Connecticut asked bilingual Mexicans to rate their personalities using both English and Spanish questionnaires. English responses emphasised openness and extroversion, while Spanish responses were more humble and reserved. “Language is such a powerful thing. It obviously makes you see yourself differently,” Ramírez-Esparza says.
According to Shai Danziger of Ben-Gurion University in Israel and Robert Ward of Bangor University in the UK, it can also influence how you think of others. They asked Arabic-Hebrew bilinguals to match Arab and Jewish names with positive or negative trait words by pressing a key. They say participants showed more involuntary positive attitudes towards Jews when tested in Hebrew than when tested in Arabic. Paula Rubio-Fernandez of the University of Oslo, meanwhile, has found that bilingual children perform better on tests that require them to understand a situation from someone else’s perspective.
Evidence is mounting that the words we speak and think shape our brains, perceptions, and personalities. Who knows what else? Perhaps our tastes, habits, or values. The door is wide open.
This article appeared in print under the headline “How your language shapes your brain and your personality”
By Megan Scudellari - Megan Scudellari is a science writer in Boston.
Magazine issue 3059 published 6 February 2016
A new kind of logic: How to upgrade the way we think
A lot of problems in today’s world are too big for our brains. An algorithm that identifies how cause and effect are linked could lead us to better solutions
IT IS not obvious what connects a falling apple and the rising sun. It took a genius to realise these two effects are the product of a single hidden cause, one that also explains the stars’ positions and the fact that our feet are firmly planted on the ground – the invisible pull of gravity.
That genius was Isaac Newton, a man gifted with “the power of holding continuously in his mind a purely mental problem until he had seen straight through it”, in the words of economist John Maynard Keynes. We could use a few of his stamp now more than ever. While Newton’s successors stutter as they try to find further unifying laws of the cosmos, the world at large grapples with problems on unprecedented scales – economic instability, poverty and disease, climate change.
Finding solutions means doing what Newton did with gravity: asking the right questions, teasing out causes and effects, and so building an intellectual framework to explain the puzzle. But how do we do that with the sheer quantity of data sloshing around in today’s world? It’s this problem that has led some to think we need to think seriously about the way we think. Only by rebooting our powers of logic and going beyond what nature has hardwired into our brain can we hope to grapple with problems that are far bigger than any of us. It’s time to install Thinking 2.0.
For most of us, Thinking 1.0 is taxing enough. We humans love to sideline logic in favour of the easy answer. We might make life decisions based on newspaper horoscopes, champion medicines that don’t really make us better or follow the same diet that didn’t work last year. Collectively, we convince ourselves we can put off making hard decisions about tackling climate change until tomorrow, and stake our futures on economic models that bear no resemblance to the real world.
Scientists, those supposed guardians of cold logic and rational thinking, are by no means immune. Here the most acute problem lies in statistical analysis – used correctly, the best way to extract meaning from a large amount of data, but all too often prone to oversimplification or flawed claims.
“Statistics is presented as a way of being careful about the world and not making claims that aren’t supported by the data, but it ends up being a licence for people to believe things that they would never otherwise have believed” says Andrew Gelman, a statistician at Columbia University in New York. “It’s an excuse for people to turn their brain off”.
He cites findings from recent eye-catching studies with statistical flaws that good-looking parents are more likely to produce female children, or that women tend to wear red clothing when ovulating. Back in 2005, researcher John Ioannidis, now at Stanford University in California, claimed that sloppy statistics could mean that more than half of scientific papers reach flawed conclusions. Neuroscience and psychology are areas that have come under particular scrutiny recently for possibly flawed assignments of cause and effect.
Now, as never before, screeds of information are logged digitally every day and stored in perpetuity, from government records to ocean surface temperatures. Couple that with the relentless rise of cheaply accessible computing power and it’s clear that the potential to find answers is greater than ever before – but the possibility of finding the wrong ones rises.
The challenge starts in asking the right questions, says social scientist Gary King of Harvard University. “Often the question isn’t well posed, or there isn’t a direct answer but there is another question that’s of interest”.
In the world around us, cause and effect are often unambiguously defined, and the right questions and answers are similarly clear. I kick a football, the football moves. The question “What made the football move?” is as obvious as the answer “I did”. But then take a cock’s crow before dawn. An incautious statistical analysis might show that the one precedes the other sufficiently often that the crowing causes the sun to rise. We only know that is nonsense because independently acquired knowledge tells us we live on a rotating planet – an entirely different assignment of cause to effect.
Big and messy
Often we don’t have that independent knowledge. Say a clinical trial involving a large group of people shows that a new drug is effective at treating an illness. Break things down, though, and it turns out that the recovery rate is lower for men who got the drug compared with those who didn’t – and it is also lower for women. “If you try to interpret this in a causal fashion, it seems paradoxical” says Robert Spekkens, a physicist at the Perimeter Institute in Waterloo, Canada. This is an example of what’s known as Simpson’s paradox, and the reason is that the drug does not cause better recovery – it’s actually your sex that does (see diagram).
|
In a structured environment such as a clinical trial, such potential pitfalls can generally be circumnavigated through means such as control groups, in which some people take no drug or a placebo. But often controls aren’t possible, and the waters may be muddied by irrelevant, confusing or missing data. The UK’s groundbreaking post-war cohort studies, for example, followed a segment of the population throughout their lives to assess their health, wealth and happiness. But how do we tease out cause and effect from what will always be incomplete information, and so draw conclusions that help others improve their lives? “Big data are messy data” says Gelman. Such data might show that quantities A and B shrink or grow together – but does that mean A is causing B? Or that both A and B have a common cause in C – or in D or E, about which we have no data?
Even professional problem-solvers fall down in such a situation. Richard Scheines of Carnegie Mellon University in Pittsburgh, Pennsylvania, recalls a researcher who came to him convinced his data showed that religious practices reduced stress and helped overcome depression. A more detailed breakdown, however, showed that stress was causing depression, and depression was causing people to turn to religion – and it was not clear that anyone was getting better anyway. “He was very disappointed” says Scheines. This, then, is the rub as we attempt to bring our homespun logic to bear on big problems. “People know that correlation does not equal causation, but they don’t grasp the depth of it” says Judea Pearl, a computer scientist and philosopher at the University of California, Los Angeles. “You simply cannot grasp causal relationships with statistical language”. Pearl has spent the past few decades pioneering more reliable tools. These are founded on a mathematical language that for the first time enables causal relationships such as “the rooster crow does not cause the sun to rise” to be coded directly into a computer program. Algorithms built using this language enable researchers to reliably find the cause-and-effect relationships between the variables in a problem. Rather than combing a data set for things you think might be cause and effect, the algorithms instead create a hypothetical set of relationships and see if they fit with the data in every instance. In a similar way, a physicist would construct a model of a real-world situation and see if its predictions match with reality in all cases. Depending on the number of variables, building such a “causal structural model” can be a mammoth task – but it provides an independent logical template into which you can see if your data fits. “It gives you mathematical guarantees” says Pearl. “We know that correlation does not equal causation, but we don’t grasp the depth of it“ This is seminal work, says Scheines, who is also a co-founder of the University of Pittsburgh’s Center for Causal Discovery, created in 2014 to exploit the new insights. “Pearl has been monumentally important to the revolution in causation” he says. Most science, he points out, involves coming up with a model of causes and effects, testing it, and modifying it when it fails – a wearying process. “We can now look to a computer to search billions and trillions of different models at the same time” he says. |
Slowly, the new techniques are gaining adherents. One early success came in 2012, when biologists and statisticians at the Swiss Federal Institute of Technology (ETH) in Zurich used causal discovery algorithms to pin down the genes responsible for determining the flowering time of plants. “They got it down to nine candidates from 25,000 genes” says Scheines. Others are applying the methods to understanding causal relationships in cell signalling, and between genetic mutations and tumour development, as well as to tease out differences in the workings of autistic and neurotypical children’s brains. “This work is starting to penetrate the mainstream” says Scheines – just as ambitious programmes such as the UK’s 100,000 Genomes Project are beginning to crunch gigantic swathes of data to find correlations, and perhaps causations, between people’s genes, diet, lifestyles and health, and the onset of cancer.
Alexander Bochman of the Holon Institute of Technology in Israel also thinks the new logical tools could be a game changer. “It is truly revolutionary” he says. “It has shown that causal reasoning works and provides clear, reasonable and systematic answers to important questions”.
Together with Vladimir Lifschitz of the University of Texas at Austin, Bochman is boring down to the next level. They have been working to reformulate Pearl’s mathematical notations into the more basic language that formal logicians use, enabling them to explore ideas of causality within logical frameworks that contain assumptions – and understand what happens when those assumptions are found wanting. “Assumptions are just beliefs, so they should be abandoned when we learn new facts that contradict them” Bochman says. If we find the grass is wet, we might assume it has just rained, for example – until we learn that a new sprinkler system has just been introduced.
The “nonmonotonic reasoning” logic scheme that he and Lifschitz have devised allows a user to keep track of their assumptions, and update a chain of reasoning if those assumptions change. This provides a surer way to work with incomplete or changing information – a boon in interpreting “the vast range of vexed questions about causation and its role in science and beyond”, says Bochman.
Spekkens is approaching that same problem from a different angle. He is a quantum physicist – and there is no field of human enquiry in which questions of cause and effect are more vexed. Quantum theory describes with perfect accuracy phenomena that occur on atomic and subatomic scales, but only at the price of particles existing in two places at once, for example, or seeming to influence each others’ properties without any physical connection.
Physicists explore this last phenomenon of “entanglement” using an experimental set-up known as a Bell test, which aims to exclude the possibility of standard, “classical” cause-and-effect influences being responsible for what is observed. Bell tests generally show that the experimental apparatus obeys our familiar classical causal rules, while the measured particles seem to be obeying a different, unfamiliar set. There are two responses to that: say “it’s weird” and carry on – or try to work out what the rules of quantum causality are, and whether they might give us a deeper insight into logic in the wider world.
What Spekkens and his colleagues have teased out so far is that the way causality works in the quantum world allows a cause-effect relationship to be distinguished from a common-cause relationship – something nigh-on impossible in the classical world without using a control group (see diagram). In other words, it allows us to sidestep those Simpson’s paradox-type situations (Nature Physics, vol 11, p 414).
Quantum Logic
| “Entanglement seems to be based on a causal relationship outside physical space and time“ That could have direct consequences in fundamental physics. Entanglement seems to be based on a causal relationship that lies outside our conventional conception of physical space and time, phenomena dealt with by Einstein’s general relativity. Find out more, and we might find a common cause that could unify relativity and quantum theory, two stubbornly incompatible theories. But Spekkens thinks it could also feed back into more rigorous logical analysis in the real world. There are undoubtedly more classical causal algorithms waiting to be discovered, he says, and perhaps the quantum analyses could allow us to discover ways to distinguish cause and effect from common cause in the classical world, too. That could potentially be of great interest, say, for clinical trials in which randomised controlled experiments can’t be carried out for cost, technological or ethical reasons. In precise evaluations of the comparative effectiveness of end-stage cancer treatments, for example, it would be unethical to deny a control group any treatment at all. “If one could deliver on this promise, it would be an interesting example of the field of quantum foundations having applications in other fields” says Spekkens. | 
|
Spekkens is keen to point out that such tools complement, rather than compete with, conventional tools from classical logic. Pearl for one welcomes any synergy between quantum and classical thinking, and any approach that can increase the rigour and effectiveness of our approach to crunching data to solve the messiest of human problems. Rather than relying on presumptions of cause and effect made by often politically compromised economists, for example, he thinks we might use new thinking tools to crunch through all conceivable combinations of economic variables to work out which should be tweaked and under what circumstances, say to bring about more equality while not strangling economic growth. “There’s a lot of research in this direction and it looks promising” says Pearl. “We haven’t yet discovered the cause of poverty, but we’re moving in the right direction”
If we can go beyond Thinking 1.0 to frame better questions and process data more reliably, we might all reasonably become geniuses – a cause that could mean a better future for all of us. “Maybe we will begin to think in more complex and subtle ways” says Gelman. “That would be great”.
This article appeared in print under the headline “Thinking 2.0” [New Scientist, 24 February 2016]. Michael Brooks is a consultant for New Scientist