System/ReleaseNotes/J901

From J Wiki
Jump to navigation Jump to search

>> << Pri JfC LJ Phr Dic Voc !: Rel NuVoc wd Help 

The j901 beta cycle started in Oct 2018 and was first made available in April 2019.

Changes to the J engine

Summary of Changes to the J Language in the release

  • Language addition: gerund"r: the left operand of m"v can now be a cyclic gerund, giving a sequence of verbs to be applied to corresponding cells of the arguments. At long last you can apply a different verb to each cell of a noun. For compatibility with the older uses of m"n, the gerund is recognized only if the ranks v are not all infinite and m is a list of boxes containing a valid gerund. This is an incompatible change only if you used m"n with finite rank and an m that looked like a gerund. In that case, you need to replace m"n with m"_"n.
  • Language change: Scatter-amend support: x m} y changed when m is a numeric array of rank>1. In that case, each 1-cell of m is the index list of a cell of y that is replaced by the corresponding cell of x. This is an incompatible change with J8.07 and earlier, but is compatible with previous J9.01 betas.
  • Language addition: New primitives for iteration: A family of primitives implements Fold, as previously proposed. Features:
    • iteration can be forward or backward through items of a noun
    • initial value provided as its own argument
    • early termination allowed
    • either one result for the whole iteration or a result item per iteration
      • result of an iteration can be a subset of the complete internal state passed to the next iteration
      • results of individual iterations can be put into the overall result or not
    • iteration can be by item or on the entire argument until a condition is met
  • Language addition: : u. and v. are added to mean 'u/v in the calling context'. They are implied locatives, carrying with them the private namespace and implied locale at the time the operands they represent were created. When they are executed, the execution occurs in the original name environment. This long-awaited change allows you to pass a private name from an explicit definition into an explicit modifier, and when the modifier executes its u./v. operand, the execution happens as if it were executed at the point just before the modifier was called.
  • Language addition: Foreigns to set script name: 4!:6 y adds an entry to the table of script names returned by 4!:3; 4!:7 y sets the script number to use for subsequent assignments.
  • New control over precision of numeric constants:
    • No numeric constant containing a period will be demoted to lower than floating-point precision (thus 2. or 2.0 is a float value)
    • no number of more than one character will be demoted below integer precision (thus, 01 is an integer value).
    • No numeric constant containing 'j' will be demoted below complex precision (thus, 1j0 is a complex value).
    • No numeric constant containing 'r' will be demoted below rational precision (thus, 2r1 is a rational value).
      This makes it easier to create DLL inputs of the correct precision, and to make the datatype of constants match that of other nouns they operate with. Linear representations preserve the distinction between precisions.
  • If an adverb/conjunction with only one defined valence refers to any of m n u v and also to x, the defined valence will be construed as dyadic. Example:
    1 : 'x u y'
    Formerly this would give domain error when executed as a dyad, or value error when executed as a monad. This is an incompatible change only if you used the variable x in a monadic adverb/conjunction. In that case, use a different name, or add a nonempty dyadic case to the end of your definition.
  • Quicker way to specify numbered locales: numbered locales can be referred to by indirect locatives that are an integer atom or a boxed integer atom (rather than a boxed string). Such locatives can be processed quicker because they do not have to be validated and converted to a number. If you have (obj =: conew parameters) you can replace that with (obj =: 0&".@> conew parameters) and get a performance improvement every time you use (name__obj).
  • Domain of x u;.0 y expanded: the selected subarray may start on any valid atom of its axis, or on the atom one position before or after the axis. In all cases the length will be limited to the largest valid value for the starting position (which will be 0 for positions starting outside the array)
  • Domain of {:: extended: 0 {:: y now returns > y rather than length error when y is an atom.
  • Language deletion: primitive conjunctions .. and .: (Even and Odd) are removed from the language
  • Language deletion: calculus primitives (d. D. D:) are removed from the language, replaced by a J addon math/calculus that provides most of the functionality
  • Language deletion: Taylor-series primitives (t. t: T.) are removed from the language
  • At the end of a for_xyz. loop, the names xyz and xyz_index remain defined. If the loop was exited by break. or goto., the names simply retain their latest values. If the loop is exited by executing all the iterations, xyz is set to an empty list and xyz_index is set to the number of iterations that were executed.
  • else. is now allowed as the last alternative in an if. control, even if there has been a previous elseif.

J901 beta-s (the current beta)

  • atom { array faster when atom is integer or float
  • x u^:_ y does not create x&u at startup
  • New foreign [x] 128!:8 y to produce a quick non-cryptographic checksum of y
  • Any numeric constant containing j has complex precision; any numeric constant containing r has rational precision
  • New foreign 2!:8 y for errno and its description string

J901 beta-p

  • boolean * integer/float recoded, 2x faster
  • u;._3 fixed to honor infinite length in x

J901 beta-n

  • x +/ . * y special code when y is short and wide, for floating-point data on AVX processors
  • x +/ . * y special code when x is 2x2, for floating-point data on AVX processors
  • x +/ . * y special code for upper-triangular matrices, for floating-point data on AVX processors
  • [x] %. y recoded for floating-point data on AVX processors

J901 beta-m

  • x +/@:*"1 y improved to use full speed of Skylake when x is repeated
  • else. is now allowed as the last alternative in an if. control, even if there has been a previous elseif.

J901 beta-l

  • x +/ . * y recoded to take advantage of Skylake-class processors
  • [x] %. y saves some processing when multiplying by a triangular matrix
  • At the end of a for_xyz. loop, the names xyz and xyz_index remain defined. If the loop was exited by break. or goto., the names simply retain their latest values. If the loop is exited by executing all the iterations, xyz is set to an empty list and xyz_index is set to the number of iterations that were executed.
  • error in u^:v^:_ fixed

J901 beta-k

  • bug fixes from previous beta:
    • stack frame malformed, causing JQt debugger to treat all definitions as one long line
  • Language addition: gerund"r: the left operand of m"v can now be a cyclic gerund, giving a sequence of verbs to be applied to corresponding cells of the arguments. At long last you can apply a different verb to each cell of a noun. For compatibility with the older uses of m"n, the gerund is recognized only if the ranks v are not all infinite and m is a list of boxes containing a valid gerund. This is an incompatible change only if you used m"n with finite rank and an m that looked like a gerund. In that case, you need to replace m"n with m"_"n.
  • FMA (fused multiply-add) instructions used for p., +/ . *, and +.@:*"1 in AVX2 version
  • Fast path coded for x , y where items of x and y have the same shape or one has the shape of an item of the other
  • [x] %. y coded to use in-place operation and reduce copying
  • inplacing supported for x * y and x % y on complex arguments

J901 beta-j

  • bug fixes from previous beta:
    • calling through locative overwrote the global locale used during execution of u./v.
    • error after executing the last sentence of a select. block nested inside a for. loop

J901 beta-i

  • Language deletion: primitive conjunctions .. and .: (Even and Odd) are removed from the language
  • Language deletion: calculus primitives (d. D. D:) are removed from the language, replaced by a J addon math/calculus that provides most of the functionality
  • Language deletion: Taylor-series primitives (t. t: T.) are removed from the language
  • Language change: Scatter-amend support: x m} y changed when m is a numeric array of rank>1. In that case, each 1-cell of m is the index list of a cell of y that is replaced by the corresponding cell of x. This is an incompatible change with J8.07 and earlier, but is compatible with previous J9.01 betas.
  • Language addition: New primitives for iteration: A family of primitives implements Fold, as previously proposed. Features:
    • iteration can be forward or backward through items of a noun
    • initial value provided as its own argument
    • early termination allowed
    • either one result for the whole iteration or a result item per iteration
      • result of an iteration can be a subset of the complete internal state passed to the next iteration
      • results of individual iterations can be put into the overall result or not
    • iteration can be by item or on the entire argument until a condition is met
  • Language addition: : u. and v. are added to mean 'u/v in the calling context'. They are implied locatives, carrying with them the private namespace and implied locale at the time the operands they represent were created. When they are executed, the execution occurs in the original name environment. This long-awaited change allows you to pass a private name from an explicit definition into an explicit modifier, and when the modifier executes its u./v. operand, the execution happens as if it were executed at the point just before the modifier was called.
  • Language addition: Foreigns to set script name: 4!:6 y adds an entry to the table of script names returned by 4!:3; 4!:7 y sets the script number to use for subsequent assignments.
  • If an adverb/conjunction with only one defined valence refers to any of m n u v and also to x, the defined valence will be construed as dyadic. Example:
    1 : 'x u y'
    Formerly this would give domain error when executed as a dyad, or value error when executed as a monad. This is an incompatible change only if you used the variable x in a monadic adverb/conjunction. In that case, use a different name, or add a nonempty dyadic case to the end of your definition.
  • Incompatible change: names defined as [: must be defined directly, rather than through a sequence of names, to be recognized as [: in forks. (The search to see if a name was defined as [: would never complete if it encountered a name defined to equal itself)
  • New control over precision of numeric constants: No numeric constant containing a period will be demoted to lower than floating-point precision (thus 2. or 2.0 is a float value); no number of more than one character will be demoted below integer precision (thus, 01 is an integer value). This makes it easier to create DLL inputs of the correct precision, and to make the datatype of constants match that of other nouns they operate with. Linear representations preserve the distinction between precisions.
  • New algorithm for /: y on integer lists
  • New foreigns 6!:14-17 for manipulating times and timestamps
  • Special code for x | y when x and y are integers and x is repeated, 5x faster usually, more if x is a power of 2
  • x I. y rewritten, 2x faster when x is a list, not quite as much faster when rank of x>1
  • Virtual block used for result of ,. y
  • Fast path created for [x] u^:_ y (DoWhile)
  • Domain of {:: extended: 0 {:: y now returns > y rather than length error when y is an atom.
  • New foreign 128!:7 for AES
  • Setup of atomic monads and dyads rewritten yet again, this time to reduce mispredicted indirect jumps
  • 256-bit AVX2 instructions used for dyads + - * on integers
  • 256-bit instructions used for monads - +: % -. >: <: on floating-point y, and on integers with AVX2
  • Order statistics (x ({ /:) y and x ({ /:~) y) recoded to avoid mispredicted branches, 6x faster
  • [x] u^:n y, when n is an atom>1, now allows inplacing on y
  • x { y fast path created when x is an integer atom and y is an integer/float/boxed list. The fast path supports inplacing on x.
  • 0!:1xx y now accepts a locked script for y
  • foreign 128!:6 optimized with assembly codes

J901 beta-h withdrawn

J901 beta-g

  • Bugs fixed from previous beta:
    • 13!:19 (cut stack) did not always stop after cutting back
    • 13!:5 (run next) sometimes exited the running definition prematurely
  • 256-bit instructions used for more arithmetic primitives on floating-point lists (dyads >. <., monad | +/ <./ >./)
  • If x f y uses 256-bit instructions when y has rank 1, so do f/ y, f/\ y, and f/\. y when y has rank > 1
  • 256-bit instructions used in floating-point x +/@:*"1 y (the recommended form for dot products of lists)
  • stack error is detected better, allowing deeper recursion and eliminating stack-related segfaults
  • 256-bit instructions used for boolean dyads *. +. = ~: > < >: <: and for x = y and x ~: y on literals
  • 256-bit instructions used for x i. y when it chooses sequential search, such as when x is a list and y is an atom or a short list, for integer and floating-point types
  • New algorithm for /: y and \: y when y is a floating-point list
  • 256-bit instructions used for /:~ y and \:~ y on integer and floating-point lists
  • Name lookup for global names skips lookup in the local symbol table if no surprise local assignments have been made
  • 256-bit instructions used for name comparisons

J901 beta-f

Incompatible change: m!:n generates an immediate domain error if the combination is not supported (previously it generated an equivalent of [:, without error)

  • Bugs fixed from previous beta:
    • In debug, assert. failures did not show up on the debug stack
    • dbr 1 executed during loading a script was ignored
    • assigning 13!:19 (cut stack) to a name caused a hang
    • anonymous verbs always executed with debug turned off
    • Performance Monitor failed with an assertion error
    • break.bat file didn't create interrupt
  • 256-bit instructions used for floating-point arithmetic primitives (dyads + - * % p., monad %:)
  • Inplacing is now supported for boolean arguments of arithmetic dyads even if rank is specified

J901 beta-e

  • Bugs fixed from previous beta:
    • In debug, the display showed the line executed before a stop rather than the line about to be executed
    • Run after stop skipped a line
  • New feature: virtual extended nouns. If a noun that is not referred to more than once is extended by appending items to the end, the extended value and the original value share the same data without copying
    • This speeds up the form (x { name,default) where values of _1 or #name in x select from the default. The default is appended as a virtual extension without copying all the values in name.
  • Multiple accumulators used for +/\ y and +/\. y when y is a float list, 2x faster
  • New foreign x 3!:9 y to get internal information about noun y

J901 beta-c

What Do I Need To Do?

Detail

Incompatible change: x m} y changed when m is numeric with rank>1. The language is going to be modified to use this case to specify scatter-modify, which previously required boxing each row of m. Until the new interpretation is available, the case will create nonce error. To remove the error, use (<"0 m)} instead of m} . If x is an atom you can work around with (, m)} .

Incompatible change: 18!:4 changed. Users should be using this under its system name cocurrent or coclass; those uses are unchanged. If you misguidedly use 18!:4 directly, you will see that its side effects have changed: previously it altered the current locale in the name calling the name that uses 18!:4 only if there was no subsequent named execution in the name executing 18!:4; now it always causes the caller to continue on in the locale in effect when the called name exits.

Coding guideline: cocurrent and coclass should be used only in scripts to set the locale for lines executed from the script, and avoided inside of verbs in favor of using locatives. Indirect locatives (like a__b) have less overhead. Using cocurrent or coclass adds a small overhead to the execution of all named verbs until the function that called the deprecated name returns.

Parser refinement: For parsing purposes, the execution of a verb is assumed to produce a noun, and the execution of a fork is assumed to produce a verb.

Parser specification change: If the right-to-left processing of the words of a sentence encounters a sequence that must inevitably lead to a syntax error, it is undefined whether or not the rest of the sentence will be parsed and executed. Examples of such sequences include & =. and ('a' 5)

Change in behavior of nonexistent numbered locales: attempting to access a nonexistent numbered locale now produces locale error rather than creating the locale.


New Features

  • [x] 128!:6 y - Add keccak, MD4, MD5. see latest Dictionary for documentation on support for Secure Hash Algorithms

Improvements

  • Quicker way to specify numbered locales: numbered locales can be referred to by indirect locatives that are an integer atom or a boxed integer atom (rather than a boxed string). Such locatives can be processed quicker because they do not have to be validated and converted to a number. If you have (obj =: conew parameters) you can replace that with (obj =: 0&".@> conew parameters) and get a performance improvement every time you use (name__obj).
  • Domain of x u;.0 y expanded: the selected subarray may start on any valid atom of its axis, or on the atom one position before or after the axis. In all cases the length will be limited to the largest valid value for the starting position (which will be 0 for positions starting outside the array)
  • New parameter types in calls to DLL (cd or 15!:0):
    • b unsigned byte integer (1 byte)
    • z J float complex (8 byte - 2 f values) (pointer only, not as result or scalar)
  • The support code that prepares for execution of verbs/adverbs/conjunctions has been restructured for speed
    • Parsing recoded, including matching of the parsing table, which is now very fast
    • Named execution recoded, reducing overhead
    • Name re-lookup avoided on names that are repeatedly executed (for example name"0)
    • Explicit-definition startup recoded, giving ~15% improvement on definitions whose sentences are quick
  • Odometer function (#: i.@:(*/)) y recoded, 2-3x faster
  • Partitioning modifiers, and some verbs, modified to use stack space rather than heap space for most cases
  • u^:v faster when v returns atomic value 0 or 1
  • x -: y faster on boxed arrays, especially when corresponding boxes refer to the same memory areas
  • the dispensation on virtual arrays, which allows them to be boxed without being realized only when they are to be immediately opened, is detected in more cases. For example in (x (f g&.> <;.1) y) the result of <;.1 may contain virtual arrays because they will be immediately opened.
  • Lookups for numbered locales much faster
  • When security level is set, 1!:43 and 1!:44 are not allowed, and 2!:55 is allowed.
  • Multiple assignment 'name1 name2...' =. value1;value2... now preprocesses the names during function definition, making execution much faster
  • Name processing polished to reduce space used and overhead in deleting a namespace
  • All loops that copy shapes or check for agreement reworked to avoid mispredicted branches when array rank <3
  • Deal (x ? y) changed, 5x faster
  • Roll on shape (x ?@$ y) changed, 2x faster for nonzero y

  • Special code for x (comparison L.) y when comparison is one of < <: > >:; stops scanning y as soon as possible. This means that the best way to see if y is boxed is 0&(>: L.) y
  • Special code for some common idioms:
    • Rank of a noun:  #@$ y    #@:$ y    ([: # $) y
    • Number of atoms in a noun:  #@, y    #@:, y    ([: # ,) y    */@$ y    */@:$ y    ([: */ $) y
    • Is noun empty?  (0 e. $) y
    • Is noun nonempty?  *@#@, y
    • Does noun have items?  *@# y
  • Startup of atomic dyads recoded yet again, this time with emphasis on hiding load latency and eliminating unpredicted branches. Startup time is now approximately the amount of time needed to add 10 pairs of integers (was 20).
  • Amend in place polished
    • Audit indexes faster: 2x improvement on list list} list
    • Does not build addressing cells for each atom, and thus many times faster when the cells selected by m are larger than atoms
  • Scatter-read (x (<"1@[ { ]) y) reworked. Faster when there is a negative index; otherwise just a bit faster, more so for small arguments
  • Multiple accumulators used where multi-cycle instructions created a carried dependency, giving speedups:
    • x +/@:*"1 y and x +/ . *"1 y on floating-point lists: 2x
    • +/ y on floating-point lists: 4x
    • <./ y and >./ y on integer and floating-point lists: 2x
    • (+/ % #) y on integer and floating-point lists: 2x
    • (i. <./) y, (i. >./) y, (i: <./) y, and (i: >./) y: 3x
  • Inplacing allowed in f/\ y and f/\. y for certain f
    • for <. and >., on integer/floating arguments
    • for * and +, on floating arguments
  • New sort algorithm added
    • Faster on float lists less than 50,000 long; 2x faster on lists shorter than 6000
    • A smidgen faster on integer lists less than 40 long
  • x # y recoded for boolean x. Faster, the more so as items of y are short, or x contains zeros. About 5x faster if y is integer or float and x is half zero.
  • I. y recoded for boolean y. 5x faster when y is half zero.
    • special code for x I.@:comparison y removed as it is no longer faster
  • Code for x +/@:comparison y polished, 2x faster on integer/boolean lists when the results of the comparison are coin-flips.
  • Code for x u;.1 y etc. recoded for boolean and ASCII x, 1.5x faster on fast verbs u
  • Code for comparisons improved:
    • x = y on integers brought up to the speed of the other integer comparisons, 5x faster
    • Intolerant comparison treated as a special case, 2x faster on random floats
    • x = y and x ~: y on one-byte characters recoded, 2x faster
  • x # y inplaceable when x is boolean; if inplaced, avoids copying unmoved leading cells
  • Comparison combinations (such as i.&1@:>) treat intolerant comparison as a special case
  • Several atomic monads recoded for integer and float cases
    • <. y and >. y inplaceable for integer and float; float recoded, 5x faster
    • * y inplaceable for integer and float; integer recoded; float recoded, 2x faster
    • | y inplaceable for integer and float; integer recoded; float recoded, 1.5x faster
    • %: y inplaceable for float

Changes to the base system

  • html documentation is frozen and all changes and new material will be in the wiki. Old html pages are linked from HTML Help.
  • remaining Help|User Manual is moved to the wiki System page
  • pacman now supports JE updates with 'upgrade' jpkg 'jengine'
  • remove socket sdasync (socket async connection)

Changes to the Qt IDE

for Qt IDE 1.8.5 and later:

  • add Edit|Configure|Save Current Positions menu item
  • add multi-screen support
  • add support for right justified toolbar button
  • help menu references the wiki, not the old html docs
  • help menu includes Release Notes, Standard Library, Window Driver
  • update mac and windows libraries to Qt 5.12.6

>> << Pri JfC LJ Phr Dic Voc !: Rel NuVoc wd Help 

Retrieved from "https://code.jsoftware.com/mediawiki/index.php?title=System/ReleaseNotes/J901&oldid=41846"