Guides/J9 Standalone
Standalone, cross-platform applications in J9
J provides an exceptionally powerful engine for complex data analysis. It is also an expressive and flexible general purpose programming language that makes it easy to develop desktop applications. The goal of this tutorial is to provide an example of taking a form-based J application that runs within the development environment to a standalone, desktop application that can be installed by end users on Windows, macOS, and Linux. This tutorial focuses on J904, but you should be able to use earlier versions of J9x with few changes. The sample application displays the contents of text files or URLs. Since we're not using the multimedia or web browser functions of Jqt, you should install the slim version of J904 so that fewer libraries need to be included in the final application.
The sample application: Viewer
The example for our standalone conversion, Viewer, simply opens a text file or URL and displays the result in an editm window. The main window, below, has a menu, toolbar, and an editm control. In addition, it includes three dialogs: About, Preferences, and Retrieve URL. An archive of the J9 project folder may be downloaded from here: https://www.jsoftware.com/download/drinkwater/viewer-3.zip
The text file or URL is opened from File>Open file or File>Open URL, respectively, and then displayed in a readonly editm control. Choosing Edit>Annotate makes the editm editable and you can toggle it back to readonly using the same menu choice or toolbar button. The File>Preferences dialog allows you to set the fonts for the application controls and the editm control, display the application in dark mode, and set the margins and orientation for printing.
The first step is to make sure that your application is fully functional within the J IDE. Our project is in the folder viewer-3. The contents of the viewer-3.jproj file are:
NB. project: viewer-3 NB. NB. defines list of source files. NB. path defaults to project directory. about.ijs geturl.ijs pref.ijs req.ijs main.ijs
In addition to the main window and the three dialog forms mentioned above, source files include req.ijs (some utility functions used by the program). The scripts have require statements (e.g. require 'general/misc/validate' in pref.ijs) to bring in required code from addons.
The build.ijs script consists of the line
writesource_jp_ '~Projects/viewer-3';'~Projects/viewer-3/viewer-3.ijs'
and run.ijs is
load jpath'~Projects/viewer-3/viewer-3.ijs' DBG=: 1 TBP=: jpath'~Projects/viewer-3/resources/icons/' view_run''
The DBG flag allows you to quit the application within the J IDE without closing the J system. The exit function within main is
view_exit_button=: 3 : 0 close'' if. -. DBG do. 2!:55 [0 end. )
TBP is the location of the folder containing the program icons for use while running the program within the IDE.
Let's assume that, at this point, you can run the project and the program does everything you want.
Windows standalone application
Our goal is to turn Viewer into a self-contained application that can be packaged for installation on Windows, macOS, or Linux by an end-user who does not have J installed on his or her system. The discussion below will focus first on Windows and then details for Linux and macOS will be provided.
After installation, our application folder structure will look like this:
Install folder |-bin | |-icons
The install folder contains two files, readme.txt and license.txt. The latter includes a link to the Jsoftware license and attribution for the icons used in our program.
The bin folder will contain j.dll, jqt.dll, pthreadVC3.dll, and jqt.exe (from your J904 slim installation), the Qt libraries, viewer.ijs (which we'll rename profile.ijs), the program icon (viewer.ico), and the icons folder (which contains all of the png files used by the toolbar as well as the program icon app.png and toucan.bmp, copied from the bmp addon and used by the About dialog).
Because our application doesn't use WebEngine, multimedia, or other advanced functions, we can use the "slim" versions for the jqt binaries (e.g., http://www.jsoftware.com/download/j904/qtide/jqt-winslim-x64.zip) and Qt libraries (e.g., http://www.jsoftware.com/download/j904/qtlib/qt62-win-slim-x64.zip). If you're using any of the advanced functions, you'll need to use the full versions for the jqt and Qt libraries.
First step: Making the application profile independent
Because our standalone application runs without a profile (or the rest of the J development environment), the paths to necessary files (e.g., the toolbar icons) need to be set at runtime. To get everything working, set up a folder like the Install folder above, and copy the necessary files into it (all except viewer.ijs, which we haven't made yet). In particular, you'll need to have the toolbar icons in bin/icons.
On Windows, we've set up our application folder as c:\Utility\Viewer 3.0.
The function view_run is in main.ijs:
view_run=: 3 : 0
IDE=: 1
NB. showide''
sysinit''
SetDefaults''
readPref''
Title=: Data=: ''
wd'fontdef ',FFont__
wd VIEW
wd'set wrap checked; set vw wrap 1'
wd'picon "',Bdir__,'icons/app.png"'
wd'set vw stylesheet *QPlainTextEdit {background-color:#eeeeee;',(font2css VFont__),'}'
wd'set vw text *',Data
if. 0=#TBP do. TBP=: Bdir,'icons/' end.
viewtb''
wd 'pshow;'
Ed=: 0
WRAP=: 1
view_wrap_button''
)
For a standalone application, we don't want to see the jqt terminal window. The command wd'ide hide' will hide the terminal window and you can place it as the first line of the run function. I prefer to be able to toggle the terminal window, so view_run starts with
IDE=: 1 NB. showide''
Where showide is the function
showide=: 3 : 0 if. IDE do. IDE=: 0 wd'ide hide' else. IDE=: 1 wd'ide show' wd'psel view;pactive' end. empty'' ) view_wctrl_fkey=: showide
The application will normally start with the terminal window hidden and it can be toggled with Ctrl-W. Note that we've commented out showide' ' for now so that we can see any errors. In view_run, sysinit' ' sets Bdir (as the directory containing the J libraries), and Sdir as the install folder. The SetDefaults and readPref functions (defined in pref.ijs) provide the fonts to be used for the form (FFont) and for the editm widget that will display the file/url contents (VFont). TBP contains the path to the toolbar icons (Bdir/icons) for use by viewtb, which adds the icons to the toolbar.
Once you're satisfied that everything is working, remove the NB. from the showide line.
Building the standalone script
For our standalone application, we need to build a single script file (viewer.ijs) that contains all of our source scripts as well as the system libraries and required addon scripts. Fortunately, this is easy to do using functions in the jp (jproject) locale. To our project folder we add a new script builds.ijs:
NB. builds.ijs PDIR=: jpath'~Projects/viewer-3/' LIBS=: 0 : 0 graphics/bmp ide/qt/gl2 graphics/color/rgb general/misc/validate web/gethttp ) TARGETHEADER=: 0 : 0 NB. Viewer 3.0 ) TARGETFILE=: 'viewer.ijs' TARGETEXTRA=: 'view_run''''' builds=: 3 : 0 load PDIR,'build.ijs' out=. TARGETHEADER out=. out, getstdenv_jp_'' out=. out, getlibs_jp_ LIBS out=. out,freads PDIR,'viewer-3.ijs' out=. out,TARGETEXTRA,LF out fwrites PDIR,'viewer.ijs' ) builds''
The build.ijs script (run by choosing Project>Build from the IDE edit window menu) collects all of our script files into viewer-3.ijs. The builds script defines the project folder path as PDIR. The other nouns include
LIBS, a list of required addon scripts
TARGETHEADER, copyright or other comments placed at the beginning of our standalone script
TARGETFILE, the name of our standalone script (different from the one used by build.ijs)
TARGETEXTRA, lines to be added at the end of the standalone (typically the run statement).
The builds function constructs our standalone script by pulling in the TARGETHEADER; adding the stdlib.ijs, task.ijs and the hostdef scripts; redefining the load, require, and script functions in the z locale to do nothing; adding the required addon scripts; adding our built source script, viewer-3.ijs; and finally adding the run statement. All of this is written out to our standalone script, viewer.ijs. Uncomment the showide' ' line in the run function of main.ijs and build the standalone script. Note that the resulting viewer.ijs is platform independent. You can simply copy this script from your Windows box to other platforms.
Running and debugging the standalone application
Copy viewer.ijs to the Viewer/bin directory of the standalone test folder (c:\Utility\Viewer for Windows) and rename it to profile.ijs. Alternatively, you could start the program with a batch script using the command
jqt.exe -jprofile viewer.ijs
Even if your application worked fine within the J development environment, you may encounter an error when you try to run it as a standalone. Most often, the application will simply fail to open for no obvious reason. These errors may result from a missing file, misconfigured paths, or failing to add a required library to builds.ijs. Debugging standalone applications is much easier with Jqt than it was with Jwd. Open the application script, profile.ijs (or viewer.ijs) in our example, search for showide' ' and comment it out. You'll now get a terminal window when you try to start the application and the error it displays should allow you to figure out how to fix the program.
Locking your application script
J scripts, like our application viewer.ijs, are readable text. If your application contains proprietary code that you don't want to reveal to end users, you can lock your script to render it unreadable as described here https://code.jsoftware.com/wiki/Guides/Locked_Scripts. After running builds as described above, you can lock the viewer.ijs script by running the following in the J terminal window:
dat=. fread jpath'~Projects/viewer-3/viewer.ijs' dat=. 3!:6 dat dat fwrite jpath'~Projects/viewer-3/viewer.ijl'
Copy the viewer.ijl locked script to the application bin folder. You will need to create a file, profile.ijs, in the bin folder with the content
0!:1 <'viewer.ijl'
Running jqt.exe will start your application.
Making an installer for distribution
Having constructed our standalone Viewer 3.0 folder by following the directions above, you could simply zip up the folder and distribute the application as a compressed archive. However, we've glossed over the fact that the Intel version of the j library comes in three flavors, depending on whether the cpu supports avx1, avx2, or neither. The various versions for j.dll are available from http://www.jsoftware.com/download/jengine/j904-beta/windows/j64/. If your program does not do computations on large data sets, you could simply include the non-avx version of the j.dll. Alternatively, you could use an installer that will provide the correct version of j.dll, depending on the capabilities of the target cpu.
I've been using NSIS (https://nsis.sourceforge.io/Main_Page) to build Windows installers. You'll need the CPU Features plug-in (https://nsis.sourceforge.io/CPUFeatures_plug-in) to allow installation of the correct version of j.dll at install time. A copy of the NSIS script for the Viewer 3.0 installer (Viewer.nsi) is in the resources directory of the viewer-3 project folder, together with a listing of all of the files within the installer folder (Viewer3-nsis-tree.txt).
Linux standalone application
If you're developing on a Linux system, download the viewer-3 project folder and copy it to j904-user/projects. You can then follow the guide above.
Linux application folder
We'll make our application folder ~/viewer3. In that folder, copy license.txt, readme.txt, and viewer3.sh from the resource directory of the project folder and make a subfolder labeled bin. The application will be started using the viewer3.sh script. The -lib flag for jqt will be used to choose the appropriate j engine library at runtime:
#!/bin/bash
set +o history
abspath=$(cd ${0%/*} && echo $PWD/${0##*/})
APP_PATH=`dirname "$abspath"`
cd $APP_PATH/bin
echo $APP_PATH
CPU=`./jconsole -jprofile cputest.ijs`
echo $CPU
if [ "$CPU" == "avx avx2" ]; then
./jqt -lib libjavx2.so
elif [ "$CPU" == "avx" ]; then
./jqt -lib libjavx.so
else
./jqt -lib libj.so
fi
The bin directory should contain the program icons subfolder; jconsole, libj.so, libjavx.so, and libjavx2.so from the linux jengine folder (http://www.jsoftware.com/download/jengine/j904-beta/linux/j64/); jqt and libjqt from jqt-slim-x64.tar.gz on the jsoftware website; and cputest.ijs from the viewer3 project resources folder. You should also copy viewer.ijs from the viewer3 project and rename it to profile.ijs. The cputest.ijs script is used in our startup script to determine the capabilities of the target cpu.
NB. cputest.ijs systype=. 9!:12'' (2!:7'') 1!:2 (4) 2!:55''
You should now be able to run the application by executing viewer3/viewer3.sh. If it fails to start, comment out the showide line in viewer.ijs (profile.ijs) and see what errors are shown in the terminal window.
Distributing the Linux application
You can simply make an archive of the viewer3 folder and distribute it to your end users. They will need to have the Qt libraries installed on their system, so you may want to edit the readme.txt file to provide that instruction. On Ubuntu 20.04 (or 22.04), they should run
sudo apt install qtbase5-dev libqt5websockets5-dev libqt5svg5-dev
assuming that the slim set of libraries suffice for your application. If you need advanced Qt features, or are using another distribution, refer to https://code.jsoftware.com/wiki/Guides/Linux_Installation.
macOS standalone application
If you're developing on a macOS system, download the viewer-3 project folder and copy it to j904-user/projects. You can then follow the guide above.
macOS application folder
The install directory contains license.txt, readme.txt, the script setup.command (more on that later), and the Viewer application. On macOS, the application is really a specialized folder, Viewer.app, that contains all of the components of the application. The contents of Viewer.app, which can be displayed in Finder by right-clicking and choosing Show Package Contents, are:
Viewer.app | |-Contents | |- Frameworks Info.plist |- MacOS |- Plugins |- Resources
Info.plist is an xml file that contains essential information about the application. For Viewer, the contents of Info.plist are:
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>CFBundleExecutable</key> <string>apprun</string> <key>CFBundleIconFile</key> <string>app.icns</string> <key>CFBundleIdentifier</key> <string>com.jsoftware.www</string> <key>CFBundleName</key> <string>Viewer</string> <key>CFBundlePackageType</key> <string>APPL</string> <key>CFBundleSignature</key> <string>????</string> <key>LSMinimumSystemVersion</key> <string>10.13</string> <key>NOTE</key> <string>This file was generated by Qt/QMake.</string> <key>NSPrincipalClass</key> <string>NSApplication</string> <key>NSSupportsAutomaticGraphicsSwitching</key> <true/> <key>NSRequiresAquaSystemAppearance</key> <true/> </dict> </plist>
The Frameworks and Plugins folders are the Frameworks and plugins folders from the Qt directory of your J904 installation, or the contents of https://www.jsoftware.com/download/j904/qtlib/qt62-mac-slim-x64.zip. Note the case change when copying the plugins folder.
The Resources folder contains app.icns from the viewer-3 project resources folder. This file provides the application icon.
The MacOS folder contains the icons folder from the project resources, the jqt and libjqt executables from https://www.jsoftware.com/download/j904/qtide/jqt-macslim-x64.zip, and libj.dylib from the J installation. In addition, it contains profile.ijs (renamed from viewer.ijs in your project folder) and apprun, an executable script with the contents:
#! /bin/bash
set +o history
cd "${0%/*}"
arch_name="$(sysctl -n machdep.cpu.brand_string)"
if [[ "${arch_name}" = *Apple* ]]; then
arch -arm64 ./jqt
else
./jqt
fi
On macOS, the j, jqt, and Qt binaries are universal binaries that will run on both Intel and Apple M1 cpus. Simply double-clicking the Viewer icon in the Viewer 3.0 folder should start the application. You can simply compress the Viewer 3.0 folder and distribute your application as the compressed archive.
There is one final wrinkle that needs to be dealt with before end users can run your application. Since macOS 10.15, Apple has ramped up the security to prevent users from running applications downloaded from the internet rather than its App Store. Assuming you're not inclined to go through Apple's App Store approval process, the end user needs to go to the General tab of Security and Privacy in System Preferences and select Allow apps downloaded from: App Store and identified developers. The "identified developers" provision means that you will need to have your application notarized by Apple, which requires an Apple Developer ID costing $99 (US) annually. If you're not a commercial developer, there is a way around the notarization requirement.
When application archives are downloaded from the internet, all of the included files have an extended attribute, com.apple.quarantine, set. If you try to run an unnotarized application after downloading it, you will get a scary message box stating that the application can't be opened because the developer can't be verified and offering to delete it from your system. To circumvent this issue, run the setup.command script in the Viewer 3.0 folder. This script deletes the extended attributes from the downloaded files:
#!/bin/bash # run from the install directory cd "$(dirname "$0")" xattr -rc Viewer.app
Run the setup.command from a terminal and the Viewer application will run. You will want to edit the readme.txt file to provide appropriate instructions to your users.
Final words
The combination of J and Qt is as close to write once run everywhere as I've been able to get. This guide is based on my experience developing Mstat (https://oncology.wisc.edu/mstat), a nonparametric statistics and graphing program, for students with no experience or knowledge of J.
Viewer files
The viewer project folder is available from https://www.jsoftware.com/download/drinkwater/viewer-3.zip
Installer files for Windows, OSX, and Linux may be downloaded from the links below.
Windows: https://www.jsoftware.com/download/drinkwater/Viewer3-setup.exe
macOS: https://www.jsoftware.com/download/drinkwater/Viewer_3.0.zip
Linux: https://www.jsoftware.com/download/drinkwater/viewer3.tar.xz
Contributed by Norman Drinkwater, 8/2022