CGI/Design

Modularity is important to promote separation and code re-use, so that each individual module, such as HTTP parameter parser, could be loaded independently by other programs with a short name 'web/cgi/module'.

Each module should distinguish between private names and public interface, marked with scriptdoc NB.* comments.

Compliance with API design guidelines is essential, addressing the needs of three levels of programmers. For example, in the same area of HTTP parameters:

• Opportunistic: it provides ready to use parameter accessors handling default values, type conversions and lists
• Pragmatic: allow to substitute custom paramer provider or handler in place of standard, as in Guides/JWebServer/HttpParser
• Systematic: provide access to raw HTTP parameters in their original sources for GET or POST, while possibly hiding the lowest protocol of whether it's stdin, environment variable or socket.

API

Programmatic interfaces for application developer.

CGI Environment

• expose CGI environment variable as high-level standard set of accessors
• hiding platform specifics, e.g. SCRIPT_URL vs SCRIPT_NAME, etc.
• provide physical paths with jpath aliases, e.g. ~CGI for root of current application, ~Page for location of the current page, etc.

HTTP Parameters

• high-level accessors to HTTP parameters regardless of source (QUERY_STRING or stdin) with support for default values and type coersing in quasi-declarative form
• handle multipart/form-data (for upload)
• provide key-value dictionary of parameters with given prefix, possibly coersed to numeric, etc.
• provide list of parameters with a given same name, as from a set of checkboxes, possibly coersed to numeric, etc.
• convert a set of parameters into a series of input fields or a URL string

HTTP Headers

• provide a accessor for request HTTP Headers
• provide a storage for output HTTP Headers
• automatically calculate output Content-Length
• maintain output Content-Type
• maintain output Content-Encoding

Output Writer

• provide stream-like interface for emitting string and typed data
• provide binary output, hide Windows text mode stdout issue
• support buffered mode with clearing and premature end of response
• support automatic encoding, specified in HTTP header
• support HTML Writer interface from utilities

Error Handling

• handle errors: with top level try/catch
• error logging
• output diagnostics of unhandled errors according to content type
• provide normal termination and programmatic abnormal termination as

special types of throw

Cookies

• provide a accessor for request cookies
• provide a storage for output cookies
• automatic encryption and decryption, possibly delegated to other addon
• automatically forward persistent cookies

Session

Optional session "object" support

• maintain a session between requests
• support different session methods: cookies, hidden input field, scrambled container
• provide accessor to save and get different string values

Utilities

Static functionality for web-related processing, such as parsing, encoding and formating. Some of the functionality could be delegated to other addons.

HTTP Parameter Processing

• decode and encode string with URL encoding
• parse and generate URL query with a matrix with shape N,2
• split and combine URL with protocol, address, port, path, query and hash parts
• split request and combine response with headers, body and mime parts

Text Encoding

• encode and decode text using HTML entity encoding
• convert between different character encodings: Unicode, UTF-8, standard national

Mime Types

• declare a list of standard mime type for validation, mapping of file extensions and support of other content generation modules, such as image converters
• decode and encode mime multipart messages

HTML Generator

This is meant to be used by non-web server programs like GUI HTML formatters, etc. Possibly this should be a separate addon: web/html.

• programmatic generation of generic HTML elements
• automatic escaping and encoding of attributes
• verbs for some concrete HTML elements
• bulk generation: table from matrix, HTML list from boxed list, etc.
• HTML Writer interface with different output writers: string, file, generic output writer

HTTP Headers

• parse and generate HTTP headers with a matrix of shape N,2

Middleware

A set of handlers for executing high-level API frameworks such as JHP, Web Services.

Handlers are selected based on configuration and current file extension or file name, etc.

Generic handler for low-level handler implementation is provided for J scripts with .ijs extension. Developers can implement handlers for output images and other specific formats.

Middleware Execution::

• invoked by CGI integration layer, see Integrated execution
• obtain basic interface to CGI integration, such as configuration properties, logging, etc.
• obtain CGI adapter interfaces, environment, request, response
• setup handler-specific interfaces: JHP, Web Services, etc.
• setup error handling try/catch block
• load and process script
• execute script
• process errors in middleware specific manner, e.g. error section in web service response
• finalize execution
• resume to Integration layer

Integration

Interfacing with low-level web-server protocols: standard (default) CGI, enhanced CGI (FastCGI, SCGI), J stand-alone Web Server, as well as abstract interface with integration adapters created in the future. E.g. ASP.NET request handler can act as such adapter in a way similar to FastCGI/SCGI invoking J via COM or DLL while leveraging IIS for load-balancing, multithreading, etc.

Integration Interfaces

• configuration: reading and exposing
• error handling
• logging
• selecting adapter
• executing handler

Integrated execution::

• read CGI config file
• select CGI adapter
• identify script and select middleware handler
• setup CGI services for handler and application code
• setup error handling try/catch block
• execute handler, see Middleware Execution
• process errors
• generic logging
• finalize execution
• CGI exit code depending on specific adapter

Adapter Interfaces

• integration interface should be very thin, delegating most of the HTTP processing to the CGI framework, and only implementing the low-level protocol, such as sockets, etc.
• integration is configurable
• much of the CGI implementation and all of the application code should be isolated from the choice of particular low-level adapter

Adapter

• error reporting

Environment

• raw environment variable access: resolving and platform-specifics is done by higher level CGI

Request

• input stream: buffering reading, parsing, splitting are done by higher-level interfaces

Response

• output stream: buffering, combining, encoding, etc. done by higher interfaces

Configuration

Setting up the front-end web server, J system, and CGI framework for mutual interaction.

Web Server

Vendor-specific host web server procedures

• register specific web application or document root
• register a file extension for CGI framework, while leaving other files to be handled by host web server
• url-rewriting
• option of a dedicated instance of web server, such as "second" in JHP
• local configuration files, e.g. .htaccess, web.config
• specifics of web user and permissions settings to allow certain functionality, such as OpenGL under Mac OS X

J System

• boot code to recognize CGI mode
• delegating to CGI framework

CGI Framework

Configuration file(s) with settings for various layers of CGI framework.

Location:: Main config file should be located near J system to be found by the CGI boot code. Other, application-specific files, may be located in folders on the path from the web root to the current script folder.

Settings:: The following sections or properties can be configured

• integration
• adapters
• handlers
• execution modes
• logging
• application parameters