THE NETSCAPE SERVER API


Implementing the design goals of the Netscape HTTP server, Netscape Communications has developed an architectural framework that will allow modular growth of the full range of Netscape Server software features independently of other applications or customizations being developed. We focused our efforts on allowing multiple programming teams, many of which will not be from Netscape Communications, to develop features and extend the functionality of the Netscape Server products for custom installations and applications.

Familiarity with Appendix B (technical information) of the Netscape server documentation will be helpful in understanding this document. For owners of Netscape Server products, this information is available on-line as /Admin/config.html. It would also be valuable for you to understand some fundamental design concepts that are integral parts of the HTTP implementation of Netscape's Server software.

WHAT IS THE NSAPI?

The Netscape Server API (NSAPI) is an extension that allows you to extend and/or customize the core functionality of the Netscape server and provide a scaleable, efficient mechanism for building interfaces between the HTTP server and back-end applications.

Netscape Server products can take advantage of the NSAPI in addition to the Common Gateway Interface (CGI). CGI is a simple interface, common across most HTTP server implementations, for running external programs, or gateways, between the information server and external applications. The NSAPI was designed to solve performance and efficiency problems common to installations that make liberal use of CGI functionality. In addition to the dicussion of the API specification in this document, we have also prepared an overview of the differences and advantages of the NSAPI over CGI.


A LOGICAL BREAKDOWN OF THE
HTTP REQUEST-RESPONSE PROCESS

After the client sends its request to a server, it is helpful to define a set of logical steps which the server must take before a response can be sent. The definition of these steps is taken from experience with the feature sets of common HTTP servers. The steps chosen should be as orthogonal as possible, with the result of one step affecting the next, but the methods employed in carrying out each step should not affect the next steps.

We have identified the following steps in the normal response process:

If at any time one of these steps fails, another step must be taken to handle the error and inform the client about what happened. In this case, the Netsite HTTP server allows the administrator to customize the response which is sent with more site-specific information about the error.


THE SERVER APPLICATION FUNCTION

To accomplish each of these steps, a set of server functions must be applied. These functions take the request, and server configuration database as input, and return a response to the client as output. The set of functions which are applied are determined by the inputs.

We have dubbed these server functions server application functions. By changing the set of functions available to the server application, the set of features that server implements is changed. Further, the base code of the server may be upgraded, or even changed to implement a different protocol with minimal changes to the server application functions. For example, the first release of the Netsite HTTP server has a function set which implements most of the NCSA httpd feature set.

Server application functions are said to have a particular class, where the class corresponds to the request-response step it helps implement. There is an additional class of application function, the initialization function, which is executed upon daemon startup and performs static data initialization for the various server modules. Server application functions have a single class and are not informed by the server which class they are being used for.

The server keeps an internal table of available functions, and maps these function pointers to unique character strings which identify them. By using this string in the configuration database, a function can be called to carry out one of the above steps. In the initial release of the Netsite HTTP server, this mapping of function pointers to names is hard-coded into the server. A mechanism has been proposed by which these functions may be loaded at run-time through the use of shared objects.


FUNCTION-SPECIFIC PARAMETERS

When specified in the Netsite object configuration database, all classes of functions are called as follows:

Directive fn=function [name1=value1] ... [nameN=valueN]
Directive identifies which class of function is being called. Functions should not be called with the wrong class.

fn=function identifies the function to be called using the function's unique character-string name.

These two parameters are mandatory. After this, there may be an arbitrary number of function-specific parameters, each of which is a name-value pair. If there are duplicate names, the results will be undefined.


THE PARAMETER BLOCK

This logically brings us to the first fundamental data structure within the server code: the parameter block. A key observation in the internal httpd source code was that many of the server variables were based on name-value pairs, where a named variable is given a specific value. The parameter block, or pblock, is a hash table keyed on the name string, which maps these name strings onto their value character strings.

DATA STRUCTURES

#include "base/pblock.h"

typedef struct {
    char *name,*value;
} pb_param;

struct pb_entry {
    pb_param *param;
    struct pb_entry *next;
};

typedef struct {
    int hsize;
    struct pb_entry **ht;
} pblock;
The pb_param structure is used to manage name-value pairs. pb_entry is a structure used to create linked lists of pb_param structures.

The pblock is the hash table which holds these pb_entry structures. Its contents are transparent to most code, however, it is described below. The hash function is subject to change and is therefore not made known to application functions.

PUBLIC FUNCTIONS

#include "base/pblock.h"

/*
 * param_create creates a parameter with the given name and value. If name
 * and value are non-NULL, they are copied and placed into the new pb_param
 * struct.
 */

pb_param *param_create(char *name, char *value);

/*
 * param_free frees a given parameter if it's non-NULL, and returns 1 if
 * p was non-NULL, and 0 if p was NULL.
 *
 * Useful for error checking pblock_remove.
 */

int param_free(pb_param *pp);

/*
 * pblock_create creates a new pblock with hash table size n.
 *
 * It returns the newly allocated pblock.
 */

pblock *pblock_create(int n);

/*
 * pblock_free frees the given pblock and any entries inside it.
 *
 * If you want to save anything in a pblock, remove its entities with
 * pblock_remove first and save the pointers you get.
 */

void pblock_free(pblock *pb);

/*
 * pblock_find finds the entry with the given name in pblock pb.
 *
 * If it is successful, it returns the param block. If not, it returns
 * NULL.
 *
 * Note: This function is actually a macro which calls a separate
 * function with a hard-coded third parameter.
 */

pblock *pblock_find(char *name, pblock *pb);

/*
 * pblock_findval finds the entry with the given name in pblock pb, and
 * returns its value, otherwise returns NULL.
 */

char *pblock_findval(char *name, pblock *pb);

/*
 * pblock_remove behaves exactly like pblock_find, but removes the given
 * entry from pb.
 *
 * Note: This function is actually a macro which calls a separate
 * function with a hard-coded third parameter.
 */

pblock *pblock_remove(char *name, pblock *pb);

/*
 * pblock_nvinsert creates a new parameter with the given name and value
 * and inserts it into pblock pb. The name and value in the parameter are
 * also newly allocated. Returns the pb_param it allocated (in case you
 * need it).
 */

pb_param *pblock_nvinsert(char *name, char *value, pblock *pb);

/*
 * pblock_pinsert inserts a pb_param into a pblock.
 */

void pblock_pinsert(pb_param *pp, pblock *pb);

/*
 * pblock_str2pblock scans the given string str for parameter pairs
 * name=value, or name="value". Any \ must be followed by a literal
 * character. If a string value is found, with no unescaped = signs, it
 * will be added with the name 1, 2, 3, etc. depending on whether it was
 * first, second, third, etc. in the stream (zero doesn't count).
 *
 * Returns the number of parameters added to the table, or -1 upon error.
 */

int pblock_str2pblock(char *str, pblock *pb);

/*
 * pblock_pblock2str places all of the parameters in the given pblock
 * into the given string (NULL if it needs creation). It will re-allocate
 * more space for the string. Each parameter is separated by a space and of
 * the form name="value"
 */

char *pblock_pblock2str(pblock *pb, char *str);
While this list may seem overwhelming, only a handful of the functions need to be used from an application class function.


PASSING PARAMETERS TO SERVER APPLICATION FUNCTIONS

All server application functions (regardless of class) are described by the following prototype:

int function(pblock *pb, Session *sn, Request *rq);
pb is the parameter block with the parameters given by the site administrator for this function invocation. This parameter should be considered read-only, and any data modification should be performed on copies of the data. Doing otherwise is unsafe in threaded server architectures, and will yield unpredictable results in multiprocess server architectures.

PUBLIC STRUCTURES AND DATA ACCESS FUNCTIONS

#include "base/session.h"

typedef struct {
    /* Information about the remote client */
    pblock *client;

    /* The socket descriptor to the remote client */
    SYS_NETFD csd;
    /* The input buffer for that socket descriptor */
    netbuf *inbuf;

    /* Raw socket information about the remote client (for internal use) */
    struct in_addr iaddr;
} Session;
We define a session as the time between the opening and the closing of the connection. The Session data structure is used to hold variables which apply session wide, regardless of the requests being sent. The client parameter block currently contains two entries:

For HTTP, there is currently only one request per session. The following structure contains the variables which applies to each request within that session:

#include "frame/req.h"

typedef struct {
    /* Server working variables */
    pblock *vars;

    /* The method, URI, and protocol revision of this request */
    pblock *reqpb;
    /* Protocol specific headers */
    int loadhdrs;
    pblock *headers;

    /* Server's response headers */
    pblock *srvhdrs;

    /* The object set constructed to fulfill this request */
    httpd_objset *os;

    /* The stat last returned by request_stat_path */
    char *statpath;
    struct stat *finfo;
} Request;
vars are the server's working variables. The set of active variables is different depending on which step of the request the server is processing, which is discussed further below.

reqpb contains the request parameters which are given by the client:

headers is a pblock which contains the HTTP headers sent by the client. HTTP sends an arbitrary number of headers of the following form (RFC 822):

       Name: value
If more than one header has the same name, then they are concatenated with commas as follows:

       Name: value1
       Name: value2
becomes:

       Name: value1, value2
The parameter block is keyed on the fully lowercase version of the name string, without the colon. Application functions should not access this pblock directly, but instead use the following function:

#include "frame/req.h"

/*
 * request_header finds the named header depending on the requesting
 * protocol. Name should be the lowercase header name string to look
 * for, and value is a pointer to your char * which should contain the
 * header. If no header with the given name was sent, value will be
 * set to NULL.
 *
 * The server may not load headers until the first is requested
 * to increase performance when talking to certain NCSA products. This
 * option is not currently active.
 *
 * Because of this, the function can return REQ_ABORTED. If it
 * does, your function should return REQ_ABORTED. On success, this
 * will return REQ_PROCEED.
 */

int request_header(char *name, char **value, Session *sn, Request *rq);
Currently, it is safe to access the pblock instead of using this function, but this behavior should not be relied upon.

srvhdrs is the set of HTTP headers for the server to send back. This pblock may be modified by any function.

The last three entries in the request structure should be considered transparent to application code since they are used by the server's base code. After the server has a path for the file it intends to return (see below), application functions should use the following call to obtain stat() information about the file:

#include "frame/req.h"

/*
 * request_stat_path tries to stat path. If path is NULL, it will look in
 * the rq->vars pblock for "path". If the stat is successful, it returns the
 * stat structure. If not, returns NULL and copies an error message into err.
 * If a previous call to this function was successful, and path is the same,
 * the function will simply return the previously found value.
 *
 * Application functions should not free this structure.
 */

struct stat *request_stat_path(char *path, char *err, Request *rq);
Using this function avoids multiple, unnecessary calls to stat().


APPLICATION FUNCTION RETURN CODES

The following integer return codes are available to server application functions:

/* The function performed its task, proceed with the request */
#define REQ_PROCEED 0
/* The entire request should be aborted: An error occurred */
#define REQ_ABORTED -1
/* The function performed no task, but proceed anyway. */
#define REQ_NOACTION -2
/* Tear down the session and exit */
#define REQ_EXIT -3
PROCEED indicates that the function has performed its task without a problem.

ABORTED indicates that an error has occurred and that the request (not necessarily the session) should be terminated.

NOACTION indicates that the function found conditions such that it did not perform its intended action. The meaning of this depends on the step being performed (see below).

EXIT should only be used when a read or write error has occured while talking to the client, and the entire session cannot continue.

ERROR REPORTING

When problems occur, the server application function should set server status codes which give the client an idea of what went wrong. Further, the function should log an error in the server error log for the administrator to find.

There are two interfaces which make this happen:


SET RESPONSE STATUS CODE

#include "frame/protocol.h"

/*
 * protocol_status sets status to the code n, with reason string r. If r is
 * NULL, the server will attempt to find one for the given status code.
 * If it finds none, it will give "Because I felt like it."
 */
void protocol_status(Session *sn, Request *rq, int n, char *r);
Generally, protocol_status will be called with a NULL reason string, and one of the following status codes defined in protocol.h:

PROTOCOL_OK
Normal status, the request will be fulfilled normally. This should only be set by Service class functions.

PROTOCOL_REDIRECT
The client should be directed to a new URL, which your function should insert into the rq->vars pblock as url.

PROTOCOL_NOT_MODIFIED
If the client gave a conditional request, such as an HTTP request with the if-modified-since header, then this indicates that the client should use its local copy of the data.

PROTOCOL_BAD_REQUEST
The request was unintelligible. Used primarily in the framework library.

PROTOCOL_UNAUTHORIZED
The client did not give sufficient authorization for the action it was trying to perform. A WWW-authenticate header should be present in the rq->srvhdrs pblock which indicates to the client the level of authorization it needs to perform its action.

PROTOCOL_FORBIDDEN
The client is explicitly forbidden to access the object and should be informed of this fact.

PROTOCOL_NOT_FOUND
The server was unable to locate the item requested.

PROTOCOL_SERVER_ERROR
Some sort of server-side error has occurred. Possible causes include misconfiguration, resource unavailability, etc. Any error unrelated to the client generally falls under this rather broad category.

PROTOCOL_NOT_IMPLEMENTED
The client has asked the server to perform an action which it knows it cannot do. Generally, this is used to indicate your refusal to implement one of the thousands of less than useful HTTP features.

If no status is set, the default is PROTOCOL_SERVER_ERROR.

ERROR REPORTING

When errors occur, it is customary to report them in the server's error log. To do this, functions call log_error:

#include "frame/log.h"

/*
 * log_error logs an error of the given degree from the function named func
 * and formats the arguments with the printf() style fmt. Returns whether the
 * log was successful. Records the current date.
 *
 * sn and rq are optional parameters. If given, information about the client
 * will be reported.
 */
int log_error(int degree, char *func, Session *sn, Request *rq,
              char *fmt, ...);
You may give log_error any printf() style string to describe the error. If an error occurs after a system call, use the following function to translate an error number to an error string:

#include "base/file.h"

char *system_errmsg(SYS_FILE fd);
Note: the fd parameter is vestigial under UNIX and may need to be changed for other operating systems.


CONDITIONS FOR EACH FUNCTION CLASS

The following list the special conditions which exist in the Request variables and return codes for each class of server function.


AuthTrans

We have split authentication into two steps. First, the authorization data sent by the client (if any) is decoded, and verified against internal databases. If the data is valid, it should be stored into the Request->vars pblock. In the second step, this data is compared against the required authorization for the requested path (performed in a PathCheck class function). There is no reason that both of these steps could not be performed in one AuthTrans or one PathCheck directive, however we split the steps for flexibility. Also, it is required in case the server is ever required to support the SecureHTTP proposal.

There are no variables active in the Request->vars pblock upon entry to an AuthTrans function. Upon successful translation and function exit, the following variables are customarily set:

auth-type
Identifies the authorization type. Currently, only basic is defined for HTTP Basic user authentication.

auth-user
Gives name of the remote user as verified by internal databases.

Of course, the flexibility of the pblock structure allows AuthTrans functions to define their own custom variables which can later be retrieved by the PathCheck function which implements the second step.

Upon return, the AuthTrans function should return REQ_ABORTED if the request is to be aborted, and either REQ_NOACTION or REQ_PROCEED if the translation was unsuccessful or successful respectively.


NameTrans

NameTrans class functions are used to translate a virtual path, as given by the client, into a physical path as used by the server.

The following variables are active in the Request->vars pblock for NameTrans functions:

Upon return, the REQ return codes have the following meaning:


PathCheck

PathCheck class functions are used to verify that a given path is safe to return to a given client. This class of functions performs various system-specific URL filtering and screening actions, authorization checks, access control, searches for CGI extra path information, and other functions.

The system only defines one variable for PathCheck functions in Request->vars. path is the path resulting from the execution of all NameTrans directives. Any variables which may have been created by previous NameTrans or AuthTrans directives are also active.

On return, a code other than REQ_ABORTED will be considered a success.


ObjectType

ObjectType functions take the path resulting from the previous directives, and locate a filesystem object for the path. If none exists, and none of the PathCheck directives have looked yet, the current ObjectType routines will return an error to the client.

ObjectType directives recieve no special variables aside from path in Request->vars, and the variables defined by the previous directives. Upon return, a code other than REQ_ABORTED indicates success.

The server's base functionality library provides mechanisms for typing files natively to the machine's operating system. Under most systems, this consists of filename extension recognition code.


Service

Service functions send the server's reply to the client. The function is generally selected by the directory the file resides in, or the type of the file being sent. Most files are simply mapped into memory and sent to the client verbatim. Some classes of documents are executed as system programs, and others are parsed before being sent to the client.

Service directives recieve no special variables aside from path in Request->vars and the variables defined by the previous directives.

The REQ return codes are defined for Service class functions as follows:

The Service class functions need to start the response process for the server. They do this through the following function:

/*
 * Starts the protocol-specific response. If this function returns
 * REQ_NOACTION, then the body response should be skipped and the
 * application function should return successfully. Otherwise, this
 * function returns REQ_PROCEED.
 */

int protocol_start_response(Session *sn, Request *rq);
If cross platform considerations are not required, then operating-system specific I/O calls may be made by Service functions.


INITIALIZATION FUNCTIONS

As described above, this class of server application functions are used to initialize static data for server modules. These functions are called by the base daemon process or thread, and their data, sockets, or file descriptors inherited by child processes or threads. This class of function is currently used to initalize the logging, file typing, and some of the name translation functions.

Init functions receive their parameters just like the other functions, except that the Session and Request parameters are NULL. The parameter pblock is filled with information from the server's technical configuration file.

Static data should be limited to read-only data if possible. Other mediums will need to employ some method of locking to ensure that only one process or thread is accessing the data at a time. Unfortunately, there is currently no OS-generic abstraction for this functionality available from the base server code.

Upon failure, these functions should return REQ_ABORTED and insert a variable into their parameter pblock named error which contains a string describing the error. Any other return code is considered success.

If the server is restarted, these modules may need to be terminated before they are started again by the server code. For this purpose, the server provides the following function:

/*
 * The given callback fn will be called by the server when the server
 * is restarted. The data pointer is given to the function as its
 * argument.
 *
 * For the curious, Magnus was the code name of the Netsite server.
 */
void magnus_atrestart(void (*fn)(void *), void *data)
Note: Netsite does not call the termination callbacks upon server termination, only upon restart.


A DYNAMIC LINKING INTERFACE TO THE
NETSCAPE SERVER SOFTWARE

The server application function and its related configuration paradigm suggest a powerful mechanism for customizing the Netsite server software without the customizing programmer having access to server source code or object modules.

Netsite uses dynamic linking or shared object functions of operating systems that support this mechanism to load Netsite API-compliant code modules into the server at run time. The mapping of character strings to function pointers already present in the server code lend themselves easily to the semantics of the dlopen and dlsym system calls, or their equivalents.

To load a Netsite-compliant code module, a new Initialization class server application function has been defined:

Init fn=load-module so=/local/netsite/lib/sm.so
     funcs="init-smokebender,use-smokebender"
This initialization function opens the given shared object, and loads the function pointers init_smokebender and use_smokebender. They are then accessible as init-smokebender and use-smokebender from the configuration files.


SUPPORTED PLATFORMS

Of the systems Netsite supports, the following systems support the mechanisms necessary to load functions into the server at run time:

Of the systems, BSDI is the only system which does not support the necessary mechanisms.


CREATING A SHARED OBJECT

When writing your server application functions, you should use the reference guide that came with the server software to access various server facilities. The server is written in ANSI C, and its header files are written that way. On some systems such as Windows NT and AIX, you must have an import list which specifies all global variables and functions you need to access from the server binary.

Otherwise, you write your code as you normally would. You will also need the Netsite header files for structure definitions and function prototypes.

UNIX

The following list describes the commands used to link object files into a shared object under the various UNIX platforms. In these examples, the object files t.o and u.o are being linked to form a shared object foo.so.

IRIX
ld -shared t.o u.o -o foo.so

SunOS
ld -assert pure-text t.o u.o -o foo.so

Solaris
ld -G t.o u.o -o foo.so

OSF/1
ld -all -shared -expect_unresolved "*" t.o u.o -o foo.so

HP-UX
ld -b t.o u.o -o foo.so

When compiling your code, you must also use the +z flag to the HP C compiler.

AIX
cc -bM:SRE -bE:module.exp -bI:module.imp -e _nostart t.o u.o -o foo.so

In the case of AIX, module.exp must be a file which lists each function that is to be exported from your library into Netsite. The functions should be listed, one per line.

module.imp lists, in the same format, each function or global variable which is to be imported from the Netsite program into your library.

WINDOWS NT

Using Visual C++ 2.0, you will need to compile a DLL. Similar to AIX, you must create a file which lists each function you intend to export to the Netsite program. You will also need the file httpd.lib which gives you the locations needed to link your library.


Corporate Sales: 415/528-2555; Personal Sales: 415/528-3777
If you have any questions, please visit Customer Service.

Copyright © 1996 Netscape Communications Corporation