Visual Basic and the Common Gateway. Article from Visual Developer Magazine

Need to create interactive Web sites? Why learn another language? VB can do anything Perl can do, and the Common Gateway Interface provides a standard mechanism for two-way browser-server communications.

One of the things I like best about Visual Basic is that it lets you do lots of things well. As the product has evolved over the past few versions, I have found myself turning to C++ less and less often. With version 6’s Web and Internet programming prowess, Visual Basic has become even more of a Swiss Army knife. Going beyond the bounds of Visual Basic itself, a knowledge of the Basic language takes you even further, with VBScript for client-side and server-side scripting, not to mention the versions of the Basic language that are used by Excel, Word, and Access. If you are going to be monolingual in programming languages, it seems like Basic is the way to go.

Basic is a central component of Microsoft’s approach to Web development, and in my opinion, Microsoft’s Web technology is excellent. To be honest, I have never done any serious Web development work with the competition’s tools, so I cannot make real comparisons—so no nasty letters from Java-heads, please! Recently, however, I did some Web database development using Active Server Pages (ASP) and ActiveX Data Objects (ADO). Anyone who thinks that Web database programming is difficult ought to give this combination a try!

There is one downside, as many of you are already thinking. Many of Microsoft’s Web technologies run only on Microsoft Web servers. Despite Microsoft’s push to take over the world, they have not yet succeeded, and many (if not most) Web servers are running non-Microsoft software. This may not present a problem for you, but for widest acceptance you should stay away from proprietary technologies as much as possible. For Web programming, this means you may want to use the Common Gateway Interface, or CGI.

What is CGI?

The most basic task of a Web server is to receive and fill user requests for specific HTML documents. This was the model of the original World Wide Web, and it was static—there was no provision for dynamic content. A way was needed for users to interact with programs that executed on the server beyond the basic act of requesting documents. If the user could enter information into the browser, and then send that information back to a specific program on the server, flexibility would be vastly improved. Based on the information sent by the user, the program on the server could perform whatever tasks were asked of it—such as searching a database—and return customized information to the user.

CGI was developed as a standard for this communication: how information gets from the user to a program on the server, and how the program sends information back to the user. At the user end, the HTML form specification was developed to permit the user to enter information through screen forms on HTML pages. Defined by the <FORM>...</FORM> tags, an HTML form contains one or more elements, such as text boxes and radio buttons, that permit the user to enter text, numbers, and other information. A form also has a “submit” button which, when clicked, causes the information that was entered on the form to be sent to the server. Also sent is the name of the program that the server should execute and pass the information to. The program uses the information to generate a response, almost always in the form of an HTML document, that is sent by the server back to the user, where it is displayed in his or her Web browser.

On the server, the CGI specification uses two methods for communication between the server software and the program. One method uses environment variables, which are named data storage locations maintained by the operating system and accessible from software. The other method uses the standard input and output streams. Referred to as stdin and stdout, these streams originated back in the elder days of C programming. Stdin was by default the keyboard, and stdout was the console, or screen—but these streams could be used for other purposes. Without going into too many details, here’s how it works:

The user submits a form from his or her browser. The information sent to the server includes the name of the program to run on the server as well as the information that the user entered into the form.
The Web server software puts the submitted information into environment variables and, in some situations, into stdin. If the submission was done with GET, all information is in environment variables and stdin is not used. If POST was used, then some information is in environment variables and some is in stdin. (More on GET and POST later on.) The server then executes the requested program.
The program retrieves the user data from the environment variables and, when required, from stdin. Based on this information it performs whatever processing is required.
The program writes its output to stdout. This output consists of the HTML document to be sent back to the user, plus some HTTP headers required by the server.
The server retrieves the output data from stdout, performs any necessary processing, and sends it on to the user.

To be used to write CGI programs, therefore, a programming language has to be able to access not only the environment variables but also stdin and stdout. In the days of 16-bit Windows, Visual Basic programs could not do this and therefore could not be used to write CGI programs. A workaround called Windows CGI was developed (see the sidebar, “Windows CGI”) but it did not develop wide support.

Windows CGI: A Relic of the 16-bit Era

The Windows CGI specification was developed to permit CGI programs to be written in languages that could not access environment variables and/or the standard streams. Win CGI placed user information in a temporary file called the CGI profile file instead of using environment variables and stdin. When the Win CGI program is started by the Web server, the name of the CGI profile file is passed to it on the command line. The program reads the user information from this file and performs the needed processing. When finished, the output is written to another temporary file, the name for which was included in the CGI profile file. The server software reads this file and then sends the HTML-formatted response back to the user.

Accessing Environment Variables

An environment variable is a global variable that is part of a CGI process. It is created by the server and lasts as long as the process exists. Each environment variable has a name and a value. For example, during a CGI session the server creates an environment variable named SERVER_NAME and assigns it the value of the server’s domain name.

In Visual Basic you use the Environ function to read environment variables. The syntax for the function is this:

val = Environ(name)

where name is the environment variable name whose value you want to retrieve. If there is no such variable then an empty string is returned. The CGI standard defines a large number of environment variables that are set during a CGI session. Table 1 describes the ones that you will use most often.

Table 1. Common CGI environment variables.
Variable Name	Description
CONTENT_LENGTH	The length in bytes of the supplemental data sent as part of a POST request.
CONTENT_TYPE	The MIME type of the supplemental data sent as part of a POST request.
GATEWAY_INTERFACE	The version number of the server’s CGI implementation (for example, CGI/1.1).
HTTP_ACCEPT	A comma delimited list of the MIME data types that the client can accept.
HTTP_REFERRER	The URL of the document that referred the user to the CGI program.
HTTP_USER_AGENT	Information on the requesting web client, including product code name, version, and operating system.
PATH_INFO	The extra part of the URL path.
PATH_TRANSLATED	The physical path corresponding to the extra path information, based on the server’s document mapping.
QUERY_STRING	The query string part of the URL.
REMOTE_ADDR	The IP address of the client computer from where the request is coming. If a proxy server is being used, then its IP address is returned.
REMOTE_HOST	Fully qualified domain name of the client computer from where the request is coming. Not always available because many client computers do not have a domain name.
REQUEST_METHOD	The method used by the client to make the request (for example, GET or POST).
SCRIPT_NAME	The location part of a URL in a request.
SERVER_NAME	The fully qualified domain name of the server.
SERVER_PROTOCOL	The name and version of the web protocol used to make the request (for example, HTTP/1.1).
SERVER_SOFTWARE	The name and version of the Web server software.

Accessing the Standard Input and Output Streams

While it is necessary to turn to the Windows API to access stdin and stdout, it is nonetheless very easy. You must first get a handle to stdin or stdout with the function GetStdHandle. Its declaration is as follows:

Public Declare Function GetStdHandle Lib "kernel32" _ (ByVal nStdHandle As Long) As Long

The argument nStdHandle specifies which handle you want; pass either STD_OUTPUT_HANDLE (its value being 11&) or STD_INPUT_HANDLE (its value 10&) as needed.

To read from stdin, you use the API function ReadFile to get the data. The declaration of this function is:

Private Declare Function ReadFile Lib "kernel32" _ (ByVal hFile As Long, _ ByVal lpBuffer As Any, _ ByVal nNumberOfBytesToRead As Long, _ lpNumberOfBytesRead As Long, _ ByVal lpOverlapped As Any) As Long

Here, the parameters are as follows:

hFile is the stdin handle obtained from GetStdHandle.
lpbuffer is a string variable where the data will be placed.
nNumberOfBytesToRead is the number of bytes to read. Its value should be one less that the size of lpbuffer.
lpNumberOfBytesRead is a type Long where the function returns the actual number of bytes read.
lpOverlapped is not used when reading from stdin; pass a value of zero.

To send data to stdout, use the API function WriteFile. Its declaration is:

Private Declare Function WriteFile Lib "kernel32" _ (ByVal hFile As Long, _ ByVal lpBuffer As Any, _ ByVal nNumberOfBytesToWrite As Long, _ lpNumberOfBytesWritten As Long, _ ByVal lpOverlapped As Any) As Long

The parameters are defined as follows:

hFile is the stdout handle obtains from GetStdHandle.
lpBuffer is a string variable containing the data to be written.
nNumberOfBytesToWrite is the number of bytes to write (the length of lpBuffer).
lpNumberOfBytesWritten is where the function returns the actual number of bytes written.
lpOverlapped is not used when writing to stdout; pass a value of 0.

Here is some code that shows you how to use these functions to access stdin and stdout:

Dim inbuf As String, lBytesRead As Long, hStdIn As Long Dim outbuf As String, hStdout As Long, lBytesWritten As Long inbuf = Space(2000) hStdIn = GetStdHandle(STD_INPUT_HANDLE) ReadFile hStdIn, inbuf, Len(inbuf) - 1, lBytesRead, 0& ' Now inbuf contains data from stdin. hStdout = GetStdHandle(STD_OUTPUT_HANDLE) ' Write data in outbuf to stdout. Call WriteFile(hStdout, (outbuf), Len(outbuf), lBytesWritten, 0&)

Using Stdin

The supplemental data that is sent as part of a POST request is placed in stdin by the server. As mentioned earlier, the stdin stream is not used with GET requests when the user data is obtained in the QUERY_STRING environment variable. When POST is used, the CONTENT_TYPE and CONTENT_LENGTH environment variables contain information about the size and format of the data available through stdin. The data in stdin is formatted as follows:

Spaces are replaced by plus signs.
Names and values are separated by equal signs.
Entries are separated by ampersands.

Here’s an example of user data returned in stdin:

keywords=monica+clinton&sort=ascending

This means that the user submission element named keywords has the value “monica clinton” and the element named sort has the value “ascending.” After reading the data from stdin, code in your program will have to separate out the individual values so they can be used.

Creating the Response

Once your CGI program retrieves the data submitted by the user, it needs to perform whatever processing is required. This may be database access or some other task—that depends on the purpose you wrote the program for. In any case, the final act of the CGI program will be to write its response to stdout.

What must be written to stdout? You must write the entire HTML page that the user will eventually see, including all HTML tags as well as content. In addition, the response must include some HTTP response headers that are required by the server. The headers you include determine whether you create a direct response or an indirect response.

A direct response is sent by the server to the user with no processing. This means that only the HTTP headers you included will be sent. To create a direct response, begin the response with a status header, which is a sequence of three text variables with the following structure:

Protocol/Version Status Reason

Protocol/version will be “HTTP/1.0”. Status is a three digit numerical code describing the results of the request, and Reason is a brief text description of the result. For example, a status header would look like this:

HTTP/1.0 200 OK

This status header indicates that the request completed successfully. Other codes and descriptions are used for various error conditions. Following the first line of a direct HTTP response, you can optionally include additional headers that provide information to the client. You can find details on other status codes and HTTP headers in any book that covers the HTTP protocol in depth, as well as some advanced HTML texts. As an example, here is a set of direct response HTTP headers:

HTTP/1.0 200 OK Server: Microsoft-IIS/4.0 Date: Friday, 9-OCT-1998 10:15:00 GMT Last-modified: Thursday, 8-OCT-1998 21:35:30 GMT Content-type: text/html

Creating an Indirect Response

You can also send an indirect response by beginning the data with the content-type header. Since your response will be an HTML document, the type is text/html and the complete header will be as follows:

Content-type: text/html

When you send an indirect response, the server recognizes it as such and will add additional headers to create a completely formed HTTP response before sending it to the client.

What comes after the headers? A blank line and then your HTML response. Here is an example of a simple but complete response that a CGI program could send back to the user’s Web browser:

Content-type: text/html

<html> <head> <title>Sample CGI output</title> </head> <body> Hello, world! </body> </html>

A Demonstration

Enough talk, let’s code! I have written a very simple CGI application. It consists of an HTML page that lets the user input his or her name and then submits it to SIMPLECGI.EXE, written in Visual Basic. The CGI program retrieves the user information and responds with a page that greets the user by name and displays some other information that is available in environment variables. Listing 1 shows the HTML document’s text, and Listing 2 contains the Basic listing for SIMPLECGI.BAS. By the way, be sure to use the same declarations of WriteFile() and ReadFile() that I use here. If you try to use the declarations of these functions returned by the Visual Basic API Text Viewer, be aware that these include errors and your program will not work.

You can run this program under the Microsoft Personal Web Server or most any other Web server. Create an EXE and place it in your CGI-BIN directory. If your CGI executable directory is something else be sure to edit the FORM tag in SIMPLECGI.HTM to point to the proper location. Place the HTML document in the root folder and navigate to it. As with all Visual Basic programs, the Visual Basic runtime must be installed on the server for the program to run.

The Special Nature of CGI Apps in Visual Basic

CGI applications in Visual Basic are unlike any Visual Basic application you have probably ever created. For one thing, they have no visual interface—no forms or controls, just code. Your Visual Basic project will contain only a code module, which must contain a procedure named Main() where execution will begin. Because there is no one to respond to a CGI application, it cannot display message boxes and in fact cannot be interactive in any way; it needs to simply run on its own.

The no-user-interaction consideration is relevant to error trapping as well. You must make sure that error trapping is enabled throughout the program so that the default error message box will never be displayed. You can write error messages to the output so the user will see them. You can also write error messages to a log file on the server.

If you need to create CGI applications, Visual Basic can be very useful. One of the reasons that PERL is so popular for CGI programming is its robust text handling capabilities, and we all know that it is hard to beat Visual Basic when it comes to text! v

During daylight hours, Peter is a research scientist at Duke University Medical Center in Durham, North Carolina. Comments and suggestions may be sent to him at paitken@acpub.duke.edu