IIS7 modules vs. IIS6 ISAPI: Memory Management

For post #3 in the IIS7 Modules vs. IIS6 ISAPI series, let’s look at one of the biggest trouble areas that bite ISAPI developers - managing memory.

If you develop software that uses Win32 APIs, you should be familiar with the feeling you get when you look at the MSDN documentation of a fairly simple function, and see that it returns a string or a set of bytes.

The problem, of course, is that you as a caller are responsible for allocating a buffer of a fixed size, and providing this buffer to the function so it can receive the data.

Sounds simple, right? Well, not really. Don’t forget that you also have the handle the case where the buffer is too small to receive the data, AND, you are responsible for cleaning up the buffer when you done.

Let’s take the ISAPI extension GetServerVariable function as an example - this function is frequently called by ISAPI extensions to obtain various request information, such as the normalized URL of the request:

BOOL WINAPI GetServerVariable( 
       HCONN hConn, 
       LPSTR lpszVariableName, 
       LPVOID lpvBuffer, 
       LPDWORD lpdwSizeofBuffer
);

Here is a simplified (!!!) code sample of a utility functions that uses GetServerVariable to obtain server variable values:

#define DEFAULT_BUFFER_SIZE   100

#define MAXIMUM_BUFFER_SIZE   512

HRESULT

GetServerVariableIIS6(

    EXTENSION_CONTROL_BLOCK *  pEcb,

    PSTR                       pszName,

    PSTR*                      ppszValue

)

{

    //

    //  Allocate a buffer of default size

    // 

    HRESULT hr        = S_OK;

    BOOL    fRet      = FALSE;

    DWORD   dwLength  = DEFAULT_BUFFER_SIZE;

    PSTR    pszBuffer = new CHAR[dwLength];

    if ( pszBuffer == NULL )

    {

        hr = HRESULT_FROM_WIN32( ERROR_NOT_ENOUGH_MEMORY );

        goto Finished;

    }

    //

    //  Try to get the server variable

    //

    fRet = pEcb->GetServerVariable( pEcb->ConnID,

                                    pszName,

                                    pszBuffer,

                                    &dwLength );

    //

    //  Check for insufficient size, but only allow up to

    //  a bounded size for the value

    //

    if ( !fRet &&

           GetLastError() == ERROR_INSUFFICIENT_BUFFER &&

           dwLength < MAXIMUM_BUFFER_SIZE )

    {

        //

        //  Re-allocate the buffer to the required size

        //

        delete [] pszBuffer;

        pszBuffer = new CHAR[dwLength];

        if ( pszBuffer == NULL )

        {

            hr = HRESULT_FROM_WIN32( ERROR_NOT_ENOUGH_MEMORY );

            goto Finished;

        }              

        //

        //  Try again

        //

        fRet = pEcb->GetServerVariable( pEcb->ConnID,

                                        pszName,

                                        pszBuffer,

                                        &dwLength );

    }

    //

    //  Check if we succeeded

    //

    if ( !fRet )

    {

        hr = HRESULT_FROM_WIN32( GetLastError() );

        goto Finished;

    }

    //

    //  Return the buffer

    //

    *ppszValue = pszBuffer;

    pszBuffer = NULL;

Finished:

    //

    //  Clean up the buffer if an error occurred

    //

    if ( pszBuffer != NULL )

    {

        delete [] pszBuffer;

        pszBuffer = NULL;

    }

    return hr;

}

So, let’s quickly recap what I had to do here:

1. Allocate a buffer
2. Call GetServerVariable to fill the buffer
3. Check whether the value was too large, and if so, clean up the first buffer, and re-allocate the buffer to be the right size, but only if the size is not too big (never trust the client)
4. Call ServerVariable again with the larger buffer
5. If we failed to return the buffer to the client, clean up the buffer

Now, things are about to get a lot worse. You notice that I return the pointer to the allocated buffer from this function to the caller, making deallocating this buffer the responsibility of the caller. This is why I also don’t just allocate the initial buffer on the stack, like David does in his old sample for this code available here: http://blogs.msdn.com/david.wang/archive/2005/08/18/ISAPI-GetServerVariable.aspx.

This means, that the caller has to clean up the buffer when they are done with it. Imagine, that your code has to get a lot of server variables (as is the case a lot of the time).

You will pretty much also have to:

1. Maintain a list of buffers you used to get server variables
2. At the end of the request, go through that list and delete all the buffers

And you thought all you were going to do is get a few little server variables. Welcome to ISAPI hell.  

It turns out that managing buffers to operate ISAPI APIs a lot of the time becomes the main theme of your development. Sure, you’ll say, you’ve written this code so many times, you have a solid pattern and maybe even a library you’ve written to do it. And you probably spent quite a few hours debugging memory leaks and access violations doing it.

Enter IIS7.

In IIS7, we made one simple improvement - we took over the management of memory for server APIs that return buffers. Instead of you keeping track of buffers with server variable values, the IIS server core does it (hey, it has to do it anyway).

Many IIS7 APIs return pointers to already allocated core server memory that contains the values you are retrieving.

With IIS7, you don’t need a utility function, or a single new or delete statement to get a server variable. You simply to this:

IHttpContext*    pContext;

PCSTR        pszValue = NULL;

DWORD       dwLength = 0;

//

//  Get the server variable

//

hr = pContext->GetServerVariable( "AUTH_USER",

                                  &pszValue,

                                  &dwLength );

if ( FAILED( hr ) )

{

    goto Finished;

}

NOTE: The dwLength parameter is not an input parameter - it receives the length of the server variable value as a convenience to you, so that you don’t need to call strlen().

After successful completion, The pszValue points to a buffer that can be used for the duration of the request. After the request is finished, the server will automatically reclaim the memory so you never need to worry about cleaning it up.

BONUS - Unicode Support: The server provides an overload of this function that returns Unicode server variable values. This replaces the really painful way to get Unicode values with ISAPI - call the GetServerVariable with “UNICODE_variablename”, and treat the returned bytes as WCHAR even though the buffer is passed in as a CHAR buffer.

Overall, the IIS7 memory management model basically means that much of the time, you don’t need to:

1. Allocate your own memory
2. Check for insufficient buffer sizes
3. Manage and clean up your own memory

This post got a little long, but really I cannot overstate how much grief this saves you when writing a module … letting you concentrate on the reason why you are actually writing it.

Until next time …