Using Lwan as a Library

Instead of being a standalone web server, Lwan is built as a reusable library that can be used by applications to expose an interface through HTTP. Although possible, Lwan isn't intended to be used to embed a HTTP server into already-existing programs; minimal changes to the API are required for this purpose.

There are various examples throughout the tree that can be used as a way to understand how to use Lwan as a library, and even how to extend its capabilities.

This is a tutorial that will present some examples, as well as point to (more complete) examples found throughout the source tree. Not everything will be explained, specially if deemed too straightforward or self-explanatory. This is a Wiki page, though, so feel free to extend it as you see fit.

Let's start with a minimal example and grow it to use more of Lwan's infrastructure.

A piece of code that sets up the HTTP server, awaits for connections, and replies to requests is pretty easy to write:

#include "lwan.h"

int main()
{
    struct lwan l;
        
    lwan_init(&l);
    lwan_main_loop(&l);
    lwan_shutdown(&l);
        
    return 0;
}

The easiest way to build this program is to create a directory in the Lwan source tree, say "minimal", add the example above as "minimal.c", and add a CMakeLists.txt file with this content:

add_executable(minimal minimal.c)
target_link_libraries(minimal
	${LWAN_COMMON_LIBS}
	${ADDITIONAL_LIBRARIES})

Add the following line to the CMakeLists.txt found in the root of the source tree:

add_subdirectory(minimal)

Enter the build directory and issue "make". You should get a new directory called "minimal" with a binary called "minimal" inside it. There's no need to run CMake again if you've built Lwan before inside this build directory, as it'll pick up changes to the root CMakeLists.txt file automatically. Refer to the README.md file for build instructions if you haven't built Lwan before.

If you run this program, it'll print something like this (output may be different if built in Debug mode, or if your computer has a different number of CPU cores):

Loading configuration file: minimal.conf.
Could not read config file, using defaults.
Using 4 threads, maximum 1024 sockets per thread.
Listening on http://127.0.0.1:8080.
Ready to serve.

One thing to notice is that Lwan inferred from the executable name that it should read its configuration from a file named "minimal.conf". Since it could not be found, defaults are being used.

If you try to access any URI under http://127.0.0.1:8080, all you're going to get is a 404 error, because no handlers are set up at the moment. Let's change that by creating a "minimal.conf" file in the same directory you're calling "minimal" from:

listener 127.0.0.1:8080 {
	prefix /hello {
		handler = hello_world
	}
}

Let's try running the minimal example again and see what happens:

~/git/lwan/build$ ./minimal/minimal 
Loading configuration file: minimal.conf.
Error on config file "minimal.conf", line 3: Could not find handler "hello_world".
Aborted (core dumped)

References to a handler in Lwan are stored in a special section in the executable file. To aid the declaration of one, the LWAN_HANDLER() macro can be used. Add this code to "minimal.c":

LWAN_HANDLER(hello_world)
{
    static const char message[] = "Hello, World!";

    response->mime_type = "text/plain";
    lwan_strbuf_set_static(response->buffer, message, sizeof(message) - 1);
    return HTTP_OK;
}

Rebuild and try running it again. Verify that it doesn't complain about a missing configuration file anymore, so something must have worked. Try opening http://127.0.0.1:8080/hello on a browser. You should get a plain text "Hello, World!" response.

Pretty simple, eh?

It's not always possible to use the automatic handler binding, though. For instance, the "data" parameter in the hello_world() function will always have a garbage value when hooking up a handler like this.

To provide some valid data, one would declare an array of struct lwan_url_map type and call the lwan_set_url_map() function. It's a pretty simple idiom, which is better illustrated in the freegeoip sample application: there, only one handler, templated_output(), answers for three prefixes: "/json/", "/xml/", and "/csv/"; what changes between them is the data parameter, which are pointers to compiled template instances for the different output formats supported by that sample application.

Go there take a look, I'll wait.

All done? Good.

You may have seen a call to the SERVE_FILES() macro, when answering to the "/" prefix. This basically serves as a fallback: files will be served from a particular directory if none of the other prefixes matches. If instead of declaring this fallback in the lwan_url_map_t array you'd like to declare in the configuration file, it's pretty simple as well. Add these lines to minimal.conf (in the same level as the "prefix /hello" block) and restart the server:

serve_files / {
    path = ./wwwroot
}

Notice that, instead of having a "prefix" block, there is a "serve_files" block. This is because file serving is a module, not a handler. Modules allow greater control of how Lwan will parse a request, while also providing means of storing per-module-instance datum. This is just a shorthand syntax for the following configuration options:

prefix / {
    module = serve_files
    path = ./wwwroot
}

Let's write our own module. Something simple, just to wet our feet. Or, better yet, let's dissect the "redirect" module, built into Lwan. Add these lines to "minimal.conf", and restart the server:

redirect /elsewhere {
    to = http://lwan.ws
}

Restart the server and try opening http://127.0.0.1:8080/elsewhere. Notice your browser went to the Lwan website.

To understand how this works, open the "lwan-redirect.c" file within the "common" directory. Take a quick glance, and notice there are only static functions with the exception of one: lwan_module_redirect().

This function returns pointer to a lwan_module_t instance, and is called only once when Lwan is initialized. It is called from within lwan_init() and passed to lwan_module_register(), which ties the name field to that instance. This way, when Lwan finds "redirect" in the configuration file, it knows what module to refer to.

You'll notice also that there are two initialization functions: redirect_init() and redirect_init_from_hash(). One takes a void pointer, and the other a pointer to a struct hash.

Each module might have its own configuration structure, and that's exactly what the first function does: accepts that configuration structure, initializes a module instance, and returns data that is passed to the handler function.

The function that takes a hash table does exactly the same setup, but it's called when the module is configured from a configuration file; notice that a settings structure is allocated and the other function is called to set up the redirect module. All the values are looked up at runtime from the hash table, that was populated while reading the configuration file.

Usually a module will have an instance structure as well, but the redirect module is pretty simple so only a copy of the URI to redirect to is stored as the instance data. That's the reason the standard free() funciton is used as a module instance shutdown function; should a structure be used instead, a proper function should be provided, to perform all required shutdown steps.

The handler function works exactly the same as the one we just wrote for the "Hello, World!" example. We're just using all parameters now: request, response, and data.

You'll notice as well that we allocate memory using coro_malloc() rather than malloc(). This wrapper around malloc() will ensure that memory will be freed automatically regardless of how the handler exited, and after Lwan is done using the data. This prevents memory leaks and some race conditions. There various other similar wrappers available and they'll be introduced as we walk forward.

Right now, the API to register a module isn't exposed to library users, so loading modules from a configuration file is only possible for built-in modules. If there's a need to have modules in your application, just use the lwan_set_url_map() function. This will change in the future once the API stabilizes a little bit more.

Another thing that you might have noticed is the "flags" parameter defined to 0 in redirect_module. This is a mask of various possible flags, and that signals Lwan to perform (or not) certain potentially expensive tasks that are not always required for that particular module. For instance, parsing the query string or checking the Range header isn't always required, so flags to perform these operations are not set. Possible flags are:

typedef enum {
    HANDLER_PARSE_QUERY_STRING = 1<<0,
    HANDLER_PARSE_IF_MODIFIED_SINCE = 1<<1,
    HANDLER_PARSE_RANGE = 1<<2,   
    HANDLER_PARSE_ACCEPT_ENCODING = 1<<3,
    HANDLER_PARSE_POST_DATA = 1<<4,
    HANDLER_MUST_AUTHORIZE = 1<<5,
    HANDLER_REMOVE_LEADING_SLASH = 1<<6,

    HANDLER_PARSE_MASK = 1<<0 | 1<<1 | 1<<2 | 1<<3 | 1<<4
} lwan_handler_flags_t;

The special value HANDLER_PARSE_MASK is used on handlers, so all handlers will parse the query string, the If-Modified-Since header, the Range header, the Accept-Encoding header, and the post data (if applicable).

That's basically it for modules and handlers.

There are various other things in Lwan that can be used to build more complicated things. The templating mechanisms are pretty well illustrated in the blog posts that illustrated them, so I'll now focus on how to use the caching mechanism.

The caching mechanism in Lwan controls the lifetime of the items it caches. Items remain in the cahce for a tunable amount of time; they're reference counted, so if someone is holding a reference the item isn't destroyed when its time to live expires.

To create a cache, one need to provide two callback functions, a pointer to some context data (passed to these callback functions) and the time to live for each cache item.

One callback has the type CacheEntryCallback, and must return a cache_entry_t*. Its parameters are a key, and a pointer to the context, passed to the cache constructor. One must embed the cache_entry_t structure in the structure containing the data to be cached; none of its fields should be touched by the user -- everything should be considered private data. The context can be cast to any structure useful for your purposes. So, for instance:

struct deep_thought {
    struct cache_entry_t base; /* must be the first element */
    int answer;
};

struct cache_entry_t *calculate_answer(const char *key, void *context) {
        struct deep_thought *dt = malloc(sizeof(*dt));
        if (!dt) return NULL;
        if (!strcmp(key, "about-life-universe-and-everything"))
	    dt->answer = 42;
	else
	    dt->answer = -1;
        return (struct cache_entry_t *)dt;
}

The other callback has a type DestroyEntryCallback, and receives a struct cache_entry_t and a pointer to a context:

void destroy_answer(struct cache_entry_t *entry, void *context) {
    free(entry);
}

With these things defined, creating the cache is pretty simple:

    struct cache_t *cache = cache_create(calculate_answer, destroy_answer,
    		NULL, 15);

To obtain something from the cache, it is advisable to do so while within a coroutine (if obtaining an item from a request handler, that's always true):

    struct deep_thought *dt = (struct deep_thought *)cache_coro_get_and_ref_entry(
    	cache, request->conn->coro, "");

This will automatically unreference the cache entry when the coroutine is done, avoiding race conditions and ensuring the entry will be properly destroyed whenever it's not being used anymore.

If not within a coroutine, one should use cache_get_and_ref_entry() and call cache_entry_unref() manually.

Ideally, the cache should be allocated in a module's initialization function, and then destroyed in its shutdown function. That's what the file serving module (lwan-serve-files.c) does, for example.