Introduction ¶

There is nothing special about Weblocks. It is merely one possible logical solution to a subset of problems faced in web application development. To get familiar with Weblocks and its approach let's start with breaking the ice.

Breaking the Ice ¶

Once Weblocks has been installed it can be loaded into the Lisp image like this:

> (asdf:operate 'asdf:load-op 'weblocks)

We can then start the webserver (Weblocks uses Hunchentoot) and the framework on the default port (8080) like this:

> (weblocks:start-weblocks)

If we point the web browser to ​http://localhost:8080/ we'll get Error 500 - an Internal Server Error[1]. We now know the web server is running, but what's wrong?

When weblocks starts up it sets up a hook that processes all client requests. The hook looks for two things - a web application definition, and a callback function named 'init-user-session' that initializes the application every time a new session is created (sessions are managed by Weblocks automatically). Let's satisfy these requirements:

> (weblocks:defwebapp 'our-application)
> (defun init-user-session (comp)
    (setf (weblocks:composite-widgets comp)
          (list "Hello!")))

Refresh the browser. If all went well you should see the word "Hello!" printed on the screen.

What did we do? We told Weblocks that our application is named 'our-application'. Weblocks will look for 'init-user-session' in the same package where the symbol 'our-application' is interned. Before we can figure out how 'init-user-session' works we need to explain the concept of 'widgets' - this will be done in the next section.

Widgets ¶

Weblocks applications are not organized into pages. They're organized into "widgets". A widget can potentially be anything that's rendered to the browser. Widgets can be very simple or very complex.

Weblocks defines a generic function 'render-widget-body' along with a number of methods that accept different objects. Any object that can be passed to 'render-widget-body' can be called a widget. New widgets can be created by adding methods to this generic function.

There is a method added to 'render-widget-body' that accepts a vector of characters. That means the simplest possible widget is a string:

> (weblocks:render-widget-body "Hello World!")

When 'render-widget-body' is called on a string, the string is simply outputted as an HTML paragraph.

Functions can also be treated as widgets. When 'render-widget-body' is called with a function an appropriate method is invoked that simply calls the function:

> (weblocks:render-widget-body (lambda (&rest args)
                                 (with-html
			           (:p "test"))))

The macro 'with-html' is a wrapper that weblocks provides for CL-WHO macro 'with-html-output'. For each client request Weblocks sets up a stream available via a special variable *weblocks-output-stream*. By writing to this stream the application can send HTML to the client. However, using 'with-html' is preferred - it automatically redirects output to *weblocks-output-stream* and takes full advantage of CL-WHO which alleviates the need to format HTML strings or to use template engines[2].

One of the widgets that Weblocks defines is a 'composite' widget. A composite is a CLOS object that contains a list of other widgets. A composite provides a simple and convenient way to group widgets. Rendering a composite renders each widget in the list, one by one:

> (setf my-composite
        (make-instance 'weblocks:composite
                       :widgets (list "a" "b")))
> (weblocks:render-widget-body my-composite)

In this case two paragraphs will be rendered. The first one will say "a" and the second one will say "b". Composites can be recursive - they can contain other composites.

Most widgets defined by Weblocks are CLOS objects. The fact that strings and functions are widgets are an exception made for convenience, rather than the rule. The base class for widgets is 'widget'. It contains a number of slots that are common to all widgets. One of these slots is 'name'. If the name of the widget isn't given during the instantiation, a unique name is generated automatically.

Widgets map to HTML really well. The way widgets are normally rendered is via a function 'render-widget', not via 'render-widget-body'. The function 'render-widget' wraps the widget body in a 'div' element. The id of the div is set to the widget name and the classes of the div are set to the CLOS hierarchy class names. For example, the header for the composite widget will be rendered something like this:

<div id="g2345" class="widget composite"> ... </div>

This works very well for CSS styling too.

Widgets that don't derive from CLOS class 'widget' like strings and functions are handled in a similar manner, except they lack the "id" attribute.

The magic behind 'init-user-session' ¶

We're now ready to understand how 'init-user-session' works.

When weblocks sees an HTTP request that does not yet have a session associated with it, a session is created. A new composite widget is instantiated and is associated with this session. This composite widget is called a 'root' - all other widgets of the application will be stored in it.

The root composite is then passed to 'init-user-session'. It's up to 'init-user-session' to add other widgets to the root composite. When it's time for Weblocks to render HTML to the client it simply calls 'render-widget' on the root composite, which ends up rendering all widgets that were added to the root in 'init-user-session'.

Recall our 'init-user-session' code:

(defun init-user-session (comp)
  (setf (weblocks:composite-widgets comp)
        (list "Hello!")))

The argument 'comp' is the root composite that Weblocks will pass to 'init-user-session'. The accessor 'composite-widgets' gives access to the slot of the root composite that contains a list of widgets. We simply set the slot to a new list of one element - a string "Hello!". We can get away with this because strings are also widgets.

We can add other widgets to this list, including other strings, functions, composites, etc[3].

Renderers ¶

A major criticism of the above approach is that it steers away from the MVC model since HTML (or the CL-WHO alternative) is mixed with the code. This does not have to be true. In principle, nothing stops you from writing widgets that invoke a template engine. However, Weblocks does not go down this path.

Weblocks treats HTML as a serialization format. Weblocks' philosophy is that neither the programmer nor the designer should have to write HTML - most of the time HTML should be generated automatically. The programmer's job is to define the data structures, the high level UI components, and the business logic. The designer's job is to create appropriate stylesheets. The actual HTML markup generation is the job of the framework.

In practice this isn't always possible because of CSS quirks and limitations, but Weblocks tries to stay true to this approach whenever the current state of W3C affairs permits. In order to make this possible Weblocks defines a set of 'renderers' - pieces of code that serialize data structures to different types of HTML.

Let's illustrate this approach by defining a data structure for a person:

> (defclass person ()
    ((id :initform (gensym))
     (first-name :accessor first-name
   	         :initarg :first-name
	         :initform nil)
     (last-name :accessor last-name
   	        :initarg :last-name
	        :initform nil)
     (age :accessor age
          :initarg :age
	  :initform nil)))
> (setf joe (make-instance 'person
                           :first-name "Joe"
	                   :last-name "Average"
			   :age 31))

Normally, to render information about the person to the screen we'd create a number of templates that we'd use throughout the application. We would probably create a template for rendering the data, a template for rendering the form, and perhaps a template for rendering a table of people. This wouldn't be too difficult except we have to do this again and again for every new data structure we come up with. We end up generating nearly the same HTML manually - all that's changing is a list of fields. Weblocks automates this work:

> (weblocks:render-data joe)
> (weblocks:render-table (list joe))

The first line renders our data structure into a list of fields. The second line renders a table of one row. In similar manner weblocks provides a renderer for serializing data structures into forms. Other renderers can be added if the need arises.

Renderers are designed to output high quality, validating HTML. Special care is taken to ensure the generated HTML follows accessibility guidelines. Due to CSS limitations renderers generate somewhat heavy HTML - extra tags and classes are put in place to allow for sufficient freedom in styling. Much of this markup will disappear when CSS3 is supported by all mainstream browsers[5].

Renderers try to respect the rules of the language. For example, if a slot has no accessor, by default this slot will be omitted from rendering.

Renderers can be controlled to easily rename, rearrange, hide/show, and custom render slots[4]. Most widgets that deal with rendering data structures (dataform widget, grid widget, etc.) use renderers to render data. Weblocks tries to provide renderers that satisfy commonly used customization cases. The programmer should resort to custom HTML only in rare special cases when there is no way to configure the renderer to generate sufficient HTML.

Actions ¶

Everything we've discussed above has been about rendering data to the client. So far we haven't discussed interactivity - the user's ability to change the state of the UI and to modify the data.

Weblocks allows the programmer to deal with client interaction without having to worry about the limitations of HTTP protocol. In Weblocks the programmer can 'render' a function, a generic function, or a closure into a link (or a button in a form). When the user clicks on the appropriate link (or button), Weblocks maps the click back to the callback. If the callback is a lexical closure, the programmer will have the full context in which the closure was created despite the fact that it was created during a completely different HTTP request. The only requirement weblocks imposes on the programmer is to convert a funcallable object into an 'action' by calling 'make-action':

> (weblocks:render-link (weblocks:make-action (lambda (&rest args)
                                                (do-something)))
	                "Modify")

In this case the lambda function is converted into an action by calling 'make-action'[6]. The action is then rendered into a link named 'Modify' via 'render-link'. When the user later clicks on 'Modify', the lambda function is called (in the appropriate lexical context, if there is one), and 'do-something' is evaluated. A similar mechanism can be invoked for rendering form buttons.

Note, 'render-link' renders an AJAX-friendly link. The callback will be made via an AJAX request if !Javascript is available. Otherwise, Weblocks will fall back to a regular request. Because Weblocks applications aren't organized by pages, links change the state of widgets, rather than move between pages. This is why rendering a non-AJAX link doesn't make sense in the context of Weblocks.

Weblocks does have a mechanism to allow for friendly URLs. This mechanism is exposed to the programmer via the 'navigation' widget. The navigation widget takes very special care to hide the fact that Weblocks applications aren't organized into pages from the user. This way the user can enjoy the rich UI that follows established conventions, while the programmer can code the application using the widget paradigm. See reference documentation for the navigation widget for more details.

More on Widgets ¶

So far we've only seen stateless widgets - string, function, and composite. However, we've hinted that widgets can have state. The simplest example of a stateful widget is a closure that increments a counter every time it's evaluated.

Some of the more complex Weblocks widgets have state associated with them. One such widget is 'dataform'. We'll talk about it here to put together everything we've discussed so far.

Dataform widget models a fairly common behavior of web applications. It renders a datastructure to HTML so the user can see some data. Below the data it renders a "Modify" link. If the user clicks on the link, the widget changes its state from 'data' to 'form' and renders a form for the user to modify the data along with a 'Submit' and 'Cancel' buttons. If the user clicks 'Submit', any changes are submitted and the state is changed back to 'data'. If the user clicks 'Cancel', the changes are discarded and the state is changed to 'data' as well.

Dataform widget is composed of everything we've discussed so far. It uses a data renderer to render the data, and a form renderer to render the form. It also uses actions to render the 'Modify' link and the buttons. Since dataform is a CLOS object, it stores its current state (whether it's displaying the structure as data or form) inside a slot. When the actions are invoked by the user this state is changed appropriately. We can change 'init-user-session' to take advantage of the dataform widget, and to finally add some interactivity to our application:

> (defun init-user-session (comp)
    (setf (weblocks:composite-widgets comp)
          (list (make-instance 'dataform :data joe))))

If we reset the session and refresh the page, we'll see information about Joe displayed in the browser. We can click on 'Modify', change some of the data in the form, and click submit or cancel.

Since each dataform widget encapsulates and manages its own state, we can add as many dataform widgets to our page as we like. We can play with their states and they will all behave correctly. This is an enormous benefit about componetized approach - once we've created a widget, we can reuse it to our heart's content.

AJAX ¶

Because actions are rendered to be AJAX-friendly, the dataform widget also ends up being updated in an AJAX request. How does this work? Why does the dataform widget redraw if 'Modify' is called asynchronously?

Weblocks uses the power of Lisp to fully automate this behavior. Every time a value of any slot of the widget changes, Weblocks detects the change using a hook in MOP and marks the widget as dirty[7]. Once the action is fully evaluated, Weblocks sends a list of dirty widgets back to the client, and each dirty widget is updated on the browser via JavaScript. Since the 'Modify' action changes the state of the dataform widget, the dataform is marked by Weblocks as dirty and the newly rendered version is sent to the client.

Note that this behavior scales back very well if the user has JavaScript? turned off. Instead of sending a list of dirty widgets to the client, the entire page is redrawn. Most of the time the web programmer, and even the widget designer, need not care about AJAX at all - everything is managed by Weblocks[8].

Troubleshooting ¶

If anything goes wrong during this tutorial (or during development), the 'Internal Server Error' message does not give great insight into why an error has occurred. Fortunately, it's very simple to turn on debug mode, so that the exact error, along with the stack trace, are displayed in the browser. Instead of

> (weblocks:start-weblocks)

run

> (weblocks:start-weblocks :debug t)

Note, if Weblocks is running you need to stop it first with 'stop-weblocks'.

[1] This is not very user friendly. The default application should provide basic information on how to get started.

[2] Many people argue that template engines are a good thing since they enforce MVC. Weblocks takes a different approach by using 'renderers' - pieces of code that generically serialize data structures into different types of HTML.

[3] Be sure to reset the session or redefining 'init-user-session' will have no effect. You can do this by restarting the browser or by calling 'weblocks::reset-sessions'.

[4] See reference documentation for 'render-data' for more details.

[5] Much of the extra markup would disappear if CSS2 was supported by IE6.

[6] See reference documentation for 'make-action' for more details on the conversion process.

[7] It is of course possible to turn this behavior off for some slots.

[8] Some widgets take care to specifically turn off AJAX. For example, the navigation widget renders non-AJAX links to allow for friendly URLs.