Introduction
There is nothing special about Weblocks. It is merely one possible logical solution to a subset of problems faced in web application development. To get familiar with Weblocks and its approach let's start with breaking the ice.
Breaking the Ice
To learn how to obtain and install Weblocks see ObtainingAndInstalling.
Once Weblocks has been installed it can be loaded into the Lisp image like this:
> (push #p"/usr/lib/sbcl/site/cl-weblocks/" asdf:*central-registry*) > (asdf:operate 'asdf:load-op 'weblocks)
Replace /usr/lib/sbcl/site/cl-weblocks/ with the path to your weblocks installation.
We can then start the webserver (Weblocks uses Hunchentoot) and the framework on the default port (8080) like this:
> (weblocks:start-weblocks)
Point the web browser to http://localhost:8080/weblocks-demo/. If all went well we should see a welcome page. We're now running Weblocks!
Our First Weblocks Application
When weblocks starts up it sets up a hook that processes all client requests. The hook looks for two things - a web application definition, and a callback function named 'init-user-session' that initializes the application every time a new session is created (sessions are managed by Weblocks automatically). If these are not provided, Weblocks uses default values that point to the welcome page.
To create our own Weblocks application we need to override the defaults. We can do it by typing the following code snippet into the REPL:
> (weblocks:defwebapp our-application :prefix "/")
> (defun init-user-session (comp)
(setf (weblocks:composite-widgets comp)
(list "Hello!")))
Now let's tell Weblocks to reset all sessions so it picks up the new application:
> (weblocks:reset-sessions)
Refresh the browser. Instead of the welcome page you should now see the word "Hello!" printed on the screen.
What did we do? We told Weblocks that our application is named 'our-application'. Weblocks will look for 'init-user-session' in the same package where the symbol 'our-application' is interned. Before we can figure out how 'init-user-session' works we need to explain the concept of 'widgets' - this will be done in the next section.
Widgets
Weblocks applications are not organized into pages. They're organized into 'widgets'. A widget can potentially be anything that's rendered to the browser. Widgets can be very simple or very complex.
Weblocks defines a generic function 'render-widget-body' along with a number of methods that accept different objects. Any object that can be passed to 'render-widget-body' can be called a widget. New widgets can be created by adding methods to this generic function.
There is a method added to 'render-widget-body' that accepts a vector of characters. That means the simplest possible widget is a string:
> (weblocks:render-widget-body "Hello World!")
When 'render-widget-body' is called on a string, the string is simply outputted as an HTML paragraph.
Functions can also be treated as widgets. When 'render-widget-body' is called with a function an appropriate method is invoked that simply calls the function:
> (weblocks:render-widget-body (lambda (&rest args)
(weblocks:with-html
(:p "test"))))
The macro 'with-html' is a wrapper that weblocks provides for CL-WHO macro 'with-html-output'. For each client request Weblocks sets up a stream available via a special variable *weblocks-output-stream*. By writing to this stream the application can send HTML to the client. However, using 'with-html' is preferred - it automatically redirects output to *weblocks-output-stream* and takes full advantage of CL-WHO which alleviates the need to format HTML strings or to use template engines. (Many people argue that template engines are a good thing since they enforce MVC. Weblocks takes a different approach by using 'views' - pieces of code that generically serialize data structures into different types of HTML.)
One of the widgets that Weblocks defines is a 'composite' widget. A composite is a CLOS object that contains a list of other widgets. A composite provides a simple and convenient way to group widgets. Rendering a composite renders each widget in the list, one by one:
(setf my-composite
(make-instance 'weblocks:composite
:widgets (list "a" "b")))
(weblocks:render-widget my-composite)
*Please note*, all code snippets that aren't prepended with > (including the one above) will singal an error when evaluated in the REPL. These snippets need a dynamic session environment set up in order to work properly. When they're called as part of the HTTP request, the session environment is present. When they're called from the REPL, the session environment is missing. If you'd like to experiment with these snippets in the REPL, see macro 'with-request' defined in 'weblocks-test'. The macro sets up a dummy environment for testing, debugging, and experimenting purposes.
In this case two paragraphs will be rendered. The first one will say "a" and the second one will say "b". Composites can be recursive - they can contain other composites.
Most widgets defined by Weblocks are CLOS objects. The fact that strings and functions are widgets are an exception made for convenience, rather than the rule. The base class for widgets is 'widget'. It contains a number of slots that are common to all widgets. One of these slots is 'name'. If the name of the widget isn't given during the instantiation, a unique name is generated automatically.
Widgets map to HTML really well. The way widgets are normally rendered is via a function 'render-widget', not via 'render-widget-body'. The function 'render-widget' wraps the widget body in a 'div' element. The id of the div is set to the widget name and the classes of the div are set to the CLOS hierarchy class names. For example, the header for the composite widget will be rendered something like this:
<div id="g2345" class="widget composite"> ... </div>
This works very well for CSS styling too.
Widgets that don't derive from CLOS class 'widget' like strings and functions are handled in a similar manner, except they lack the "id" attribute.
The magic behind 'init-user-session'
We're now ready to understand how 'init-user-session' works.
When weblocks sees an HTTP request that does not yet have a session associated with it, a session is created. A new composite widget is instantiated and is associated with this session. This composite widget is called a 'root' - all other widgets of the application will be stored in it.
The root composite is then passed to 'init-user-session'. It's up to 'init-user-session' to add other widgets to the root composite. When it's time for Weblocks to render HTML to the client it simply calls 'render-widget' on the root composite, which ends up rendering all widgets that were added to the root in 'init-user-session'.
Recall our 'init-user-session' code:
> (defun init-user-session (comp)
(setf (weblocks:composite-widgets comp)
(list "Hello!")))
The argument 'comp' is the root composite that Weblocks will pass to 'init-user-session'. The accessor 'composite-widgets' gives access to the slot of the root composite that contains a list of widgets. We simply set the slot to a new list of one element - a string "Hello!". We can get away with this because strings are also widgets.
We can add other widgets to this list, including other strings, functions, composites, etc (Be sure to reset the session or redefining 'init-user-session' will have no effect. You can do this by restarting the browser or by calling 'weblocks::reset-sessions').
Views
A major criticism of the above approach is that it steers away from the MVC model since HTML (or the CL-WHO alternative) is mixed with the code. This does not have to be true. In principle, nothing stops you from writing widgets that invoke a template engine. However, Weblocks does not go down this path.
Weblocks treats HTML as a serialization format. Weblocks' philosophy is that neither the programmer nor the designer should have to write HTML - most of the time HTML should be generated automatically. The programmer's job is to define the data structures, the high level UI components, and the business logic. The designer's job is to create appropriate stylesheets. The actual HTML markup generation is the job of the framework.
In practice this isn't always possible because of CSS quirks and limitations, but Weblocks tries to stay true to this approach whenever the current state of W3C affairs permits. In order to make this possible Weblocks defines a set of 'views' - pieces of code that serialize data structures to different types of HTML.
Let's illustrate this approach by defining a data structure for a person:
> (defclass person ()
((weblocks:id)
(first-name :accessor first-name
:initarg :first-name
:initform nil)
(last-name :accessor last-name
:initarg :last-name
:initform nil)
(age :accessor age
:initarg :age
:initform nil)))
> (setf joe (make-instance 'person
:first-name "Joe"
:last-name "Average"
:age 31))
Normally, to render information about the person to the screen we'd create a number of templates that we'd use throughout the application. We would probably create a template for rendering the data, a template for rendering the form, and perhaps a template for rendering a table of people. This wouldn't be too difficult except we have to do this again and again for every new data structure we come up with. We end up generating nearly the same HTML manually - all that's changing is a list of fields. Weblocks automates this work:
;; note: this has to be invoked in function passed as widget to dynamic environment (weblocks:render-object-view joe '(data person)) (weblocks:render-object-view (list joe) '(table person))
The first line renders our data structure into a list of fields. The second line renders a table of one row. In similar manner weblocks provides a view for serializing data structures into forms. We specify the class of the object (person) to allow the scaffolding infrastructure to generate an appropriate view. Other views can be added if the need arises.
Views are designed to output high quality, validating HTML. Special care is taken to ensure the generated HTML follows accessibility guidelines. Due to CSS limitations views generate somewhat heavy HTML - extra tags and classes are put in place to allow for sufficient freedom in styling. Much of this markup will disappear when CSS3 is supported by all mainstream browsers (much of the extra markup would disappear if CSS2 was supported by IE6).
Views try to respect the rules of the language. For example, if a slot has no reader, by default this slot will be omitted from rendering.
Views provide an extremely flexible customization mechanism to easily rename, rearrange, hide/show, and custom render slots (see example code for details). Most widgets that deal with rendering data structures (dataform widget, grid widget, etc.) use views to render data. Weblocks tries to provide views that satisfy commonly used customization cases. The programmer should resort to custom HTML only in rare special cases when there is no way to configure the view to generate sufficient HTML.
Actions
Everything we've discussed above has been about rendering data to the client. So far we haven't discussed interactivity - the user's ability to change the state of the UI and to modify the data.
Weblocks allows the programmer to deal with client interaction without having to worry about the limitations of HTTP protocol. In Weblocks the programmer can 'render' a function, a generic function, or a closure into a link (or a button in a form). When the user clicks on the appropriate link (or button), Weblocks maps the click back to the callback. If the callback is a lexical closure, the programmer will have the full context in which the closure was created despite the fact that it was created during a completely different HTTP request:
(weblocks:render-link (lambda (&rest args)
(do-something))
"Modify")
In this case the action is rendered into a link named 'Modify' via 'render-link'. When the user clicks 'Modify', the lambda function is called (in the appropriate lexical context, if there is one), and 'do-something' is evaluated. A similar mechanism can be invoked for rendering form buttons.
Note, 'render-link' renders an AJAX-friendly link. The callback will be made via an AJAX request if JavaScript is available. Otherwise, Weblocks will fall back to a regular request. Because Weblocks applications aren't organized by pages, links change the state of widgets, rather than move between pages. This is why rendering a non-AJAX link doesn't make sense in the context of Weblocks.
Weblocks does have a mechanism to allow for friendly URLs. This mechanism is exposed to the programmer via the 'navigation' widget. The navigation widget takes very special care to hide the fact that Weblocks applications aren't organized into pages from the user. This way the user can enjoy the rich UI that follows established conventions, while the programmer can code the application using the widget paradigm. See reference documentation for the navigation widget for more details.
Stores
Many of the built in widgets operate on data peristed to a backend store. Weblocks supports a number of stores including SQL databases (via CLSQL) and XML files (via cl-prevalence). New stores can easily be added by implementing a simple API.
In this tutorial we will use a memory store for simplicity (in this case the data does not get persisted after the application quits). Before we can go on to use advanced widgets we need to define out store:
> (weblocks:defstore *scratch-store* :memory)
This line will create a memory store, bind it to *scratch-store*, and make it a default store for the application to use. A similar command can be used to define a SQL store in a production system. Defined stores will be opened automatically when the server is started and closed when the sever is stopped.
*Please note*, once you define a new store you should stop and start Weblocks to ensure the store is opened and ready to operate.
More on Widgets
So far we've only seen stateless widgets - string, function, and composite. However, we've hinted that widgets can have state. The simplest example of a stateful widget is a closure that increments a counter every time it's evaluated.
Some of the more complex Weblocks widgets have state associated with them. One such widget is 'dataform'. We'll talk about it here to put together everything we've discussed so far.
Dataform widget models a fairly common behavior of web applications. It renders a datastructure to HTML so the user can see some data. Below the data it renders a "Modify" link. If the user clicks on the link, the widget changes its state from 'data' to 'form' and renders a form for the user to modify the data along with a 'Submit' and 'Cancel' buttons. If the user clicks 'Submit', any changes are submitted and the state is changed back to 'data'. If the user clicks 'Cancel', the changes are discarded and the state is changed to 'data' as well.
Dataform widget is composed of everything we've discussed so far. It uses a data view to render the data, and a form view to render the form. It also uses actions to render the 'Modify' link and the buttons. Since dataform is a CLOS object, it stores its current state (whether it's displaying the structure as data or form) inside a slot. When the actions are invoked by the user this state is changed appropriately. We can change 'init-user-session' to take advantage of the dataform widget, and to finally add some interactivity to our application:
> (defun init-user-session (comp)
(setf (weblocks:composite-widgets comp)
(list (make-instance 'weblocks:dataform :data joe))))
If we reset the session and refresh the page, we'll see information about Joe displayed in the browser. We can click on 'Modify', change some of the data in the form, and click submit or cancel.
Since each dataform widget encapsulates and manages its own state, we can add as many dataform widgets to our page as we like. We can play with their states and they will all behave correctly. This is an enormous benefit about componetized approach - once we've created a widget, we can reuse it to our heart's content.
AJAX
Because actions are rendered to be AJAX-friendly, the dataform widget also ends up being updated in an AJAX request. How does this work? Why does the dataform widget redraw if 'Modify' is called asynchronously?
Weblocks uses the power of Lisp to fully automate this behavior. Every time a value of any slot of the widget changes, Weblocks detects the change using a hook in MOP and marks the widget as dirty (it is of course possible to turn this behavior off for some slots). Once the action is fully evaluated, Weblocks sends a list of dirty widgets back to the client, and each dirty widget is updated on the browser via JavaScript. Since the 'Modify' action changes the state of the dataform widget, the dataform is marked by Weblocks as dirty and the newly rendered version is sent to the client.
Note that this behavior scales back very well if the user has JavaScript turned off. Instead of sending a list of dirty widgets to the client, the entire page is redrawn. Most of the time the web programmer, and even the widget designer, need not care about AJAX at all - everything is managed by Weblocks (Some widgets take care to specifically turn off AJAX. For example, the navigation widget renders non-AJAX links to allow for friendly URLs).
Useful Hints
Troubleshooting
If anything goes wrong during this tutorial (or during development), the 'Internal Server Error' message does not give great insight into why an error has occurred. Fortunately, it's very simple to turn on debug mode, so that the exact error, along with the stack trace, are displayed in the browser. Instead of
> (weblocks:start-weblocks)
run
> (weblocks:start-weblocks :debug t)
Note, if Weblocks is running you need to stop it first with 'stop-weblocks'.
Creating a New Project
When you're ready to do more than just play around in the REPL, Weblocks provides a script to help you get started. The ASDF operation 'make-app-op' located in the 'wop' package allows you to quickly create an asdf-based project with necessary files, default stylesheets, etc. that you can start modifying.
First, load weblocks:
(asdf:operate 'asdf:load-op :weblocks)
You can then create a new project like this:
(asdf:operate 'wop:make-app-op :weblocks :name 'myapp :target "/home/coffeemug/projects/")
Or, if you don't like ASDF syntax, you can use the 'make-app' shortcut:
(wop:make-app 'myapp "/home/coffeemug/projects/")
This will create a directory '/home/coffeemug/projects/my-app/' and add necessary weblocks files to it. Note, some Lisp implementations don't understand shell-based shortcuts (like tilde for home directory), so you might have to type the directory in full. Alternatively you can change into the directory where you want to create the new project in your implementation (in slime, switch to REPL and type ,cd RET), and omit the target argument to the operation.
Now, all you have left to do is add your project source directory to ASDF central registry. You can do it by adding the following line to your implementation init file:
(push #p"/home/coffeemug/projects/my-app/" asdf:*central-registry*)
That's it! You can now load the newly created application and enjoy the ride:
(asdf:operate 'asdf:load-op :my-app) (my-app:start-my-app) ; an alias for 'start-weblocks'
