Application Load Balancing

The objective with load balancing is that two or more servers should share the load of incoming requests, to provide increased response time to the user and avoid overloading any single server.

With web based applications, the simplest way to reduce load from a single server is to distribute components to other servers. A simple example is placing the web server or application server on a separate server to the database server. Other examples are message queues such as JMS, or the use of RMI, EJB or SOAP. With these solutions, an entire server can be dedicated to servicing incoming HTTP requests.

In some cases however, this solution is not practical. For example:

• a single server may not be capable of handling all incoming HTTP requests
• availability is reduced, because now there are many points of failure in the overall system, any of which can go offline
• the software may be written in such a way that components cannot be moved to other servers, or doing so would not distribute load in a useful way

Therefore, companies often look at load balancing the application server itself. This involves sending incoming HTTP requests to 2 or more servers.

Two of the major problems to consider when load balancing a web based application are as follows:

• HTTP sessions are often used to maintain user state. Session state can be lost if the user is directed to a different server than the one where their session information is kept. Maintaining state over multiple servers is known as session-failover.
• If a server goes offline, and HTTP requests are still being sent to that server, those requests will fail. Protecting against this problem is known as request-failover.

Request-failover is usually handled by a heart-beat system between the servers and the load balancer, so that the balancer is aware of which servers are capable of handling requests.

Session-failover is more complicated. A number of solutions are listed below.

Option 1 - Don't use sessions

Google uses this technique in their search engine, placing all required information for each request in the URL. There is no need to store any information about the user in this case. The URL provides all details required to fulfill each request.

Session-failover is not required - there are no sessions.

Option 2 - Sticky-sessions

An intelligent router can select a target server for a user based on their IP address or session ID. The user is sent to that same server each time, removing the problem of redirection to a different server.

Session-failover is not supported. If a server goes offline, all session for all users on that server are lost, and the users all receive errors. If the server is removed from the list of available servers by the router, the user will be sent to a new server, and asked to log in again or be sent to an error page.

Option 3 - Central storage of session data

With this technique, session data is stored in a separate server, database, network attached storage (NAS). As each request comes in, the server retrieves the session data from the central storage. Adobe Cold Fusion is capable of using this method when storing session data in a database.

Session-failover is automatic. If a server goes offline, another server simply takes over.

Option 4 - Session-replication (such as with Tomcat clustering)

With this technique, sessions are replicated to other members of the cluster. The user can be sent to any server, as all servers have a copy of the user's session. Tomcat uses this method to enable web applications to be distributable.

Session-failover is automatic. If a server goes offline, another server simply takes over.