Metawerx Java Hosting Small Logo

Decreasing Tomcat startup time with multi-threading

Author: Neale Rudd, Metawerx

Date: 4-Jan-2007

These days, multiple physical CPUs and multiple cores are becoming much more readily available. By changing your configuration, Tomcat can take advantage of this power to decrease startup time.

By default, Tomcat allocates a single thread to perform deployment and management of applications. When multiple applications are handled by a single Tomcat instance, this can cause the startup time to increase considerably, as each application is started in sequence by the management thread. Additionally, if a single application performs processing during it's startup, Tomcat will wait for it to finish before attempting to deploy the next application.

In a single-CPU environment, this is an effective way of starting each application. However, with multiple physical CPUs and multiple cores, it is obvious from watching the CPU utilisation that the startup performance could be increased dramatically by using more threads.

The Test Server

This server used for testing is a Dell PowerEdge 6850, with the following configuration:

  • 24gb available RAM
  • 3 x 147gb 15,000 RPM SCSI drives arranged in a RAID-5 set
  • 4 Dual-Core 3GHz CPUs (8 cores total)
  • Windows 2003 Server R2 Enterprise 64 bit Edition

The test environment is as follows:

  • Tomcat 5.5.20
  • JDK 1.5
  • 34 mid-sized Apache Struts based applications, many using Hibernate, each running as separate tomcat virtual hosts

To benchmark effectively, no other applications were running during the Tomcat startup process.

Before Shots

The following screenshot was taken before making any changes.

http://www.metawerx.net/images/screenshots/tomcatmultithread_standard.png

As can be seen, there is some light activity on 3 of the cores, but only one of the cores is under heavy load.

The total startup time in this case was 92 seconds.

After Shots

The following screenshot was taken after introducing a multithreaded startup.

http://www.metawerx.net/images/screenshots/tomcatmultithread_threaded.png

As can be seen, all 8 cores are now maxed out during the startup process.

The startup time was reduced to 51 seconds.

Obviously there was a lot of thrashing, and better performance times could be achieved with some better tuning according to the number of available cores and the startup time of each application. In this case, all 34 applications started at exactly the same time, instead of in sequence.

Understanding the Change

The default behaviour of Tomcat, is to start each application when Tomcat is starting up.

We will modify this behaviour, so that no applications are automatically started.

Instead, we will rely on the Background Processing Thread to start them - and we're going to start a lot of those.

To understand the changes, it is first necessary to understand the Background Processing Thread and the way it is allocated.

Background Processing Thread

The Background Processing Thread runs periodically looking for new applications to deploy.

It is this thread that is responsible for automatic deployments with the autoDeploy feature of Tomcat.

The way these threads are allocated, is as follows:

  • For each <Engine> defined in server.xml, a Background Processing Thread is allocated.
  • The attribute backgroundProcessorDelay specifies the number of seconds between starting the engine, and starting the thread.
  • By default, all child elements of <Engine> have a backgroundProcessorDelay of -1, which means they don't start their own thread, and will instead rely on this master thread.
  • The thread calls the backgroundProcess method on the Engine and its child elements.
  • Elements which have their own backgroundProcessorDelay attribute set to a value other than -1 are skipped.
  • Those elements have separate threads allocated, which will start after the number of seconds defined in their backgroundProcessorDelay attribute

Making the Changes

Instead of starting a single thread per Engine, we are going to allocate a single thread per <Host>, because in the test environment, our applications are each using a separate Host element. An alternative is to allocate a single thread per <Context>, which can be done using the server-wide context.xml file, or the DefaultContext sub-element of Engine. This is achieved by adding a backgroundProcessorDelay attribute to each <Host> element. In this test, we set the value to 5 seconds, so that the thread starts up 5 seconds after the <Host>. The hosts in this case all start within 3-4 seconds, so 5 is an appropriate value.

backgroundProcessorDelay="5"

The deployOnStartup attribute of <Host> defaults to true. This means that as Tomcat is starting, each Host will deploy all of its applications. We will disable this in the <Host> element using the following attribute. Instead, the Background Processing Thread will be starting our applications, 5 seconds after the Host starts.

deployOnStartup="false"

Finally, we will manually set autoDeploy to true, in case it has been set to false elsewhere. This flag defaults to true anyway, so this step can probably be left out. We are including it for safety, to make sure our applications are actually deployed.

autoDeploy="true"

After making the change, our Host tag now looks like this. The last 3 attributes are the important ones.

<Host name="testA.metawerx.net" appBase="g:\testA" backgroundProcessorDelay="5" deployOnStartup="false" autoDeploy="true">

Further Tuning

The setup can be tuned using the backgroundProcessorDelay attribute, and by modifying the list of applications or Hosts that should deploy on startup, and the list which should wait for the Background Processor Thread. The exact tuning will depend on your configuration and requirements.

As we are starting 34 hosts, and have 8 cores, there is obviously too much contention during the startup with the above test case, where each application is set to start at exactly the same time. This results in a faster startup, but extreme CPU usage during the startup (8 cores, but we're starting 34 apps).

To create a better balance for example, we could keep 2 cores free by adjusting the timing on each host to allow a maximum of 6 hosts to start at the same time. This would require checking the start time of each application, and adjusting the timing accordingly to group sets of hosts according to the order we want them to start.

Good luck!

--Neale Rudd

More Information

navigation
metawerx specific
search
Share
tools
help

referring pages
...nobody

Share