JBoss Clustering

There are a number of areas to consider when clustering JBoss. Briefly:

Basic Setup

The easiest way to get started is to take a standard JBoss installation and deploy into its jboss/server/all folder. You can run this instance by doing run -c all.

The all configuration is JBoss with everything installed, including clustering support. JBoss instances started in the all configuration will automatically find each other and cluster together.

Note: every JBoss instance must be set up identically!

Simulating A Cluster

If you want to simulate a JBoss cluster on a single machine, take a copy of the jboss/server/all folder (call it, say, jboss/server/cluster2). Then update jboss/server/cluster2/conf/jboss-service.xml and comment in the ServiceBindingManager block. This has an attribute ServerName which you can change to ports-default, ports-01, ports-02 etc. to easily re-assign all of JBoss' ports to avoid conflicts. The mappings are defined in jboss/docs/examples/binding-manager/sample-bindings.xml. You can then run multiple instances by doing run -c cluster2.

JBoss 5 update: you no longer need to edit jboss-service.xml. You can just run a new instance using run -c cluster2 -Djboss.service.binding.set=ports-01

Simulating Several Clusters

The file jboss/server/all/deploy/cluster-service.xml defines an attribute PartitionName which dictates which instances of JBoss cluster together. You can simulate several clusters on a single machine by assigning them different partition names.

It also defines the multicast port, and this should be different for different clusters (else you will see a lot of warning messages in the logs).

Web Clustering

To cluster Web sessions, in theory, you just add an empty distributable element to your web.xml.

Clustering Web sessions creates considerable replication traffic around the cluster, degrades performance, and is much less scalable. If you are using sticky sessions and/or are happy for your users to have to re-login upon a server failure, not clustering Web sessions may be a better option.

JMS Clustering

To cluster JMS (also referred to as High Availability JMS or HA-JMS) it is important there is only one instance of your JMS queue/topic in the entire cluster, so you must:

• use an external database for storing your queue rather than the Hypersonic one JBoss normally uses (in its data folder). You will need to copy the relevant xxx-jdbc2-service.xml file from jboss/docs/examples/jms into jboss/server/all/deploy-hasingleton/jms
• delete jboss/server/all/deploy/hsqldb-ds.xml and add your own datasource, making sure you name it DefaultDS. The contents of jboss/server/all/deploy/hsqldb-ds.xml are a little scary. All you really need are (for MySQL):

<?xml version="1.0" encoding="UTF-8"?>

<datasources>
	<local-tx-datasource>
	
		<jndi-name>DefaultDS</jndi-name>
		<connection-url>jdbc:mysql://localhost/jboss-cluster</connection-url>
		<driver-class>com.mysql.jdbc.Driver</driver-class>
		<check-valid-connection-sql>select 1 from dual</check-valid-connection-sql>
		
		<security-domain>MySqlDbRealm</security-domain>
		
		<metadata>
			<type-mapping>mySQL</type-mapping>
		</metadata>
		
	</local-tx-datasource>
</datasources>

• add your queue/topic definitions to jboss/server/deploy-hasingleton/jms/jbossmq-destinations-service.xml (not anywhere else)
• update jboss/server/all/conf/login-config.xml and change the HsqlDbRealm to suit the one defined in your new datasource
• instead of doing a JNDI lookup to ConnectionFactory, do it to java:/JmsXA
• your queue JNDI names are now all jnp://localhost:1100/queue/bob. It may be easier (albeit JBoss-specific) to change your code to use:

	// Old: Queue queue = (Queue) new InitialContext().lookup( "jnp://localhost:1100/queue/bob" );
	Queue queue = new SpyQueue( "bob" );

JBoss 5 update: the xxx-jdbc2-service.xml files are now named xxx-persistence-service.xml. They additionally contain an attribute 'Clustered' which must be set to true to cluster the JBoss Messaging PostOffice. When you do this, you'll also need to set -Djboss.messaging.ServerPeerID=1. You'll then need to add an attribute <attribute name="Clustered">true</attribute> to each queue in destinations-service.xml. Finally instead of java:/JmsXA you need to use java:/ClusteredConnectionFactory

Troubleshooting

• If you get ...queue name... not bound during startup, make sure your jboss.xml is pointing to the HA queue instance (ie. jnp://localhost:1100) not the local queue instance. In a singleton environment, there will not be a local queue instance on anything but the primary cluster node
• If you're getting clustered delivery, but not clustered consumption of messages (ie. you have 3 nodes, each gets a 1/3 of the messages, and you have to wait for the slowest to finish) make sure you aren't using the 'Clustered=true' attribute in your destinations-service. The concept of 'clustered JMS queues' is very different to 'load balanced MDBs': one is a delivery methodology and one is a consumption methodology

JMX Clustering

If you need to use JMX against your JMS queue (eg. to see how many messages are in a queue), you will need to JNDI lookup to jnp://localhost:1100/jmx/invoker/SingletonRMIAdaptor rather than jmx/invoker/RMIAdaptor. To set this up, create a file called jboss/server/all/deploy-hasingleton/singleton-jms-adapter-service.xml and put in it:

<server>
  <mbean code="org.jboss.invocation.jrmp.server.JRMPProxyFactory" name="jboss.jmx:type=singletonadaptor,name=Invoker,protocol=jrmp,service=proxyFactory">
    <!-- Use the standard JRMPInvoker from conf/jboss-service.xxml -->
    <depends optional-attribute-name="InvokerName">jboss:service=invoker,type=jrmp</depends>
    <!-- The target MBean is the InvokerAdaptorService configured below -->
    <depends optional-attribute-name="TargetName">jboss.jmx:type=adaptor,name=Invoker</depends>
    <!-- Where to bind the RMIAdaptor proxy -->
    <attribute name="JndiName">jmx/invoker/SingletonRMIAdaptor</attribute>
    <!-- The RMI compabitle MBeanServer interface -->
    <attribute name="ExportedInterfaces">org.jboss.jmx.adaptor.rmi.RMIAdaptor,org.jboss.jmx.adaptor.rmi.RMIAdaptorExt</attribute>
    <attribute name="ClientInterceptors">
      <interceptors>
        <interceptor>org.jboss.proxy.ClientMethodInterceptor</interceptor>
        <interceptor>org.jboss.proxy.SecurityInterceptor</interceptor>
        <interceptor>org.jboss.jmx.connector.invoker.client.InvokerAdaptorClientInterceptor</interceptor>
        <interceptor>org.jboss.invocation.InvokerInterceptor</interceptor>
      </interceptors>
    </attribute>
  </mbean>
</server>

Follow @metawerx