How To: Apache Tomcat / Apache WebServer

A few years ago I was asked to configure an intranet to use Apache Tomcat and Apache HTTP Server. For a true IT person this is probably a walk in the park, but for a Java web dev acting as an IT person -not so much. After a lot of experimenting, googling and reading spotty documentation, I got it all working. Recently I started a new job and was tasked with setting this up again. Since I had done this before I had a notional idea of what to do, but the details had long since been flush away. I figured that after a couple of years the documentation would be improved and someone would have provided a nice walk through. If such a document exists, I could not find it, so I decided to record how I went about solving this problem.

Audience

I suspect that I am not the only Java dev thrust into this situation. So this document is geared for someone who knows a bit about Tomcat, and Apache HTTP, but does not configure them often. If you are an IT person looking for tips, your probably best suited looking somewhere else, but constructive criticism is always appreciated.

Scenario

Let's say you have an intranet, and you want to run several Tomcat apps. You also want to use Apache HTTP server to route the requests to each Tomcat server. Why would you bother with Apache HTTP Server, when standalone Tomcat is faster than the two combined? A couple of reasons; by fronting Tomcat with Apache, you can easily offload webapps to additional Tomcat servers, by merely updating the workers file on Apache to point to the new instance. Also, we are going to be using Catalina Base to create separate instances of our web apps. Using Apache to route to each of these instances is a snap! Also Apache HTTP can load balance your Tomcat apps if the need arises. Note that we decided to not have our Apache server serve any static content, we just use it to intelligently pass the requests to the appropriate Tomcat instance.

For this example you have two internal apps with DNS entries mapped to "bugreports" and "purchasing". Your users simply have to browse to http://bugreports or http://purchasing. No ugly ports tagged on or convoluted urls like: http://bugreports.foo.com:8080. Configuring the DNS is outside the scope of this blog, but I have done it using Windows Server and it was pretty straight forward.

Environment

Apache 2.2
Tomcat 5.5

Apache HTTP Server

In your "%APACHE_HOME%/conf directory create a workers.properties file. This is the file that links your Apache server to the Tomcat.
example:

worker.list=bugreportWorker, purchasingWorker
# worker "bugreportWorker" will talk to Tomcat listening on machine "staging" at port "8091"
worker.bugreportWorker.host=staging
worker.bugreportWorker.port=8091

# worker "bugreportWorker" uses connections, which will stay no more than 10mn in the connection pool
worker.bugreportWorker.connection_pool_timeout=100
worker.bugreportWorker.connection_pool_size=100


#mount can be used as an alternative to the JkMount directive
#note I could not get this to work properly, so I am mapping this in http.conf
#worker.bugreportWorker.mount=/bugreport /bugreport/*

#same config for purchasing but different port
worker.purchasingWorker.host=staging
worker.purchasingWorker.port=8092
worker.purchasingWorker.connection_pool_timeout=100
worker.purchasingWorker.connection_pool_size=100

To be honest I am not completely sure about the ramifications of the timeout and cache settings. I did experiment with them, but we ran into an issue where running one app on Tomcat pegged the cpu at 50%, running two web apps straight lined it at 100%. The only way I could get the Tomcat server to behave was to disable the connection reuse in http.conf (JkOptions +DisableReuse). This is documented to have a performance hit under heavy load, but so far we have not had any issues.

Next we update the http.conf to snag the incoming requests, and route them to the appropriate worker.

# Allows for several virual host to be mapped to the same ip.
# Otherwise you will get "overlaps with VirtualHost" in error log
NameVirtualHost *

# Load mod_jk module
LoadModule jk_module modules/mod_jk-1.2.26-httpd-2.2.4.so
# Where to find workers.properties
JkWorkersFile C:\apache2.2\conf\workers.properties
# Where to put jk logs
JkLogFile C:\apache2.2\logs\mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel debug
# Select the timestamp log format
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "

# NOTE we had to add this option because after a request was forwarded to Tomcat, excessive chatter
# between Tomcat and the webserver ensued. The documentation states that this will be a performance drag
# so we should try to figure it out in the near future.
JkOptions +DisableReuse


#these allow us to map hosts to a worker. Virtual hosts can only map to local
#resources, so we map them to the worker using JkMount, which in turn maps to the
#remote Tomcat server.

<virtualhost>
ServerName bugreport-staging
CustomLog c:/apache2.2/logs/bugreport_vhost_access.log combined
JkMount /* bugreportWorker
</virtualhost>

<virtualhost>
ServerName purchasing-staging
CustomLog c:/apache2.2/logs/purchasing_vhost_access.log combined
JkMount /* purchasingWorker
</virtualhost>

Tomcat

For the Tomcat side of the equation we are going to create separate base instances for each of our apps. Why? Well, from what I understand the jvm does not handle large memory footprints well. We noticed this while trying to load four fairly good sized applications under the same Tomcat instance. We kept getting out of memory errors even after maxing the initial and max heap settings. Another reason to run base instances is that you sandbox the applications. So a misbehaving app will have a harder time affecting the performance of your other apps since they are running in separate jvms.

Catalina Base

Catalina base is a very powerful feature that allows you to run several apps under under different jvms. This is a particularity powerful feature when you are developing different applications that have different Tomcat configurations. For example, if you are a consultant and have a client that uses JOSSO authentication, and another client that used Acegi. You can have have a Catalina base instance for each application, one configured for JOSSO, another for Acegi. Also after attending a SpringSource webinar about Tomcat, I now deploy each app as ROOT (i.e. just add it as ROOT.war), which simplifies the host mappings.

To setup my Catalina base instances I will do the following:

• Create two directories for my base instances to live
  
• c:\apps\bugreports
• c:\apps\purchasing


• Copy and/or create the following directories from my main Tomcat install to each of my base apps
• conf (copy)
• webapps (create)
• the app we want to run e.g. c:\apps\bugreports\webapps\ROOT.war

Each port that Tomcat uses must be unique per instance. So edit the server.xml file in each of the new conf directories and change the following ports:

• http
• shutdown
• ssl (if necessary)
• AJP 1.3

<Server port="8001" shutdown="SHUTDOWN"> ...
<Service name="Catalina">
<Connector port="81" maxHttpHeaderSize="8192" maxThreads="150" minSpareThreads="15" maxSpareThreads="75" enableLookups="false" redirectPort="8443" acceptCount="100" connectionTimeout="20000" disableUploadTimeout="true" />
<!-- Define an AJP 1.3 Connector on port 8091 -->
<Connector port="8091" enableLookups="false" redirectPort="8443" protocol="AJP/1.3" />
<!-- Define the top level container in our container hierarchy -->
<Engine name="Catalina" defaultHost="ROOT">
<Host name="ROOT" appBase="webapps" unpackWARs="true" autoDeploy="true" xmlValidation="false" xmlNamespaceAware="false">
<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="bugreport_access_log." suffix=".txt" pattern="common" resolveHosts="false"/> <Alias>bugreport</Alias>
</Host>
</Engine>
</Service>
... <Server>

For the bugreports site I changed:

• the http port from the default of 8080, 81
• the shutdown port from default of 8005 to 8001
• the AJP 1.3 connector port from the default of 8090 to 8091.
  

For purchasing I will change it to 82, 8002 and 8092. Note that the AJP 1.3 port must match the port you provided in your workers.properties file on the Apache HTTP Server.

You can start the separate instances simply by setting the %CATALINA_BASE% environment variable to the appropriate base instance. For example, in Windows open a new command window and type CATALINA_BASE=c:\apps\bugreports

Then run startup.bat from your Tomcat directory. In a non-dev environment you will want to setup each of these base installs as a separate service.

Ready to go!

Startup your Apache HTTP server, and start your Catalina base instances on the Tomcat server. You should be able to simply type:
http://bugreports
in your browser. The Apache HTTP server will field the request, match the request to a virtual host, which in turn will call the worker for that request. The worker, forwards the request to the appropriate Catalina base instance on the Tomcat server.