As a requirement for Continuous Delivery, changes to applications should be deployed as frequently as possible. GitHub is a well-known example: on their “busiest day,” GitHub “saw 563 builds and 175 deploys.” To do this with a Java web application, zero-downtime deployment needs to be done to avoid service disruption.
To achieve zero-downtime deployment, there are several solutions. Both commercial tools, such as LiveRebel, and container features, such as Tomcat 7’s parallel deployment, can make this happen. This post focuses on the approach that uses Apache’s auto failover feature to achieve zero-downtime deployment by performing round-robin updates on multiple Tomcat instances.
In order for auto failover to work seamlessly without users noticing the redeployments of applications, sessions must be replicated between application servers. Without session replication, users lose sessions, i.e. getting logged out, as soon as the request is served by a different application server.
Tomcat has built-in facilities, including
that enable session replication. In practice, I found both of these complicated
to configure, and I have not been able to maintain sessions with
upon auto failover.
As an alternative, I found that memcached-session-manager not only works great but is also very easy to configure. As a bonus, it offers good performance and scalability. This post therefore documents the simple steps to set up Tomcat auto failover using Apache, memcached and memcached-session-manager on a single machine. In theory, the steps documented here would apply equally well to a multi-machine clustered environment.
1. Set up Tomcat Instances
For auto failover to work, at least 2 Tomcat instances need to be set up. To use an existing Tomcat installation, just make a copy of it. Otherwise, download it from here and also make 2 copies.
a. Install Required Jars
For each copy, download the following jars and install them to the
- memcached-session-manager-x.y.z.jar, available here
- memcached-session-manager-tc7-x.y.z.jar (for Tomcat 7) or
memcached-session-manager-tc6-x.y.z.jar (for Tomcat 6), also available here
- spymemcached-x.y.z.jar, available here
At the time of this writing, the latest version of memcached-session-manager is 1.6.3 and spymemcached is 2.8.4.
Update Dec. 13, 2013: As the comments below indicate, later versions of memcached-session-manager have a hard dependency on Couchbase Client, which can be downloaded here.
b. Tomcat Configurations
For each copy, open
tomcat_dir/conf/context.xml, and add the
following lines inside the
<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager" memcachedNodes="n1:localhost:11211" requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$" />
If you already have a memcached instance setup that is on a different server
and/or listens on a different port, change the value in
Otherwise, the setting here are the defaults of the memcached instance to be
installed in the step below.
tomcat_dir/conf/server.xml, look for the following lines:
<Server port="8005" ...> ... <Connector port="8080" protocol="HTTP/1.1" ...> ... <Connector port="8009" protocol="AJP/1.3" ...>
Change the ports, so the two installations listen to different ports. This is
optional, but I would also disable the HTTP/1.1 connector by commenting out its
<Connector> tag, as the setup documented here only requires the AJP connector
to be enabled.
Finally, look for this line, also in
<Engine name="Catalina" defaultHost="localhost" ...>
jvmRoute property, and assign it a value, that is different between
the two installations. For example:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1" ...>
And, for the second instance:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm2" ...>
That’s it for Tomcat configuration. This configuration uses memcached-session-manager’s default serialization strategy and enables sticky session support. For more configuration options, refer to the links in the references section.
2. Configure Apache
For Apache, open
/etc/httpd/conf/httpd.conf (for Fedora, CentOS and Red Hat
Enterprise Linux, your distribution may have it elsewhere), add the following
<Proxy balancer://cluster> BalancerMember ajp://localhost:8009 route=jvm1 BalancerMember ajp://localhost:8109 route=jvm2 </Proxy> ProxyPass / balancer://cluster/ stickysession=JSESSIONID|jsessionid
These lines may also be placed inside a
VirtualHost directive. Change the path
if your application does not run on
/. For example:
ProxyPass /app/ balancer://cluster/app/ stickysession=JSESSIONID|jsessionid
Note that the
BalancerMember lines point to the ports and
configured in step 1. This step sets up a load balancer that dispatches web
requests to the two Tomcat installation. When one of the Tomcat instance gets
shutdown, requests will be served by the other one that is still up. As a
result, user does not experience downtime when one of the Tomcat instances is
taken down for maintenance or application redeployment.
This step also sets up sticky session. What this means is that, if user begins session with instance 1, she would be served by instance 1 throughout the entire session, unless of course this instance goes down. This can be beneficial in a clustered environment, as application servers can use session data stored locally without contacting a remote memcached.
3. Install memcached
On Fedora/CentOS/Red Hat Enterprise Linux, memcached can be installed using
yum install memcached. If memcached is not available, add
Remi as a yum repository and try again. For
other distributions, refer to
memcached wiki. The default
settings should work fine out of box; no additional configuration for memcached
would be required.
4. Test Auto Failover
Deploy any web applications that use sessions onto both Tomcat installations.
Start Tomcat instances, Apache and memcached. Open the web address proxied by
Apache in a browser. Check the cookies set. With Chrome, this can be done with
the built-in Developer Tools, as the screenshot above shows. In my case, the
JSESSIONID value reveals that my current Tomcat instance for the session is
jvm1. Find out which instance is used to serve you, and shut it down. Refresh
web browser, and the cookie value should indicate that the other Tomcat instance
is taking over. Check if session values set by the application remain. For
example, if the application deployed allows user to log in, you should be able
to remain logged in after auto failover.
As this post shows, memcached-session-manager is simple to configure. Together with memcached and Apache (or any other application load balancer), it is possible to perform round-robin deployments of web applications on Tomcat with zero downtime. This capability allows us to achieve continuous delivery and frequently push out new features.
Update Jan. 1, 2013: To allow Tomcat to shut down gracefully, i.e.
let existing requests finish first, the
unloadDelay attribute may be set to
give Tomcat the time to complete fulfilling existing requests. This attribute
may be set in the
tomcat_dir/conf/context.xml file. For example, the
declaration below will give Tomcat 30 seconds to finish off with existing
requests and then shut down:
Update Dec. 1, 2013: A note about the dependency on Couchbase Client has been added.