Xerces ClassCastExceptions in multiple deployments

Today I'd like to talk about strange XML ClassCastExceptions one sometimes gets when deploying the same web application twice in the same Tomcat.

The problem occurs when you deploy for example a newer xerces library in your webapp's WEB-INF/lib directory. The problem has to do with the mechanism the XML factories are instantiated dynamically. I've always thought (and wondered) why this is so, because obviously the xerces classes have been loaded properly by the webapp classloader specific for the webapp.

After not having to deal with the problem in deegree 2 (the xerces was not specifically needed there, and the problem could be solved to remove it from WEB-INF/lib) I ran into the problem again with deegree 3, where xerces is a central and necessary library to parse XML Schemas.

I've found that there is a debug setting for the JVM -Djaxp.debug=true, which was essential for finding the problem. It showed that the ClassCastException did not actually occur when instantiating the DocumentBuilderFactory for the second time, but when XSLT was used. So what happened?

The built-in XSLT-solution in Java is xerces' counterpart, xalan. Like the built-in xerces this is an older version, apparently preconfigured to use the built-in xerces. I think what probably happens is that the used class is somewhere cached within the parent class loader (one of the global class loaders of Tomcat), but loaded with the webapp class loader of the first deployed webapp. Once the second webapp tries to use XSLT, it fails, because a different (and inaccessible) class loader was used to instantiate the parser in question. But that's pure speculation...

... which led to a hunch on my side. I thought that if each webapp had its own xalan as well, it might solve the problem, because the dynamic loading mechanism for XML factories prefers factories that can be loaded via SPI. So I deployed a recent xalan as well (plus the xalan serializer jar) and the problem was gone.

To conclude, it seems good practice to deploy recent versions of all needed (directly or indirectly) XML factories in each webapp. An indirect need for XSLT can be created as quickly as converting a OMElement to an XMLStream, so it's a good guess you need it.

PS: Another solution would be to deploy all the XML factory libs centrally in the Tomcat, but that solution has obvious disadvantages (having to fiddle with the Tomcat, not being able to use different versions across webapps).

Maven, Java and Yet Another Memory Problem

Everyone working with Java has probably experienced an OutOfMemoryError at one time or another. Since the default values of the JVM are often ridiculously low (used to be 64MB for a standard JVM), increasing the value through startup options usually solved the problems.

Experienced users know that there are different kinds of OutOfMemory errors, the most common being 'Java heap space'. People playing with Tomcat and redeploying webapps a couple of times run into the 'PermGen space' variant pretty fast.

But recently we encountered yet another memory problem, which even resulted in a vm crash, with problems in libjvm.so. This was reproducible on our build server, where we also release new versions using the maven-release-plugin.

It seems that during the maven run within maven (when running mvn release:prepare) the reserved code cache memory area runs full. The documentation about -XX:ReservedCodeCacheSize is a bit unclear on its workings:

Reserved code cache size (in bytes) - maximum code cache size. [Solaris 64-bit, amd64, and -server x86: 48m; in 1.5.0_06 and earlier, Solaris 64-bit and and64: 1024m.]

But it just so happens that increasing it to half a gigabyte fixed the problem for good.

I'm all for being able to configure the JVM in all ways imaginable, but being forced to control it just can't be the right way. To run the release plugin we now have to configure three different memory settings. Can't we have a maximum memory setting that limits heap, stack, permgen and code cache size?

Raster Pyramids in deegree

Today I would like to talk about raster pyramids in deegree. Gone are the times where you have to use our custom tools to create pyramids and manually configure the different solutions.

In order to make the handling of raster pyramids easy we now support using standard GeoTIFF pyramids with overlays. Let me show you how to prepare your data and use it as coverage data source in deegree.


To prepare your data, you can use standard tools like GDAL. There are a couple of requirements that the processed data needs to meet:

• it must be a GeoTIFF, containing the extent and coordinate system of the data
• overlays must be multiples of 2

Assume you have a couple of raster files lying around in a directory, eg. png with .wld. First, build a GDAL virtual raster:

gdalbuildvrt virtual.vrt *png

Then, build the base GeoTIFF using gdalwarp:

gdalwarp -t_srs EPSG:26912 -co BIGTIFF=YES -co TILED=YES virtual.vrt merged.tif

The BIGTIFF option enables you to create files bigger than 4GB. There are a whole lot of other options (you can also compress the TIFF if you want), see the GDAL documentation for details.

Finally, calculate the overlays using gdaladdo. Take care to only use multiples of 2 for the overlays, like in this example:

gdaladdo -r average merged.tif 2 4 8 16 32

That's it, now you have a GeoTIFF ready to run. To configure it in deegree, simply add a new coverage data source using the web console and configure the location of your file.

All done! Now you can use it like any other coverage data source to configure your layers (or a WCS). The files can be very big, I've already tested it with ~150GB files (my lack of disk space prevented testing bigger files).

The implementation was done using the standard TIFF driver from imageio-ext, which has support for BigTIFF without the need for JNI libraries (imageio-ext also has support for GDAL using JAI, but I didn't use that one). Many thanks to the developers for making this easy!

Edit: Removed references to images. I was wondering where all my screenshots went. Seems it had something to do with joining Google+. But I'm not taking the effort to recreate the screenshots here (you should read the deegree handbook anyway, and use the new tile stores instead of this one).