Dr. Axel Rauschmayer is a freelance software engineer, blogger and educator, located in Munich, Germany. Axel is a DZone MVB and is not an employee of DZone and has posted 246 posts at DZone. You can read more from them at their website. View Full User Profile

File System Storage and Servlets

03.15.2010
| 8459 views |
  • submit to reddit

One of the blessings of the JVM is that there are many “pure” databases available for it; Sesame (RDF) and Apache Derby (relational) come to mind. Alas, these databases need to save their files somewhere. This becomes a challenge with Servlets: Where should one put these files? Pre-packaged, read-only, files usually reside in SERVER/webapps/app/WEB-INF/, but mutable files? Putting them in WEB-INF makes upgrading a web application more difficult. I can easily imagine myself accidentally removing such data during an upgrade. The best solution, that I was able to come up with, is:

  • By default, the data directory of a web application “app” is SERVER/webapps/app.data/WEB-INF/ (WEB-INF is necessary to protect against serving secret files).
  • This can optionally be configured via a file SERVER/webapps/app.properties. There, one can specify the servlet data directory via a relative or an absolute path. The former case is useful if you want this directory to be a sibling of SERVER/webapps. The latter case is useful for putting this directory anywhere on a system.

I do realize that there are security issues with this scheme: While it worked for me on Tomcat, I would expect other servlet containers to prevent servlets from writing files in this manner. Other options are:

  • Make the data directory a servlet parameter in web.xml: Complicates upgrades.
  • Specify a directory via the Java preference API: Prevents installing the same web application multiple times on the same system.
  • Storing the data in a temporary (as in File.createTempFile()) directory: Not a solution for long-term persistence.

Are there other solutions out there? Let us know in the comments.

From http://2ality.blogspot.com

Published at DZone with permission of Axel Rauschmayer, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Tags:

Comments

Maik Jablonski replied on Mon, 2010/03/15 - 1:36am

I'm using embedded object-databases (like db4o) in my web-applications (e.g. JeaseCMS) and the best solution I've found so far: store the data-file(s) in directories relative to the home directory of the user who runs the server-process. This works nicely for development and production (even for mixed scenarios with Windows and Linux), because you don't have to worry about changing configs when deploying an application.

Daniel Kvasnička replied on Mon, 2010/03/15 - 1:57am

One option might be to access the database through a JNDI-provided resource. This way you can upgrade your app whenever you want because you don't work with filesystem paths or anything, just JNDI name. Have your database anywhere on the disk, your app doesn't need to know this at all. This is an obvious solution for full-blown app servers (mainly in case of JDBC accessible databses like Derby you mentioned) but should work also with Tomcat as it can expose beans over JNDI (server.xml / web.xml).

Nicolas Bouillon replied on Mon, 2010/03/15 - 3:58am

Yes, JNDI is in my opinion the best solution. And even for lightweight servers, like Jetty, JNDI is available.

I'm always very suprised when trying to install J2EE applications, such as XWiki, Pentaho, Alfesco and so on. They always provide a full zip package with Tomcat embedded. And if you want to install just the WAR on your own J2EE server (for example Glassfish), you start a very complicated setup, with extraclasspath containing configuration, or even worse unpacking the WAR and modifing WEB-INF config files...

I know, JNDI is not so easy to setup, but IMO it is the best way...

oded peer replied on Mon, 2010/03/15 - 3:59am

How is this problem different than setting a directory for log files? What was your solution for that? (keeping log files under the WEB-INF directory is bad practice in my opinion).

What about not-exploded applications (not opening the war file)? You solution forces your user to use exploded wars.

For out of the box integration I'm with Maik. Use the "user.home" JVM property, which is always present, but allow your user to change it.

Upgrade is a complex process, don't be afraid of it, be prepared. Have a nice GUI, it always helps.

Axel Rauschmayer replied on Mon, 2010/03/15 - 4:38am in response to: oded peer

"What about not-exploded applications (not opening the war file)? You solution forces your user to use exploded wars."

The WAR of "app" can be unexploded, i.e., I do not require it to be exploded. Conversely, app.dir/ is a pseudo web application called "app.dir" which is exploded. But that should only be a problem if the servlet container only supports non-exploded webapps. What I'm really looking for is a servlet container giving my web application a directory where it can store its data.

I do like the homedirectory-relative approach (where mine could be considered webapp-directory-relative) and using JNDI.

Marc Ende replied on Mon, 2010/03/15 - 9:20am

If you really need the configuration of derby in the servlet you sholud use the home-dir way. But I would configure the database in the servlet-container.

This has some good points:

a) it's easy to update (you can deploy a exploded or non-exploded war) it should only contain the relevant entry for the jndi-path

b) your application can make use of the connection-pooling capabilities of the servlet-container (performance!!)

c) you doesn't have to worry about file-locations (they are only once configured and only reconfigured when setting up a new version of the servlet-container)

d) you can place your database everywhere (sufficient permissions assumed) and not only relative to the server or to the home-dir.

marc

 

Sudhakar Ramasamy replied on Mon, 2010/03/15 - 12:50pm

How about passing in a system property to the JVM such as -Dmyapp.datadir=/var/myapp/data?

The application then uses this environment property to store. I've seen this technique commonly used by products such as those from Atlassian. It makes the installation/upgrade process pretty painless.

LOL DUDE replied on Wed, 2010/03/17 - 5:40am

File tmpDir = new File(System.getProperty("java.io.tmpdir"), ".my-app-folder")

Brent Crammond replied on Wed, 2010/03/17 - 6:12pm

The home directory or some subdir of it has been my solution of choice for quite a while. This the location the hudson folks took and seems to work ok.  I do have an alternative to this.

This is one of the major failings of the JEE specs, I know you shouldn't use the file system as it is not transactionalised, but in reality you always need to.

The other is JNDI is not a good general application configuration mechanism, I have moved to storing my config in a table in the DB, I can then maintain it through a db utility and I then don't care what server ot OS I am running on.

I do bootstrap the DB config through JNDI. And this is the basis for my alternative, which is to bootstrap the DB and then get all locations from the Config table in my database.

Why don't I think JNDI is a great way to go, well it is a very limited standard, with until recently no standard JNDI naming scheme and no standard defined way of externally maintaining it.

Ruby on Rails and Grail, set up a Development, Testing and Production environment, but this is still quite limited and assumes you knowyou locations when you package it.

Sorry this was a bit of a rant.  

 

David Whitmore replied on Thu, 2010/03/18 - 8:56am

Could you use the same approach used by logging frameworks such as log4j?  Have a properties file on your classpath which specifies the directory to use for storing your files. 

 Your application could read this on startup using something like getResourceAsStream.

This is the sort of thing that I find myself doing quite a lot - as I'm a Java web-app developer.

 It has the advantage that your app is properly 'packaged' (the configuration file lives under WEB-INF/classes).  It also works whether or not you use 'exploded'  deployment of web-apps.  And the classpath would usually not be accessible to users of your app.

 Not sure of the best location to actually store the files.  As you say, under WEB-INF is a possibility, and keeps them private, but might hamper upgrades.  It might be better to store them somewhere other than the web-server directories.

The other thing I'd add, is that I use Maven to build my web-apps.  This has the advantage of enabling me to build machine-specific web-apps.

(For the technically inclined, you need to use the 'profile' element of the pom file, and turn resource-filtering on.  You just use a different 'profile' for each machine that you want to deploy on)

Just my 2 cents worth!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.