Geertjan is a DZone Zone Leader and has posted 466 posts at DZone. You can read more from them at their website. View Full User Profile

In-Memory Virtual Filesystems

11.12.2008
| 17345 views |
  • submit to reddit
Alex Miller recently wrote an interesting piece about the need for Java to have an in-memory virtual filesystem (here), in the context of JSR-203. Alex notes that athough "[a] virtual filesystem will not be included as part of JSR 203", "the API is designed to be extensible and all of the hooks exist to provide such an implementation".

I really like his use case for having such a filesystem:

Having a virtual memory system to plug into would be fantastic for unit testing. You could create files to your heart’s content and the file access would be fast while also saving you from all the annoying issues with deleting temporary files, Windows file locking, etc.

Makes sense to me. In the comments to his blog entry, a few existing solutions are mentioned, in particular, Commons VFS and the JBoss Microcontainer project. Also, another use case for a filesystem of this kind can be found there:

It may also be useful when running on operating systems that don’t have /tmp or equivalent on swap.

Another solution, much less known in this context, is provided by two standalone JARs that are used within the NetBeans Platform (full disclosure: I work there). These two JARs, org-openide-filesystems.jar and org-openide-util.jar, can simply be dropped in your classpath and then... you have access to an in-memory virtual filesystem. Those two JARs are all that it takes. In other words, there are no dependencies of any kind on any part of the NetBeans Platform. These two JARs can simply be copied/pasted from the NetBeans Platform (i.e., which is a folder within NetBeans IDE) to your own classpath.

The filesystem that you then have is hierarchical. Basically, imagine an XML file in memory and you've got the idea. You can write folders and files into this filesystem and then you can ascribe attributes to those files. These can then be used as part of your unit tests, exactly as described above by Alex. Here's a small example:

package demo;

import java.io.IOException;
import junit.framework.TestCase;
import org.junit.Before;
import org.junit.Test;
import org.openide.filesystems.FileObject;
import org.openide.filesystems.FileSystem;
import org.openide.filesystems.FileUtil;
import org.openide.util.Exceptions;

public class FsJFrameTest extends TestCase {

FileSystem fs = FileUtil.createMemoryFileSystem();
FileObject root = fs.getRoot();

@Before
@Override
public void setUp() {
try {

//Create a virtual folder:
FileObject testDataFolder = root.createFolder("TestData");

//Create a virtual file:
FileObject testData1 = testDataFolder.createData("testData1");
//Create three virtual attributes for the file:
testData1.setAttribute("name", "John");
testData1.setAttribute("age", 27);
testData1.setAttribute("employed", true);

//Create a second virtual file:
FileObject testData2 = testDataFolder.createData("testData2");
//Create three virtual attributes for the file:
testData2.setAttribute("name", "Jane");
testData2.setAttribute("age", 34);
testData2.setAttribute("employed", false);

} catch (IOException ex) {
Exceptions.printStackTrace(ex);
}
}

//This test will pass because all attributes match the test data.
@Test
public void testData1() {
FileObject testData1 = root.getFileObject("TestData/testData1");
assertEquals(testData1.getAttribute("name"), "John");
assertEquals(testData1.getAttribute("age"), 27);
assertEquals(testData1.getAttribute("employed"), true);
}

//This test will fail because age = 34 in the test data.
@Test
public void testData2() {
FileObject testData2 = root.getFileObject("TestData/testData2");
assertEquals(testData2.getAttribute("name"), "Jane");
assertEquals(testData2.getAttribute("age"), 33);
assertEquals(testData2.getAttribute("employed"), false);
}

}

 

Here you can see that even though you haven't written any actual physical files anywhere on disk and even though you haven't retrieved anything from actual physical files on disk, you do have the important thing: the test data that you need for your unit tests. Plus, that data can be organized as easily as if one were working with a physical hierarchical filesystem. The two JARs provide a lot more besides, such as access to ZIP/JAR archives, via filesystems provided by the selfsame JARs. Just get them from your NetBeans IDE distro and then remove the distro and use another IDE, if that's your preference. (By the way, a screencast of the NetBeans Platform's filesystem [together with a transcript] can be found here.) Looking forward to seeing an in-memory virtual filesystem of this kind available in the JDK, though.

 

Published at DZone with permission of its author, Geertjan Wielenga.
Tags:

Comments

Fuqiang Zhao replied on Wed, 2008/11/12 - 8:00pm

interesting and helpful:)

Radek Jun replied on Thu, 2008/11/13 - 2:53am

Interesting, it would be nice to have it as part of JDK :).

Roman Pichlik replied on Thu, 2008/11/13 - 8:59am

+1 for in-memory file system. Maybe i missed something, but i don't see any addition value in using of Open IDE API. Usually you test some API depending on JDK abstraction as java.io.File, so classes from org.openide.filesystems do not help at all ;-).

Geertjan Wielenga replied on Thu, 2008/11/13 - 10:44am in response to: Roman Pichlik

[quote=rp117107]+1 for in-memory file system. Maybe i missed something, but i don't see any addition value in using of Open IDE API. Usually you test some API depending on JDK abstraction as java.io.File, so classes from org.openide.filesystems do not help at all ;-). [/quote]

Classes from  org.openide.filesystems give you an in-memory virtual filesystem.

kackbratze1 kac... replied on Sat, 2008/11/15 - 1:28pm in response to: Geertjan Wielenga

I think rp117107 is raising the same point that I would raise: that having an in-memory "file system" is all very well, but if it doesn't have the same interface (or a bridging interface available) to make it look like regular Java file access (and/or JSR203), it's not much help for code already written that writes/reads files to the file system in the usual Java way!

 

 

Aaron Digulla replied on Tue, 2008/11/25 - 8:32am

Am I the only one who is missing the "file" part here? How do I use this code to actually write data to these "files"? Do they operate nicely with java.io.File? I/O streams? Can I use them directly in existing code that wants to read, say, config files?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.