In-Memory Virtual Filesystems

Alex Miller recently wrote an interesting piece about the need for Java to have an in-memory virtual filesystem (here), in the context of JSR-203. Alex notes that athough "[a] virtual filesystem will not be included as part of JSR 203", "the API is designed to be extensible and all of the hooks exist to provide such an implementation".

I really like his use case for having such a filesystem:

Having a virtual memory system to plug into would be fantastic for unit testing. You could create files to your heart’s content and the file access would be fast while also saving you from all the annoying issues with deleting temporary files, Windows file locking, etc.

Makes sense to me. In the comments to his blog entry, a few existing solutions are mentioned, in particular, Commons VFS and the JBoss Microcontainer project. Also, another use case for a filesystem of this kind can be found there:

It may also be useful when running on operating systems that don’t have /tmp or equivalent on swap.

Another solution, much less known in this context, is provided by two standalone JARs that are used within the NetBeans Platform (full disclosure: I work there). These two JARs, org-openide-filesystems.jar and org-openide-util.jar, can simply be dropped in your classpath and then... you have access to an in-memory virtual filesystem. Those two JARs are all that it takes. In other words, there are no dependencies of any kind on any part of the NetBeans Platform. These two JARs can simply be copied/pasted from the NetBeans Platform (i.e., which is a folder within NetBeans IDE) to your own classpath.

The filesystem that you then have is hierarchical. Basically, imagine an XML file in memory and you've got the idea. You can write folders and files into this filesystem and then you can ascribe attributes to those files. These can then be used as part of your unit tests, exactly as described above by Alex. Here's a small example:

package demo;

import java.io.IOException;
import junit.framework.TestCase;
import org.junit.Before;
import org.junit.Test;
import org.openide.filesystems.FileObject;
import org.openide.filesystems.FileSystem;
import org.openide.filesystems.FileUtil;
import org.openide.util.Exceptions;

public class FsJFrameTest extends TestCase {

    FileSystem fs = FileUtil.createMemoryFileSystem();
    FileObject root = fs.getRoot();

    @Before
    @Override
    public void setUp() {
        try {

            //Create a virtual folder:
            FileObject testDataFolder = root.createFolder("TestData");

            //Create a virtual file:
            FileObject testData1 = testDataFolder.createData("testData1");
            //Create three virtual attributes for the file:
            testData1.setAttribute("name", "John");
            testData1.setAttribute("age", 27);
            testData1.setAttribute("employed", true);

            //Create a second virtual file:
            FileObject testData2 = testDataFolder.createData("testData2");
            //Create three virtual attributes for the file:
            testData2.setAttribute("name", "Jane");
            testData2.setAttribute("age", 34);
            testData2.setAttribute("employed", false);

        } catch (IOException ex) {
            Exceptions.printStackTrace(ex);
        }
    }

    //This test will pass because all attributes match the test data.
    @Test
    public void testData1() {
        FileObject testData1 = root.getFileObject("TestData/testData1");
        assertEquals(testData1.getAttribute("name"), "John");
        assertEquals(testData1.getAttribute("age"), 27);
        assertEquals(testData1.getAttribute("employed"), true);
    }

    //This test will fail because age = 34 in the test data.
    @Test
    public void testData2() {
        FileObject testData2 = root.getFileObject("TestData/testData2");
        assertEquals(testData2.getAttribute("name"), "Jane");
        assertEquals(testData2.getAttribute("age"), 33);
        assertEquals(testData2.getAttribute("employed"), false);
    }

}

 

Here you can see that even though you haven't written any actual physical files anywhere on disk and even though you haven't retrieved anything from actual physical files on disk, you do have the important thing: the test data that you need for your unit tests. Plus, that data can be organized as easily as if one were working with a physical hierarchical filesystem. The two JARs provide a lot more besides, such as access to ZIP/JAR archives, via filesystems provided by the selfsame JARs. Just get them from your NetBeans IDE distro and then remove the distro and use another IDE, if that's your preference. (By the way, a screencast of the NetBeans Platform's filesystem [together with a transcript] can be found here.) Looking forward to seeing an in-memory virtual filesystem of this kind available in the JDK, though.