Jakub is a Java EE developer since 2005 and occasionally a project manager, working currently with Iterate AS. He's highly interested in developer productivity (and tools like Maven and AOP/AspectJ), web frameworks, Java portals, testing and performance and works a lot with IBM technologies. A native to Czech Republic, he lives now in Oslo, Norway. Jakub is a DZone MVB and is not an employee of DZone and has posted 155 posts at DZone. You can read more from them at their website. View Full User Profile

Using Java as Native Linux Apps – Calling C, Daemonization, Packaging, CLI (Brian McCallister)

10.06.2012
| 7150 views |
  • submit to reddit

This is a summary of the excellent JavaZone 2012 talk Going Native (vimeo) by Brian McCallister. Content: Using native libraries in Java and packaging them with Java apps, daemonization, trully executable JARs, powerful CLI, creating manpages, packaging natively as deb/rpm.

1. Using Native Libs in Java

Calling Native Libs

Calling native libraries such as C ones was hard and ugly with JNI but is very simple and nice with JNA (GPL) and JNR (Apache/LGPL)

JNR and JNA use the functions dlopen, dlsym to dynamically load a shared library and symbols (i.e. functions) in it.

The JNA code belows shows how to load the library MagickWand (/usr/lib/libMagickWand.so) and call some methods mapped to native functions on it, e.g. MagickWandGenesis.

// 1. The interfaceto use a dynamically loaded native lib:
import jnr.ffi.Pointer;
public interface MagickWand {
  public void MagickWandGenesis();
  /* MagickWand* */ Pointer NewMagickWand();
  int MagickReadImage(Pointer wand, String path);
}
// 2. Using it:
int MagickFalse = 0;
final MagickWand w =
Library.loadLibrary("MagickWand", MagickWand.class);
w.MagickWandGenesis();
Pointer magick_wand = w.NewMagickWand();
int rs = w.MagickReadImage(magick_wand, "bunny.jpg");
if (rs == MagickFalse) {
   fail("bad exit code");
}

See Getting Started with JNA for more examples, this time based on JNA.

Packaging Native Libs with Your Java App

Ok, so we can call a native library but how we get to it? One way to ensure the native lib is on the target system is to package it with your java application. Hawt-JNI is a library and Maven plugin that (among others) makes it possible to build a native library, attach it to you application and load it at runtime.

Leveldbjni uses JNI to access the native leveldb library and Hawt-JNI to generate JNI code from annotations (instead of using the more user-friendly JNA or JNR) and to include it inside its JAR and to access it.

The auto-loading of the packaged native library is done by the Hawt-JNI call

private static final Library LIBRARY = new Library("simple", Simple.class);
static {
  LIBRARY.load();
}

- it does the same thing as System.loadLibrary("simple") but also extracts the library from the JAR and loads if from there (copied from the doc).

You might want to check out the example Hawt-JNI maven project and perhaps alos How to Add HawtJNI to an Existing Maven Build. Notice you don’t need to use Hawt to build you library, you can use is as well to only package it in (goal package-jar) and to load it.

This is how a maven library with baked-in native libs looks like:

$ unzip -l leveldbjni-all-1.2.jar | grep libleveldbjni
655754 02-27-12 12:44 META-INF/native/linux32/libleveldbjni.so
707423 02-27-12 12:44 META-INF/native/linux64/libleveldbjni.so
446052 02-27-12 12:44 META-INF/native/osx/libleveldbjni.jnilib

2. Daemonization

Daemonization of Java apps wasn’t easy. You’d often use commons-daemon or create a wrapper script leveraging e.g. Ubuntu’s start-stop-daemon. Daemonization is difficult because you need to do lot of stuff that is difficult from the JVM: fork, setsid so you parent doesn’t own you, dup to protect your files, fork again, …. Fork in Java is risky (you’d miss all system threads such as gc; risk of incosistencies), the rest can be faked via posix – but you can basically use posix_spawn instead (via jnr-ffi). Read on!

Gressil: The Daemonization Library (~25min in the talk)

Gressil is a Java library created by Brian for simple and nice daemonization of Java apps (it basically spawns a new JVM via jnr-ffi, handles pid file, stdin/stderr/out etc.)

public static void main(String[] args) throws IOException
{
  new Daemon().withMainArgs(args)
              .withPidFile(new File("/tmp/chatty.pid"))
              .withStdout(new File("/tmp/chatty.out"))
              .withExtraMainArgs("hello", "world,")
              .withExtraJvmArgs(remoteDebugOnPort(5005))
              .daemonize();
  ...
}

Isn’t that cool?

3. Using Java As a Proper Linux Program (27min)

JAR is just a ZIP file and zip allows for arbitrary crap at the beginning (until a magic number) => you can prepend e.g. a shell script to expand the jar (beware: you need the empty line after the exec otherwise jar gets confused):

$ head -4 ./target/dwarf # this is actually a .jar
#!/bin/sh

exec java -jar "$0" "$@"

$

You can use Brian’s Maven plugin really-executable-jars to create such an executable JAR for you.

End result:

  # Instead of executing this:
$ java -jar ./dwarf.jar --waffles=yes
  # You can execute this:
$ dwarf --waffles=yes

Check out Brian’s project Dwarf (link at the bottom of this post) for a practical example using the Maven plugin.

4. Better CLI With Airline (~31 min)

Airline is a Java library for creating better command-line interfaces. It has also support for commands (such as git’s git add, git log) that are implemented by individual classes. It even generates bash autocompletion.

public static void main(String[] args)
{
  CliBuilder builder = Cli.buildCli("git", Runnable.class)
          .withDescription("the stupid content tracker")
          .withDefaultCommand(Help.class)
          .withCommands(Help.class, Add.class);
  builder.withGroup("remote")
         .withDescription("Manage set of tracked repositories")
         .withDefaultCommand(RemoteShow.class)
         .withCommands(RemoteShow.class, RemoteAdd.class);
  Cli gitParser = builder.build();
  gitParser.parse(args).run();
}

// And the add command implementation:
@Command(name="add", description="Add file contents to the index")
public static class Add implements Runnable
{
  @Option(type=OptionType.GLOBAL, name="-v", description="Verbose mode")
  public boolean verbose;

  @Arguments(description = "Patterns of files to be added")
  public List patterns;

  @Option(name="-i", description="Add modified contents interactively.")
  public boolean interactive;

  public void run()
  {
    System.out.println(getClass().getSimpleName());
  }
}

And the use of it:

$ git help
usage: git [-v] <command> []
The most commonly used git commands are:
    add       Add file contents to the index
    help      Display help information
    remote    Manage set of tracked repositories
See 'git help <command>' for more information on a specific command.

5. Creating Manpages

The Man pages native format is terrible but fortunately you can use Markdown and the command-line utility ronn to turn it into a manpage. (It basically turns md into html and then via some black magic incantantions into grof.)

6. Packaging (37min)

You have an executable, a man page, perhaps some scripts and now you want to install them on the target system. The best way is to use its native package manager.

  •  .deb: mvn plugin jdeb (deb ~ tar)
  • .rpm: mvn plugin rpm-maven-plugin, needs locally installed rpmbuild
  • mac: a mess, use e.g. homebrew manually

Example Project: Dwarf

Brian used many of this techniques in his open source project Dwarf so check it out. It uses f.ex jdeb, manpage generation from Markdown via ronn (manually), executable jar.

That’s all, folks! Go and watch the Going Native video!

Related

Brian’s posts

Acknowledgement: Thanks to Brian for providing his slides and granting the permission to use the examples from them.

 

 

 

 

 

 

 

 

 

 

 

 

Published at DZone with permission of Jakub Holý, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Karl Peterbauer replied on Mon, 2012/10/08 - 3:27am

Good stuff!!!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.