The goal of Project Panama is to enrich the connections between the Java virtual machine and well-defined but ‘foreign’, i.e. non-Java APIs.

At a high level, there are 2 things to deal with when accessing native APIs from Java, i.e. accessing the foreign memory, and invoking the foreign code.

Until now, there were different approaches to access native memory from Java. For example, one can use ByteBuffer.allocateDirect. One of the problem with this approach is that the native memory allocated via allocateDirect is only freed when the ByteBuffer is garbage collected! Another ‘solution’ us to rely on undocumented and unsupported Unsafe classes but this is brittle and not recommended!

To invoke native code (ex. C, C++, etc.) from Java, Java Native Interface (JNI) has always been the de-facto solution but JNI is cumbersome.

Panama solves those issues by introducing a supported, efficient, and secure way to invoke native code from Java. Panama has 2 fundamental APIs, the Foreign-Memory Access API, currently incubating in JDK 15, and the Foreign Linker API (candidate JEP). This post discusses 2 aspects of Panama: the Foreign Linker API but also the jextract tool.

JNI AKA the “old way”

Let’s first look at the current situation with JNI. The following example illustrates how to call the getpid C function from Java.

1. Write the Java class

class Main {
  public static void main(String[] args) {
    System.out.println("my process id: " + getpid());
  }

  private static native int getpid();
}

2. Compile the class, and generate the corresponding header file

$ javac -h . Main.java

The -h flag instruct javac to generate a C header file along with the compiled class. This generated header file (Main.h) looks as follows.

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class Main */

#ifndef _Included_Main
#define _Included_Main
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     Main
 * Method:    getpid
 * Signature: ()I
 */
JNIEXPORT jint JNICALL Java_Main_getpid
  (JNIEnv *, jclass);

#ifdef __cplusplus
}
#endif
#endif

3. Implement the C function

Main.c

#include <unistd.h>
#include "Main.h"

JNIEXPORT jint JNICALL Java_Main_getpid
  (JNIEnv *env, jclass cls) {
  // call the actual C function to get the process id!
  return getpid();
}

4. Compile the C code into a dynamic library so that JVM can load it

# Note: JAVA_HOME is the directory where your JDK is installed
# Following is the macOS command to compile it into a dynamic library
# This step is OS and compiler dependent!

$ cc -shared -o libmain.dylib -I $JAVA_HOME/include -I $JAVA_HOME/include/darwin Main.c

5. Load the dynamic library from within the Java program

class Main {
  public static void main(String[] args) {
    System.loadLibrary("main"); // <--- load dynamic library
    System.out.println("my process id: " + getpid());
  }

  private static native int getpid();
}

6. Run the program

$ java Main.java 
my process id: 86733

This basic example shows what is necessary to invoke, using JNI, a native function from Java. In a nutshell, we had to

1. Declare native method(s) in your Java class
2. Compile your Java class with -h flag to generate a C header
3. Implement the generated C declaration
4. Compile a dynamically loaded library
5. Load this dynamic library using System.loadLibrary

So we had to implement an intermediate native code wrapper to call the original native function! In other words, we had to write and compile and native code to be able (!) to call an existing native library! That is, at best, cumbersome!

Panama Foreign Linker API

This example is using Panama’s Foreign Linker API to invoke the same native function.

import java.lang.invoke.*;
import jdk.incubator.foreign.*;

class PanamaMain {
  public static void main(String[] args) throws Throwable {
    // get System linker
    var linker = CLinker.getInstance();
    var lookup = LibraryLookup.ofDefault();

    // get a native method handle for 'getpid' function
    var getpid = linker.downcallHandle(
           lookup.lookup("getpid").get(),
           MethodType.methodType(int.class),
           FunctionDescriptor.of(CLinker.C_INT));

    // invoke it!
    System.out.println((int)getpid.invokeExact());
  }
}

The following command is used to compile and run this Java program.

$ java -Dforeign.restricted=permit --add-modules jdk.incubator.foreign  PanamaMain.java
WARNING: Using incubator modules: jdk.incubator.foreign
WARNING: using incubating module(s): jdk.incubator.foreign
1 warning
87543

💡 -Dforeign.restricted=permit is required to permit native method handles from Java

💡 --add-modules is required as Panama APIs are still incubating

💡 This example uses JEP 330 to compile and run the class in a single step

As we can see, Panama’s Foreign Linker API is straight forward as it doesn’t require to write (and compile!) intermediate native wrapper like JNI does! To try this example, simply install the latest Panama Early Access build.

jextract

In the previous example, we managed to invoke getpid from Java without writing any native code wrapper. But we had to deal with method handle, FunctionDescriptor, method handle type, C symbol name, … just to call a simple C API.

This is where jextract comes in! It is a Panama tool that generates Java class(es) from C header file(s). Those generated classe(s) handle(s) native symbol lookup, calculate(s) function descriptor from C declaration method handle creation, and present simpler Java static method(s) to invoke the underlying C function(s). In short, jextract hides some of the underlying details of the Panama Foreign Linker API.

Let’s take the same getpid example but using jextract.

// simple header file that contains C declaration
// you can extract arbitrary C header file btw.
int getpid();

The following command extracts a Java interface for the above C header.

$ jextract -t com.unix getpid.h
WARNING: Using incubator modules: jdk.incubator.foreign, jdk.incubator.jextract

💡 -t com.unix is used to specify the target package.

Now let’s use com.unix.* from a new Main class (Main2.java).

import static com.unix.getpid_h.*;

class Main2 {
   public static void main(String[] args) {
      System.out.println(getpid());
  }
}

No method handle lookup, no invokeExact, etc. It couldn’t be more simple!

The following command will run the example.

$ java -Dforeign.restricted=permit --add-modules jdk.incubator.foreign  Main2.java
WARNING: Using incubator modules: jdk.incubator.foreign
warning: using incubating module(s): jdk.incubator.foreign
1 warning
87716

getpid is a basic example, more interesting examples combining jextract with the technologies below can be found here.

• Python
• SQLite
• OpenGL
• TensorFlow
• LAPACK
• BLAS
• libgit2

Conclusion

This post uses 2 approaches to invoke from Java the native getpid function, using the old JNI approach, and using the new Panama Foreign Linker API. We can see that the Foreign Linker API is simple and straight forward as it does not require to deal with an intermediate native code wrapper. Moreover, jextract is a Panama tool that simplifies things further as it parses a C header file to generate a Java class that presents a simpler Java static method to invoke the underlying C function(s).