The Java Platform’s NIO channels currently only support I/O operations on synchronous channels with byte buffer views over confined memory segments. While somewhat of a limitation, this reflects a pragmatic solution to API constraints, while simultaneously pushing on the design of the Foreign Memory Access API itself.

Photo by Thomas Tucker

With the latest evolution of the Foreign Memory Access API (targeting JDK 17), the lifecycle of memory segments is deferred to a higher-level abstraction, a resource scope. A resource scope manages the lifecycle one or more memory segments, and has several different characteristics. We’ll take a look at these characteristics in detail, but most notably there is now a way to render a shared memory segment as non-closeable for a period of time. Given this, we can revisit the current limitation on the kinds of memory segments that can be used with NIO channels, as well as the kinds of channels that can make use of segment views.

This writeup introduces the resource scope abstraction, describes its characteristics, and finally how it can be leveraged to provide better interoperability with the different kind of NIO channels. While much of the details are specific to NIO channels, many of the concerns and approaches described here are general enough so might also be applied to other low-level frameworks or libraries operating with byte buffers.

Before discussing how NIO channels can leverage resource scope and views over memory segments, we first need to understand the resource scope abstraction.

Resource Scope

A resource scope models the lifecycle of one or more associated resources, such as memory segments. A newly-created resource scope is alive, which means that all of its associated resources can be accessed safely. A resource scope can be closed, which means that access to its associated resources is no longer allowed. Once closed, all of its associated resources are freed, such as deallocation of memory associate with native memory segments. A resource scope has a number of characteristics, outlined here:

1. Confinement

   • Confined - thread confinement, only the owner thread can manipulate the resources associated with this kind of resource scope

   • Shared - no thread confinement, any thread can manipulate the resources associated with this kind of resource scope

   A resource scope is either a confined scope or a shared scope.

2. Cleanup

   • Implicit - an implicit scope is automatically closed at some point after it becomes unreachable. Additional cleanup actions are handled after the scope has been closed. Invoking close on an implicit scope will fail.

   • Explicit - an explicit scope may be closed by invoking the close method. Additional cleanup actions, if any, are handled when the scope is closed, by the thread invoking the close method. Explicit scopes can be associated with a user-provided Cleaner, to allow for resource cleanup in case the scope instance becomes unreachable and the close method has not been invoked. Either way, the scope is closed exactly once.

There are six static factory API points that allow to retrieve or create resource scope objects that correspond to a combination of the above characteristics. Here they are:

	API point	Confinement	Cleanup
1	newConfinedScope()	confined	explicit
2	newConfinedScope(Cleaner)	confined	explicit
3	newSharedScope()	shared	explicit
4	newSharedScope(Cleaner)	shared	explicit
5	newImplicitScope()	shared	implicit
6	globalScope()	shared	implicit

We can see that there is no API point for a confined resource scope that features implicit cleanup. Such scopes aren’t all that useful, since they are thread-confined and can easily be deterministically closed.

While the global scope features implicit cleanup it is in effect non-closeable, since it is always guaranteed to be strongly reachable (so will never be closed).

NIO channels

NIO channels perform I/O operations with byte buffers. These byte buffers can be backed by memory in the Java heap, off-heap (direct), or views over memory segments.

There are two broad categories of NIO channels that perform (read/write) I/O operations:

1. Synchronous channels - DatagramChannel, FileChannel, SocketChannel
2. Asynchronous channels - AsynchronousFileChannel, AsynchronousSocketChannel

The first category, synchronous channels; read and write operations are surfaced in the API in synchronous form. An I/O operation initiated on thread T will either i) complete successfully returning an appropriate return value, or ii) throw an exception if an error occurs, either outcome occurs on thread T. When a read or write operation is invoked on a channel, the method invocation is passed a byte buffer, or aggregate of byte buffers, to read into or write from, respectively. At the point of method invocation (either read or write) there is a logical transfer of control, the passed byte buffer(s) are effectively under control of the channel until the method invocation completes, at which point control is passed back to the caller. All this occurs synchronously on thread T.

The second category, asynchronous channels; read and write operations are surfaced in the API in asynchronous form. An I/O operation initiated on thread T may schedule that I/O operation to complete at some later time, and on some thread other than T. Similar to synchronous channels, when a read or write operation is invoked on an asynchronous channel, the method invocation is passed a byte buffer, or aggregate of byte buffers, to read into or write from, respectively. At the point of method invocation (either read or write) there is a logical transfer of control, the passed byte buffer(s) are effectively under control of the channel until the operation completes, at which point control of the byte buffers is passed back to the user code. Dissimilar to synchronous channels, I/O operations on asynchronous channels commonly do not complete immediately, but at some later time and on a thread other than the thread that initiated the I/O operation.

With the understanding of the two broad categories of NIO channels and the various characteristics of resource scopes from the previous section, we are now in a position to discuss how they can play well together.

Synchronous channels

Synchronous channels are more straightforward than their sibling asynchronous channels, since “all the action” takes place in a synchronous fashion on a single thread. That is to say, once the I/O operation is initiated and control of the byte buffer(s) is handed over to the channel, the byte buffer(s) should no longer be susceptible to influences of other user code - it would be a bug in the user code if this were the case. That said, however, the Java Platform offers a strong safety guarantee - it should never crash. To adhere to this guarantee, the synchronous channel implementation needs to protect itself when accessing memory backed by byte buffers.

Byte buffers created by the factory methods in the ByteBuffer class, regular byte buffers, do not feature deterministic deallocation - their backing memory is only deallocated when the buffer becomes unreachable. Once the channel implementation holds a strong reference to the buffer for the period of the I/O operation, it can be sure that the memory backing the buffer will not be deallocated.

For byte buffer views over memory segments things are a little more complicate, since a segment’s backing memory is associated with a resource scope. Segments associated with an implicit scope cannot be closed explicitly, so it is sufficient to hold a strong reference to the buffer (and transitively to the resource scope), similar to regular byte buffers. Memory segments with a confined scope can only be accessed by the owning thread, so once control of the buffer is transferred to the channel, it cannot be closed by another thread for the duration of the I/O operation. So far, so good. This leaves us with byte buffer views over memory segments associated with shared scopes. For these kinds of buffers the channel implementation can acquire a resource scope handle to temporarily render the scope non-closeable. This prevents the backing memory of resources associated with the scope from being deallocated for the period of the I/O operation. After which, the resource scope handle is released.

With this, synchronous channels can perform I/O operations with byte buffer views over segments associated with all the different kinds of resource scopes. The current limitation (as of JDK 16), where synchronous channels only support I/O operations with byte buffer views over confined memory segments can be removed.

Asynchronous channels

Asynchronous channels are, by definition, at odds with thread confinement, since I/O operations initiated on one thread commonly complete on another thread. Therefore, the NIO asynchronous channels cannot really work well with byte buffer views over segments that are associated with a thread-confined scope. In fact, I/O operations initiated with such byte buffers should eagerly fail (throw an exception with a suitable detail message). That leaves just shared scopes.

Byte buffer views over segments associated with an implicit scope require the channel implementation to hold a strong reference to the buffer (and transitively to the resource scope), similar to synchronous channels. This prevents the backing memory from being deallocated while the I/O operation is outstanding, regardless of whether the I/O operation completes on a thread other than that which initiated it. Byte buffer views over segments associated with an explicit scope, again similar to synchronous channels, can acquire a resource scope handle to temporarily render the scope non-closeable. This prevents the backing memory of resources associated with the scope from being deallocated for the period of the I/O operation, after which the handle can be released to make the scope closeable again. The acquire and release can occur on different threads.

With this, asynchronous channels can perform I/O operations with byte buffer views over segments associated with shared resource scopes (not confined). This is an improvement over JDK 16, where asynchronous channel do not support I/O operations with any byte buffer views over segments.

Implementation details

So far we’ve outlined how the different kinds of channels can interoperate with buffer views over segments associated with resource scopes, but (as always) there are practicalities of the code and a nod to “yet to be proven” micro optimizations, that influence decision making. By applying a number of small restrictions and simplifications, we can more easily write a straightforward implementation without hindering usability. These simplifications are:

1. Always acquire a resource scope handle for byte buffer views over segments associated with an explicit scope. As outlined above, this is not strictly necessary for synchronous channels, but will simplify the code paths. This restriction can be removed later if there is sufficient evidence that is it problematic.

2. For scattering and gathering I/O operations with byte buffer views over segments from multiple explicit scopes, retain the resource scope handles as a trivial linked-list-like structure of runnables/closeables. Quite often all the buffers will be from a single scope, in which case a single runnable/closeable will be sufficient.

3. Unconditionally acquire the resource scope handle for explicit scopes, even if the handle for a particular scope is already held. Again, this is a simplification that helps keep the code uniform, but can be revisited later if proven to be an issue.

Mircobenchmarks comparing the various aspects of performing (scattering/gathering) I/O operations will be used to investigate the performance aspects of the implementation.

The following Pull Request tracks the code changes: https://github.com/openjdk/panama-foreign/pull/512

Conclusion

Enhancements in Panama’s Foreign Memory Access API (in JDK 17), most notably the resource scope abstraction, vastly improves the interoperation of byte buffer views over memory segments and NIO channels. The NIO channel implementations can now support all byte buffer views that are logically applicable to the programming model offered by the channel.

Originaly posted on panama-dev.