When working with the Foreign Memory API, clients might need to prevent other clients from closing the scope associated with a resource they are actively working on (e.g. a memory segment). The ResourceScope API provides a way to do this; in Java 17 a pair of methods — ResourceScope::acquire and ResourceScope::release, respectively — has been added to achieve that goal. These methods have morphed, in a later incarnation of the API, into a single method, namely ResourceScope::keepAlive, which can be used in a similar fashion. In this document we will discuss scope dependencies in greater details, by showing how they are most commonly used, and we will propose a new strategy to model scope dependencies in a more explicit fashion.

Use cases

In this section, we introduce a set of reference examples where scope dependencies are required. This set of examples will help the reader understand in which ways scope dependencies are currently used, and will also serve as a way to assess possible alternative approaches.

Critical regions

Perhaps the most common use case for scope dependencies is a critical region — that is, a block of code which needs to execute while a number of resources are alive. Two relevant instances of critical regions are:

• foreign function calls: values passed by-reference need to be kept alive for the entire duration of the call;
• asynchronous buffer reads/writes: when passing a segment to an async IO operation, we need to make sure that the segment is kept alive (across multiple threads) for the entire duration of the IO operation.

Resource scope dependencies allow clients to model critical regions, as follows:

void m(MemorySegment m1, MemorySegment m2, MemorySegment m3) {
   try (ResourceScope criticalScope = ResourceScope.newConfinedScope()) {
      criticalScope.keepAlive(m1.scope());
      criticalScope.keepAlive(m2.scope());
      criticalScope.keepAlive(m3.scope());
      // critical section
   }
}

Above we have a method m which takes three memory segments. The method creates a new confined scope (criticalScope), which is then used to keep alive the segments’ scopes. After the critical operation, criticalScope is closed, which makes the segments’ scopes closeable again.

Non-closeable scopes

Another common use case for scope dependencies consists in creating a resource that cannot be closed by external clients. For instance, a library might be managing a segment, and return slices of that segment back to its clients. One client might decide to call the close() method on the segment scope, which will, as a side effect, also close all the other segments generated by the library. With scope dependencies, clients can create a non-closeable segment, as follows:

ResourceScope libScope = ReosurceScope.newConfinedScope();
MemorySegment libSegment = MemorySegment.allocateNative(..., libScope);
ResourceScope privateScope = ResourceScope.newConfinedScope();
privateScope.keepAlive(libScope);
...
MemorySegment getSlice() {
    return libSegment.asSlice(...);
}

Here, the library creates a native segment (libSegment), backed by some scope (libScope). Before slices are returned to clients, the library creates another private scope (privateScope), which is used to keep libScope alive. This way, any attempt from clients to close libScope through the returned slice will fail with IllegalStateException. When the library is ready to deallocate the segment, it can first close privateScope, and then proceed to close libScope. Since external clients do not have access to privateScope, all segments returned by the library in this way will appear to clients as non-closeable.

Pooled allocators

Some allocators might attempt to recycle memory allocations through a memory pool, to speed up allocation. Pooled allocators are typically used as follows:

MemoryPool pool = MemoryPool.create(ResourceScope.newSharedScope()); // creates memory pool
...
try (ResourceScope scope = ResourceScope.newConfinedScope()) {
    SegmentAllocator allocator = pool.pooledAllocator(scope); // obtains pooled allocator for 'scope'
    ...
} // memory returned to the pool

As the above snippet shows, the memory pool itself has a scope, so that a client can release the pool memory, if required. Clients interact with the memory pool by first creating a scope, and then requesting a pooled allocator from the memory pool, with given scope. When the client scope is closed, the memory allocated through the pooled allocator will be returned to the pool, and can be reused by a different client.

There is a subtle invariant in the above code: the memory pool scope cannot be closed before the client scope. A failure to enforce this invariant might result in crashes, as clients attempt to access memory in the pool that is no longer present.

In fact, the implementation of pooledAllocator takes advantage of scope dependencies, to prevent premature closure of the pool scope:

ResourceScope poolScope = ...

SegmentAllocator pooledAllocator(ResourceScope scope) {
    scope.keepAlive(poolScope);
    ...
}

Since the client scope keeps poolScope alive, poolScope cannot be closed while the client scope is still active.

Spaghetti scopes

Scope dependencies are a great tool to manage the lifecycle of complex resources. The ResourceScope::keepAlive method shown above is a step in the right direction, as it allows clients to reason about dependencies between scopes, rather than forcing client to set up complex chains of acquire/release via ResourceScope::addCloseAction.

That said, it can be hard to visualize the full graph of scope dependencies, given that said dependencies are, in essence, fully dynamic and be added or removed at any time. Because of this, having a predicate which allows clients to query whether two scopes are related is problematic, especially in a multi-threaded environment. In most cases this means that (as in the allocator example shown above) that clients would have to act defensively, by always adding a scope dependency between the client scope and the pool scope.

Since we cannot check if any two scopes are related, it also follows that it is not possible to determine whether adding a new dependency would form a cycle. Consider the following code:

ResourceScope scope1 = ResourceScope.newConfinedScope();
ResourceScope scope2 = ResourceScope.newConfinedScope();
ResourceScope scope3 = ResourceScope.newConfinedScope();
scope1.keepAlive(scope2);
scope2.keepAlive(scope3);
scope3.keepAlive(scope1); // whoops
scope1.close(); // error
scope2.close(); // error
scope3.close(); // error

The above code creates three scopes, and then sets up a dependency loop. After the third dependency is added, all the scopes will become non-closeable, and it will no longer be possible to release memory associated with them (at least not explicitly). Worse, there’s no way for a client to detect the issue; no error occurs when the problematic dependency is added, as checking for cycles in a graph of dynamically updated (possibly by multiple threads) scopes would be prohibitively expensive.

Note that the above code is deliberately simplistic; in real world uses, segments will be passed around to multiple APIs, which might add dependencies to their scopes; in such cases, the user might not necessarily be aware that a dependency loop has been created.

Finally, since scope dependencies are set up in isolation, it is possible for clients to set up more dependencies than strictly required. Consider the critical section example shown above: a new dependency is added to each of the segments’ scopes. What if all segments share the same scope? What if there is a pre-existing dependency between two of the scopes? Since a client cannot answer these questions, it follows that the safe approach is to just add three separate dependencies, which can be suboptimal, especially when working with shared scopes, whose dependency status has to be updated atomically.

Enter, ancestors!

All the problem described in the previous section have a common cause: scope dependencies are completely dynamic. There is no way for clients to statically set up a scope dependency graph; also, as dependencies come and go, there is no way for clients to ask whether two scopes are related. These issues can be addressed by allowing clients to specify a set of scope dependencies upon the creation of a new scope; that is, a scope can be created with zero, one or more ancestors. If the scope is created correctly, all its ancestors cannot be closed before the scope is (either implicitly or explicitly).

Creating a scope with multiple ancestors can be done with a new factory in ResourceScope [¹], as follows:

ResourceScope scope1 = ...
ResourceScope scope2 = ...
ResourceScope scope12 = ResourceScope.ofConfined(Set.of(scope1, scope2));

Creating a scope with one or more ancestors might succeed or fail, depending on whether, at creation, the runtime can successfully establish a dependency on all the specified ancestors. There is also some basic validation which occurs when creating a dependent scope such as the one above: for instance, while a confined scope can have a shared scope ancestor, the reverse is not tolerated.

Scope dependencies setup in this way form a directed acyclic graph; absence of cycles stems from the fact that dependencies on ancestors can only be expressed by creating a new resource scope. As such, it is not possible to create a scope which has itself as ancestor, either directly or indirectly.

The ResourceScope API can also provide a new predicate method, namely ResourceScope::isAncestorOf(ResourceScope) which returns true if the receiver scope is an ancestor of the parameter scope. We say that a scope S1 is an ancestor of another scope S2, if it is possible to find a path from S2 to S1, by recursively walking up the ancestor chain. We also say that the global scope (ResourceScope::globalScope) is an ancestor of any other scope S. It is easy to show that this relationship is both reflexive and transitive (in fact the ancestor-ship relation defines a partial order on scopes).

Let’s now turn again at the set of use cases discussed previously, to see how they can be handled in a world where scope dependencies are captured explicitly, using the API shown above.

Critical regions

Handling critical regions is easy: clients must create a scope whose ancestors are the scopes that need to be kept alive. If the creation of such a scope succeeds, work on the critical section can begin, as follows:

void m(MemorySegment m1, MemorySegment m2, MemorySegment m3) {
   try (ResourceScope criticalScope = ResourceScope.newConfinedScope(Set.of(m1.scope(), m2.scope(), m3.scope()))) {
      // critical section
   }
}

This is not too dissimilar from what was shown above, except that now dependencies are specified in bulk, when creating the critical scope.

Non-closeable scopes

Dealing with non-closeable scope doesn’t change much. All a library has to do is to create a private scope whose ancestor is the external scope which will be later exposed to clients, as follows:

ResourceScope libScope = ReosurceScope.newConfinedScope();
MemorySegment libSegment = MemorySegment.allocateNative(..., libScope);
ResourceScope privateScope = ResourceScope.newConfinedScope(Set.of(libScope));
...
MemorySegment getSlice() {
    return libSegment.asSlice(...)
}

Pooled allocators

In the memory pool example there are two relevant resource scopes: the pool scope and a client scope. The pool scope is long-lived, while the client scope is short-lived. Whenever the client scope is closed, some memory is returned to the pool. If scope dependencies are static, it is now possible for the memory pool API to probe the relationship between the pool and the client scopes, as follows:

ResourceScope poolScope = ...
    
SegmentAllocator pooledAllocator(ResourceScope scope) {
    if (!poolScope.isAncestorOf(scope)) {
        throw new IllegalArgumentException("Bad scope!");
    }
    ...
}

In other words, the allocator API can now make sure that the client scope doesn’t outlive the pool scope. When that happens, an exception is thrown, as it wouldn’t be safe for the client to use the pooled memory. The onus of coming up with a well-formed scope falls on the client:

MemoryPool pool = MemoryPool.create(ResourceScope.newSharedScope());
...
try (ResourceScope scope = ResourceScope.newConfinedScope(Set.of(pool.scope()))) {
    SegmentAllocator allocator = pool.pooledAllocator(scope); //ok
    ...
}

In the special, but common, case where the pool scope is indeed the global scope, the above code reduces to a simpler form:

MemoryPool pool = MemoryPool.create(ResourceScope.globalScope());
...
try (ResourceScope scope = ResourceScope.newConfinedScope()) {
    SegmentAllocator allocator = pool.pooledAllocator(scope); //ok
    ...
}

Here, no scope dependency is required — after all, the global scope is an ancestor of every other possible scope.

Soft dependencies

Setting up more ad-hoc relationships between scopes is still possible, thanks to ResourceScope::addCloseAction. For instance, it is possible to establish a soft dependency between two scopes S1 and S2 so that closing S1 also closes S2:

ResourceScope scope1 = ...
ResourceScope scope2 = ...
scope1.addCloseAction(scope2::close);

The code above might behave in surprising ways: consider the case where both scope1 and scope2 are confined scopes, but where scope1 is associated with a Cleaner instance. In such case it is possible that the close actions associated with scope1 will be executed from the cleaner thread. Since scope2 is also a confined scope, calling its close method from the cleaner thread is bound to fail. Because of this, idioms such as the one shown in the above snippet should not be relied upon, unless a client has full control of all the scopes involved.

Conclusions

In this document we have explored an extension which makes relationship between scopes a first-class concept in the ResourceScope API. This move allows scope dependencies to be captured in a stable fashion (scope dependencies form an acyclic graph), which can be queried by clients. This latter point is especially important, as it gives more power to APIs to reject invalid scope combinations (see the pooled allocator example above).

Having scope relationships clearly marked in the API also gives clients an option to reason about the lifecycle associated with multiple memory resources. Is it safe, for instance, to store a memory address obtained from one segment S1 into another segment S2? Again we can build on the ancestor-ship relation defined above: if the scope of S1 is an ancestor of S2, then the store is valid, as it is never possible for a client of S2 to observe a pointer to an already-freed memory location (since S1 can only be closed after S2).

In other words, the changes described in this document enable safer uses of the Foreign Memory Access API, all while retaining the same basic expressiveness of previous API iterations.