1B Rows with the Memory API - JEP Cafe #25

In this JEP Café episode, we take on the well-known ‘1 Billion Rows Challenge’ and implement it using the standard Memory API introduced in JDK 22. The goal of this implementation is not to break any speed records, but to show you how can process binary data using standard patterns offered by the JDK, in a clean and efficient way.

You will see how the different elements of the Memory API are working together. First, you will see which Arena implementation to choose from, among the four given to you by the API. Then you will create Memory Segments to map your multi gigabytes file in the off-heap memory. And then, you will define MemoryLayouts to describe their content in a structured way. Lastly, you will use VarHandles and parallel streams to access and process this data. You see how fast you can go with these clean and standard patterns, that you can use in your own application.

Make sure to check the show-notes!