ONNX Based Generative AI LLMs in Java with Project Babylon

The Open Neural Network Exchange (ONNX) serves as a universal format for representing machine learning models, facilitating their deployment across diverse platforms. Traditionally, Large Language Models (LLMs) are developed in Python using frameworks like PyTorch, TensorFlow, or scikit-learn, and then exported to ONNX for execution.

In this presentation, we demonstrate how Java, a language not traditionally associated with AI modeling, can be utilized to produce ONNX models. We also explore the concept of ONNX-based Generative AI LLMs in Java, leveraging Project Babylon's code reflection capabilities.

The presentation showcases a practical Java example of an LLM, detailing its transformation into the ONNX format and subsequent execution.

Recorded at Devoxx Belgium 2025.