Blazingly Fast GenAI App Development using Spring AI

Blazingly Fast GenAI App Development With
Java and Spring AI
F/ND _Tech Conference 2024
Juarez Barbosa Junior @juarezjunior
Sr Principal Java Developer Evangelist
ORACLE
Copyright © 2024, Oracle and/or its affiliates

Juarez Barbosa Junior
Senior Principal Java Developer Evangelist
• Speaking from Dublin, Ireland
• 28 years of experience in SW Engineering & DevRel
• Oracle, Microsoft, IBM, Nokia, Unisys, Accenture, startups
• Microsoft Azure Developer Relations Lead
• IBM Watson Tech Evangelist & Cloud Rockstar
• IBM Mobile Tech Evangelist & Global Thought Leader
• Nokia Developers Global Champion
• Java, Python, Cloud, DevOps, SRE, Cloud-native, IoT, AI,
Blockchain, Rust
• Speaker at conferences
• Oracle CloudWorld, Oracle Code, Microsoft Ignite, Microsoft
TechX, jPrime, JCON, DevConf.cz, GeeCon, DevOpsDays,
DeveloperWeek, DevOps Institute, CloudLand, DWX, The
Developer’s Conference (TDC), Sec4Dev, JSNation, NodeConf,
Conf42, Shift Conf, Global Azure, Open-Source Lisbon,
CodeFrenzy, Mêlée Numérique, React Summit, Test.js Summit,
Porto TechHub Conf, Pyjamas, MiTechCon, JUGs, meetups, etc.
@juarezjunior
@juarezjunior

Agenda
• AI and GenAI – Synthetic Content
• Artificial Intelligence and GenAI
• OpenAI
• Spring AI
• Spring AI – Demo
• Enabling GenAI with AI Vector Search in
Oracle Database
• Spring AI with Vector Search - Demo
• Java App Dev with the Oracle Database
• Technical References
• Audience: SW Engineers + Business
audiences (not necessarily ML Engineers)

AI and GenAI –
Synthetic Content
Craiova the largest city in southwestern
Romania pictured in a futuristic scenario

Artificial Intelligence and GenAI
• Core concepts in a nutshell
• GenAI - Artificial intelligence algorithms and transformer models with the
capability of generating content.
• Content - Text, images, videos, code, and others.
• Generation – a result of an input prompt (prompt engineering).
• Training data – dataset
• Apps – Applied currently in the fields of content development, entertainment,
drug discovery, software development, healthcare, material generation
sciences, and others.
• Ethical issues – misuse, deep fakes, bias and discrimination, automation and
job replacements, cybersecurity and cybercriminals.
• AI is now a commodity!

OpenAI
• Generative AI with OpenAI
• Natural Language Processing (NLP)
• Powerful, pre-trained and customizable Machine Learning Models
• OpenAI API (REST interface)
• Scalable infrastructure (GPUs)
• Artificial General Intelligence (AGI) Research – ultimate goal
• https://openai.com/sora

Spring AI (0.8.1)
• Spring AI (AI Concepts)
• A Java app framework for building AI-related applications. It provides a robust
abstraction layer that encapsulates many of the complexities required to interact with
various GenAI providers, such as OpenAI, Cohere, Google Vertex AI/Gemini, and
others.
• API support across AI providers for chat, text-to-image, and embedding models.
• Its goal is to apply to the AI domain Spring ecosystem design principles such as
portability and modular design and promote using POJOs as the building blocks of an
application to the AI domain.
• https://spring.io/projects/spring-ai

Spring AI
• RAG (Retrieval-Augment Generation) with Spring AI
• LLMs' baseline knowledge might have gaps based on their training data. If you ask
an LLM to write about a recent trend or event, the LLM won't know what you're
talking about, so you may get meaningless responses.
• There's a way to provide LLMs with additional information (including business-
specific data)- that is what RAG is for!
• Oracle - What Is Retrieval-Augmented Generation (RAG)?
• Oracle – What Is Generative AI? How Does It Work?

Spring AI
• RAG (Retrieval-Augment Generation) with Spring AI
• In a nutshell, RAG is a technique that:
• Reads unstructured data;
• Converts the data into tokens;
• It uses embeddings to transform the tokens into floating-point
arrays or vectors so that LLM can process and interpret them;
• It stores the vectors in Vector Databases.

Spring AI
• The Spring AI API covers a wide range of functionalities
• Chat Completion API
• Embeddings API
• Image Generation API
• Transcription API
• Vector Databases
• Function Calling API

Demo - Spring AI with Open AI
• This is a demo of a Spring AI application that uses
Open AI, along with a quick code walkthrough of some
simple examples.

Spring AI
• Spring AI – Blazingly Fast Performance
• GraalVM
• Native Image - compile Java code ahead of
time (AOT) to a binary – a native executable
• Developing an Oracle JDBC app with
GraalVM Native Image
• mvn -Pnative native:compile
• ./target/spring-ai-graalvm-oai.exe

Spring AI
• Spring AI – Blazingly Fast Performance
• Spring with Virtual Threads
• Add the following entry in the application.properties file:
spring.threads.virtual.enabled=true
• Class VirtualThreadTaskExecutor
• Embracing Virtual Threads
• All together now: Spring Boot 3.2, GraalVM native images, Java 21, and
virtual threads with Project Loom

Oracle Database 23ai Free available for
download now
• Oracle Database 23ai Free accelerates
Oracle’s mission to make it simple to
develop and run all data-driven apps
• Provides developers easy early access to
23ai AppDev features
• Limited database size, memory, and CPU
threads
http://oracle.com/23aiFree
http://oracle.com/23aiFree

Enabling Generative AI
with AI Vector Search in
Oracle Database 23ai
Background behind semantic search and
Generative AI leveraging Oracle AI Vector Search
15 Copyright © 2024, Oracle and/or its affiliates

The following is intended to outline our general product
direction. It is intended for information purposes only, and may
not be incorporated into any contract. It is not a commitment to
deliver any material, code, or functionality, and should not be
relied upon in making purchasing decisions. The development,
release, timing, and pricing of any features or functionality
described for Oracle’s products may change and remains at the
sole discretion of Oracle Corporation.
The materials in this presentation pertain to Oracle Health, Oracle, Oracle Cerner, and Cerner Enviza which are all wholly owned
subsidiaries of Oracle Corporation. Nothing in this presentation should be taken as indicating that any decisions regarding the
integration of any EMEA Cerner and/or Enviza entities have been made where an integration has not already occurred.

Generative AI and AI Vector Search Democratizes AI
Using Vector Databases plus LLMs is SIMPLE
• No longer requires data scientists to build AI solution
• Use standard pre trained embedding models
• Generate vectors, add to private Vector Database
• Augment LLM search with private knowledge
AI vector search is SUPERFAST
• In order to provide AI responses in real time

Finding what you’re looking for can be challenging…
18
I have a photo of an
artwork, but don’t know
the artist or work’s name
I heard a catchy tune
and can even hum a
few bars, but don’t
know the title or artist
I got this error message
ORA-XXXX, but don’t
know how to resolve it

A technology called Vector Search
enables end-user questions
to be mapped to relevant data
in your database
19

Vector search continues to evolve…
20
Vector Space Model
1960s – 1970s
Word Embeddings
e.g., LSA and LDA
2000s
Deep Learning Era
e.g., Word2Vec and GloVe
2010s
Efficient Search Algorithms
e.g., ANN, HNSW, PQ, Annoy
2010s
Transformers and Contextual Embeddings
e.g., BERT, GPT, etc.
late 2010s
33
42
16
21
50
Early Multidimensional Indexing
e.g., KD trees, R-trees, Ball trees
1970s – 1990s
Dedicated Vector Databases
e.g., Milvus, Pinecone, Vespa
late 2010s …
Relational Database
Integration
e.g., Oracle
late 2010s – 2020s

Vectors in AI represent semantics of
unstructured data such as images, documents, videos, etc.
A vector is a sequence of
numbers, called dimensions,
used to capture the important
"features" of the data
Vectors represent the semantic
content of data, not the
underlying words or pixels
Vectors generated using deep
learning embedding models
Vector
33
42
16
21
50
21

Word relatedness in two dimensions
An intuitive example
22
blackberry
strawberry
apple
raspberry
pear
plum
cat
kitten
dog
puppy
wolf
lion
Maine
Vermont
New York
kiwi
d1
d2
elephant
tiger

Convert an image to one or more vector embeddings and search vector database
Embeddings for images
23
0.5, 1.5, 2.6, -1.1, …
Compare these against other
vectors in the database to find
similar content

Generating embeddings
Use an embedding model
Text documents
Natural Language
Embedding
Model
Images
Image
Embedding
Model
Text Vector Table
id vector Image
1 [0.5, 1.5, 2.6, -1.1, …]
2 [1.0, 0.9, 1.6, -1.3, …]
3 [0.6, 1.1, 1.3, -0.9, …]
… … …
Image Vector Table
id vector text
1 [0.8, 0.5, 1.6, -2.5, …] “It was the best of times, it was the
worst of times, it was..”
2 [1.1, 0.3, 0.6, -1.3, …] “It is a truth universally
acknowledged, that a single man..”
3 [1.3, 0.1, 0.2, -1.1, …] “It was a bright cold day in April, and
the clocks were striking..”
… … …

Simple vector search
Pipeline
Embedding
Model
Input Data
Object
Data Object Retrieval
Vector ID
Matches
Relevant
content
Vector Search
embedding
What we’re looking for
Text description, image, etc.
…

Role of vector databases with LLMs
Address the hallucination problem inherent in LLM responses
Augment prompt with enterprise-specific content to produce better responses
Avoid exceeding LLM token limits by using most relevant content
Broad range of data
from the internet
Snapshot of data from a point in time
Private enterprise data
Current data - frequently updated
LLM
Vector Database

Role of vector databases with LLMs
Avoid using sensitive customer data for LLM training and fine-tuning
Cheaper than fine-tuning LLMs, which can be expensive to update
Real-time updated knowledgebase
Cache previous LLM prompts/responses to improve performance and reduce costs
LLM
Better business outcomes
Vector Database

LLM-based chatbot with “enterprise knowledge”
Extended pipeline using retrieval architecture
Prompt
Chat History
Embedding
Model
”What’s the policy?”
”What’s my corporate
401k policy?”
Enhanced
Prompt
Similarity Search
Relational Database
Vector ID
Matches
Fetch docs for matching IDs
Augmented
Prompt
Enhanced prompt + policy details +
scenario examples + known concerns +
expected policy changes…
LLM
Private
content
RESPONSE
Vector Database
embedding
embedding

Use cases
What do embeddings and vector databases enable?
31
Similarity Search
Match candidates with jobs
Image and video retrieval
Find patients with similar issues
Legal e-discovery
Content-Based
Filtering
Personalized recommendation
Find retail items from image
Natural Language
Processing
Text Classification and Clustering
SQL generation
Data Analytics
Anomaly detection
Pattern recognition
Computer Vision
Face recognition
Biometric identification
Object detection
Biomedical Research
Gene/DNA similarity analysis
Molecular structure search
Geographic
Information Systems
Spatial analysis
Map rendering
Industrial Applications
Quality control
Predictive maintenance
Machinery malfunction

AI Vector Search can augment your prompt
to enable accuracy with Generative AI

Demo - Spring AI with Open AI
• Spring AI application that uses Oracle AI Vector Search
and the Oracle Database 23ai

Java App
Dev with
Oracle
Database

Overview of Oracle DB Access with Java

Oracle JDBC - Support for the Latest Java
Versions
• Support to the latest Java versions
(11, 17, 21) and GraalVM
• Project Loom - Virtual Threads
support
• Java Virtual Threads and Pipelined
Database Operations
• JDBC Standards - 4.2 and 4.3
• Reactive Streams - Java Flow API
support
• R2DBC, Reactive Streams Ingestion
(RSI)
• Pipelined Database Operations
Data access is critical in
mission-critical applications!

Technical References
• Spring AI - https://spring.io/projects/spring-ai
• OpenAI - https://openai.com/
• Oracle AI Vector Search - https://rb.gy/sp41uk
• Oracle AI Vector Search for Java Developers with the Oracle Database 23ai -
https://rb.gy/7ivwxr
• Java / JDBC – https://www.oracle.com/jdbc
• Introduction to Oracle JDBC Driver Support for Virtual Threads -
https://bit.ly/3UlNJWP
• Developing an Oracle JDBC app with GraalVM Native Image - https://rb.gy/iy3sgh
• Getting Started with Reactive Relational Database Connectivity and the Oracle R2DBC
Driver - https://rb.gy/42dnz5
• Getting Started with the Java library for Reactive Streams Ingestion (RSI) -
https://bit.ly/3rEiRnC
• Introduction to JDBC Reactive Extensions with the Oracle Database — Developer
Release – https://rb.gy/qxlrbx
• Pipelined Database Operations - https://rb.gy/iy3sgh

Oracle LiveLabs
Showcasing how Oracle’s solutions can
solve your business problems
500+
free workshops,
available or in
development
3.5 million
people have already visited
LiveLabs
developer.oracle.com/livelabs
learn something new …at your pace!
600+
events run
using LiveLabs
workshops

3 membership tiers
Connect: @oracleace facebook.com/OracleACEs
aceprogram_ww@oracle.com
500+ technical experts &
community leaders helping peers globally
The Oracle ACE Program recognizes & rewards individuals for
their technical & community contributions to the Oracle community
Nominate
yourself or a candidate:
ace.oracle.com/nominate
Learn more - ace.oracle.com
blogs.oracle.com/ace

Create your FREE
Cloud Account
• Go to
https://signup.cloud.oracle.com/

Blazingly Fast GenAI App Development using Spring AI

More Related Content

Similar to Blazingly Fast GenAI App Development using Spring AI

More from Juarez Junior

Recently uploaded

Blazingly Fast GenAI App Development using Spring AI