EY has developed a multimodal retrieval-augmented generation framework for enterprise AI systems, according to SiliconANGLE, with text and illustrative content indexed separately before a knowledge graph reconnects related evidence.

The business name of Ernst & Young LLP is trying to address a common RAG limit: many corporate records place important information in charts, tables, engineering diagrams, equations, and images.

The framework changes how material is prepared, indexed, related, and supplied to a language model at inference time; it does not change the underlying model.

EY Framework Separates Text And Illustrations

Dipanjan Sengupta, a consulting technologist at EY Global Delivery Services, said RAG works well for textual content, while many industries also keep critical information in illustrative formats.

Industrial engineering drawings and life-sciences graphs are examples of information that can be missed when retrieval systems focus mainly on text.

The framework starts with separate ingestion pipelines.

Text is segmented and enriched through keyword extraction and named-entity resolution, while illustrations receive descriptive metadata from captions, nearby text, bounding-box analysis, optical character recognition, and language-model descriptions.

Text and illustrations are then stored in separate vector indexes.

Each element is represented as graph data, adding weighted edges where passages and illustrations are related.

White Paper Lists Three Relationship Methods

The white paper describes deterministic keyword matching, semantic similarity based on embeddings, and machine-learning inference for implicit associations as the three ways to build relationships across the graph.

A gleaning process is used to find missing links, resolve ambiguous entities, and identify related information across documents.

Retrieval happens in stages.

The paper states the system first performs a similarity search against the relevant modality-specific index, then uses the resulting identifiers to traverse neighbouring graph nodes.

The search can remain local, expand into graph communities, or combine both approaches before a multimodal re-ranker orders the evidence for the LLM prompt.

The configuration is intended to vary by use case.

A compliance application may favour narrower deterministic retrieval; a research application may need broader semantic exploration.

The paper treats chunking choices, embedding models, relationship construction, reranking, and retrieval scope as settings that can change by application.

AI Agents Still Need Benchmark Evidence

Sengupta connected the method to enterprise AI agents, saying agents need current, domain-specific information to make decisions and select actions.

He also rejected the idea that larger model context windows remove the need for RAG, arguing that broad context does not by itself find the most relevant evidence.

The boundary remains important because the evidence is EY's own research and client experience.

The public record still lacks comparative benchmarks, quantified accuracy gains, named client deployments, and third-party validation results.