VENDORiQ: Graph-Based Retrieval-Augmented Generation: Business Impact of GraphRAG Release

Why It’s Important

The biggest challenge of generative AI is its tendency to hallucinate or wander off-topic. RAG – or retrieval augmented generation – is an approach that harnesses an LLM vector database and an organisation’s proprietary data to ‘refine prompts’ used in generative AI to provide more specific results. However, such RAG pipelines have their limitations, as detailed in an upcoming IBRS advisory paper “Enhancing Large Language Models: The Power Trio of Prompt Engineering, Retrieval-Augmented Generation, and Fine-Tuning” – request early access from [email protected]. 

GraphRAG leverages detailed knowledge graphs from datasets. Adding graph AI capabilities to the RAG models enables far more nuanced insights without predefined queries. 

IBRS has been expecting graph capabilities to come down to a price point and ease of use where they would invoke a pivotal point in adoption, with initial expectations to see this happen in mid-2025. With this announcement, IBRS is bringing that prediction forward to late 2024.  

Given the ease of the new service and the large number of enterprise IT groups experimenting with RAG approaches, IBRS believes that graph-based RAG will quickly gain popularity.  

However, IBRS does not believe that the addition of graph technology alone will be sufficient for highly accurate enterprise generative AI solutions based on the RAG model. Instead, we will see the emergence of RAG-TAG-GAG – a combination of prompt engineering, traditional vector-based RAG, fine-tuned (trained) LLMs, and graph. The layering and quality measures and different models will drive the most accurate, reliable, generative AI solutions.

The Impact of Graph on RAG Implementations

The ability to answer global questions, queries that address the entire dataset rather than specific chunks of text, is particularly groundbreaking. Traditional RAG methods often falter here, as they rely on text chunks that are semantically similar to the question rather than a holistic view. GraphRAG’s community summaries ensure that all input texts are considered, creating more accurate and complete answers.

Evaluations have shown that GraphRAG outperforms naive RAG on metrics such as comprehensiveness and diversity, especially when using intermediate and low-level community summaries. Moreover, its performance is competitive with hierarchical source-text summarisation, providing similar results at substantially lower token costs.

This technology also indicates a shift towards making advanced data management tools more accessible. The GitHub repository and solution accelerator lower the barrier for adoption, allowing organisations to deploy this technology rapidly and without extensive coding requirements.

Who’s Impacted?

• Development groups
• Information management teams
• CIO and ICT strategy group
• AI teams

What’s Next?

• Explore the GraphRAG GitHub repository for implementation.
• Assess current data querying processes for potential integration with GraphRAG.
• Plan for the deployment of the Azure-hosted solution accelerator for rapid prototyping.
• Expect the other hyperscale cloud vendors to release graph models tightly coupled with existing cloud databases by the end of the year.
• Google is heavily investing in AI-based enterprise search services with Gemini. Expect GraphRAG approaches to be integrated into this service by the end of the year.
• Third-party generative tools, such as Toothfairy and Afforia, among others, will soon add GraphRAG to their services, although such capabilities may be more costly than existing vector-based RAG approaches.