Businesses often store important information across websites, databases, ERP systems, documents, and internal records. However, that information is only useful when teams can quickly find the right answer, understand where it came from, and use it confidently.
This use case explores how a retrieval-augmented AI workflow can help users ask questions in plain language, retrieve relevant information from approved sources, maintain context across conversations, and return structured responses that can be displayed in a chat interface or dashboard.
The Problem
Internal data, including website content, database records, and ERP figures, is only useful if people can ask questions in plain language and receive a trustworthy, sourced answer.
Many chat-with-your-data systems can produce unsupported answers, return unstructured responses that are difficult for a frontend to display, or lose context when a conversation continues across multiple messages. This can make it difficult for users to rely on the information returned by the system.
The Solution
The RAG Data Insight Agent is a retrieval-augmented chat API with persistent session management.
It receives a question through a webhook, tracks the conversation under a session ID, searches connected knowledge and data sources before responding, and uses a fallback data request when retrieval confidence is too low.
The workflow returns a structured JSON response that can include a natural-language answer, table-ready data, confidence information, and sources for the frontend to render directly.
How the Agent Works
- Receive a question and identify the session. The workflow receives a user’s question together with a session ID, allowing the conversation to remain organised across multiple messages.
- Check available knowledge sources. The agent searches indexed website content and database records before generating an answer.
- Use a fallback data request when needed. Where the retrieved information is not strong enough, the workflow can call a separate data-insight process to retrieve additional information.
- Return a structured response. The agent provides an answer in a consistent format, including a written response, table data where relevant, source references, and a confidence score.
- Maintain conversation context. Session-based memory helps the agent understand follow-up questions and maintain continuity throughout the conversation.
Technical Workflow
1. Intake and session extraction
A webhook (/ingest_data) receives a POST request with the user’s message text, a session_id, and an optional session_title. A Set node extracts these into clean fields.
2. Session lookup and branching
A Postgres query checks whether the session_id already exists in a chat_sessions table. An IF node branches: existing sessions pull the stored title and prior message straight through; new sessions get an LLM (GPT-4.1-mini) generated 3–6 word title, safely parsed, and inserted as a new row marking the session created.
3. The core RAG agent
Both branches converge on a single AI Agent (running on Claude Sonnet 4.5) governed by a strict system prompt: it must always search Pinecone first, checking a website namespace and a database_records namespace, before ever answering.
If retrieval confidence comes back at or below 40, it is required to fall back to an HTTP tool calling a separate n8n webhook as a last resort. It is explicitly forbidden from answering out of general knowledge.
4. Structured, tool-visible output contract
The agent’s output format is rigidly enforced: a JSON object with assistant_reply, columns/rows (populated only when the answer is genuinely tabular), a confidence score (0–100), and sources (URLs or record IDs, never fabricated).
The agent is also told never to mention Pinecone, tools, or confidence scoring inside the natural-language reply itself.
5. Conversation memory
A Postgres-backed chat memory node, keyed by session ID with a 10-message context window, gives the agent continuity across turns within the same session, stored in the same database as the session-title tracking.
6. Response formatting and reply
A code node parses the agent’s raw JSON output, falls back gracefully to a plaintext wrapper if parsing fails, and assembles the final payload: session ID, session title, reply, columns/rows, confidence, and sources.
The response is then sent back to the caller through Respond to Webhook.
Technology and Integrations
Built with: n8n, Pinecone, Postgres, OpenAI GPT-4.1-mini, Anthropic Claude Sonnet 4.5, webhooks, and a connected data-insight fallback workflow.
Outcome
The RAG Data Insight Agent creates a more reliable way for users to ask questions about internal knowledge and business data.
Instead of relying on unsupported AI responses, the workflow is designed to retrieve information from indexed sources or explicitly call a fallback data process when retrieval confidence is low. It can also return structured data for dashboards or chat interfaces while maintaining conversation context across multiple questions.
Explore Custom AI Automation for Your Business
The RAG Data Insight Agent is one example of how AI automation can make internal knowledge and business data easier to access and use.
Divverse Labs designs and builds custom AI agents, automation workflows, internal tools, and connected systems for a wide range of business processes.
From sales, marketing, recruitment, customer support, and reporting to finance, operations, and internal team workflows, each solution is designed around the way your business works.
Request a Quote for a Custom AI Automation Solution to get started.