Omni-Agent · Tran Quoc Truong

Overview

An Applied AI platform engineered from real operational needs.

Omni-Agent is an Applied AI platform born from solving concrete operational bottlenecks in document intelligence, retrieval, agent workflows, and AI assurance, rather than speculative or experimental use cases. Architected as a Python monorepo with clean control-plane / data-plane separation, everything is scoped to an authenticated, isolated Project workspace. The platform pairs agentic document intelligence with behavioral verification across two flagship capabilities:

Knowledge Builder — document intelligence featuring Docling layout parsing and OCR, Qdrant multi-vector indexing, a Streamable HTTP Model Context Protocol (MCP) server, and a LangGraph-based multi-agent runtime with citation-grounded RAG.
Agent Assurance (IVS Hackathon 2026 2nd Place) — a behavioral verification and observability engine capturing GenAI execution evidence through LiteLLM and OpenTelemetry-based telemetry, designing trace reconstruction pipelines, and enforcing deterministic evaluation quality gates.

Architecture

Monorepo topology and strict domain boundaries.

The monorepo enforces clean dependency boundaries with independent package management per application. The Django/DRF control plane manages project workspaces, tenant boundaries, identity, and permissions, while stateless async workers and FastAPI services execute technical workloads:

Omni-Agent (monorepo)
├── apps/
│   ├── oa-platform/                Django/DRF control plane, project workspaces, identity & React console
│   ├── oa-knowledge-builder/
│   │   ├── docling-server/         Docling layout parsing and OCR service
│   │   ├── doc-processor/          Stateless Celery worker for Markdown chunking & snapshots
│   │   ├── tool-server/            Streamable HTTP MCP server (port 8020) for Qdrant retrieval
│   │   └── knowledge-chat/         LangGraph-based multi-agent runtime (port 8030)
│   └── oa-agent-assurance/
│       ├── api/                    FastAPI backend & OTLP trace ingest endpoint (/otlp/v1/traces)
│       ├── llm-gateway/            LiteLLM proxy with OpenTelemetry GenAI instrumentation
│       └── worker/                 Async evaluation worker (LLM-as-a-Judge & quality gates)
└── packages/
    ├── oa-contracts/               Shared Pydantic transport DTOs
    ├── oa-worker-shared/           Celery, MinIO/S3 storage, and dependency injection lifecycle
    ├── oa-agent-assurance-contracts/
    └── oa-agent-assurance-domain/

Control Plane (oa-platform) owns:

Project tenancy, identity, role-based permissions, and API credential management
Document lifecycles and committed-text snapshot versioning with UUID v7 keys
Generic async job dispatching across decoupled Celery task queues
S3/MinIO artifact metadata and presigned URL access control

Capability 1: Knowledge Builder

Document processing, Streamable HTTP MCP, and LangGraph-based multi-agent runtime.

Knowledge Builder solves unstructured technical document processing and conversational grounded retrieval through three integrated services:

1. Pipeline: Ingestion → Chunking → Vector Indexing

Uploaded technical specifications, PDFs, and manuals are parsed by a dedicated Docling server for layout detection, table structure extraction, and OCR. A stateless Celery worker (doc-processor) creates reproducible Markdown chunk snapshots stored in S3-compatible object storage (MinIO) and generates multi-vector embeddings indexed in Qdrant collections scoped by project.

2. Streamable HTTP Model Context Protocol (MCP) Server

The tool-server service exposes standard Streamable HTTP MCP on port 8020. This allows external coding agents (Claude, Cline, Cursor, Antigravity) and internal workflows to discover and execute project-scoped document retrieval tools with explicit tenant isolation and vector search filtering.

3. LangGraph-Based Multi-Agent Orchestration

The conversational runtime (knowledge-chat on port 8030) implements a stateful LangGraph workflow running over MongoDB session persistence, coordinating specialized agent roles:

OrchestratorAgent — routes incoming user queries between clarification, general chat, or deep research paths.
Research & Navigation Subgraph — CorpusNavigatorAgent, MapSelectorAgent, ExpanderAgent, and ResearchAgent interact with MCP tools to locate relevant documents and extract precise evidence snippets.
EvidenceCuratorAgent & CitationRegistry — deduplicates evidence, validates source provenance, and structures citations into an auditable registry.
AnswerAgent & VerifierAgent — generates grounded answers with structured blocks; the Verifier agent cross-checks claims against cited evidence to prevent hallucinations before emitting the final response.
Streaming Events — streams real-time text deltas, thinking notifications, activity updates, and final grounded blocks over SSE.

Capability 2: Agent Assurance

Runtime trace capture, reconstruction, and deterministic evaluation gates.

Awarded 2nd Place (2nd Runner-up) at IVS Hackathon 2026, Agent Assurance applies software testing discipline to agentic systems across the end-to-end assurance lifecycle (Intake → Strategy → Requirements → Test Design & Oracle Definition → Measurement Qualification → Validation → Reporting → Re-assurance). Unlike traditional tests that only assert final text outputs, Agent Assurance audits the decision trajectory:

Team Nova4Test awarded 2nd Runner-up at IVS Hackathon 2026 — Award Ceremony: Team Nova4Test (IVS HCM) receives 2nd Runner-up at IVS Hackathon 2026

Tran Quoc Truong pitching Agent Assurance on stage — Technical Defense: Truong presenting the agent evaluation and verification architecture

1. OpenTelemetry-Based Trace Capture

Agent interactions pass through a LiteLLM proxy boundary instrumented with OpenTelemetry GenAI semantic conventions (gen_ai.* / genai.*). Traces are exported via OTLP to a dedicated FastAPI endpoint (/otlp/v1/traces) with idempotency enforcement on (trace_id, span_id) to prevent duplicate evidence collection.

2. Trace Reconstruction

The project_evidence pipeline reconstructs raw OTel spans into chronological, multi-turn execution trajectories. It normalizes system instructions, user prompts, assistant completions, tool calls, tool results, model parameters, token usages, and span latencies into immutable evidence packages.

3. Multi-Stage Evaluation & Security Governance

Runs diagnostic evaluation contracts across behavioral dimensions covering context, tool use, trajectory, response quality, security (prompt injection testing), privacy (data leakage evaluation), and safety. Multi-agent evaluation roles (Planner, Judge, Critic, Meta-Judge) prevent hallucinated scores and judge bias.

4. Deterministic Quality Gates & Archival Casefiles

A strict "No Average Hiding" policy evaluates percentile thresholds (P10/P35/P50) and critical blockers. A prompt injection or malformed tool call immediately triggers a deterministic BLOCKED or FAIL verdict. All judgments, traces, and metrics are preserved in an archival "Assurance Casefile" evidence ledger with immutable report snapshots for regression tracking.

Platform foundations

Idempotent async orchestration and reliable data contracts.

Reliability & Data Integrity — designed idempotent evidence ingestion keyed by trace/span identity, with PostgreSQL-backed execution metadata and S3-compatible artifact storage.
Typed Transport Contracts — worker commands and event envelopes live in oa-contracts as framework-free Pydantic DTOs, decoupling Celery tasks from application business logic.
Worker DI & Runtime Helpers — oa-worker-shared standardizes Celery connection management, S3-compatible storage abstractions, and dependency injection lifecycles across data-plane executors.
Network Segmentation — Compose topologies enforce private internal planes for PostgreSQL, Redis, MongoDB, and Qdrant; external access is restricted to Nginx and Cloudflare tunnel routes.
Storage Boundary & Security — raw MinIO/S3 object keys are concealed behind internal aliases; client browsers interact exclusively through short-lived presigned URLs.

Stack & infrastructure

Applied AI technology stack.

Agent Orchestration & AI — LangGraph, LangChain, Model Context Protocol (MCP), LiteLLM, Qdrant Vector DB, Ollama
Observability & Evaluation — OpenTelemetry (OTel), OTLP Ingest, LLM-as-a-Judge, Langfuse, Deterministic Quality Gates
Backend & Workers — Python, FastAPI, Django/DRF, Celery, Redis, PostgreSQL, MongoDB, Pydantic, SQLAlchemy
Documents & Storage — Docling OCR & Layout Engine, MinIO S3 Object Storage
Frontend & Tooling — React 19, TypeScript, Vite, Tailwind CSS, Docker, Docker Compose, Nginx

Stack

LangGraphFastAPIModel Context Protocol (MCP)OpenTelemetry (OTel)LLM-as-a-JudgeQdrantDoclingDjango/DRFCeleryPostgreSQLRedisDocker