AI Operational Intelligence Prototype — demo pack¶

This page assembles sections relevant to demos and stakeholder presentations: overview, role, architecture, decisions, roadmap, and demonstration.

Contents¶

Summary
Overview
Role and Responsibilities
Architecture and Integrations
Decisions, Trade-offs, and Risks
Roadmap and Demonstration

Summary¶

Status¶

Technical PoC / working prototype, June 2026

Role¶

System Designer, AI-assisted Prototype Engineer

Stack¶

Type: Enterprise AI / controlled LLM execution prototype / evidence-backed analytics prototype

Python, FastAPI, LangGraph, Open WebUI, PostgreSQL, Qdrant, MinIO, Redis, Docker Compose

Project value¶

A technical PoC of controlled LLM execution for evidence-backed analytics. The prototype explores how an executive analytics assistant could be built: the LLM does not answer freely from memory and does not get direct access to data. It operates inside a backend-mediated tool environment over prepared synthetic scenarios.

The prototype validated the architectural idea on single-turn analytical requests. It is not a production platform, not a complete decision-support product, and not a multi-turn conversational agent.

What was implemented¶

chat-like UI based on Open WebUI;
experimental LangGraph-based execution flow;
backend tools for accessing prepared synthetic data;
initial tool registry / tool description concept;
synthetic financial and cross-functional management scenarios;
single-turn analytical requests;
evidence-backed response pattern;
basic execution trace / run details;
architectural documentation and future direction.

Current limitations¶

The most important limitation: each new message in Open WebUI was effectively processed as a new independent request rather than continuation of the same analytical session.

What this demonstrates¶

understanding of enterprise AI risks;
controlled LLM execution instead of free chat;
separation of chat UI and execution layer;
tool-mediated analytics;
evidence-backed response design;
execution trace as a trust/debugging mechanism;
ability to build a working prototype quickly;
ability to honestly document limitations.

Overview¶

Summary¶

AI Operational Intelligence Prototype — a technical PoC / working prototype of controlled LLM execution for evidence-backed analytics.

The original working name was AI Operational Intelligence Platform / Executive Decision Intelligence. That name described the intended product direction, not the maturity of what was built.

The prototype explores how an executive analytics assistant could be built. A user asks an analytical question in a chat-like UI; the backend runs a controlled execution flow over prepared synthetic data. The LLM is intended to act inside a backend-mediated tool environment, not as a free-form chatbot with arbitrary data access.

This is not a production platform and not a complete multi-turn decision-support product. It is an experimental enterprise AI architecture prototype that validated a bounded execution idea.

What was prototyped¶

chat-like UI based on Open WebUI;
experimental LangGraph-based execution flow;
backend tools for accessing prepared synthetic data;
initial tool registry / tool description concept;
synthetic financial and cross-functional management scenarios;
single-turn analytical requests;
evidence-backed response pattern;
basic execution trace / run details;
architectural documentation and future direction.

Technology Stack¶

Layer	PoC choice	Role
Chat-like UI	Open WebUI	Temporary demo interface
Execution flow	FastAPI + LangGraph	Experimental single-request diagnostic flow
Tool execution	FastAPI tool-server	Controlled backend tools, validation, structured output
Structured data	PostgreSQL	Synthetic finance, delivery, ITSM, PMO, meetings data
Document store	MinIO + Qdrant	Document evidence direction for RAG
Runtime/cache	Redis	Lab runtime support
LLM	OpenAI-compatible API	Planning and synthesis; not a source of truth
Infra	Docker Compose	Reproducible lab stand

Architecture patterns explored: Tool Gateway, Tool Registry concept, prototype playbook routing, RAG direction, run trace, evidence trail.

Development approach: AI-assisted prototyping, synthetic data generation, scenario-driven PoC validation.

Role and Responsibilities¶

Role¶

System Designer, AI-assisted Prototype Engineer.

This was prototype architecture work: translating an enterprise AI idea into a constrained lab stand, not production architecture ownership.

Scope of work¶

translated the product idea into a prototype architecture model;
formulated the controlled LLM execution concept;
chose a temporary lab stack for the PoC;
designed the synthetic dataset;
designed a prototype playbook / diagnostic-path approach;
designed the controlled tool-access model (see Security and Access Model);
implemented and evolved the initial Tool Registry concept (see Architecture);
prepared synthetic enterprise demo data (see Domain Model);
developed the evidence and transparency approach (see Integration principles);
configured Open WebUI as a temporary chat-like interface (see Trade-offs);
documented limitations, including the absence of a working multi-turn session loop;
shaped a future direction toward reports, signal cards, evidence views, and backend-native orchestration. These items were not implemented.

Work performed¶

designed the prototype architecture: chat-like harness, experimental execution flow, diagnostic-path routing concept, tool layer, synthetic data layer, document-evidence direction, and execution trace;
built a laboratory FastAPI + LangGraph runtime for experimental single-request diagnostic flows;
established the principle that the LLM does not query PostgreSQL, Qdrant, or MinIO directly;
implemented / laid down controlled tool execution through a FastAPI tool-server;
prepared a synthetic dataset for a fashion/retail/manufacturing company;
expanded the dataset toward cross-domain diagnostics: finance, delivery, ITSM, PMO, meetings, documents;
recorded the target direction without presenting it as delivered scope.

Use of AI¶

The project was developed with AI-assisted prototyping.

LLMs were used to speed up:

draft code generation;
synthetic data preparation;
prompt and process-logic drafts;
architecture option analysis;
documentation;
test scenario drafts.

Key decisions remained under manual control:

architectural boundaries;
data-access model;
lab versus target runtime;
verifiability requirements;
synthetic dataset structure;
meaning of diagnostic paths;
PoC limits;
result review;
project positioning.

Architecture and Integrations¶

Architecture idea¶

The architecture describes a prototype flow, not a completed product architecture.

User request
→ controlled execution flow
→ tool selection
→ backend tool call
→ structured result
→ evidence-backed answer
→ execution trace

The LLM was intended to act inside a controlled backend-mediated tool environment, not as a free-form chatbot with direct arbitrary data access.

OpenWebUI
  → experimental LangGraph / FastAPI flow
  → tool selection / prototype playbook routing
  → Tool Registry concept
  → tool-server / Tool Gateway
  → PostgreSQL / Qdrant / MinIO
  → structured result
  → evidence-backed answer
  → execution trace

This is a single-request prototype loop. A second user message in Open WebUI was not treated as continuation of the same analytical session.

Context diagram¶

flowchart TB
    User[User]
    Prototype[AI Operational Intelligence Prototype]
    Synth[(Synthetic enterprise data)]
    Docs[Synthetic documents]
    LLM[OpenAI-compatible LLM]

    User --> Prototype
    Prototype --> Synth
    Prototype --> Docs
    Prototype --> LLM

The diagram shows the lab stand. Real ERP, ITSM, PMO, or document-system connectors were not implemented.

Tool Gateway pattern¶

All data access goes through controlled HTTP tools with explicit input contracts, validation, structured output, and metadata.

Prototype playbook concept¶

The prototype explored routing questions into domain-specific diagnostic paths instead of exposing all tools to the LLM at once.

Each domain was intended to expose a bounded set of allowed tools, diagnostic steps, constraints, and expected evidence. This was an experimental routing concept, not a complete playbook engine.

Tool Registry¶

Implemented and evolved an initial Tool Registry / tool-description concept as a machine-readable catalog of available tools, schemas, domains, and constraints.

One diagnostic run¶

sequenceDiagram
    autonumber
    participant U as User
    participant UI as OpenWebUI
    participant AG as experimental flow
    participant LLM as LLM
    participant TG as tool-server
    participant PG as PostgreSQL
    participant QD as Qdrant
    participant MN as MinIO

    U->>UI: Analytical question
    UI->>AG: POST /agent/check-hypothesis
    AG->>LLM: plan next diagnostic step
    LLM-->>AG: selected tool
    AG->>TG: controlled tool call
    TG->>PG: execute named query
    PG-->>TG: metric result
    TG-->>AG: structured result
    AG->>LLM: evaluate evidence
    LLM-->>AG: optional document evidence
    AG->>TG: rag_search
    TG->>QD: vector search with filters
    QD-->>TG: chunks + scores
    TG->>MN: resolve object refs
    MN-->>TG: source metadata
    TG-->>AG: document evidence
    AG->>LLM: synthesize answer with limitations
    AG-->>UI: final answer + run details
    UI-->>U: evidence-backed answer + execution trace

The sequence describes one request. It does not describe a durable conversational loop.

Integration principles¶

The LLM does not execute SQL.
The LLM does not read documents directly.
The LLM should not receive the full unrestricted tool list.
The backend / tool-server validates input parameters.
Tools return structured JSON, metadata, warnings, and status.
Evidence is linked to a tool call, document, period/entity, and claim where possible.
Debug visibility is available through run details and should not expose private chain-of-thought.

Evidence-first answers¶

Final responses are expected to rest on tool outputs, document evidence, calculations, or explicitly stated limitations.

Run trace as a trust layer¶

Each diagnostic run preserves, at a basic level, the selected diagnostic path, tool calls, parameters, outputs, and run details for debugging and discussion.

Evidence and transparency approach¶

The prototype explored an evidence and transparency pattern: selected diagnostic path, tool calls, parameters, tool results, execution timeline, run details, and JSON-level debug visibility.

Decisions, Trade-offs, and Risks¶

Key Decisions¶

Backend as control plane¶

The backend should define available tools, permissions, validation rules, execution boundaries, auditability, and response structure. The LLM is not the data-access system.

This was prototyped as a lab control plane. Production authentication, authorization, and audit were not implemented.

Lab runtime separated from target architecture¶

LangGraph and Open WebUI were used as fast lab tools for an experimental execution flow. The target product architecture would assume a backend-native control plane, a dedicated UI, Tool Gateway, semantic layer, report service, and audit trail.

LangGraph was useful for a single-request tool loop. It was not used as durable conversation memory.

Open WebUI as temporary interface¶

Open WebUI was configured as a temporary chat-like interface for PoC demonstration rather than building a dedicated executive UI upfront.

The most important limitation: each new message in Open WebUI was effectively processed as a new independent request rather than continuation of the same analytical session. Even a clarification question such as “which scenario did you mean?” did not produce a working continuation when the user answered.

Prototype playbook concept instead of a free tool set¶

The prototype did not expose all tools to the LLM at once. A diagnostic path was intended to define the domain frame and allowed tools. This remained a prototype concept, not a complete playbook engine.

Synthetic data instead of real enterprise connectors¶

The PoC uses a synthetic enterprise dataset to demonstrate cross-domain diagnostics without live client data.

RAG as document evidence, not as a SQL replacement¶

Structured metrics are calculated in PostgreSQL/tools. RAG is used for documents, decisions, reports, minutes, and context.

Trade-offs¶

1. Interface¶

Context. The PoC needed a fast way to show the main user scenario: ask an analytical question, run a controlled tool flow, and return an evidence-backed answer.

Decision. Open WebUI as a temporary interface.

Rejected alternative. Building a dedicated UI first would have added frontend, auth, and conversation-history work before the architectural idea was validated.

Trade-off. Open WebUI did not provide durable session state for this prototype. Run details had to be generated on the backend and opened by a link. There is a risk that the PoC is perceived as “just another LLM chat”.

2. Execution harness¶

Context. A single request needed planning, tool calls, evidence evaluation, possible extra tool calls, and a final answer.

Decision. LangGraph + FastAPI as an experimental lab runtime.

Rejected alternatives. A Java Spring Boot backend was premature. n8n and Dify were either too rigid or too heavy for this PoC.

Trade-off. LangGraph should not become the production core. Workflow logic would later need a backend-native control plane. Per-request graph state is not the same as multi-turn conversation state.

3. Synthetic dataset¶

Context. A realistic demo needed linked finance, sales, delivery, ITSM, PMO, meetings, and documents. Real corporate data is sensitive, incomplete, and expensive to connect.

Decision. Use a synthetic enterprise dataset with planted cross-domain scenarios.

Trade-off. The PoC does not prove behavior on dirty, incomplete, or contradictory real data. Real integration problems were not tested.

4. Data-access layer¶

Context. The LLM must not get direct access to data.

Decision. A lightweight FastAPI tool-server.

Trade-off. The layer is sufficient to check the hypothesis “LLM selects an action, backend executes, result returns as evidence”. It is not a mature Tool Gateway with RBAC, quotas, or approval gates.

5. LLM API¶

Context. The PoC needed usable reasoning and fast iteration. A local model would have required hardware and serving work first.

Decision. External OpenAI-compatible LLM API.

Trade-off. External API cost, latency, and data-policy limits. Acceptable here because the PoC uses synthetic data only.

Main risks¶

Scope explosion — the idea easily spreads into finance, process mining, goal-setting, personal assistant, meeting transcription, and daily reports. A hard vertical slice is required.
Missing digital traces — real companies may not have the needed data, or access may be politically blocked.
Weak verifiability — without a bind to tool results, an answer can look convincing and still be poorly checkable.
Conversation state — without multi-turn session handling, clarification questions cannot become a real analytical dialogue.
Corporate security — a production version would need separate work on RBAC, ACL, audit, secrets, deployment, and operations.

Roadmap and Demonstration¶

Roadmap¶

Phase	Goal	Exit criteria
v0.1 Lab PoC	Show controlled LLM execution	Request → diagnostic path → tool calls → answer + run details. PROTOTYPED
v0.2 Stabilized run	Stabilize one financial and one operational scenario	Demo can run without hand-substituting the result. PROTOTYPED
v0.3 Document evidence layer	Strengthen evidence through RAG	Answer can refer to documents/chunks. PROTOTYPED as a direction
v0.4 Tool Registry v0.1	Move hardcoded tools toward a registry	Tools described through a manifest; separate tool-server. PROTOTYPED as an initial concept
v0.5 Executive report	Move from chat answer to report artifact	Executive brief + signal cards + evidence appendix. NOT IMPLEMENTED
v0.6 Cross-domain scenario	Show finance → delivery → ITSM → PMO chain	Prototype can explore a planted cross-domain cause. EXPLORED, NOT FULLY STABILIZED
v1 Real MVP	See below	Not reached

Next step toward MVP¶

To become a real MVP, the prototype would need:

multi-turn session state;
correct handling of clarifying questions;
continuation of the same analytical run;
persisted run/session context;
more formal tool registry schema;
evaluation harness;
read-only connectors to realistic enterprise data sources;
authentication and authorization model;
audit and observability layer;
demo scenarios with measurable expected outcomes.

The current work is a technical PoC / working prototype. It is not yet a real MVP.

Demo Scenarios¶

Demo scenarios below are single-turn analytical requests on synthetic data. They show the intended flow, not a complete conversational product.

Financial Performance Diagnosis¶

Example question:

Why did gross margin drop in March?

Expected prototype flow: route into a financial diagnostic path, call metric tools for gross margin, revenue, discounts, COGS, and product mix, then produce a structured summary with evidence and limitations.

Operational / KPI Anomaly Diagnosis¶

Example question:

Why is time-to-market unstable while local team KPIs look normal?

Expected prototype flow: route into an operational diagnostic path and inspect delivery, PMO, ITSM, meeting decisions, and related evidence for cross-functional bottlenecks that are not visible in isolated KPI dashboards.

Cross-Domain Management Hypothesis¶

Target scenario:

Identify the top problematic projects, explain the selection criteria, describe the issue for each project, and prepare a meeting agenda for product owners.

This scenario shows the intended product direction: not only retrieving delayed tasks, but turning structured and document evidence into a management-ready diagnostic brief. It was explored as a target demo-flow and still needs stabilization of cross-domain linkage and evidence quality.

UI Screenshots¶

"What can you do?"¶

UI_1 — Prototype catalog of diagnostic paths and tools

Financial diagnostic path: gross margin drop hypothesis¶

UI_2 — Financial performance diagnosis with tool-based evidence

UI_3 — Financial performance diagnosis usage report

UI_4 — Financial performance diagnosis: called tools and run details

Operational diagnostic path: KPI anomaly¶

UI_5 — Operational anomaly diagnosis across delivery, ITSM, PMO, and documents

What This Demonstrates¶

This project demonstrates the ability to take an ambiguous enterprise AI idea and turn it into a constrained, demonstrable prototype.

It shows:

understanding of enterprise AI risks;
controlled LLM execution instead of free chat;
separation of chat UI and execution layer;
tool-mediated analytics;
evidence-backed response design;
execution trace as a trust/debugging mechanism;
ability to build a working prototype quickly;
ability to honestly document limitations.

The core value is not "using an LLM". The core value is designing a system where AI reasoning is bounded by architecture, evidence, tool contracts, and auditability — and stating clearly what the prototype did and did not implement.