AI Operational Intelligence Prototype — all-in-one

This page is assembled from compact project documentation sections. Individual section files remain the source of truth; this page is for sequential reading, review, and PDF-style export.

Contents

• Summary
• Overview
• Context and Problem
• Goals, Requirements, and Constraints
• Role and Responsibilities
• System Model
• Architecture and Integrations
• Security, Quality, and Operations
• Decisions, Trade-offs, and Risks
• Roadmap and Demonstration
• Architecture Decision Records

Summary

Status

Technical PoC / working prototype, June 2026

Role

System Designer, AI-assisted Prototype Engineer

Stack

Type: Enterprise AI / controlled LLM execution prototype / evidence-backed analytics prototype

Python, FastAPI, LangGraph, Open WebUI, PostgreSQL, Qdrant, MinIO, Redis, Docker Compose

Project value

A technical PoC of controlled LLM execution for evidence-backed analytics. The prototype explores how an executive analytics assistant could be built: the LLM does not answer freely from memory and does not get direct access to data. It operates inside a backend-mediated tool environment over prepared synthetic scenarios.

The prototype validated the architectural idea on single-turn analytical requests. It is not a production platform, not a complete decision-support product, and not a multi-turn conversational agent.

What was implemented

• chat-like UI based on Open WebUI;
• experimental LangGraph-based execution flow;
• backend tools for accessing prepared synthetic data;
• initial tool registry / tool description concept;
• synthetic financial and cross-functional management scenarios;
• single-turn analytical requests;
• evidence-backed response pattern;
• basic execution trace / run details;
• architectural documentation and future direction.

Current limitations

The most important limitation: each new message in Open WebUI was effectively processed as a new independent request rather than continuation of the same analytical session.

What this demonstrates

• understanding of enterprise AI risks;
• controlled LLM execution instead of free chat;
• separation of chat UI and execution layer;
• tool-mediated analytics;
• evidence-backed response design;
• execution trace as a trust/debugging mechanism;
• ability to build a working prototype quickly;
• ability to honestly document limitations.

Overview

Summary

AI Operational Intelligence Prototype — a technical PoC / working prototype of controlled LLM execution for evidence-backed analytics.

The original working name was AI Operational Intelligence Platform / Executive Decision Intelligence. That name described the intended product direction, not the maturity of what was built.

The prototype explores how an executive analytics assistant could be built. A user asks an analytical question in a chat-like UI; the backend runs a controlled execution flow over prepared synthetic data. The LLM is intended to act inside a backend-mediated tool environment, not as a free-form chatbot with arbitrary data access.

This is not a production platform and not a complete multi-turn decision-support product. It is an experimental enterprise AI architecture prototype that validated a bounded execution idea.

What was prototyped

• chat-like UI based on Open WebUI;
• experimental LangGraph-based execution flow;
• backend tools for accessing prepared synthetic data;
• initial tool registry / tool description concept;
• synthetic financial and cross-functional management scenarios;
• single-turn analytical requests;
• evidence-backed response pattern;
• basic execution trace / run details;
• architectural documentation and future direction.

Technology Stack

Layer	PoC choice	Role
Chat-like UI	Open WebUI	Temporary demo interface
Execution flow	FastAPI + LangGraph	Experimental single-request diagnostic flow
Tool execution	FastAPI tool-server	Controlled backend tools, validation, structured output
Structured data	PostgreSQL	Synthetic finance, delivery, ITSM, PMO, meetings data
Document store	MinIO + Qdrant	Document evidence direction for RAG
Runtime/cache	Redis	Lab runtime support
LLM	OpenAI-compatible API	Planning and synthesis; not a source of truth
Infra	Docker Compose	Reproducible lab stand

Architecture patterns explored: Tool Gateway, Tool Registry concept, prototype playbook routing, RAG direction, run trace, evidence trail.

Development approach: AI-assisted prototyping, synthetic data generation, scenario-driven PoC validation.

Context and Problem

Context

Enterprise AI can help with synthesis, but a free-form chat over corporate data is a weak and unsafe model for managerial analytics.

In executive questions, the answer text is not enough. It also matters:

• which data the conclusion rests on;
• which calculation was performed;
• which documents were used;
• which limitations the analysis has;
• what is a fact and what is a hypothesis;
• why this diagnostic path was chosen.

This prototype explores how LLM-based managerial analytics could be made controlled, traceable, and useful for enterprise discussion.

Problem

Typical executive analytics still requires manual assembly of a picture from BI reports, spreadsheets, task trackers, ITSM, PMO materials, documents, meeting notes, and domain experts.

LLMs can speed up synthesis, but unconstrained chat over corporate data creates risks:

• hallucination;
• wrong source selection;
• no reproducible calculation;
• no audit path for the conclusion;
• mixing facts and interpretation;
• unsafe model access to data.

The prototype was built to check a narrower idea: the LLM should not “answer from memory” and should not get direct access to business data. It should operate inside a controlled execution loop: select an allowed tool path, call backend tools, retrieve structured results, preserve a basic execution trace, and produce an evidence-backed answer.

The intended product direction remains an evidence-backed decision-support prototype for executives and domain owners. That direction was not delivered as a complete product in this work.

Goals, Requirements, and Constraints

Goals and Non-Goals

Primary project goals

• Check whether an LLM can work with prepared enterprise traces through controlled backend tools, not through direct data access.
• Build a reproducible lab stand with synthetic enterprise data.
• Implement an experimental vertical slice: user request → controlled execution flow → tool selection → backend tool call → structured result → evidence-backed answer → execution trace.
• Show an approach to managerial diagnostics where conclusions rest on facts, documents, calculations, and explicit analysis limits.
• Document an architectural direction for a future product: backend control plane, Tool Gateway, Tool Registry, playbook-based scenarios, semantic layer, audit trail.

What this PoC is not

• Not a real MVP.
• Not a production-ready enterprise platform.
• Not a complete multi-turn conversational agent.
• Not an autonomous "AI executive".
• Not a BI replacement.
• Not a complete process-mining product.
• Not a production authentication and authorization model.
• Not a set of real enterprise data connectors.
• Not a complete playbook engine.
• Not an evaluation pipeline for answer quality.
• Not a production deployment model.
• Not a full observability and audit model.

What was implemented

• chat-like UI based on Open WebUI;
• experimental LangGraph-based execution flow;
• backend tools for accessing prepared synthetic data;
• initial tool registry / tool description concept;
• synthetic financial and cross-functional management scenarios;
• single-turn analytical requests;
• evidence-backed response pattern;
• basic execution trace / run details;
• architectural documentation and future direction.

The prototype also explored a playbook-routing concept: a question could be directed into a bounded diagnostic path with a limited tool set. This was a prototype playbook concept, not a complete playbook engine.

What was not implemented

• full multi-turn session state;
• continuation of an analytical scenario after a clarification question;
• correct handling of the user’s answer to system clarification;
• durable conversation memory;
• persisted analytical run context;
• production authentication and authorization;
• real enterprise data connectors;
• complete playbook engine;
• evaluation pipeline for answer quality;
• production deployment model;
• full observability and audit model.

Current limitations

The most important limitation: each new message in Open WebUI was effectively processed as a new independent request rather than continuation of the same analytical session.

Even when the prototype asked a clarification question such as “which scenario did you mean?”, the next user answer was processed as a new independent request, not as continuation of the previous run.

The limitation was intentionally documented. The prototype validated a single-turn controlled execution idea; it did not deliver a conversational analytics product.

Requirements

The items below describe the PoC intent. They should not be read as a claim that every requirement was fully implemented.

• BR-001. Evidence-backed managerial analytics The prototype should investigate a managerial question and return a conclusion grounded in calculations, documents, tool results, and explicit analysis limits.

• BR-002. Faster first-pass analysis The prototype should reduce the time of a first-pass hypothesis check by routing the request, calling tools, collecting evidence, and forming a structured answer.

• BR-003. Transparency instead of opaque AI The user should see not only the final text, but also the basis of the conclusion: selected diagnostic path, tool calls, parameters, results, sources, and limits.

• BR-004. Safe data access model The LLM should not receive direct access to databases, documents, or arbitrary SQL.

• BR-005. Domain diagnostic paths Analysis should be framed as bounded diagnostic paths, not as free chat over the full tool catalog.

• BR-006. Demonstration without client data The PoC should demonstrate the approach on a synthetic enterprise dataset.

• BR-007. Extensibility direction New analysis domains should be addable through tools, tool descriptions, and diagnostic paths rather than one giant prompt.

• BR-008. Future enterprise constraints Architectural choices should leave room for later audit, RBAC/ACL, controlled integrations, private deployment, and workflow portability. These were not implemented in the PoC.

Functional requirements

• FR-001. Chat-like interface for an analytical question The user should be able to ask a free-form managerial question through a chat-like UI.

• FR-002. Diagnostic path selection The prototype should determine the domain of the question and select a bounded diagnostic path, or ask a clarification. Clarification was explored, but the user’s answer was not handled as continuation of the same run.

• FR-003. Tool restriction by diagnostic path A selected path should limit the available tools so the LLM does not work with the full catalog at once.

• FR-004. Data access only through the tool-server The LLM should receive data through controlled backend tools.

• FR-005. Structured tool results The tool-server should return structured JSON with result, metadata, status, and enough technical detail for a run log.

• FR-006. Financial metric calculation Financial metrics should be calculated from structured database data, not generated by the LLM.

• FR-007. RAG as document evidence direction Documents should be usable as additional evidence through a MinIO/Qdrant/RAG contour.

• FR-008. Structured analytical answer The final answer should include facts, interpretation, hypotheses, analysis limits, and recommended actions where possible.

• FR-009. Run transparency The prototype should expose the selected diagnostic path, tool calls, inputs, outputs, and basic run details.

• FR-010. Same-language response The prototype should answer in the same language as the user question.

• FR-012. Clarifying question If the question is incomplete, too broad, or missing critical parameters, the prototype may ask for the missing information. This was not a working multi-turn loop: the next message was treated as a new request.

• FR-013. Reproducible synthetic data import The lab stand should restore demo state from the repository: PostgreSQL seed, MinIO objects, Qdrant artifacts, manifests, and scripts.

• FR-014. Several connected synthetic domains The demo dataset should contain linked domains: finance, sales, products, documents, delivery, ITSM, PMO, meetings, and tasks.

Constraints

• CON-001. Verifiable conclusions A material conclusion should be linked to a tool result, document, calculation, or an explicit limit.

• CON-002. Controlled data access The LLM should not get direct access to PostgreSQL, Qdrant, MinIO, the filesystem, or arbitrary SQL.

• CON-003. Execution traceability A run should preserve the selected diagnostic path, tool calls, parameters, results, and the final answer at a basic level.

• CON-004. Synthetic data only The PoC uses synthetic data. Real client data is not connected.

• CON-005. Limited PoC scale The PoC is sized for demo scenarios and architectural validation, not production load, mass users, or SLA.

• CON-006. LLM usage A local LLM is a target for a future product. The PoC may use an external LLM and should remain adaptable to model replacement.

Assumptions

• The prototype works on a synthetic dataset, not on live client data.
• Some cross-domain scenarios are demonstrative and need further stabilization.
• The RAG contour shows a document-evidence direction; it is not a mature ingestion/lifecycle solution.
• Tool Registry and related factory ideas are early: part implemented, part described architecturally.
• The UI is a temporary chat-like harness, not a target executive cockpit.
• The prototype does not prove production readiness. It proves that a controlled LLM analytics loop can be assembled and demonstrated on synthetic data.

Role and Responsibilities

Role

System Designer, AI-assisted Prototype Engineer.

This was prototype architecture work: translating an enterprise AI idea into a constrained lab stand, not production architecture ownership.

Scope of work

• translated the product idea into a prototype architecture model;
• formulated the controlled LLM execution concept;
• chose a temporary lab stack for the PoC;
• designed the synthetic dataset;
• designed a prototype playbook / diagnostic-path approach;
• designed the controlled tool-access model (see Security and Access Model);
• implemented and evolved the initial Tool Registry concept (see Architecture);
• prepared synthetic enterprise demo data (see Domain Model);
• developed the evidence and transparency approach (see Integration principles);
• configured Open WebUI as a temporary chat-like interface (see Trade-offs);
• documented limitations, including the absence of a working multi-turn session loop;
• shaped a future direction toward reports, signal cards, evidence views, and backend-native orchestration. These items were not implemented.

Work performed

• designed the prototype architecture: chat-like harness, experimental execution flow, diagnostic-path routing concept, tool layer, synthetic data layer, document-evidence direction, and execution trace;
• built a laboratory FastAPI + LangGraph runtime for experimental single-request diagnostic flows;
• established the principle that the LLM does not query PostgreSQL, Qdrant, or MinIO directly;
• implemented / laid down controlled tool execution through a FastAPI tool-server;
• prepared a synthetic dataset for a fashion/retail/manufacturing company;
• expanded the dataset toward cross-domain diagnostics: finance, delivery, ITSM, PMO, meetings, documents;
• recorded the target direction without presenting it as delivered scope.

Use of AI

The project was developed with AI-assisted prototyping.

LLMs were used to speed up:

• draft code generation;
• synthetic data preparation;
• prompt and process-logic drafts;
• architecture option analysis;
• documentation;
• test scenario drafts.

Key decisions remained under manual control:

• architectural boundaries;
• data-access model;
• lab versus target runtime;
• verifiability requirements;
• synthetic dataset structure;
• meaning of diagnostic paths;
• PoC limits;
• result review;
• project positioning.

System Model

Domain Model

Main entities

Entity	Role
Executive User	Asks an analytical question and receives a prototype answer
Domain Owner	Intended reviewer of conclusions in a domain: finance, delivery, PMO, ITSM
Platform Admin	Target-system role for sources, access, diagnostic paths, and tools; not implemented in the PoC
Diagnostic path / playbook concept	Bounds the diagnostic process, available tools, and domain frame
Tool	Controlled operation over data: metric, RAG, aggregated result
Diagnostic Run	One analysis run with run details, selected path, tool history, and final answer
Evidence Item	Fact, calculation, document chunk, or limitation linked to a conclusion
Claim	Statement in the answer that should refer to evidence

A diagnostic run in this prototype is a single-turn analytical request. The model does not describe a durable multi-turn conversation.

Synthetic enterprise dataset

Prepared synthetic enterprise demo data for a fashion/retail/manufacturing company, conceptually extended toward FashionCo Group / fashionco-enterprise.

Company domain:

• premium apparel / small-to-medium manufacturing;
• B2B / B2B2C through distributors, boutiques, showrooms, and marketplace partners;
• data period: 2024-2025;
• business domains linked in one synthetic enterprise contour.

Connected business domains:

Domain	Content
core	customers, products, sales orders, order items
crm	companies, contacts, deals, activities, tasks
finance	invoices, payments, accounts receivable, COGS
production	production orders, operations, materials, supplier deliveries
documents	document objects, invoice files, metadata
rag	RAG documents and chunks
delivery	epics, tasks, transitions, rework, cycle time
itsm	incidents, SLA, affected services, business impact
pmo	roadmap items, milestones, slippage, status reports
meetings	decisions, action items, decision-to-action gaps
goals	KPI, objectives, ownership, conflicts — target/extension
semantic	metric definitions, business entities, calculation rules
system	dataset version, runtime metadata
eval	scenario truth, expected claims, forbidden claims — not exposed to normal tools

Data Model

Simplified ERD logic:

flowchart LR
    SalesOrder[core.sales_orders] --> SalesItem[core.sales_order_items]
    SalesOrder --> Invoice[finance.invoices]
    SalesOrder --> Payment[finance.payments]
    SalesOrder --> Deal[crm.deals]
    Deal --> Roadmap[pmo.roadmap_items]
    Roadmap --> Epic[delivery.epics]
    Epic --> Task[delivery.tasks]
    Task --> Transition[delivery.task_transitions]
    Roadmap --> Incident[itsm.incidents]
    Roadmap --> Decision[meetings.decisions]
    Decision --> Action[meetings.action_items]
    Document[documents.document_objects] --> Chunk[rag.rag_chunks]
    Chunk --> Roadmap
    Chunk --> Incident
    Chunk --> Decision

API Contracts

Agent endpoint

POST /agent/check-hypothesis
Content-Type: application/json

Example request:

{
  "question": "Why did gross margin drop in March 2025?",
  "hypothesis": "The margin drop is related to discounts",
  "context": {
    "period": "2025-03",
    "domain": "finance"
  }
}

Example response:

{
  "selected_playbook": "financial_operations",
  "verdict": "partially_supported",
  "final_answer": "...",
  "evidence": [
    {
      "type": "metric",
      "tool_id": "metric_gross_margin",
      "period": "2025-03",
      "summary": "gross margin decreased compared with baseline"
    }
  ],
  "tool_calls": [
    {
      "tool_id": "metric_gross_margin",
      "args": { "period": "2025-03" },
      "status": "ok"
    }
  ],
  "limitations": [
    "Analysis is based on synthetic dataset only"
  ]
}

The selected_playbook field reflects the prototype playbook-routing concept, not a complete playbook engine.

Tool-server health

GET /health

Purpose: tool-server availability check.

Gross margin tool

POST /tools/metric/gross-margin
Content-Type: application/json

Supported request forms:

{ "period": "2025-03" }

{ "start_date": "2025-02-01", "end_date": "2025-03-31", "group_by": ["month"] }

Calculation logic:

revenue = SUM(core.sales_orders.net_amount_rub)
cogs = SUM(core.sales_orders.cogs_amount_rub)
gross_margin = revenue - cogs
gross_margin_rate = gross_margin / revenue

RAG search tool

POST /tools/rag-search
Content-Type: application/json

Example request:

{
  "query": "Definition of Ready decision for the promo feature",
  "filters": {
    "domain": "delivery",
    "source_type": "meeting_minutes",
    "period": "2025-Q1"
  }
}

Expected response:

{
  "status": "ok",
  "results": [
    {
      "document_id": "DOC-PMO-2025-03-12",
      "chunk_id": "CHUNK-001",
      "title": "PMO weekly meeting notes",
      "score": 0.82,
      "object_key": "executive-demo-docs/pmo/2025-03-12.md",
      "snippet": "..."
    }
  ]
}

Architecture and Integrations

Architecture idea

The architecture describes a prototype flow, not a completed product architecture.

User request
→ controlled execution flow
→ tool selection
→ backend tool call
→ structured result
→ evidence-backed answer
→ execution trace

The LLM was intended to act inside a controlled backend-mediated tool environment, not as a free-form chatbot with direct arbitrary data access.

OpenWebUI
  → experimental LangGraph / FastAPI flow
  → tool selection / prototype playbook routing
  → Tool Registry concept
  → tool-server / Tool Gateway
  → PostgreSQL / Qdrant / MinIO
  → structured result
  → evidence-backed answer
  → execution trace

This is a single-request prototype loop. A second user message in Open WebUI was not treated as continuation of the same analytical session.

Context diagram

flowchart TB
    User[User]
    Prototype[AI Operational Intelligence Prototype]
    Synth[(Synthetic enterprise data)]
    Docs[Synthetic documents]
    LLM[OpenAI-compatible LLM]

    User --> Prototype
    Prototype --> Synth
    Prototype --> Docs
    Prototype --> LLM

The diagram shows the lab stand. Real ERP, ITSM, PMO, or document-system connectors were not implemented.

Tool Gateway pattern

All data access goes through controlled HTTP tools with explicit input contracts, validation, structured output, and metadata.

Prototype playbook concept

The prototype explored routing questions into domain-specific diagnostic paths instead of exposing all tools to the LLM at once.

Each domain was intended to expose a bounded set of allowed tools, diagnostic steps, constraints, and expected evidence. This was an experimental routing concept, not a complete playbook engine.

Tool Registry

Implemented and evolved an initial Tool Registry / tool-description concept as a machine-readable catalog of available tools, schemas, domains, and constraints.

One diagnostic run

sequenceDiagram
    autonumber
    participant U as User
    participant UI as OpenWebUI
    participant AG as experimental flow
    participant LLM as LLM
    participant TG as tool-server
    participant PG as PostgreSQL
    participant QD as Qdrant
    participant MN as MinIO

    U->>UI: Analytical question
    UI->>AG: POST /agent/check-hypothesis
    AG->>LLM: plan next diagnostic step
    LLM-->>AG: selected tool
    AG->>TG: controlled tool call
    TG->>PG: execute named query
    PG-->>TG: metric result
    TG-->>AG: structured result
    AG->>LLM: evaluate evidence
    LLM-->>AG: optional document evidence
    AG->>TG: rag_search
    TG->>QD: vector search with filters
    QD-->>TG: chunks + scores
    TG->>MN: resolve object refs
    MN-->>TG: source metadata
    TG-->>AG: document evidence
    AG->>LLM: synthesize answer with limitations
    AG-->>UI: final answer + run details
    UI-->>U: evidence-backed answer + execution trace

The sequence describes one request. It does not describe a durable conversational loop.

Integration principles

1. The LLM does not execute SQL.
2. The LLM does not read documents directly.
3. The LLM should not receive the full unrestricted tool list.
4. The backend / tool-server validates input parameters.
5. Tools return structured JSON, metadata, warnings, and status.
6. Evidence is linked to a tool call, document, period/entity, and claim where possible.
7. Debug visibility is available through run details and should not expose private chain-of-thought.

Evidence-first answers

Final responses are expected to rest on tool outputs, document evidence, calculations, or explicitly stated limitations.

Run trace as a trust layer

Each diagnostic run preserves, at a basic level, the selected diagnostic path, tool calls, parameters, outputs, and run details for debugging and discussion.

Evidence and transparency approach

The prototype explored an evidence and transparency pattern: selected diagnostic path, tool calls, parameters, tool results, execution timeline, run details, and JSON-level debug visibility.

Security, Quality, and Operations

Security and Access Model

Implemented in the PoC

• Only synthetic data is used.
• The LLM does not get direct access to PostgreSQL, Qdrant, or MinIO.
• Data access goes through controlled HTTP tools.
• Tools have explicit input contracts.
• Financial calculations are performed by named queries / backend logic, not by arbitrary SQL from the LLM.
• Run details can show the selected diagnostic path, tool calls, inputs, and outputs.

Target production model — not implemented

• SSO / IdP integration.
• RBAC / ABAC.
• Tenant isolation.
• ACL-aware RAG retrieval.
• Tool permissions by diagnostic path, user role, and domain.
• Read-only mode by default.
• Approval gates for write actions.
• Full audit log: run_id, user_id, tool_id, params hash, result hash, source refs.
• Secrets management.
• On-prem/private deployment option.

Controlled LLM execution

The LLM can plan the next step, but data access is delegated to controlled backend tools — not free chat over corporate data.

Controlled tool access model

The LLM never queries PostgreSQL, Qdrant, or MinIO directly.

Non-Functional Requirements

The PoC was not sized or proven against production NFR. The relevant lab constraints were:

• demonstrable single-request flow;
• traceability of tool calls;
• no direct model access to data stores;
• reproducible Docker Compose stand;
• synthetic data only.

Failure Modes

Failure mode	How it appeared	Notes
Wrong diagnostic-path routing	An operational question could go into a finance path	Clarification was explored, but the user’s answer was not continued as the same session
Unsupported question	Question outside dataset/tool coverage	Explicit limitation is preferable to a fabricated answer
Duplicate tool calls	Same tool called with the same parameters	Fingerprint tool_id + canonical_json(args), run-local cache
Stub/empty tool response	Tool returned no data or a stub	Status handling, warning, insufficient-evidence verdict
Missing RAG evidence	Document layer finds no confirmation	Explicit limitation: document evidence not found
Hallucinated conclusion	LLM states a conclusion without evidence	Evidence-first prompt; no evaluation pipeline was implemented
Incomplete cross-domain linkage	Metrics exist, but finance ↔ delivery ↔ ITSM is not proven	Needs a stronger semantic layer; not fully stabilized
External LLM unavailable	API down or rate-limited	Retry/backoff and local-model option are future work
Context overflow	Tool manifest/evidence too large	Context budget, summarization, retrieval filters — early
Data leakage risk	Model sees extra data	Tool-level permissions and no direct data access; no production ACL

Sizing and Cost Notes

The current PoC is sized for demonstration:

• 1-3 concurrent users;
• a few diagnostic runs during a demo;
• synthetic dataset for 2024-2025;
• tens/hundreds of thousands of rows at most in lab data;
• one run usually expected to stay within 1-10 tool calls;
• cost is driven by LLM API calls and Docker/VPS/local infrastructure;
• production sizing was not performed.

Production would need separate estimates for source volume, document corpus, ingestion frequency, users, RPS, SLA, LLM routing cost, and private deployment. That work was not done.

Decisions, Trade-offs, and Risks

Key Decisions

Backend as control plane

The backend should define available tools, permissions, validation rules, execution boundaries, auditability, and response structure. The LLM is not the data-access system.

This was prototyped as a lab control plane. Production authentication, authorization, and audit were not implemented.

Lab runtime separated from target architecture

LangGraph and Open WebUI were used as fast lab tools for an experimental execution flow. The target product architecture would assume a backend-native control plane, a dedicated UI, Tool Gateway, semantic layer, report service, and audit trail.

LangGraph was useful for a single-request tool loop. It was not used as durable conversation memory.

Open WebUI as temporary interface

Open WebUI was configured as a temporary chat-like interface for PoC demonstration rather than building a dedicated executive UI upfront.

The most important limitation: each new message in Open WebUI was effectively processed as a new independent request rather than continuation of the same analytical session. Even a clarification question such as “which scenario did you mean?” did not produce a working continuation when the user answered.

See also Architecture Decision Records.

Prototype playbook concept instead of a free tool set

The prototype did not expose all tools to the LLM at once. A diagnostic path was intended to define the domain frame and allowed tools. This remained a prototype concept, not a complete playbook engine.

Synthetic data instead of real enterprise connectors

The PoC uses a synthetic enterprise dataset to demonstrate cross-domain diagnostics without live client data.

RAG as document evidence, not as a SQL replacement

Structured metrics are calculated in PostgreSQL/tools. RAG is used for documents, decisions, reports, minutes, and context.

Trade-offs

1. Interface

Context. The PoC needed a fast way to show the main user scenario: ask an analytical question, run a controlled tool flow, and return an evidence-backed answer.

Decision. Open WebUI as a temporary interface.

Rejected alternative. Building a dedicated UI first would have added frontend, auth, and conversation-history work before the architectural idea was validated.

Trade-off. Open WebUI did not provide durable session state for this prototype. Run details had to be generated on the backend and opened by a link. There is a risk that the PoC is perceived as “just another LLM chat”.

2. Execution harness

Context. A single request needed planning, tool calls, evidence evaluation, possible extra tool calls, and a final answer.

Decision. LangGraph + FastAPI as an experimental lab runtime.

Rejected alternatives. A Java Spring Boot backend was premature. n8n and Dify were either too rigid or too heavy for this PoC.

Trade-off. LangGraph should not become the production core. Workflow logic would later need a backend-native control plane. Per-request graph state is not the same as multi-turn conversation state.

3. Synthetic dataset

Context. A realistic demo needed linked finance, sales, delivery, ITSM, PMO, meetings, and documents. Real corporate data is sensitive, incomplete, and expensive to connect.

Decision. Use a synthetic enterprise dataset with planted cross-domain scenarios.

Trade-off. The PoC does not prove behavior on dirty, incomplete, or contradictory real data. Real integration problems were not tested.

4. Data-access layer

Context. The LLM must not get direct access to data.

Decision. A lightweight FastAPI tool-server.

Trade-off. The layer is sufficient to check the hypothesis “LLM selects an action, backend executes, result returns as evidence”. It is not a mature Tool Gateway with RBAC, quotas, or approval gates.

5. LLM API

Context. The PoC needed usable reasoning and fast iteration. A local model would have required hardware and serving work first.

Decision. External OpenAI-compatible LLM API.

Trade-off. External API cost, latency, and data-policy limits. Acceptable here because the PoC uses synthetic data only.

Main risks

1. Scope explosion — the idea easily spreads into finance, process mining, goal-setting, personal assistant, meeting transcription, and daily reports. A hard vertical slice is required.
2. Missing digital traces — real companies may not have the needed data, or access may be politically blocked.
3. Weak verifiability — without a bind to tool results, an answer can look convincing and still be poorly checkable.
4. Conversation state — without multi-turn session handling, clarification questions cannot become a real analytical dialogue.
5. Corporate security — a production version would need separate work on RBAC, ACL, audit, secrets, deployment, and operations.

Roadmap and Demonstration

Roadmap

Phase	Goal	Exit criteria
v0.1 Lab PoC	Show controlled LLM execution	Request → diagnostic path → tool calls → answer + run details. PROTOTYPED
v0.2 Stabilized run	Stabilize one financial and one operational scenario	Demo can run without hand-substituting the result. PROTOTYPED
v0.3 Document evidence layer	Strengthen evidence through RAG	Answer can refer to documents/chunks. PROTOTYPED as a direction
v0.4 Tool Registry v0.1	Move hardcoded tools toward a registry	Tools described through a manifest; separate tool-server. PROTOTYPED as an initial concept
v0.5 Executive report	Move from chat answer to report artifact	Executive brief + signal cards + evidence appendix. NOT IMPLEMENTED
v0.6 Cross-domain scenario	Show finance → delivery → ITSM → PMO chain	Prototype can explore a planted cross-domain cause. EXPLORED, NOT FULLY STABILIZED
v1 Real MVP	See below	Not reached

Next step toward MVP

To become a real MVP, the prototype would need:

1. multi-turn session state;
2. correct handling of clarifying questions;
3. continuation of the same analytical run;
4. persisted run/session context;
5. more formal tool registry schema;
6. evaluation harness;
7. read-only connectors to realistic enterprise data sources;
8. authentication and authorization model;
9. audit and observability layer;
10. demo scenarios with measurable expected outcomes.

The current work is a technical PoC / working prototype. It is not yet a real MVP.

Demo Scenarios

Demo scenarios below are single-turn analytical requests on synthetic data. They show the intended flow, not a complete conversational product.

Financial Performance Diagnosis

Example question:

Why did gross margin drop in March?

Expected prototype flow: route into a financial diagnostic path, call metric tools for gross margin, revenue, discounts, COGS, and product mix, then produce a structured summary with evidence and limitations.

Operational / KPI Anomaly Diagnosis

Example question:

Why is time-to-market unstable while local team KPIs look normal?

Expected prototype flow: route into an operational diagnostic path and inspect delivery, PMO, ITSM, meeting decisions, and related evidence for cross-functional bottlenecks that are not visible in isolated KPI dashboards.

Cross-Domain Management Hypothesis

Target scenario:

Identify the top problematic projects, explain the selection criteria, describe the issue for each project, and prepare a meeting agenda for product owners.

This scenario shows the intended product direction: not only retrieving delayed tasks, but turning structured and document evidence into a management-ready diagnostic brief. It was explored as a target demo-flow and still needs stabilization of cross-domain linkage and evidence quality.

UI Screenshots

"What can you do?"

Prototype catalog of diagnostic paths and tools

Financial diagnostic path: gross margin drop hypothesis

Financial performance diagnosis with tool-based evidence

Financial performance diagnosis usage report

Financial performance diagnosis: called tools and run details

Operational diagnostic path: KPI anomaly

Operational anomaly diagnosis across delivery, ITSM, PMO, and documents

What This Demonstrates

This project demonstrates the ability to take an ambiguous enterprise AI idea and turn it into a constrained, demonstrable prototype.

It shows:

• understanding of enterprise AI risks;
• controlled LLM execution instead of free chat;
• separation of chat UI and execution layer;
• tool-mediated analytics;
• evidence-backed response design;
• execution trace as a trust/debugging mechanism;
• ability to build a working prototype quickly;
• ability to honestly document limitations.

The core value is not "using an LLM". The core value is designing a system where AI reasoning is bounded by architecture, evidence, tool contracts, and auditability — and stating clearly what the prototype did and did not implement.

Architecture Decision Records

See Architecture Decision Records.