Skip to main content

Workflow Infographics

This page provides visual walkthroughs of the primary user journeys in Oversight. Each workflow is shown as a two-lane infographic: the User lane on the left shows what the user does, and the System lane on the right shows what Oversight does in response. Decision points and error states are called out inline.

Workflow 1: Analyst File Upload​

This is the primary workflow for analyzing a pre-recorded chatbot conversation.

Analyst File Upload
User
System
Select JSON file
On the /upload page · max 5 MB
Client-side validation
File type, size, JSON structure
Invalid format or size → inline error shown
Configure analysis
Mode (Gemini / Groq / Both) · categories · optional ground truth
Click Upload
POST /api/upload
Rate limit (5/min or 40/day) → HTTP 429
Store file & create upload record
Vercel Blob + DB row with status = processing
Run analysis pipeline
Hallucination · Bias · Toxicity in parallel via selected provider(s)
Update upload status
Did any analysis succeed?
Yesstatus = completed
Nostatus = failed · error stored
View analysis dashboard
/dashboard/[id] · Overview · Hallucination · Bias · Toxicity tabs

Text description of the flow:

  1. User selects a JSON file on the /upload page
  2. Client-side validation — File type, size (≤ 5 MB), and JSON structure are checked
    • Error state: Invalid format or size → inline error message shown
  3. User configures analysis settings — Analysis mode (Gemini/Groq/Both), analysis types (hallucination/bias/toxicity), and optional ground truth document
  4. User clicks Upload → POST /api/upload fires
    • Error state: Rate limit exceeded (5/min or 40/day) → HTTP 429 shown
  5. File is stored to Vercel Blob storage
  6. Upload record is created in the database with status processing
  7. Analysis pipeline runs — Each selected category (hallucination, bias, toxicity) is analyzed in parallel using the selected provider(s)
  8. User sees the processing page (/processing/[id]) with a polling animation
    • Note: The progress bar is simulated — it does not reflect actual backend progress
  9. Upload status updated to completed or failed
    • Error state: All analyses fail → status set to failed, error message stored
    • Success: At least one analysis category succeeds → status set to completed
  10. User is redirected to the analysis dashboard (/dashboard/[id])
  11. Dashboard displays results across tabs: Overview, Hallucination, Bias, Toxicity

Workflow 2: Live Chat Monitoring​

This workflow covers a customer interacting with the live chatbot and the automatic quality monitoring.

Live Chat Monitoring
User
System
Customer opens /chat
No login required
Send message
POST /api/chat
Rate limit (5/min or 40/day per IP) → HTTP 429
Generate bot reply
Groq Llama
Run live monitoring in parallel
Hallucination check (true/false) · Bias score (0–100)
Evaluate violation rules
Violation detected? (hallucination = true OR bias ≥ threshold)
NoConversation continues · results shown in panel
YesEnd session · show "Live agent required" · run full analysis · email analysts
End chat (normal path)
If no violation occurred
Finalize session
POST /api/chat/[id]/complete · full batch analysis · upload record (source: "chat")
Email all analysts
Resend → every analyst with alertEmail configured

Text description of the flow:

  1. Customer visits /chat (no login required)
  2. Customer types a message → POST /api/chat
    • Rate limit check: 5/min, 40/day per IP address
    • Error state: Rate limit exceeded → HTTP 429 returned
  3. Bot reply generated using Groq Llama
  4. Live monitoring runs in parallel on the latest assistant turn:
    • Hallucination check → returns true or false
    • Bias score check → returns a score from 0 to 100
  5. Monitoring results displayed in the monitoring panel below the chat
  6. Decision point: Violation detected?
    • No violation — Conversation continues, monitoring result shown
    • Violation (hallucination = true OR bias score ≥ analyst threshold):
      1. Session is automatically ended
      2. "Live agent required" message shown to customer
      3. POST /api/chat/[id]/complete fires with violation details
      4. Full batch analysis runs on the entire conversation
      5. Alert emails sent to all analysts with alertEmail configured
  7. On normal "End Chat" (no violation path):
    1. POST /api/chat/[id]/complete fires
    2. Full batch analysis runs on the entire conversation
    3. Alert emails sent to analysts
    4. Upload record created with source: "chat", visible in all analysts' upload history

Workflow 3: Analyst Live Monitoring Dashboard​

This workflow covers an analyst observing active and recent chat sessions.

Analyst Live Monitoring Dashboard
User
System
Analyst opens /monitor
Load session sidebar
GET /api/monitor/sessions · polls every 30 s · last 30 min window
Click a session
Badge (Active / Violated / Ended) · message count · timestamp
Load message history
GET /api/chat/[id]
Session still active?
YesPoll every 5 s for new messages
NoShow static history
Render per-message monitoring
Bias bar (green/yellow/red) · hallucination flag · violation banner if flagged
Adjust bias threshold
On /settings · tunes sensitivity for future sessions

Text description of the flow:

  1. Analyst visits /monitor
  2. Session Sidebar loads and polls GET /api/monitor/sessions every 30 seconds
    • Shows sessions from the last 30 minutes
    • Each session shows: status badge (Active/Violated/Ended), last message preview, message count, relative timestamp
  3. Analyst clicks a session in the sidebar
  4. Session Detail Panel loads the full message history via GET /api/chat/[id]
    • If the session is active: polls every 5 seconds for new messages
  5. Per-message monitoring results are displayed:
    • Bias score bar (color-coded: green/yellow/red based on threshold)
    • Hallucination flag indicator
    • Violation banner if the session was flagged
  6. Analyst adjusts threshold in /settings to tune sensitivity for future sessions

Workflow 4: Batch Upload​

This workflow covers uploading and analyzing multiple files at once.

Batch Upload
User
System
Select up to 10 JSON files
On the /upload page
Configure shared settings
Mode · categories · optional ground truth (applied to all files)
Upload files sequentially
POST /api/upload per file · shared batchId
Track per-file status
Per-file outcome
SuccessPending → Uploading → Done
FailureMarked error · batch continues
Click "View Batch Report"
Shown when all files complete
Render batch report
/batch/[batchId] · aggregate KPIs · per-file table · subtype breakdown · drill-down links

Text description of the flow:

  1. User selects up to 10 JSON files on the /upload page
  2. Shared settings configured — Analysis mode, analysis types, optional ground truth (same for all files)
  3. Files uploaded sequentially to POST /api/upload, each with a shared batchId
    • Per-file status shown inline: Pending → Uploading → Done / Error
    • Error state: Individual file fails → marked as error, other files continue
  4. When all files complete — "View Batch Report" button appears
  5. Batch report at /batch/[batchId] shows:
    • Aggregate KPIs (total issues, detection rates across all files)
    • Per-file results table sorted by issue count
    • Subtype breakdown across all files
    • Link to individual dashboard per file