Unified Interaction Representation

Protocols are modality-agnostic. All input modalities convert to a Unified Interaction Representation (UIR) before protocol analysis. One protocol. Any modality. Same measurement.

Abstraction Principle

Regardless of source modality, the protocol engine receives a normalized representation containing:

Semantic Content

What was communicated

Temporal Structure

When it happened

Actant Involvement

Who was involved

Reconstruction Chain

Glass-box provenance

Interaction Types

Human ↔ Human

Conversations, meetings, negotiations, interviews, written exchanges

Human ↔ System

Voice assistants, chatbots, UI interactions, forms, navigation

System ↔ System

API calls, event streams, log sequences, data pipelines

Human ↔ Environment

Sensor data, biometrics, IoT interactions, physical-digital bridging

Protocol Modalities

Text Documents

● Available

.pdf, .docx, .pptx, .rtf, .txt

Full extraction with page/slide segmentation

Structured Data

● Available

.xlsx, .csv, JSON, XML

Sheet/table parsing with structure preservation

Scanned Documents

● Available

Image-based PDFs, photos

OCR extraction via advanced handler

Audio

● Available

Calls, meetings, voice notes

Transcription with speaker diarization and timing

Images

● Available

Screenshots, diagrams, embedded

VLM description; OCR for text-in-image

Video

◐ Partial

Screen recordings, video meetings

Frame extraction and context; continuity in progress

System Logs

◐ Partial

Application logs, error traces

Text-based logs supported; event semanticization planned

Event Streams

○ Roadmap

API request/response, webhooks

Real-time ingestion pipeline

Behavioral Signals

○ Roadmap

Click streams, navigation paths

Session reconstruction

Sensor Data

○ Roadmap

IoT telemetry, biometrics

Signal abstraction

Available Partial Roadmap

Protocols

Learn how protocols process UIR to extract measurements

API

Send content for analysis programmatically