Modalities - Blankstate

The same SGM model underlies every modality. A Protocol authored against text works against audio, video, or agent world output without modification.

Supported modalities

Modality	Input	Notes
Text	UTF-8 string	Native. Base modality.
Transcript	Diarised JSON	Speaker turns mapped to actants automatically.
Audio	WAV / MP3 / Opus stream	Transcribed then sensed. Transcript available in response.
Video	MP4 stream	Transcribed and framed. Visual axes available on supporting Protocols.
World	Agent action/observation log	For Stage-generated and programmatic interaction.

The SGM’s spectral basis is stable across modalities. A reading of the same participant across written messages and a voice call will produce comparable Signature updates.

Sending a modality

Specify modality in the request body. The IBF Engine selects the correct preprocessing pipeline.

{
  "content": "...",
  "modality": "text",
  "protocols": ["empathy"]
}

For audio and video, send a URL or base64-encoded binary. The Engine handles transcription and framing inside the IBF pipeline.

⌘I