> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blankstate.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Modalities

> The SGM reads any signal type through one instrument.

The same SGM model underlies every modality. A Protocol authored against text works against audio, video, or agent world output without modification.

## Supported modalities

| Modality       | Input                        | Notes                                                                  |
| -------------- | ---------------------------- | ---------------------------------------------------------------------- |
| **Text**       | UTF-8 string                 | Native. Base modality.                                                 |
| **Transcript** | Diarised JSON                | Speaker turns mapped to actants automatically.                         |
| **Audio**      | WAV / MP3 / Opus stream      | Transcribed then sensed. Transcript available in response.             |
| **Video**      | MP4 stream                   | Transcribed and framed. Visual axes available on supporting Protocols. |
| **World**      | Agent action/observation log | For Stage-generated and programmatic interaction.                      |

## Cross-modal consistency

The SGM's spectral basis is stable across modalities. A reading of the same participant across written messages and a voice call will produce comparable Signature updates.

## Sending a modality

Specify `modality` in the request body. The IBF Engine selects the correct preprocessing pipeline.

```json theme={null}
{
  "content": "...",
  "modality": "text",
  "protocols": ["empathy"]
}
```

For audio and video, send a URL or base64-encoded binary. The Engine handles transcription and framing inside the IBF pipeline.