The SSYNC protocol · version 2.0

One file holds the whole storybook.

SSYNC is to storybooks what PDF is to documents: an open format for immersive, narrated, illustrated, musical reading. It is deliberately not video — still images, layered audio and timing, so a page weighs kilobytes and a voice survives being re-read at any speed.

A .storysync file is a ZIP holding one manifest.json and its assets. The player is plain web. Nobody needs our servers — or our app — to hear a story.

  • ContainerZIP · manifest.json + assets/
  • ManifestUTF-8 JSON · schema v2.0
  • NarrationAAC-LC in M4A (WAV fallback)
  • RuntimePlain web · no server required

Anatomy

Seven layers in one packet

A story is not a blob of media. It is seven layers stacked in one manifest, each one optional except the first — so a reader can implement as far up the stack as it likes and still play every SSYNC file it meets.

The SSYNC seven-layer packetA stack of seven labelled bands, numbered one to seven: Metadata, Visual, Text, Voice, Sound, Behavior and Signature. Each band names the manifest block that carries that layer. The list below the diagram describes every layer in full.ONE FILE · SEVEN LAYERS.storysync01Metadatatitle · author · language · covermetadata02Visualstills · alt text · entrancepages[].illustration03Textthe words, page by pagepages[].text.content04Voicerecorded narration · audioCodecpages[].text.audioUrl05Soundthe music bed, per pagepages[].music06BehaviorautoPause · highlight · a11ysettings · pages[].timing07Signatureownership · remix · consentsignatureV2.1 · DRAFTEvery layer is one JSON block in manifest.json. Unknown fields are ignored, never fatal.
Fig. 1 — the SSYNC packet. A reader that understands only layers 1–3 still shows the story; each layer above adds a sense.
  1. 01

    Metadata

    metadata

    Who made it, what it is called, what language it speaks. The only required field in the whole format is metadata.title — everything else is optional, so a one-page story is a legal story.

  2. 02

    Visual

    pages[].illustration

    Still images, not video: a photograph of a crayon drawing, ≤ 2048px on the long edge, with alt text so the page is describable. An optional entrance animation per page — never required to understand the story.

  3. 03

    Text

    pages[].text.content

    The words on the page, kept as text rather than baked into the picture, so they can be re-typeset, resized, read aloud, or highlighted word by word as they are spoken.

  4. 04

    Voice

    pages[].text.audioUrl

    The recording of a real person reading this page — the reason the format exists. Published narration is always AAC/M4A (WAV as the only fallback), so the tape a parent records on Android plays on a grandparent's iPad.

  5. 05

    Sound

    pages[].music

    A mood name for the music bed under the voice — Wonder, Calm, Adventure, Hush. The player generates it and ducks it about −12 dB while narration plays. No mood on a page means silence, on purpose.

  6. 06

    Behavior

    settings · pages[].timing

    How the story moves: how long to wait after narration ends, whether to chime on the page turn, whether words light up as they are read, and the accessibility timing multiplier a slower reader needs.

  7. 07 · v2.1 draft

    Signature

    signature

    Who owns this tape, what it was remixed from, the share URL it lives at, and the consent record for every recorded voice inside it. Consent travels with the story, not in some server's database. Draft in v2.1 — additive, and v2.0 readers ignore it.

The container

A ZIP you can open with anything

the-brave-little-star.storysync
├── manifest.json            REQUIRED · SSYNC v2 manifest (UTF-8 JSON)
└── assets/                  all bundled media, flat
    ├── cover.jpg
    ├── page-1.jpg
    ├── narration-1.m4a      AAC-LC · 48 kHz mono · ~96 kbps
    ├── page-2.jpg
    └── narration-2.wav      the only permitted fallback
Fig. 2 — the archive layout
  • manifest.json MUST sit at the archive root and validate against the v2 schema.
  • Every assets/… path the manifest references MUST exist. A reader MUST reject dangling references.
  • Assets SHOULD be stored uncompressed (deflate level 0) — media is already compressed, so recompression costs time for nothing.
  • Unknown files and unknown manifest fields MUST be ignored, never fatal. Forward compatibility is the whole point of a format.
  • Reference budget: an 8-page story with ~5 minutes of narration fits in about 15 MB.

One page of that manifest:

{
  "id": 3,
  "illustration": { "url": "assets/page-3.jpg", "alt": "A drawing of a red boat" },
  "text": {
    "content": "The little boat sailed on.",
    "audioUrl": "assets/narration-3.m4a",
    "audioCodec": "aac"
  },
  "music": "Calm",
  "timing": { "autoPause": "3s" }
}

Normative

The published-audio codec rule

MediaRecorder disagrees with itself across browsers: Safari records AAC/MP4, Chrome and Android record Opus/WebM — and iOS Safari cannot reliably play Opus/WebM. Left alone, a tape recorded on an Android phone is silent on an iPhone: exactly the cross-device moment this format exists to deliver.

In a published .storysync

  1. Narration MUST be AAC-LC in an MP4/M4A container (audioCodec: "aac"). Reference target: mono, 48 kHz, ~96 kbps.
  2. Where AAC cannot be encoded, narration MAY fall back to 16-bit PCM WAV, 16 kHz mono (audioCodec: "wav"). That is the only permitted fallback.
  3. Narration MUST NOT be Opus, WebM or Ogg. Those are valid only in editor drafts that never leave the authoring device.

Normalization happens at publish time, in the browser — a child’s voice is never sent to a server just to be transcoded:

How each recording format is normalized when a story is published
Recorded asAt publish
AAC/MP4 — Safari, iOSPasses through. Already compliant.
Opus/WebM — Chrome, Android, with a WebCodecs AAC encoderDecoded and re-encoded to AAC-LC, 48 kHz mono, ~96 kbps, in M4A.
Opus/WebM — no AAC encoder available (Firefox today)Decoded to 16-bit PCM WAV, 16 kHz mono. Bigger, but it plays everywhere.

Normative for the reference reader

A child’s voice is part of the format’s job

The flagship user is four years old and recording over their own drawings. So the rules are in the spec, not only in a policy page: publishing is gated behind a parental gate, deleting a story deletes all of it — manifest, images, narration, share code — and the player ships zero third-party trackers. The Signature layer exists so the consent record travels inside the tape rather than in somebody’s database.

Hold one in your hand

The demo tape, as a file

The Brave Little Star
by Island Development Crew
SIDE A · 8 PAGES · SSYNC v2.0

Take the tape apart

This packs the demo story into a real container in your browser — the manifest, plus every illustration bundled into assets/. Rename it to .zip and read the whole thing; drop it back on the reader and it plays.

Nothing is uploaded — the ZIP is built on this device.