Docs: Timed media

Timed media

A timed-media section pairs a video or audio stimulus with a cue timeline. As the media plays, cues reveal items, pause playback until the learner answers, or record that a moment was reached. It is a section type, so it renders in the section player layouts you already use.

How it fits together

The stimulus

An ordinary passage with class stimulus. Its content mounts the media, usually @pie-element/video-stimulus; its accessibility catalogs carry the transcript and signed alternates.

The timeline

timedMedia.cues names time ranges and the items each one acts on. Items no cue reveals or gates are delivered as usual.

The layouts

Splitpane, vertical, tabbed and kernel-host layouts all render it. Without sectionType: "timed-media" a section behaves exactly as before.

A cued section

At 42.5 seconds the video pauses on q1 and resumes only once it is answered correctly; at 118 seconds q2 appears without stopping playback.

Cue activation

Activation	When the cue fires
`reveal`	Its items appear and stay visible. Reveals never retract, even after a seek backwards.
`gate`	Its items appear and playback pauses until every one satisfies `releaseOn`, such as `"correct"`. A gate never resumes playback on its own, and seeking past it still trips it. `onUnknownCorrectness` decides what an item that cannot report correctness does.
`metadata`	Records that the moment was reached and reveals nothing.

To split must-answer items from optional ones at the same moment, author two cues at one timestamp: a gate over the first set and a reveal over the second. A gate on correctness reads the result of formative delivery(opens in new tab), so the section needs maxTries: "unlimited"; with a finite budget a learner could exhaust their tries and never release playback.

Playback policy

Field	Default	Effect
`allowSeekAhead`	`false`	The learner cannot seek past the furthest position reached, including across a reload.
`pauseOnRequiredCue`	`true`	Playback pauses when a gate activates.
`requireMediaCompletion`	`false`	The section is complete only once the media has played to the end.

scoringPolicy.strategy (sum-child-outcomes, average-child-outcomes or host-defined) is validated and carried with the section. PIE derives no aggregate score from it; the host does.

Media Time Source

The section reaches media only through a Media Time Source: current time, duration, play, pause, seek and events, plus two capabilities, canPause and canRestrictSeeking. The stimulus card attaches one for the media element its passage mounted. A host with its own player, a third-party video service for instance, implements the same interface from @pie-players/pie-players-shared/timed-media and attaches it.

When a port reports canPause: false or canRestrictSeeking: false, the matching policy becomes advisory: cues still fire and state is still recorded, the projection reports enforcement: "advisory", and a recoverable timed-media framework warning names the policy that cannot be enforced.

Session and errors

Cue state persists in the section session as a versioned timedMedia slice: the media position, the furthest position reached, whether the media completed, visited and completed cues, and the active cue. It hydrates with the rest of the snapshot, so a reload resumes at the same gate. A slice with an unknown version is rejected whole. A cued item is mounted and hidden until its cue fires, so its session survives seeking.

Three authoring mistakes fail loudly with a timed-media framework error, after which the section delivers as an ordinary section with every item visible: a stimulusRef that resolves to nothing, a stimulus that mounts no media, and a correctness gate over an item without unlimited tries.

Accessibility

Cue activations are announced, and focus moves to a gated item when playback pauses on it.
Captions and the transcript stay available throughout; signed video rides on the stimulus's catalogs.
Read-aloud and media audio never play at once, and the learner's latest action wins: starting read-aloud pauses the media, starting the media stops read-aloud. Neither resumes what it silenced.

Accessibility covers read-aloud and catalogs.

Status

The section contract and runtime are implemented. @pie-element/video-stimulus is at 0.1.x. Still open: an authoring experience for composing cued sections, a QTI serialization (QTI has no native cue primitive, so PIE carries it as its own profile), media control styling, a fallback when the stimulus fails to play, and scorer and review views.

Timed-media section contract(opens in new tab) · Section player README(opens in new tab)