Sim-to-Stage

Section 1Introduction

A live theatrical performance is a coordinated multi-modal artifact: script, blocking, lighting cues, sound cues, video projections, scenic transitions, performer interpretation, audience interaction — all unfolding in real time over an evening of performance. Producing such an artifact at high quality has historically been organized through human craft alone: directors, designers, technicians, and performers iterating across weeks of rehearsal, accumulating implicit knowledge that lives in production journals, cue sheets, and individual memory.

In the past three years, three forces have arrived simultaneously that change the underlying engineering question.

First, large language models and multi-modal generative models have become sufficiently capable to participate as collaborators in theatrical creative work — not merely as tools for narrow tasks, but as departmental specialists.

Second, the robotics community has formalized Sim-to-Real as a paradigm for transferring policies trained in simulation to physical deployment, complete with mitigation techniques (domain randomization, system identification, residual learning) and a vocabulary of "gap" diagnosis.

Third, the AI infrastructure community has converged on world models — large-scale simulations of physical or social environments — as a primary research direction.

These three forces converge on a question: what would it look like to produce a live performance the way modern robotics produces a deployed policy? That is — to simulate the production end-to-end in a director-gated environment, to validate the simulated rehearsal against a single authoritative truth source, and to compile the validated simulation into one-shot executable artifacts that human performers and technical crew deploy on the live stage. We call this paradigm Sim-to-Stage.

This paper makes four contributions:

1. We define the Sim-to-Stage paradigm precisely (Section 3).
2. We construct a structural homomorphism between Sim-to-Real (robotics) and Sim-to-Stage (live performance), showing the two are not merely analogous in metaphor but structurally homologous as engineering pipelines (Section 3.2).
3. We define the Sim-to-Stage gap as the paradigm's central open problem and decompose it into seven distinct sub-gaps that admit independent research (Section 4).
4. We propose seven architecture primitives — one useful decomposition that a mature Sim-to-Stage system is likely to instantiate variants of (Section 5).

We are deliberate about what this paper does not claim. We do not claim the Sim-to-Stage gap has been closed. We do not claim the reference implementation has solved the seven primitives in their general form. We do not claim the paradigm subsumes all live-performance production — only that it provides a useful frame for one mode of production-augmented work. Section 8 details limitations and open questions in full.

Section 2Background

2.1The Sim-to-Real Paradigm

The Sim-to-Real paradigm originates in robotics, where the cost and risk of training control policies directly on physical robots motivated a transfer-learning approach: train policies in a physics simulator (Isaac Sim, MuJoCo, Habitat, CARLA), then deploy to physical hardware. The central technical problem is the Sim-to-Real gap — the discrepancy between the simulator and the physical world that causes a policy trained in sim to fail on real hardware.

Three families of mitigation are well-established. Domain randomization trains the policy across a distribution of simulator parameters (mass, friction, lighting) so the real world appears as one of many seen environments. System identification estimates the parameters of the physical world to bring the simulator into closer correspondence with reality. Residual policy learning trains a corrective policy on top of the sim-trained policy that learns to account for residual sim-to-real discrepancy.

The Sim-to-Real paradigm has become canonical in robotics because it makes a previously intractable engineering question tractable: it gives the field a vocabulary (the gap, the mitigations), a benchmark structure (sim eval + transfer eval), and an architectural pattern (sim trainer, transfer step, deployment policy). This paper proposes that live performance production stands today in a similar position to robotics circa 2015, and that an analogous paradigm — Sim-to-Stage — provides similar vocabulary, structure, and pattern.

2.2Live Performance Production Today

Many productions unfold over weeks or months of rehearsal, technical preparation, and previews. The artifact that ultimately runs on stage is the joint product of dozens of specialists: director, dramaturg, lighting designer, sound designer, video designer, scenic designer, costume designer, stage manager, technical director, and a cast of performers. The coordination problem alone is substantial; the creative problem is, by traditional account, the entirety of theatrical art.

Several technological currents already exist in live-production practice. Stage management software (QLab, ETC Eos, Disguise) captures cue scripts and operates them at performance time. Digital previz tools (Capture, WYSIWYG, depence²) allow lighting and projection designers to preview their work before installation. Collaborative document tools and shared cue sheets manage cross-department coordination. However, these tools operate as departmental islands: each specialist's tool optimizes that specialist's work in isolation, with handoffs across department boundaries handled through unstructured human communication.

The Sim-to-Stage paradigm proposes a different architecture: a single director-gated truth source representing the production, against which all specialist work is reconciled; a multi-modal simulation environment in which the production can be rehearsed end-to-end across modalities; and a compiler that translates the validated simulation into one-shot executable artifacts deployable by the human performers and technical crew.

2.3Why "Sim-to-Stage" Now

Three preconditions for Sim-to-Stage have arrived approximately together in 2024–2026:

1. Multi-modal generative models are now capable of producing draft artifacts (lighting plots, sound compositions, video sequences, scenic visualizations) at quality sufficient for director review and iteration. This makes specialist agents tractable.
2. Long-context and persistent-memory language models can hold an entire production's worth of script, notes, blocking, and rationale in working memory, enabling a Director Intelligence Profile that remembers across rehearsals.
3. Sim-to-Real as a paradigm has matured to the point that its vocabulary and architectural lessons (gap diagnosis, mitigation families, benchmark structure) can be ported to adjacent transfer domains.

The Sim-to-Stage paradigm therefore arrives at a specific historical moment, and we believe the moment is genuinely open: we are not aware of a widely adopted canonical paradigm for this domain.

Section 3The Sim-to-Stage Paradigm

3.1Definition

Sim-to-Stage is a paradigm for producing live performances in which:

1. The production is represented as an authoritative, director-gated, multi-modal artifact (the Authoritative Truth Source — Section 5.1) maintained as a single source of truth across all departments.
2. The production is rehearsed end-to-end in a simulation environment that models, to varying fidelity, lighting state, sound state, video state, scenic state, blocking, and performance timing.
3. Specialist work is contributed by department-level agents (human or AI) whose proposals are reconciled against the truth source through director-mediated confirmation gates.
4. The validated simulation is compiled into one-shot executable artifacts (cue scripts, packages, briefings) that human performers and technical crew deploy on the live stage.
5. Every decision, every gate, every revision is recorded in a provenance ledger that maintains audit traceability from director intent to live execution.

A Sim-to-Stage system is one that instantiates variants of these five properties. An early reference implementation is StageR (simtostage.com), but the paradigm is intended to be larger than any single implementation, and we invite external groups to instantiate it independently.

3.2Homomorphism with Sim-to-Real

Sim-to-Stage is not merely analogous to Sim-to-Real in metaphor; the two paradigms exhibit a structural homomorphism — a systematic correspondence of roles between the two transfer pipelines. Table 1 documents the correspondence.

We claim this homomorphism is more than a stylistic flourish: it identifies an engineering correspondence we believe is load-bearing. Each row of Table 1 maps a Sim-to-Real role to a Sim-to-Stage role that occupies the same structural position in the transfer pipeline. We argue the homomorphism licenses the import of Sim-to-Real's accumulated engineering wisdom — randomization-style mitigation, identification-style mitigation, residual-style mitigation, deployment-gate governance — into Sim-to-Stage practice. The remaining sections develop the specifics.

Section 4The Sim-to-Stage Gap

4.1Definition

We define the Sim-to-Stage gap as the family of discrepancies between a simulated rehearsal and the live performance that cause a Sim-validated production package to fail on the live stage. Like the Sim-to-Real gap in robotics, the Sim-to-Stage gap is not a single phenomenon but a structural collection of distinct sub-gaps, each amenable to its own analysis and mitigation. Closing the Sim-to-Stage gap is the central open research program of the paradigm.

4.2Seven Identified Sub-Gaps

We identify seven sub-gaps. We do not claim the list is exhaustive; we claim it is useful: each sub-gap is independently studyable, and progress on any one is progress on the paradigm. Each sub-gap is named, defined, and connected to a candidate research direction.

These seven sub-gaps are the proposed map of the Sim-to-Stage research territory. We invite extension, refinement, and challenge.

Section 5Architecture Primitives

We propose the following seven architecture primitives as one useful decomposition of a Sim-to-Stage system. We expect a mature Sim-to-Stage system to instantiate variants of these primitives, but we do not claim this decomposition is unique or exhaustive (see Section 8). We define each primitive at the paradigm level — the role it plays, the interface it presents to the rest of the system — without committing to implementation details, which are properly the subject of later technical reports.

Section 6Toward a Sim-to-Stage Benchmark

A paradigm is established not only by its definition but by the benchmark community against which progress is measured. We sketch the structure of a Sim-to-Stage benchmark here; the full proposal will land separately as benchmark.md in this repository.

We propose that a Sim-to-Stage benchmark should evaluate, at minimum:

• Cross-modal coherence — does the Sim-rehearsed production exhibit consistent lighting / sound / video / scenic / blocking coordination at each cue point?
• Transfer fidelity — does the executed live performance match the Sim-rehearsed state on observable dimensions (cue timing, cross-modal coordination, executed-vs-planned divergence)?
• Director-intent retention — does the executed production preserve the director's declared aesthetic and stylistic priorities as recorded in the DIP?
• Audit traceability — for any property of the executed production, can the Provenance Ledger trace it back to a declared director decision?
• Production efficiency — does the paradigm reduce the wall-clock effort to producible state, relative to the unaided baseline?

A serious Sim-to-Stage benchmark will require a benchmark corpus of productions across genres, scales, and traditions. We invite participation in defining this corpus.

Section 7Related Work

The Sim-to-Stage paradigm draws on, and is meant to extend, several adjacent literatures.

• Sim-to-Real in robotics is the immediate ancestor; the homomorphism in Section 3.2 is the formal statement of the inheritance.
• World models in machine learning provide simulation substrate techniques applicable to Sim-to-Stage rehearsal environments.
• Multi-agent systems literature informs the specialist production agent layer.
• Theater production scholarship — production journals, design process literature, the history of stage management — provides the ground truth against which any Sim-to-Stage system must be evaluated.
• Live production tooling — QLab, ETC Eos, Disguise, Capture, WYSIWYG, depence² — represents the departmental-island state of the practice that Sim-to-Stage proposes to unify under a shared truth source.

We will expand related-work coverage in subsequent revisions.

Section 8Discussion and Limitations

We are explicit about what this position paper does not claim and where the paradigm's open problems remain.

• The seven sub-gaps are unsolved. We have named them, not closed them. Each is a research direction, not a result.
• The seven architecture primitives are not unique. Other decompositions of a Sim-to-Stage system are possible. We claim ours is useful, not exclusive.
• Aesthetic judgment will remain partially human. Sub-gap 7 is, in our view, partially irreducible; we do not propose the paradigm fully formalizes theatrical taste, only that the DIP and the Confirmation Gate provide the right interfaces for human aesthetic judgment to enter the pipeline.
• The paradigm has not been validated at scale. The reference implementation StageR is in development. We are honest that this paper is, at its publication date, ahead of empirical evidence. Subsequent papers in this repository will report on empirical progress as it accumulates.
• The paradigm may not cover all live production. Some traditions — devised work, improvisation-heavy forms, audience-participatory performance — may fit awkwardly into the ATS / Confirmation Gate / Cue Compiler structure. We do not claim universal applicability.

Section 9Conclusion

We have introduced the Sim-to-Stage paradigm: a structural homomorphism of Sim-to-Real adapted to the domain of live performance production. We have defined the Sim-to-Stage gap as the paradigm's central open problem and decomposed it into seven sub-gaps. We have proposed seven architecture primitives as one useful decomposition that a mature Sim-to-Stage system will likely instantiate variants of. We have sketched the structure of a Sim-to-Stage benchmark.

We are at the beginning of this research program, not the end. We invite extensions, refinements, challenges, and alternative formulations from external research groups, theater companies, and live-production industrial implementers. The paradigm belongs to its community; this paper is a starting coordinate.

CitationHow to Cite

If you reference the term "Sim-to-Stage" or this paradigm in academic or industrial work, please cite:

@misc{simtostage2026,
  title  = {Sim-to-Stage: A Paradigm for Live Performance Production},
  author = {{Sim-to-Stage Research}},
  year   = {2026},
  url    = {https://simtostage.com},
  note   = {Position paper v0.1.
            Canonical repository: https://github.com/sim-to-stage/sim-to-stage}
}

A machine-readable CITATION.cff is provided in the canonical repository and powers GitHub's "Cite this repository" auto-export.

·License

This work is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). You may use, share, adapt, translate, and build upon this material — including for commercial purposes — provided you give appropriate credit. The paradigm belongs to its community; the license invites contribution.

·Versioning

This is Position Paper v0.1. We expect substantial revision in v0.2 and v0.3 as the paradigm matures, additional empirical evidence accumulates, and external contributors challenge or extend the formulation. We welcome issues and pull requests on the canonical repository.