Notes · 17 March 2026
Shadow scripts outside the orchestrator
If a notebook still lands the overnight load, your coverage matrix is incomplete — even if the orchestrator UI looks tidy.
Orchestration coverage audits begin with an uncomfortable inventory: every cron, notebook, and vendor webhook that still mutates production data. Teams are often surprised by how many critical paths live beside the DAG catalogue rather than inside it.
Shadow scripts are not always negligence. They start as temporary bridges and outlive the project that created them. The risk is operational blindness — no shared retry policy, no consistent logging, no single place to pause the estate during an incident.
We score migration candidates by blast radius and change frequency, not by how embarrassing the script looks. Leadership gets a short risk note; engineering gets a sequenced plan. That pairing keeps coverage work from becoming a museum of tidy DAGs while the real load still runs elsewhere.