Research and Policy Analyst · EI Data Conversion
The documentation set
Code that few people can read is hard to change safely. The new process comes with about 550 pages of documentation, written by AI agents from the code and checked against it.
What was written · 1
The map
One page for the whole run, linking to every other page.
What was written · 2
A page for every program
What it reads, each piece of code, what stops it and what it writes.
What was written · 3
The data dictionary
Every output column: its meaning, its source and how it is made.
What was written · 4
The input guides
What the code reads from the raw inputs.
What was written · 5
The column chart
Every output column against every step, on one page.
How it stays true
Built, checked, fact-checked
The pages are built by a script from the programs. Each code excerpt is checked automatically against the program, and a second agent fact-checks each page against the code.
My part
I approve
No page is shared until I have approved it.
Every program has a page
Its inputs, each piece of code, what can stop it, what it writes and evidence from a run. A one-page map links them all.
Every column can be traced
A data dictionary says what each output column means, where it comes from and how it is made. A guide shows what the code reads from the raw inputs.
Built and checked, not written by hand
A script builds the pages from the programs. Each code excerpt is checked automatically against the program, and a second agent fact-checks each page against the code.
In more detail. The programs are Jupyter notebooks in R. The pages are made as web pages and as PDFs for printing. The second agent reads each page against the code and reports what it cannot confirm; those points are fixed before I approve the page. Each change to the code is recorded with its effect on a run, and a runbook says how to run the process.
An explanatory reconstruction of the documentation. The outlines are drawn without data.