Data Officer · Paper excerpt

An Improved Data
Exploration Pipeline

I argued that reproducible preparation gives analysts more time to explore their data and communicate useful findings.

Edited excerpt from the paper · pages 2–4Preparation · exploration · communication
Prepare, explore, communicateImport, tidy and manage data; repeatedly transform, visualize and model it; communicate the resulting analysis. This is a conceptual redraw of the framework in Thomas’s paper. A FRAMEWORK FOR REPRODUCIBLE ANALYSIS PrepareMake inputs usable Import Tidy Manage ExploreAsk, test and refine Transform Visualize Model Communicate the finding Automate reproducible steps.Keep room for judgement.

Prepare

Import → Tidy → Manage

Explore

Transform · Visualize · Model

Repeat as the questions develop.

Communicate

Explain the finding.

Framework redrawn from Figure 1 of the paper: prepare the inputs, explore through a repeating cycle, then communicate the finding. No operational data is shown.
01 / RETRIEVE

Make preparation reproducible

Use programmatic access and tidy structures to make retrieval and transformation reproducible across collection cycles.

02 / MANAGE

Keep the rules with the data

Use structured storage, explicit relationships and verification rules to support consistent calculations and readable outputs.

03 / DEVELOP

Invest in exploration

Choose tools for the problem, assess the process and keep learning so more effort can go into analysis and communication.

The paper describes programmatic retrieval and structured data management already in use, then argues for continued investment in tools, learning and exploration.

Condensed and edited from An Improved Data Exploration Pipeline, sections 2–6. The paper cites R for Data Science for its exploration framework.