Data-Intensive Astronomy

The Astronomical Data Query Language, a dialect of SQL extended with the geometry of the sky, lets astronomers ask precise questions of enormous catalogues — every source within a region, or every object matching a set of criteria. It is the language that the Virtual Observatory's Table Access Protocol serves.

Modern surveys have turned astronomy into a data-intensive science. Gaia charts over a billion stars, the Rubin Observatory's LSST will image the whole southern sky every few nights, and the Square Kilometre Array will generate data faster than any prior instrument — forcing new ways to store, move, and analyse information at petabyte scale.

The recorded lineage of a data product: which raw observations, software versions, and processing steps produced it. Provenance lets a result be traced back to its origins and trusted or reproduced — distinct from a persistent identifier, which names the product, provenance describes how it came to be.

The recognition that scientific software deserves the same care as any other research output: version control, testing, documentation, code review, and open licensing. As astronomy has grown more computational, sustaining and citing the software behind results has become a discipline in its own right.

The orchestration of a multi-step analysis — each stage's inputs, outputs, and dependencies made explicit — so that a whole computation can be re-run automatically and at scale. Workflow systems turn a tangle of scripts into a repeatable pipeline, the backbone of processing survey data through many stages.

The vision of an integrated, online ecosystem where data, software, and computing come together — the Virtual Observatory's interoperable archives joined to cloud science platforms and shared notebooks. In such an environment a researcher can find, access, and analyse data from many facilities without ever leaving the browser.