Throughout this guide, we explored the major components of a reproducible data acquisition workflow, including study discovery, accession systems, metadata acquisition, data download, validation, cloud storage, and reference dataset assembly.
This chapter integrates those components through a practical case study inspired by a common challenge in microbiome research:
How can we build a healthy reference microbiome dataset from public repositories?
The objective is not to perform downstream microbiome analysis, but to demonstrate the acquisition workflow required to construct a reusable reference dataset.
The goal is to assemble a healthy human gut microbiome reference dataset suitable for future comparative analyses.
Potential applications include:
Before searching repositories, define the target cohort.
These criteria establish the scope of the reference dataset.
The next step is identifying candidate studies.
flowchart TD
A[Healthy Reference Objective]
--> B[Repository Search]
B --> C[Candidate Studies]
C --> D[Study Evaluation]flowchart TD A[Healthy Reference Objective] --> B[Repository Search] B --> C[Candidate Studies] C --> D[Study Evaluation]
Potential repositories include:
At this stage, the objective is to identify studies that may contain eligible samples.
For each candidate study:
flowchart TD
A[BioProject]
--> B[BioSamples]
B --> C[Experiments]
C --> D[Runs]flowchart TD A[BioProject] --> B[BioSamples] B --> C[Experiments] C --> D[Runs]
Accession identifiers provide the link between studies, samples, metadata, and downloadable sequencing data.
Metadata determine whether samples satisfy project requirements.
| Variable | Example |
|---|---|
| Health Status | Healthy |
| Body Site | Stool |
| Age | Adult |
| Antibiotic Use | No |
| Host | Human |
Metadata acquisition often becomes the most time-consuming stage of the workflow.
A typical filtering process may resemble:
2,000 Samples
↓
Remove Disease Samples
↓
Remove Antibiotic-Treated Samples
↓
Remove Missing Metadata
↓
850 Eligible SamplesFiltering transforms candidate studies into a defined cohort.
Once eligible samples are identified:
Eligible Samples
↓
Retrieve Run Accessions
↓
Download Data
↓
Organize FilesDownloaded data should be stored using a reproducible directory structure.
Validation confirms:
Only validated samples should proceed to dataset assembly.
Validated samples are combined into a unified resource.
flowchart TD
A[Validated Study 1]
--> D[Reference Dataset]
B[Validated Study 2]
--> D
C[Validated Study 3]
--> Dflowchart TD A[Validated Study 1] --> D[Reference Dataset] B[Validated Study 2] --> D C[Validated Study 3] --> D
Metadata are harmonized and provenance information is preserved.
The resulting dataset contains:
The dataset is now suitable for downstream analysis and interpretation.
This case study highlights several important principles:
flowchart TD
A[Public Data Landscape]
--> B[Study Discovery]
B --> C[Accession Systems]
C --> D[Metadata Acquisition]
D --> E[Data Download]
E --> F[Data Validation]
F --> G[Cloud Storage and Transfer]
G --> H[Reference Dataset Assembly]
H --> I[Healthy Reference Dataset]flowchart TD A[Public Data Landscape] --> B[Study Discovery] B --> C[Accession Systems] C --> D[Metadata Acquisition] D --> E[Data Download] E --> F[Data Validation] F --> G[Cloud Storage and Transfer] G --> H[Reference Dataset Assembly] H --> I[Healthy Reference Dataset]
The AlphaBiomics case study demonstrates how each component of the CDI Data Acquisition System contributes to the creation of a reproducible reference dataset.
The workflow presented in this guide provides a foundation for many downstream activities, including microbiome analysis, RNA-Seq studies, benchmarking projects, clinical evidence generation, and commercial product development.
The acquisition process is complete. The resulting reference dataset is now ready to support the next stage of scientific investigation.