Building and validating
spatialscape build dlpfc.yaml # -> bundles/dlpfc next to the yaml (or -o <dir>)build updates the bundle from the configuration and source files, validates it and renders thumbnails. Rerun it after editing the YAML or replacing an input. Unchanged samples are reused; names, colors and display defaults are written to the dataset metadata. The table below lists which changes rebuild sample data.
What happens per sample
- Read the AnnData / SpatialData table (R objects were converted first, see below): only the expression layer the sample uses,
obs,varand theobsmkeys it needs are loaded, not every layer and pairwise matrix in the file. Resolvefields: autoandimages: auto, declare any undeclared fields. An optional layer whose file is not there (an image,polygons.path, a segmentation,molecules.path) is left out with a note, not an error: the sample builds from its object alone and gains the layer on the nextbuildonce the file exists. The expression file itself is never optional. - Expression: detect counts vs log-normalized, drop all-zero genes, compute each gene's range, quantize to 8 bits (
round(255 · x / max)for a non-negative gene, symmetric around zero for a signed feature, with any non-zero value kept ≥ 1), and write a gene-major Zarr v3 array sharded into ~128 MB files. Only the stored values are quantized (one pass over the non-zeros, scattered straight into the shards in the shuffled row order); the matrix is never densified. Optionally keep a float16 copy (keep_f16). Tissue sections also getrows.zarr, the same codes stored one cell at a time along a Morton curve, so a test on a lasso fetches only its cells (expression.rows; off for embeddings unless asked). - Geometry: convert coordinates to microns, apply
transform, shift the origin to the bounding box corner, record the bounding box and a default point radius per platform. - Observations: shuffle rows once with a fixed seed (the viewer draws a prefix of the shuffled rows at low zoom), write positions, categorical codes against the dataset vocabularies, numeric columns, and the cell ids in blocks.
- Cell shapes, segmentations and molecules (when the sample has them:
polygonsor anobsmarray of boundaries,segmentations,molecules): matchcell_idto the object and cut the shapes and the transcripts into spatial tiles. - Images: convert to an OME-Zarr pyramid with the same flip/rotation as the points, record pixel size and offset in microns. A pyramidal source (OME-Zarr, pyramidal or OME-TIFF) is read at the level
max_sizeasks for and downsampled in strips, so a full-resolution scan is never decoded whole. - Fingerprint: write
build.jsoninto the sample folder (what the next build compares against).
Then the dataset-level files: manifest.json, genes.json, features.json, thumbnail.png, and thumbnail.png per sample.
What a rebuild does
Every built sample folder carries build.json, a fingerprint of what went into it: the resolved sample settings, the size and modification time of each input file, the declarations of the fields it maps (type, vocabulary, category order, aliases), the shard settings and the format version. The next build recomputes the fingerprint from the yaml and the files and decides per sample:
| Situation | Action | Reason shown |
|---|---|---|
| not in the bundle yet | build | new |
| an input file changed (re-exported h5ad, new image next to it, edited parquet) | build | source changed |
| its settings changed (fields mapping, coords, microns, transform, images, polygons, expression, ...) | build | settings changed |
a field it maps was given a different type, vocabulary, categories or aliases | build | field declarations changed |
sharded or shard_genes changed (also --no-shard on or off) | build | sharding changed, shard size changed |
| built before the row store existed, and the sample wants one | build | row store added |
| the bundle format or build recipe changed with a new spatialscape version | build | format changed |
| none of the above | skip | unchanged |
Samples the yaml no longer lists are deleted. Names, group labels, the palette, the opening gene and annotation, the layout, feature groups and thumbnail_sample are not part of the fingerprint: they live in manifest.json, which every build rewrites together with genes.json, features.json and the thumbnails. Renaming a field or changing its palette therefore does not rebuild the sample arrays.
build --dry-runprints this table for the current state and writes nothing.build --forcerebuilds every sample.- R objects listed under
samples:are converted intobundles/.cache/<id>/on the first build and reused afterwards; the cache is replaced when the object or its conversion settings change (see R objects as samples). - A bundle built before fingerprints existed is rebuilt once in full, then behaves as above.
Vocabularies stay stable
When some samples are kept, the vocabularies they use are kept too: a rebuilt sample's labels get the same codes as before and new labels are appended, so colors never shift under samples that did not change. When every sample is rebuilt, vocabularies start fresh in the declared order.
Other commands on a bundle
| Task | Command |
|---|---|
| Re-render thumbnails around another section | set thumbnail_sample and build, or spatialscape thumbnails bundles/dlpfc --hero vis_151673 |
Validation
build ends with the same checks as spatialscape validate bundle:
- manifest parses; every referenced vocabulary, field and sample exists
- expression arrays have the right shape and gene-major chunking, and one chunk reads back
- observation arrays, id blocks and polygon files match
nObs; every cell shape tile decodes to the rings its index claims, of rows inside the sample - categorical codes are below their vocabulary's size
- images open and match their recorded size
Against a URL, validate also fetches the manifest and one expression chunk with a Range header and reports missing CORS headers or a non-206 response. Run it after every upload.
Example bundle sizes
These examples give approximate sizes; gene counts, sparsity, compression and optional layers affect the result.
| Data | Per sample |
|---|---|
| Xenium, 250k cells × 366 genes | 20 to 30 MB |
| Visium, 30k spots × 36k genes | 90 to 100 MB (90% zeros compress well) |
| snRNA-seq, 15k nuclei × 34k genes | about 110 MB |
| Visium lowres H&E OME-Zarr | 1 to 3 MB |
The first page view of a dataset downloads positions (8 bytes per cell), the default annotation (2 bytes per cell) and one gene (1 byte per cell). Everything else loads on demand.