Skip to content

Building and validating ​

bash
spatialscape build dlpfc.yaml              # -> bundles/dlpfc next to the yaml (or -o <dir>)

build updates the bundle from the configuration and source files, validates it and renders thumbnails. Rerun it after editing the YAML or replacing an input. Unchanged samples are reused; names, colors and display defaults are written to the dataset metadata. The table below lists which changes rebuild sample data.

What happens per sample ​

  1. Read the AnnData / SpatialData table (R objects were converted first, see below): only the expression layer the sample uses, obs, var and the obsm keys it needs are loaded, not every layer and pairwise matrix in the file. Resolve fields: auto and images: auto, declare any undeclared fields. An optional layer whose file is not there (an image, polygons.path, a segmentation, molecules.path) is left out with a note, not an error: the sample builds from its object alone and gains the layer on the next build once the file exists. The expression file itself is never optional.
  2. Expression: detect counts vs log-normalized, drop all-zero genes, compute each gene's range, quantize to 8 bits (round(255 · x / max) for a non-negative gene, symmetric around zero for a signed feature, with any non-zero value kept ≥ 1), and write a gene-major Zarr v3 array sharded into ~128 MB files. Only the stored values are quantized (one pass over the non-zeros, scattered straight into the shards in the shuffled row order); the matrix is never densified. Optionally keep a float16 copy (keep_f16). Tissue sections also get rows.zarr, the same codes stored one cell at a time along a Morton curve, so a test on a lasso fetches only its cells (expression.rows; off for embeddings unless asked).
  3. Geometry: convert coordinates to microns, apply transform, shift the origin to the bounding box corner, record the bounding box and a default point radius per platform.
  4. Observations: shuffle rows once with a fixed seed (the viewer draws a prefix of the shuffled rows at low zoom), write positions, categorical codes against the dataset vocabularies, numeric columns, and the cell ids in blocks.
  5. Cell shapes, segmentations and molecules (when the sample has them: polygons or an obsm array of boundaries, segmentations, molecules): match cell_id to the object and cut the shapes and the transcripts into spatial tiles.
  6. Images: convert to an OME-Zarr pyramid with the same flip/rotation as the points, record pixel size and offset in microns. A pyramidal source (OME-Zarr, pyramidal or OME-TIFF) is read at the level max_size asks for and downsampled in strips, so a full-resolution scan is never decoded whole.
  7. Fingerprint: write build.json into the sample folder (what the next build compares against).

Then the dataset-level files: manifest.json, genes.json, features.json, thumbnail.png, and thumbnail.png per sample.

What a rebuild does ​

Every built sample folder carries build.json, a fingerprint of what went into it: the resolved sample settings, the size and modification time of each input file, the declarations of the fields it maps (type, vocabulary, category order, aliases), the shard settings and the format version. The next build recomputes the fingerprint from the yaml and the files and decides per sample:

SituationActionReason shown
not in the bundle yetbuildnew
an input file changed (re-exported h5ad, new image next to it, edited parquet)buildsource changed
its settings changed (fields mapping, coords, microns, transform, images, polygons, expression, ...)buildsettings changed
a field it maps was given a different type, vocabulary, categories or aliasesbuildfield declarations changed
sharded or shard_genes changed (also --no-shard on or off)buildsharding changed, shard size changed
built before the row store existed, and the sample wants onebuildrow store added
the bundle format or build recipe changed with a new spatialscape versionbuildformat changed
none of the aboveskipunchanged

Samples the yaml no longer lists are deleted. Names, group labels, the palette, the opening gene and annotation, the layout, feature groups and thumbnail_sample are not part of the fingerprint: they live in manifest.json, which every build rewrites together with genes.json, features.json and the thumbnails. Renaming a field or changing its palette therefore does not rebuild the sample arrays.

  • build --dry-run prints this table for the current state and writes nothing.
  • build --force rebuilds every sample.
  • R objects listed under samples: are converted into bundles/.cache/<id>/ on the first build and reused afterwards; the cache is replaced when the object or its conversion settings change (see R objects as samples).
  • A bundle built before fingerprints existed is rebuilt once in full, then behaves as above.

Vocabularies stay stable

When some samples are kept, the vocabularies they use are kept too: a rebuilt sample's labels get the same codes as before and new labels are appended, so colors never shift under samples that did not change. When every sample is rebuilt, vocabularies start fresh in the declared order.

Other commands on a bundle ​

TaskCommand
Re-render thumbnails around another sectionset thumbnail_sample and build, or spatialscape thumbnails bundles/dlpfc --hero vis_151673

Validation ​

build ends with the same checks as spatialscape validate bundle:

  • manifest parses; every referenced vocabulary, field and sample exists
  • expression arrays have the right shape and gene-major chunking, and one chunk reads back
  • observation arrays, id blocks and polygon files match nObs; every cell shape tile decodes to the rings its index claims, of rows inside the sample
  • categorical codes are below their vocabulary's size
  • images open and match their recorded size

Against a URL, validate also fetches the manifest and one expression chunk with a Range header and reports missing CORS headers or a non-206 response. Run it after every upload.

Example bundle sizes ​

These examples give approximate sizes; gene counts, sparsity, compression and optional layers affect the result.

DataPer sample
Xenium, 250k cells × 366 genes20 to 30 MB
Visium, 30k spots × 36k genes90 to 100 MB (90% zeros compress well)
snRNA-seq, 15k nuclei × 34k genesabout 110 MB
Visium lowres H&E OME-Zarr1 to 3 MB

The first page view of a dataset downloads positions (8 bytes per cell), the default annotation (2 bytes per cell) and one gene (1 byte per cell). Everything else loads on demand.

MIT licensed.