Bundle format
formatVersion: 1. A bundle is a directory of static files; everything is addressable by URL with plain GET and Range requests.
Stability
Format 1 is final. Everything this page lists, every file, array, shape, dtype, encoding and precision, keeps its meaning: a bundle built today opens in every later viewer, and a reader written against this page keeps reading bundles built later.
- Changes within format 1 only add. A new optional file or manifest key may appear; a reader ignores keys it does not know, and nothing listed here changes meaning. A change that would break a reader bumps
formatVersionto 2, and the viewer refuses a version it does not know. - Required and optional. Required:
manifest.json,genes.json,features.json, and per samplegenes.json,expr.zarr/{u8, gmax, gmin},obs.zarr/{xy, order, cat, num}andids/. Optional, each announced by its manifest entry:rows.zarr,zero_genes.json,f16, images,shapes,segmentations/,molecules,tracks/,comparisons/,thumbnail.png.build.jsonandbuild-info.jsonare build aids, not part of the format: a reader never needs them and an upload may leave them out. Bundles built before 0.6 have no row store and no zero-gene list and may carry whole-samplepolygonsfiles instead of shape tiles; the viewer still reads them. - Conformance.
spatialscape validate <bundle | url>checks a bundle against this page: shapes and dtypes, gene-major chunking, codes within their vocabularies, tile indices that decode, id blocks, the row store, the zero-gene list, images, and over HTTP the CORS andRangebehaviour a static host must provide.python -m spatialscape.fidelitymeasures a bundle against the data it was built from, to the precision stated below. - Reference decoders.
python/src/spatialscape/{expression, rows, obs, shapes, molecules}.pywrite and read back every store;app/src/data/{worker, rows, shapeTiles, moleculeTiles}.tsare the viewer's readers. Where the two could disagree, the Python decoder is the specification.
<dataset>/
manifest.json dataset description (python: manifest.py, app: data/manifest.ts)
genes.json sorted union of plain gene symbols (autocomplete)
features.json {genes, groups:[{id, name, units, features:[{id, label}]}]}
thumbnail.png one section colored by the default annotation
build-info.json spatialscape version, config path, build time
samples/<sampleId>/
build.json fingerprint of the inputs and settings this sample was built from (incremental builds)
genes.json gene order of the expression rows for this sample
zero_genes.json features the sample measured and found zero everywhere: part of its panel, not stored
(manifest samples[].nZeroGenes counts them; the viewer draws them as zero, a test counts their cells)
expr.zarr/ Zarr v3 group
u8 uint8 (nGenes, nObs) chunk (1, nObs), shard (shardGenes, nObs), zstd
gmax float32 (nGenes) value = gmin + u8 / 255 * (gmax - gmin)
gmin float32 (nGenes) 0 for non-negative features; -gmax for features with negative values (quantized symmetrically, zero at the centre)
precision: a value is within (gmax - gmin) / 510 of the source, except a positive value under
half a step, stored as code 1 so an expressed cell never draws as zero; zeros are exact
(checked by `python -m spatialscape.fidelity`, see benchmarks/)
f16 float16 (nGenes, nObs) optional; not read by the viewer
rows.zarr/ Zarr v3 group, optional (`expression.rows`): the same codes one cell at a time
indptr uint32 (nObs + 1) stored values before each row of the store
gene uint16|uint32 (nnz) kept-gene index (row of u8) of every stored value, as the step from the row's previous id (first absolute); chunk 2^20, shard 32 chunks
value uint8 (nnz) the u8 code of that value; zeros are not stored
pos uint32 (nObs) position in the store of bundle row i; rows follow a Morton curve over xy
obs.zarr/ Zarr v3 group
xy float32 (nObs, 2) microns, sample-local, y down, origin at bbox min
order uint32 (nObs) source row of each bundle row (rows are shuffled once)
cat/<fieldId> uint16 (nObs) codes into manifest.vocabularies[field.vocabulary].categories
num/<fieldId> float32 (nObs)
ids/<block>.json obs names in blocks of idBlock (default 65536)
shapes.json cell shape tile index: {version, grid (µm per unit, 0.1), tile (µm), origin [x, y] µm,
nx, ny, reach (µm), layout "packed"|"files", tiles: [[ix, iy, offset, bytes, rings, vertices]]}
shapes.bin the tiles' zstd frames back to back, row-major (layout "packed", read with byte ranges);
shapes/<ix>_<iy>.bin or one file per tile (layout "files", from `build --no-shard`)
segmentations/<id>/shapes.json, shapes.bin another segmentation of the section, in the shape tile format; its rings are
numbered 0..n-1 (not cell rows); the manifest entry carries cells, vertices, tile, tiles, bytes, medianArea
molecules.json molecule tile index: {version, grid (µm per unit, 1/64), tile (µm), origin, nx, ny, genes
(names; a molecule's gene id indexes them), counts (per gene), count, flags ["assigned", "nucleus"],
layout, tiles: [[ix, iy, offset, bytes, molecules]]}
molecules.bin the tiles' zstd frames back to back, or molecules/<ix>_<iy>.bin
<imageId>.ome.zarr/ OME-Zarr NGFF 0.4 (zarr v2), axes c,y,x, 512² chunks (an OME-Zarr source passed through
keeps its own axes and chunks)
tracks/ ATAC samples built with `tracks:` (manifest samples[].tracks: genome, bin, units, all?, by)
all.bw every cell, binned (bigWig): built from fragments, or copied when `bigwigs.all` was given
<field>/<label>.bw one per category of each `tracks.by` field: insertions per million from fragments
(summed over runs before scaling), or exported bigWigs copied as they are
peaks.bed, links.bedpe, chrom.sizes
genes/index.json gene names per chromosome (search); genes/<chrom>.json: [name, strand, start, end, exon starts, exon ends]
thumbnail.pngCell shape tiles
A ring belongs to the tile holding its centroid and is never clipped; a reader widens its view by reach, the largest distance from a ring's centroid to one of its vertices. One tile, after zstd, is little endian, with every multi-byte stream split into byte planes (low byte first):
u32 rings, u32 vertices
row steps u32[rings], 4 planes bundle rows ascending; the first step is the first row itself
counts u8[rings] vertices per ring, the closing vertex not repeated
start x, y i16[rings] each, 2 planes first vertex, grid units from the tile origin
step x, y i16[vertices - rings] each, 2 planes steps between consecutive vertices of a ringThe tile size is the largest of 1024, 512, 256, 128 or 64 µm whose densest tile holds at most 2,048 rings. Every ring records its cell's bundle row, so the viewer colours shapes from the per-cell arrays it already holds. python/src/spatialscape/shapes.py writes and decodes it; app/src/data/shapeTiles.ts is the viewer's decoder.
Molecule tiles
Per sample, in the sample frame. The tile size is the largest of 512, 256, 128 or 64 µm whose densest tile holds at most 50,000 molecules. One tile, after zstd, little endian, multi-byte streams split into byte planes:
u32 n, u32 runs
run gene u16[runs], 2 planes gene ids ascending (indexes molecules.json genes)
run count u32[runs], 4 planes molecules of that gene
dx, dy i16[n] each, 2 planes steps between consecutive molecules on the 1/64 µm grid, the first from the tile
origin; within a gene the order is a Morton curve, so steps are short
flags u8[n] bit 0 assigned to a cell, bit 1 overlaps the nucleusAbout 3.5 bytes per molecule on Xenium data. python/src/spatialscape/molecules.py writes and decodes it; app/src/data/moleculeTiles.ts is the viewer's decoder.
Design notes
- Gene-major uint8. Coloring N cells by a gene uses N bytes of decoded values. Each gene has its own inner chunk; shards reduce the file count. Reading a gene may also require shard metadata when it is not cached. The 8-bit values are a lossy representation of the source matrix. Stored ranges reconstruct approximate values for coloring, tooltips and summaries; retain the source data for analysis.
- Rows for tests on a selection. A differential expression test on a lasso needs every gene of a few cells, the opposite access pattern, so a sample may carry the matrix a second time as sparse rows ordered along a space-filling curve: a compact region is a few contiguous runs of rows, fetched with a few range requests. It costs about as much as the gene-major store (on the sections we measured 0.96×: the dense store's zeros compress well, and the codes themselves are most of the rows);
expression.rows: autowrites it for tissue sections and not for embeddings. - Shuffled rows. Any prefix of an array is a uniform random subsample, so the viewer draws the first k rows at overview zoom with no extra data.
- Dataset-level vocabularies. Codes are shared across samples and platforms; the same label has the same code and color everywhere.
- Images carry their geometry in the manifest (
pixelSizeµm per level-0 pixel,translateµm), independent of NGFF transforms. NGFF 0.4 / zarr v2 is used because that is what the image renderer reads. - Reserved:
expr.kind: "csc"is declared in the schema for a future sparse layout; the viewer only readsu8today.
manifest.json
Field names are camelCase and mirrored one-to-one between the Python and TypeScript models.
type Manifest = {
formatVersion: 1; id: string; name: string; description?: string;
defaultGene?: string; defaultColor: {kind: "gene"|"field", gene?, field?};
defaultChannels?: string[]; // image channels drawn on open, by name; absent: the first six
layout: {mode: "grid"|"strip", gutterFraction: number, order: string[], columns?: number};
colormaps: string[];
vocabularies: Record<string, {categories: string[], colors: string[], aliases?: Record<string,string>}>;
fields: Array<{id, name, type: "categorical", vocabulary, description?}
| {id, name, type: "continuous", range?, colormap?, description?}>;
featureGroups: Array<{id, name, units?, count}>;
thumbnailSample?: string; // the sample drawn on thumbnail.png
meta: Array<{id, name, levels: string[]}>; // sample metadata keys the viewer can split the canvas by
comparisons: Array<{id, kind: "contrast"|"markers", name, description?, path, groups, contrast?, effect, units?,
population?: {field, value}, split?, link?, facet?: {column, field?},
levels: Array<{id, name, path, nGenes, nSignificant, nUnmatched, nReplicates, hasPseudobulk,
population?: {field, value}}>,
nGenes, nReplicates, nUnmatched, nMatrixGenes,
hasEffect, hasP, hasPadj, hasMean, hasPct, hasScore, hasPseudobulk}>;
samples: Array<{
id, name, platform, kind: "spatial"|"embedding", group?, meta?: Record<string, string>, nObs, nGenes,
bbox: [minx, miny, maxx, maxy], toDataset: number[9], pointRadius,
expr: {kind: "u8"|"csc", sharded, shardGenes, normalized?: "lognorm"|"log1p_cp10k"|"raw"}, hasF16, fields: string[],
images: Array<{id, name, path, kind, pixelSize, translate, size: [h, w], channels?: Array<{name, color, window}>, defaultOpacity}>,
shapes?: {path, cells, vertices, tile, tiles, bytes, medianArea?},
molecules?: {path, count, genes, tile, tiles, bytes},
segmentations: Array<{id, name, path, cells, vertices, tile, tiles, bytes, medianArea?}>,
rows?: {nnz, chunk, order: "morton"}, // rows.zarr
tracks?: {genome, bin, units, all?, by: Array<{field, categories: Record<string, string>}>, peaks?, genes?, links?, chromSizes},
nZeroGenes, suggestedGene?, idBlock
// polygons?: {path, cells, vertices, scale}: whole-sample polygons of bundles built before shape tiles (still drawn)
}>;
};comparisons/
One folder per comparisons: entry, next to samples/:
comparisons/<id>/
comparison.json the manifest entry (kind, groups, facet, levels[]) plus replicates per level:
{<level id>: [{id, group, subject, samples: [sample ids]}]}, the columns taken from each
table, the source file names and the match counts
matrix.json {genes, levels, stats, unmeasured}: the row order of matrix.zarr (every gene of every
level once), the level ids in column order, which statistics are present
matrix.zarr/<stat> float32 (genes, levels) per statistic (effect, padj, p, pct_in, pct_out, rank), NaN where
a level's table has no row for the gene; Blosc (zstd), chunked by 4096 genes, so the
panel's matrix is one fetch per statistic for most tables
levels/<level>/ one folder per level: "all" for a comparison without a facet, else one per cluster or
cell type (the level's label made file-safe)
results.json {genes, effect, p, padj, mean, pct_in, pct_out, score, unmeasured}: column arrays in one
order, rows sorted by padj then p then |effect| (a ranked list without statistics keeps
its file order); arrays the table lacks are null; unmeasured lists the row indices of
genes no sample measures
pseudobulk.zarr float32 (nGenes, nReplicates) in results order, Blosc (zstd), chunked by 1024 genes; only when
a pseudobulk table was given for the level
build.json fingerprint of the tables and settings (the incremental rule)Reading a bundle elsewhere
Python:
import zarr, json
g = zarr.open_group("bundle/samples/coronal/expr.zarr", mode="r")
genes = json.load(open("bundle/samples/coronal/genes.json"))
gmax = g["gmax"][:]
gad1 = g["u8"][genes.index("Gad1"), :] / 255 * gmax[genes.index("Gad1")]R (Rarr / pizzarr) reads the same arrays; obs.zarr/order maps rows back to the source object.
Each sample may carry suggestedGene, its most variable gene; build uses the first spatial sample's as the dataset defaultGene when default_gene is not set. The manifest may carry thumbnailSample, the sample drawn on thumbnail.png.
Next to the bundle
spatialscape build writes bundles/<id> by default and keeps one thing beside it that is not part of the bundle: bundles/.cache/<id>/<key>/ holds R objects converted into AnnData files (one folder per conversion key, with an index.json describing the source). Upload bundles/<id> alone; find_bundles and site build ignore the cache because it has no manifest.json.