Hosting
The app is static and the data is static. They can live on different hosts, and the hosted viewer at mictott.github.io/spatialscape can open bundles from anywhere.
What a data host must do
- CORS: allow
GETandHEADfrom the viewer's origin (or*), allow theRangerequest header, and exposeContent-Range,Content-Length,Accept-RangesandETag. - HTTP Range requests must return
206 Partial Content. Sharded Zarr depends on them. - Serve
.jsonasapplication/json; everything else can beapplication/octet-stream.
Check any host with:
spatialscape validate https://host/path/my_atlasCloudflare R2 (recommended)
Free egress, so a popular dataset does not run up a bill; storage is about $0.015 per GB-month.
- Create a bucket and upload one bundle folder at a time:
rclone copy bundles/my_atlas r2:bucket/my_atlas(notbundles/itself, whose.cache/sibling is local only). - Attach a custom domain (the
r2.devendpoint is rate-limited and not cached). - Set the bucket CORS policy:
[{"AllowedOrigins": ["*"], "AllowedMethods": ["GET", "HEAD"],
"AllowedHeaders": ["Range", "Content-Type"],
"ExposeHeaders": ["Content-Range", "Content-Length", "Accept-Ranges", "ETag"],
"MaxAgeSeconds": 86400}]AWS S3 + CloudFront
A private bucket read by a CloudFront distribution: free egress up to a terabyte a month, byte ranges served natively, and a CORS policy applied by CloudFront so cached responses always carry the headers. Five calls with the AWS CLI set up the bucket and the two policies (BUCKET is a name you choose; export it first):
aws s3api create-bucket --bucket $BUCKET --region us-east-1
aws s3api put-public-access-block --bucket $BUCKET --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"
aws s3api put-bucket-cors --bucket $BUCKET --cors-configuration '{"CORSRules":[{"AllowedOrigins":["*"],"AllowedMethods":["GET","HEAD"],"AllowedHeaders":["Range","Content-Type","Origin"],"ExposeHeaders":["Content-Range","Content-Length","Accept-Ranges","ETag"],"MaxAgeSeconds":86400}]}'
OAC=$(aws cloudfront create-origin-access-control --origin-access-control-config "Name=$BUCKET,SigningProtocol=sigv4,SigningBehavior=always,OriginAccessControlOriginType=s3" --query OriginAccessControl.Id --output text)
RHP=$(aws cloudfront create-response-headers-policy --response-headers-policy-config '{"Name":"spatialscape-cors-range","CorsConfig":{"AccessControlAllowOrigins":{"Quantity":1,"Items":["*"]},"AccessControlAllowHeaders":{"Quantity":3,"Items":["Range","Content-Type","Origin"]},"AccessControlAllowMethods":{"Quantity":3,"Items":["GET","HEAD","OPTIONS"]},"AccessControlAllowCredentials":false,"AccessControlExposeHeaders":{"Quantity":4,"Items":["Content-Range","Content-Length","Accept-Ranges","ETag"]},"AccessControlMaxAgeSec":86400,"OriginOverride":true}}' --query ResponseHeadersPolicy.Id --output text)Then create the distribution in the console (the CLI needs a long JSON for this): origin $BUCKET.s3.us-east-1.amazonaws.com with the origin access control $OAC, the managed CachingOptimized cache policy, the managed CORS-S3Origin origin request policy, the response headers policy spatialscape-cors-range, GET, HEAD, OPTIONS allowed, and index.html as the default root object. Note its id and domain, then let it read the bucket:
export ID=<the distribution id, E1ABC...>
export DISTRIBUTION=$(aws cloudfront get-distribution --id $ID --query Distribution.DomainName --output text) # dxxxx.cloudfront.net
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
aws s3api put-bucket-policy --bucket $BUCKET --policy "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"cloudfront.amazonaws.com\"},\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::$BUCKET/*\",\"Condition\":{\"StringEquals\":{\"AWS:SourceArn\":\"arn:aws:cloudfront::$ACCOUNT:distribution/$ID\"}}}]}"With GetObject alone, S3 answers 403 rather than 404 for a key that is not there; the viewer treats that 403 as "absent" for the files a bundle may lack (older bundles have no gmin array), so this policy is enough. Add s3:ListBucket on arn:aws:s3:::$BUCKET to the same statement if you want true 404s in the browser's console.
Upload each bundle folder (not bundles/ itself: its .cache/ sibling stays home) and the site, built with data_url set to the distribution's domain:
aws s3 sync bundles/my_atlas s3://$BUCKET/my_atlas --exclude "*/build.json" --exclude "build-info.json"
spatialscape site build site.yaml -o site && aws s3 sync site s3://$BUCKET/ # site.yaml: data_url: https://dxxxx.cloudfront.net
spatialscape validate https://$DISTRIBUTION/my_atlasCloudFront caches for a day; after re-uploading a changed bundle, invalidate its paths (aws cloudfront create-invalidation --distribution-id $ID --paths "/my_atlas/*"). The same datasets.json can be served from the distribution root and from a GitHub Pages copy of the viewer: the data is reached cross-origin either way. For public scientific data, the AWS Open Data Sponsorship Program covers storage and egress entirely.
GitHub Pages
Fine for the app and for bundles that fit the 1 GB site limit; it does not guarantee Range support, so the bundles must be unsharded (sharded: false in the dataset yaml, or build --no-shard), which makes each gene its own file. Publish your own site has the branch-and-push route; site build writes the .nojekyll file Pages needs.
Your own server
Any static server that supports Range requests works (nginx, Apache, Caddy all do). Add the CORS headers above. spatialscape serve is a development server with both and is not meant for production traffic.
Globus
Globus Connect Server v5 collections can expose files over HTTPS. A guest collection with anonymous access gives plain URLs that work for curl and, with CORS enabled on the collection, for the viewer as well. Useful when the authoritative copy already lives on institutional storage.
Deploying the viewer
You do not need Node. spatialscape site build writes a folder with the viewer, a registry and your bundles (or references to bundles on object storage); upload it as-is. The full walkthrough is in Publish your own site. Developers who change the app build it with npm run build and deploy app/dist; the repository's Pages workflow does this for the demo and the docs.