Hosting
The viewer and data bundles are static files and can be hosted together or separately. The hosted viewer at mictott.github.io/spatialscape can open remote bundles when the data host provides the headers listed below.
What a data host must do
- CORS: allow
GETandHEADfrom the viewer's origin (or*), allow theRangerequest header, and exposeContent-Range,Content-Length,Accept-RangesandETag. - HTTP Range requests must return
206 Partial Content. Sharded Zarr depends on them. - Serve
.jsonasapplication/json; everything else can beapplication/octet-stream.
Check any host with:
spatialscape validate https://host/path/my_atlasCloudflare R2
R2 can serve public bundles through a custom domain. Check R2 pricing for storage, request and transfer charges before choosing a host.
- Create a bucket and upload one bundle folder at a time:
rclone copy bundles/my_atlas r2:bucket/my_atlas(notbundles/itself, whose.cache/sibling is local only). - Attach a custom domain (the
r2.devendpoint is rate-limited and not cached). - Set the bucket CORS policy:
[{"AllowedOrigins": ["*"], "AllowedMethods": ["GET", "HEAD"],
"AllowedHeaders": ["Range", "Content-Type"],
"ExposeHeaders": ["Content-Range", "Content-Length", "Accept-Ranges", "ETag"],
"MaxAgeSeconds": 86400}]AWS S3 + CloudFront
Use a private S3 bucket with a CloudFront distribution to serve the bundles. Apply the CORS policy at CloudFront so it is included on cached responses. The following commands create the bucket, origin access control and response headers policy. Choose a unique bucket name and export it as BUCKET first. See CloudFront pricing and S3 pricing for the costs that apply to your account.
aws s3api create-bucket --bucket $BUCKET --region us-east-1
aws s3api put-public-access-block --bucket $BUCKET --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"
aws s3api put-bucket-cors --bucket $BUCKET --cors-configuration '{"CORSRules":[{"AllowedOrigins":["*"],"AllowedMethods":["GET","HEAD"],"AllowedHeaders":["Range","Content-Type","Origin"],"ExposeHeaders":["Content-Range","Content-Length","Accept-Ranges","ETag"],"MaxAgeSeconds":86400}]}'
OAC=$(aws cloudfront create-origin-access-control --origin-access-control-config "Name=$BUCKET,SigningProtocol=sigv4,SigningBehavior=always,OriginAccessControlOriginType=s3" --query OriginAccessControl.Id --output text)
RHP=$(aws cloudfront create-response-headers-policy --response-headers-policy-config '{"Name":"spatialscape-cors-range","CorsConfig":{"AccessControlAllowOrigins":{"Quantity":1,"Items":["*"]},"AccessControlAllowHeaders":{"Quantity":3,"Items":["Range","Content-Type","Origin"]},"AccessControlAllowMethods":{"Quantity":3,"Items":["GET","HEAD","OPTIONS"]},"AccessControlAllowCredentials":false,"AccessControlExposeHeaders":{"Quantity":4,"Items":["Content-Range","Content-Length","Accept-Ranges","ETag"]},"AccessControlMaxAgeSec":86400,"OriginOverride":true}}' --query ResponseHeadersPolicy.Id --output text)Then create the distribution in the console (the CLI needs a long JSON for this): origin $BUCKET.s3.us-east-1.amazonaws.com with the origin access control $OAC, the managed CachingOptimized cache policy, the managed CORS-S3Origin origin request policy, the response headers policy spatialscape-cors-range, GET, HEAD, OPTIONS allowed, and index.html as the default root object. Note its id and domain, then let it read the bucket:
export ID=<the distribution id, E1ABC...>
export DISTRIBUTION=$(aws cloudfront get-distribution --id $ID --query Distribution.DomainName --output text) # dxxxx.cloudfront.net
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
aws s3api put-bucket-policy --bucket $BUCKET --policy "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"cloudfront.amazonaws.com\"},\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::$BUCKET/*\",\"Condition\":{\"StringEquals\":{\"AWS:SourceArn\":\"arn:aws:cloudfront::$ACCOUNT:distribution/$ID\"}}}]}"With GetObject alone, S3 answers 403 rather than 404 for a key that is not there; the viewer treats that 403 as "absent" for the files a bundle may lack (older bundles have no gmin array), so this policy is enough. Add s3:ListBucket on arn:aws:s3:::$BUCKET to the same statement if you want true 404s in the browser's console.
Upload each bundle folder (not bundles/ itself: its .cache/ sibling stays home) and the site, built with data_url set to the distribution's domain:
aws s3 sync bundles/my_atlas s3://$BUCKET/my_atlas --exclude "*/build.json" --exclude "build-info.json"
spatialscape site build site.yaml -o site && aws s3 sync site s3://$BUCKET/ # site.yaml: data_url: https://dxxxx.cloudfront.net
spatialscape validate https://$DISTRIBUTION/my_atlasCloudFront caches for a day; after re-uploading a changed bundle, invalidate its paths (aws cloudfront create-invalidation --distribution-id $ID --paths "/my_atlas/*"). The same datasets.json can be served from the distribution root and from a GitHub Pages copy of the viewer: the data is reached cross-origin either way. Public research datasets may be eligible for the AWS Open Data Sponsorship Program; check its application and eligibility requirements.
GitHub Pages
Fine for the app and for bundles that fit the 1 GB site limit; it does not guarantee Range support, so the bundles must be unsharded (sharded: false in the dataset yaml, or build --no-shard), which makes each gene its own file. Publish your own site has the branch-and-push route; site build writes the .nojekyll file Pages needs.
Your own server
Any static server that supports Range requests works (nginx, Apache, Caddy all do). Add the CORS headers above. spatialscape serve is a development server with both and is not meant for production traffic.
Globus
Globus Connect Server v5 collections can expose files over HTTPS. A guest collection with anonymous access gives plain URLs that work for curl and, with CORS enabled on the collection, for the viewer as well. Useful when the authoritative copy already lives on institutional storage.
Deploying the viewer
You do not need Node. spatialscape site build writes a folder with the viewer, the gallery and your bundles (or references to bundles on object storage); upload it as-is. The full walkthrough is in Publish your own site. Developers who change the app build it with npm run build and deploy app/dist; the repository's Pages workflow does this for the hosted viewer and the docs.