Skip to content

Hosting ​

The viewer and data bundles are static files and can be hosted together or separately. The hosted viewer at mictott.github.io/spatialscape can open remote bundles when the data host provides the headers listed below.

What a data host must do ​

  1. CORS: allow GET and HEAD from the viewer's origin (or *), allow the Range request header, and expose Content-Range, Content-Length, Accept-Ranges and ETag.
  2. HTTP Range requests must return 206 Partial Content. Sharded Zarr depends on them.
  3. Serve .json as application/json; everything else can be application/octet-stream.

Check any host with:

bash
spatialscape validate https://host/path/my_atlas

Cloudflare R2 ​

R2 can serve public bundles through a custom domain. Check R2 pricing for storage, request and transfer charges before choosing a host.

  1. Create a bucket and upload one bundle folder at a time: rclone copy bundles/my_atlas r2:bucket/my_atlas (not bundles/ itself, whose .cache/ sibling is local only).
  2. Attach a custom domain (the r2.dev endpoint is rate-limited and not cached).
  3. Set the bucket CORS policy:
json
[{"AllowedOrigins": ["*"], "AllowedMethods": ["GET", "HEAD"],
  "AllowedHeaders": ["Range", "Content-Type"],
  "ExposeHeaders": ["Content-Range", "Content-Length", "Accept-Ranges", "ETag"],
  "MaxAgeSeconds": 86400}]

AWS S3 + CloudFront ​

Use a private S3 bucket with a CloudFront distribution to serve the bundles. Apply the CORS policy at CloudFront so it is included on cached responses. The following commands create the bucket, origin access control and response headers policy. Choose a unique bucket name and export it as BUCKET first. See CloudFront pricing and S3 pricing for the costs that apply to your account.

bash
aws s3api create-bucket --bucket $BUCKET --region us-east-1
aws s3api put-public-access-block --bucket $BUCKET --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"
aws s3api put-bucket-cors --bucket $BUCKET --cors-configuration '{"CORSRules":[{"AllowedOrigins":["*"],"AllowedMethods":["GET","HEAD"],"AllowedHeaders":["Range","Content-Type","Origin"],"ExposeHeaders":["Content-Range","Content-Length","Accept-Ranges","ETag"],"MaxAgeSeconds":86400}]}'
OAC=$(aws cloudfront create-origin-access-control --origin-access-control-config "Name=$BUCKET,SigningProtocol=sigv4,SigningBehavior=always,OriginAccessControlOriginType=s3" --query OriginAccessControl.Id --output text)
RHP=$(aws cloudfront create-response-headers-policy --response-headers-policy-config '{"Name":"spatialscape-cors-range","CorsConfig":{"AccessControlAllowOrigins":{"Quantity":1,"Items":["*"]},"AccessControlAllowHeaders":{"Quantity":3,"Items":["Range","Content-Type","Origin"]},"AccessControlAllowMethods":{"Quantity":3,"Items":["GET","HEAD","OPTIONS"]},"AccessControlAllowCredentials":false,"AccessControlExposeHeaders":{"Quantity":4,"Items":["Content-Range","Content-Length","Accept-Ranges","ETag"]},"AccessControlMaxAgeSec":86400,"OriginOverride":true}}' --query ResponseHeadersPolicy.Id --output text)

Then create the distribution in the console (the CLI needs a long JSON for this): origin $BUCKET.s3.us-east-1.amazonaws.com with the origin access control $OAC, the managed CachingOptimized cache policy, the managed CORS-S3Origin origin request policy, the response headers policy spatialscape-cors-range, GET, HEAD, OPTIONS allowed, and index.html as the default root object. Note its id and domain, then let it read the bucket:

bash
export ID=<the distribution id, E1ABC...>
export DISTRIBUTION=$(aws cloudfront get-distribution --id $ID --query Distribution.DomainName --output text)   # dxxxx.cloudfront.net
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
aws s3api put-bucket-policy --bucket $BUCKET --policy "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"cloudfront.amazonaws.com\"},\"Action\":\"s3:GetObject\",\"Resource\":\"arn:aws:s3:::$BUCKET/*\",\"Condition\":{\"StringEquals\":{\"AWS:SourceArn\":\"arn:aws:cloudfront::$ACCOUNT:distribution/$ID\"}}}]}"

With GetObject alone, S3 answers 403 rather than 404 for a key that is not there; the viewer treats that 403 as "absent" for the files a bundle may lack (older bundles have no gmin array), so this policy is enough. Add s3:ListBucket on arn:aws:s3:::$BUCKET to the same statement if you want true 404s in the browser's console.

Upload each bundle folder (not bundles/ itself: its .cache/ sibling stays home) and the site, built with data_url set to the distribution's domain:

bash
aws s3 sync bundles/my_atlas s3://$BUCKET/my_atlas --exclude "*/build.json" --exclude "build-info.json"
spatialscape site build site.yaml -o site && aws s3 sync site s3://$BUCKET/      # site.yaml: data_url: https://dxxxx.cloudfront.net
spatialscape validate https://$DISTRIBUTION/my_atlas

CloudFront caches for a day; after re-uploading a changed bundle, invalidate its paths (aws cloudfront create-invalidation --distribution-id $ID --paths "/my_atlas/*"). The same datasets.json can be served from the distribution root and from a GitHub Pages copy of the viewer: the data is reached cross-origin either way. Public research datasets may be eligible for the AWS Open Data Sponsorship Program; check its application and eligibility requirements.

GitHub Pages ​

Fine for the app and for bundles that fit the 1 GB site limit; it does not guarantee Range support, so the bundles must be unsharded (sharded: false in the dataset yaml, or build --no-shard), which makes each gene its own file. Publish your own site has the branch-and-push route; site build writes the .nojekyll file Pages needs.

Your own server ​

Any static server that supports Range requests works (nginx, Apache, Caddy all do). Add the CORS headers above. spatialscape serve is a development server with both and is not meant for production traffic.

Globus ​

Globus Connect Server v5 collections can expose files over HTTPS. A guest collection with anonymous access gives plain URLs that work for curl and, with CORS enabled on the collection, for the viewer as well. Useful when the authoritative copy already lives on institutional storage.

Deploying the viewer ​

You do not need Node. spatialscape site build writes a folder with the viewer, the gallery and your bundles (or references to bundles on object storage); upload it as-is. The full walkthrough is in Publish your own site. Developers who change the app build it with npm run build and deploy app/dist; the repository's Pages workflow does this for the hosted viewer and the docs.

MIT licensed.