High Level Overview
Amazon S3 has a maximum signature expiry of 7 days. In order to upload signed URLs to Kili Technology that don’t expire, we recommend proxying URLs through an endpoint on your server.
The exercise below demonstrates how to proxy through a simple flask app. We provided a one-click deploy through Heroku, however, you could also build this simple handler into your existing web service.
Step 1: Deploy a proxy endpoint
This endpoint will accept a signed URL with our JWT secret and will return a new signed s3 URL to an asset.
First, you’ll need to get IAM information to be able to create pre-signed URLs:
- a bucket name.
Make sure this IAM user can LIST and GET files in the bucket.
Check out the example proxy we made, https://github.com/kili-technology/signed-url-example. You can deploy it with one click here:
Step 2: Generate signed URLs pointing at our proxy
For each asset in our s3 bucket will generate a signed URL with our JWT secret that points to our server endpoint.
From Heroku, get the host URL of your new app by “open app”. Then, get the generated secret (settings > reveal config vars).
git clone https://github.com/kili-technology/generate-tokenized-urls cd generate-tokenized-urls/ // confirm you have node.js installed node --version npm install node cli.js --bucket <your-aws-bucket-name> --host https://<your-new-heroku-url>.herokuapp.com/ --secret <heroku-generated-config-secret> --output kili-import.json
Then you can import the urls in
kili-import.json to Kili and you’re good to go.