Import data for labeling

Once you've created a labeling project, it’s time to add some data! The Centaur API currently supports importing data from Amazon S3, Azure and Google Cloud Storage (for image or video projects) or creating assets directly (for text or HTML projects).

🚧

Hidden File Filtering (Dotfiles)

Files with names beginning with . will be skipped during import, as this typically indicates a hidden file. If you wish to label files with names beginning with ., please rename them first.

📘

API Credentials

The examples in this guide require having API credentials set up. If you haven't done so already, please follow the instructions in API credentials and Initiate API session..

Import image or video assets

Media assets can be imported from an S3 location by making a request containing the path to the S3 location, eg. s3://my-bucket/path/to/data/directory.

Prior to creating an import, you will need to add a policy to your S3 bucket allowing the Centaur importer to access your data. The policy can be retrieved with a GET request to /imports/public/v1/policy/get:

curl --location --request GET 'https://api.centaurlabs.com/imports/public/v1/policy/get?uri={S3_URI}&source_type=s3' \
--header 'X-API-KEY: YOUR_API_KEY_HERE' \
--header 'Authorization: Bearer YOUR_TOKEN_HERE' \
--header 'Content-Type: application/json'

After applying this policy to your bucket, confirm that Centaur can access your bucket with a GET request to /imports/public/v1/validateAccess:

curl --location --request GET 'https://api.centaurlabs.com/imports/public/v1/validateAccess?uri={S3_URI}&source_type=s3' \
--header 'X-API-KEY: YOUR_API_KEY_HERE' \
--header 'Authorization: Bearer YOUR_TOKEN_HERE' \
--header 'Content-Type: application/json'

Once access is confirmed, you can create an import from this source using a POST request to /imports/public/v1/add:

curl --location --request POST 'https://api.centaurlabs.com/imports/public/v1/add' \
--header 'X-API-KEY: YOUR_API_KEY_HERE' \
--header 'Authorization: Bearer YOUR_TOKEN_HERE' \
--header 'Content-Type: application/json' \
--data '{
  "uri": "{S3_URI}",
  "project_id": {PROJECT_ID},
  "source_type": "s3",
  "source_region": "us-east-1"
}'

Importing text assets directly

Text or HTML assets may be added to a project directly by sending a request with a JSON payload containing an array of objects with origin and content values for each asset. This POST request should be made to /assets/public/v1/add:

curl --location --request POST 'https://api.centaurlabs.com/assets/public/v1/add' \
--header 'X-API-KEY: YOUR_API_KEY_HERE' \
--header 'Authorization: Bearer YOUR_TOKEN_HERE' \
--header 'Content-Type: application/json' \
--data '{
  "project_id": {PROJECT_ID},
  "text_assets": [
    {"origin": "identifier_1", "content": "A drop in cells < normal was observed"},
    {"origin": "identifier_2", "content": "A drop in cells > normal was observed"}
  ]
}'

After import, text assets for a specific project may be listed with a GET request to /assets/public/v1/list:

curl --location --request GET 'https://api.centaurlabs.com/assets/public/v1/list?project_id={PROJECT_ID}' \
--header 'X-API-KEY: YOUR_API_KEY_HERE' \
--header 'Authorization: Bearer YOUR_TOKEN_HERE' \
--header 'Content-Type: application/json'

This will return a paginated list of assets. The number of assets returned defaults to 100, but may be changed by including a limit query parameter in the request (with a max of 1,000). If there are additional assets, the has_next value in the response will be true and the last_sequence_id from the response may be included in the following request in order to retrieve the next page.

List imports

You can list the imports for your project by using the project's unique identifier, project_id in a GET request to /imports/public/v1/list:

curl --location --request GET 'https://api.centaurlabs.com/imports/public/v1/list?project_id={PROJECT_ID}' \
--header 'X-API-KEY: YOUR_API_KEY_HERE' \
--header 'Authorization: Bearer YOUR_TOKEN_HERE' \
--header 'Content-Type: application/json'

You can request the status of a specific import by using the import's unique identifier, import_id in a GET request to /imports/public/v1/get:

curl --location --request GET 'https://api.centaurlabs.com/imports/public/v1/get?import_id={IMPORT_ID}' \
--header 'X-API-KEY: YOUR_API_KEY_HERE' \
--header 'Authorization: Bearer YOUR_TOKEN_HERE' \
--header 'Content-Type: application/json'