Understanding Pre-labels

Prelabels are initial annotations provided to labelers to serve as a starting point for the annotation process.

Prelabels are available for the following tasks:

• Segmentation
• Range

Why Use Pre-labels?

Model Feedback: You may want to assess the accuracy of your model or a foundation model by comparing its outputs with human-corrected annotations.

Efficiency Gains: If most of a model’s predictions are correct, having pre-labels preloaded allows annotators to make only small adjustments, which may speed up the process.

When Should You Use Prelabels?

The decision to use prelabels depends on your specific needs:

• Do you want structured feedback on your model’s performance?
  • If yes, consider using prelabels
• Do you believe your model’s outputs are mostly correct and only need minor adjustments?
  • If yes, consider using prelabels

Considerations When Using Prelabels

While prelabels can be useful, there are some important factors to keep in mind:

• Potential for Bias: Prelabels may influence annotators, leading them to accept model-generated labels even when they are incorrect.
• Accuracy Trade-offs: If a model is highly sensitive, annotators may follow its mistakes instead of identifying them.
• Efficiency Risks: If prelabels are often incorrect, it may take more time to correct them than to create annotations from scratch. If you expect your model to make frequent mistakes, prelabels could reduce efficiency instead of improving it.

Prelabels Generated by Centaur

You do not have to supply prelabels yourself. Centaur can generate segmentation prelabels for a task, either by running an established model or by training one on the gold standards you provide and applying it across the remaining cases.

Labelers then correct what the model produced instead of annotating from scratch, and the corrected annotation becomes the labeled result.

Models are checked against held-out cases before they are used on a task, since a weak model costs labelers more time than it saves. Inference and any training run inside Centaur's own infrastructure, so your data is not sent to an outside AI service.

This is set up per task. Talk to your project manager if you would like it applied to yours.

Setting Up Prelabels

Once in your task navigate to Add labels in the left sidebar. Then select Prelabels. You'll have the option to upload via API or with a CSV.

Add Using API

Reference our API documentation for instructions on uploading prelabels via API.

Upload with CSV

Follow the instructions to download the CSV template.

Add answers to each row where you'd like to assign an prelabel. Use the Case Id and Customer Origin columns to match the cases.

• Do not change the values in any of the other fields (i.e., "Case Id", "Customer Origin", and "Labeling State").
• The "Notes" column is also available as an optional place for metadata you'd like to store.
• Any rows with a blank Answer column will not change. If you've already set a prelabel for the case, you can update that row. The prelabel will only be changed, not be removed if it is you do not need to include that row in the new import.

Review here for additional detail on CSV formatting.