Labeling states
| Labeling State | Definition |
|---|---|
| Ready | An unlabeled case that we are not labeling yet. |
| In Progress | An unlabeled case that is actively being reviewed by labelers. |
| Gold Standard | A case that was submitted with a label by the customer team or an admin. |
| Labeled | A case that has been labeled by Centaur’s network of experts. |
| Canceled | A case that has been removed from labeling. |
| Flagged | A case that labelers have flagged as incorrect, inappropriate, or irrelevant. |
Task labeling stage. The states above describe individual cases. Separately, each task has a labeling stage that controls whether labeling is open at all:
- Pending — labelers can read the instructions but cannot label yet.
- Open — labeling is active.
- Assignment only — labeling is active, but labelers only work on cases assigned to them.
- Paused — labeling is on hold. Labelers can still see the instructions and their own performance.
- Closed — labeling has finished.
Your project manager sets the stage, and can attach a note that labelers see alongside it, for example to say when labeling will open. If you bring your own labelers to a task, they can only label while the task is open or assignment only.
Learn more here.
Updated 11 days ago
