Web Labeling

Web labeling lets consultants or your own team members annotate your data directly in the Centaur Labs portal, in a browser, with nothing to install. Centaur monitors labeler performance on the task, counts reads only from labelers who meet the accuracy bar, and aggregates multiple opinions together to form a final label.

Web labeling covers images, video, audio, text and PDF. Tasks built on DICOM data open in the Centaur DICOM Viewer instead.

Begin Web Labeling

Notify your project manager if web labeling is your desired annotation method. To begin labeling, navigate to your task and select the Add labels tab. From there, select Web Labeling and then select the box containing Web Labeling. Invite labelers from your team by selecting Copy link to share and sharing the link. Note: labelers must have a Centaur Labs account to access the link.

In the web labeler

A labeler works one case at a time. For each case they can View Instructions to reread the task instructions without leaving the case, Add a comment to record a note, or Flag Case to mark it as incorrect, inappropriate or irrelevant, which removes it from circulation pending review.

Selecting Submit Findings records the read and moves to the next case. The web labeler queues all cases that are In Progress.

Getting good results

Keep each case answerable on its own. A labeler should be able to answer the prompt from the case in front of them. Where a single asset is long, split it into shorter units before import.
Write a specific prompt. Say what the labeler is looking for and what to do at the edges. Ambiguity in the prompt shows up as disagreement in the results.
Provide enough gold standards. Performance measurement needs cases with known answers. Without them accuracy cannot be assessed and reads cannot be qualified.
Start small. Launch a small batch, review the results, then scale. Problems in the instructions are far cheaper to fix before the whole dataset has been labeled.

Users

You can use the Web Labeler internally, or Centaur can contract consultants from its database of trusted labelers.