Manage task

Task view

You can view labels and accuracy metrics by navigating to the task view. After logging into the Centaur Portal, select your project and task.

The Overview tab provides insights on the task name, task prompt, task activity, and labelers instructions

The Data imports tab is where you can add and manage your data in this task

The Cases tab is where you view all your cases in this task. When discussing "Task view" we refer to being in this tab.

The Add Labels tab is where you create Gold Standards or begin Web Labeling.

The Analysis tab shows key metrics on labeling results (e.g., agreement, labeling rate)

Task progress

Two views answer how far along a task is: a completion summary on the Overview tab, and a labeling rate chart on the Analysis tab.

Completion

The Overview tab shows how many cases in the task are complete, as a count and a percentage, with a Finished or In progress badge. The colors match the labeling states you see on the Cases tab.

Expand View by import to see the same breakdown for each data import, so you can tell which batch is still being labeled. Each import is listed with its own progress and a status of Finished, Labeling or Queued.

A batch here means a single data import. If you organise your work by tags rather than by import, the breakdown will not follow your tags.

How completion is calculated. Gold standards, QA answer keys and canceled cases are excluded from both the numerator and the denominator, so a batch that has been fully labeled reads 100%.

If you do not see the completion summary on your tasks, ask your project manager to enable it for your organisation.

Labeling rate

The Analysis tab has a Labeling Rate chart showing how many cases were labeled over time. Use the selector to switch between:

• Cumulative, the running total since the task began. This is the default, and it always covers all time.
• Per day, cases labeled each day.
• Per week, cases labeled each week.

For the per-day and per-week views, use the date range picker to narrow the chart to a specific window. It reads All time until you set one. On very large tasks the first load can be slow, and setting a date range is the quickest way to get a responsive chart.

The chart counts a case on the date its correct answer was assigned, and covers a single task. You can download the underlying data as a CSV.

Filtering Task view

Once in the Cases tab, there are several ways you can filter your cases on the Centaur Portal. You can filter by case identifiers, labeling state, class (number of findings or answer choice), or tags.

By Labeling State

Select the States button on the top of the screen. Then select your desired labeling state and only the cases within this labeling state appear in the case list. You can select more than one labeling state at a time. Refer to the Labeling States article for more information.

By Class

To filter by class, select the desired filter from the top of the screen under Classes or Findings.

For segmentation and NER tasks, cases can be filtered by the number of findings. Select the desired number of findings and the cases with this number of findings will populate. You can select more than one number of findings at a time. In case of multi-class NER and segmentation, you can filter by number of findings and classes.

For classification tasks, cases can be filtered by answer choice. Select the desired answer choice and the cases with this answer choice will populate. You can select more than one answer choice at a time. Note: if your task has variable answer choices (answer choices that change according to the case), you will be unable to filter by answer choice.

By Case Identifiers

Cases can be filtered by customer origin or Centaur Case ID by using the search function in the case list.

Sorting task view

By Labeler Feedback

Labeler feedback is received in comments and flags. To sort by these metrics, click More Columns located at the top of the screen. Select which columns you'd like to add and apply. The headers of those columns will sort the cases by ascending or descending number of flags / comments.

By Accuracy Metrics

Cases can be sorted by agreement and difficulty. To sort by these metrics, click More Columns located at the top of the screen. Agreement exists for In Progress, Labeled, Gold Standard, and/or QA cases. Difficulty exists for only Gold Standard cases. Once the Agreement and/or Difficulty columns are selected, choose ascending or descending order to sort.

By Case Identifiers

Cases can be sorted by customer origin or case ID by using the header at the top of the case view. Select customer origin or case ID and sort in ascending or descending order.

Other Sorting Metrics

Select More Columns in case view to sort by Number of qualified reads, Created date, and Last Labeled Date.