Results CSV

Identifiers and Case Structure

Term	Definition
Case ID	Unique identifier for cases in our system.
Origin	Original file identifier or filepath.
Origin Created At	Date and time file was uploaded to our Centaur system.
Content ID	Unique identifier for the content (e.g. image, text, audio, etc.) in the case.
URL	Click this URL to view the case on the Centaur Portal.
Labeling State	Click the information icon next to “Status” to see the definitions of labeling states.
Series (if applicable)	Unique identifier for the video or content sequence (e.g. image frames) that the case is associated with. A series can contain multiple cases, and a case can cover part or all of a series.
Series Index (if applicable)	Unique identifier for the content as part of its series (e.g. image frames).
Patch (if applicable)	Area of interest on an image to be annotated for the case, in WKT format with coordinates rescaled to a 0-100 scale.
Study UID (if applicable)	Unique ID of the DICOM study that the annotated image was from.
Series UID (if applicable)	Unique ID of the series within the DICOM study that the annotated image was from.
Instance UID (if applicable)	Unique ID of the image within the DICOM series that was annotated.

Reads and Annotations

Term	Definition
Qualified Reads	Number of reads (“opinions”) on the case. Qualified Reads are those which meet a predefined accuracy criterion.
Has Consensus?	The case has reached consensus. Read more about consensushere.
Correct Label	If Gold Standard case, the label as assigned by you and/or an admin. If Labeled case, the Majority Label when the case has met a predefined consensus criterion.
Majority Label	Annotation derived from Qualified Reads.
Number of Entities	Number of entities highlighted (NER) or selected (time range selection) on the case in the Correct Label.

All ranges are 0-indexed, include the start index, and exclude the end index (meaning the end index should be greater than the position of the final character or deci-second by 1).

For named entity recognition, your results will be delivered as character position ranges along with the highlighted entity.

🚧
White Space
Each line feed character (\n) counts as 1 character. A carriage return plus line feed (\r\n) counts as two characters.

Your results download will include the answer range and entity highlighted.

Sample text	Sample correct answers	Correct Label
Label this drug	this drug	[Results CSV **Ide]
Label this drug Label that drug	this drug, that drug	[Results CSV **Ide,tifiers and Case St]
Label this drug Label that drug Label a third drug	this drug, that drug, a third drug	[Results CSV Ide,tifiers and Case St,ucture \| Term ]
Label this drug	(no finding)	[]

For time range selection, your results will be delivered as deci-second ranges.

In multi-class range selection, results are delivered as one row per class for each case.

Measures of Confidence

Term	Definition
Difficulty	- average IoU] \ (Intersection over Union) between all of the Qualified Reads and the Correct Label.
Agreement	Average IoU (Intersection over Union) between all of the Qualified Reads and the Majority Label.

Other Information

Term	Definition
Internal Notes	Any notes internal to your team or tracked by Centaur relevant to the case.
Comments	Comments on the case by our labelers.
Flags	Flags on the case by our labelers.
Explanation	Notes written by you and/or an admin to help train and educate labelers. The explanation is shown to labelers after they have submitted a read.
Pre-answer explainers	Additional text information provided to labelers about a case prior to viewing.
Static Highlight	Range and entity highlighted for any static highlights present on the case (e.g., [{'range':[6,15], 'text': "this cell"}]).

Reads CSV

Term	Definition
Case ID	Unique identifier for cases in our system.
User ID	Unique identifier for users in our system.
Read ID	Unique identifier for reads in our system
Qscore	The trailing average accuracy for the user when they submitted an answer to the associated case ID. Read more about trailing average accuracy (Qscore) here.
Frame ID	Identifier for the frame number in the case.
Label Class	The class assigned to the label.
User Label	Read provided by the user.

_Note: Only the top reads used to create the majority label are included.