Polygons and Pixel

Majority Label

Individual qualified reads on a task are aggregated to create the Majority Label. The Majority Label is created once we have a single qualified read.

Two values determine how polygon and pixel segmentation reads are aggregated:

• Number of reads considered: Maximum number of qualified reads used to create the ‘majority’ shape(s)
• Required reads overlapping: Minimum number of qualified reads that need to include a given area (ROI) for that area to be a part of the Majority Label

Consider an example where the number of reads considered is 4 and the required reads overlapping is 3.

The top 4 qualified reads (based on each user’s performance on the task) will be aggregated to create the Majority Label. If 3 overlapping reads are required, any region where at least 3 of the top 4 reads overlap will be included in the Majority Label.

Here, only user 1 - 4’s reads will be included in the aggregation, and user 5’s read will not be included, as that user’s performance is the lowest.

Then, the Majority Label is created by identifying any areas where at least 3 of these 4 reads overlap to create the final Majority Label shape.

In the case of multi-class segmentation, this process is performed independently for each of the classes present in the image.

Correct Label

For Gold Standard cases, the Correct Label is what was provided by you or consultants contracted by you.

For Labeled cases, the Correct Label is the same as the Majority Label once it has received the minimum number of qualified reads (e.g., 3) that reach a certain level of agreement with one another (e.g., 75%).