Create instructions
To ensure high-quality results, labelers need clear and concise instructions explaining how to best complete your labeling task. Good instructions will result in consistent labels across cases that align with your specifications.
About
Instructions are informative slides that give labelers specific instructions about how to complete your task. Labelers are required to complete your instructions before they can start labeling.
You can add, edit and rearrange slides as needed, and then re-publish when you're ready. Depending on the updates you make, you can require labelers to redo the instructions before they attempt your task.
Using Gold Standard cases, labelers will get a chance to practice and receive feedback before they start labeling.
Capabilities
Import instructions from other tasks
Import instructions from other tasks you've worked on by selecting Import on the top right hand side then selecting the project and task you'd like to import the instructions from.
Upload Images / GIFs from File Picker
Upload images or .gifs from your file picker by selecting the import icon (imaged below), then selecting the file of interest.
Duplicate Slides
Duplicate a slide by selecting the duplicate icon (imaged below) on the slide of interest.
Rearrange Slides
Rearrange a slide by selecting and holding the rearrange icon (imaged below) and dragging the slide across the screen to your desired position.
Delete Slides
Delete a slide by selecting the X in the top right corner of the instruction slide. This deletion is final and cannot be undone.
HTML
You can use HTML for styled text (e.g., colored text) and embedded links. Toggle on HTML in the bottom left hand corner of the slide to write in HTML.
Colored Text Example:Red
Embedded Link Example:Centaur Labs
HTML Hot Keys:
- Bold: Control (Windows) or Command (Mac) + B
- Italicize: Control (Windows) or Command (Mac) + I
- Underline: Control (Windows) or Command (Mac) + U
- Strikethrough: Control (Windows) or Command (Mac) + Shift + S
- Bulleted List: Type - then a space
- Numbered List: Type 1. then a space
- Heading: Type # before your text
- Code: Type
**desired text**
Preview
Prior to publishing your instructions you can preview how your instructions, title screen, and case view will look in the DiagnosUs App by selecting the Preview on the top right hand slide of the screen.
Publish
Select Publish changes to save your work. Only people with access to your project will be able to view your instructions until your project manager launches a contest on the DiagnosUs app. Please let your project manager know once your instructions are final.
Best Practices
Instructions should follow this general structure:
- General introduction: introduce the general task of your labeling task. What is the topic of this contest? Why does it matter clinically?
- Contest introduction: introduce what specific task the labeler will be doing in the contest.
- Case-based instruction slides: Note: avoid using gold standards from your task in the instructions, unless you have at least 100 gold standards. Otherwise, labelers may memorize cases they'll be graded on later!
- Start with obvious, typical examples, along with basic definitions of each class or finding. Upload images or text that identify each finding (segmentation) or class (classification).
- Include examples for edge cases. Edge cases are rare, difficult cases that may be mis-classified without specific instructions. Upload images or text that point out specific findings for identifying any edge cases in your task.
- If you want some cases flagged (for being too blurry, too dark, etc), include this in the instructions, and upload example images or text if needed.
- Why it matters: the final slide should reiterate to labelers why this labeling task is important. What disease are labelers helping to detect? How do their contributions count towards real, lifesaving AI development?
Tips
- Keep instructions slides short and concise.
- If possible, write for a wide audience. Use basic language and clear examples to illustrate how to create your task.
- When possible, start sentences with the action (“Trace around …“, “Avoid…“, “Look for…“)
- Instructions do support emojis! Keep instructions fun by sprinkling some in ❤️.
- Limit the instructions slides to cover the essentials of your task, rather than providing an exhaustive list. Since labelers will continue to learn from Gold Standard cases as they complete your task, there's no need to cover everything here. Aim for 8 - 12 slides; you can create a maximum of 25 instructions slides.
- Already have instructions set up elsewhere? Use the (+) button to import instructional slides from other tasks.
Examples
Updated 3 months ago