Assessment Metrics¶
NOTE: Task1 and Task2 will be ranked and awarded separately.
Task 1: Cancer Segmentation¶
Evaluation Metrics: Jaccard Index¶
¶
- Jaccard Index * HT(Jaccard Index; 0.65) where HT(Jaccard Index; 0.65) = 1
- if Jaccard Index >= 0.65 and HT(Jaccard Index; 0.65) = 0 otherwise.
- Missing data will be scored as 0
Rank Computation Method¶
- The mean of Jaccard Index with threshold value 0.65
- If tied, the median of Jaccard Index with threshold value 0.65 (tie-breaker)
- If tied, the mean of Jaccard Index with threshold value 0.35 (tie-breaker)
- If tied, the organizers will subjectively assess the quality of the results.
- Bootstrapping (resampling) approaches will be used only for award candidates
- A paired t-test will be used only for award candidates
- (updated) Submissions with any missing case will not be scored.
Task 2: Viable Tumor Burden Estimation¶
Evaluation Metrics¶
Each Task1 case score is used as a weight for each Task2 case score.
Rank Computation Method¶
- The mean of Reversed Absolute Errors
- If tied, the median of Reversed Absolute Errors (tie-breaker)
- If tied, the organizers will subjectively assess the quality of the results.