Assessment Metrics

NOTE: Task1 and Task2 will be ranked and awarded separately.

Task 1:  Cancer Segmentation

Evaluation Metrics: Jaccard Index

• Jaccard Index * HT(Jaccard Index; 0.65) where HT(Jaccard Index; 0.65) = 1
• if Jaccard Index >= 0.65 and HT(Jaccard Index; 0.65) = 0 otherwise.
• Missing data will be scored as 0

Rank Computation Method

1. The mean of Jaccard Index with threshold value 0.65
2. If tied, the median of Jaccard Index with threshold value 0.65 (tie-breaker)
3. If tied, the mean of Jaccard Index with threshold value 0.35 (tie-breaker)
4. If tied, the organizers will subjectively assess the quality of the results.

• Bootstrapping (resampling) approaches will be used only for award candidates
• A paired t-test will be used only for award candidates
• (updated) Submissions with any missing case will not be scored.

Task 2: Viable Tumor Burden  Estimation

Evaluation Metrics

Rank Computation Method

1. The mean of Reversed Absolute Errors
2. If tied, the median of Reversed Absolute Errors (tie-breaker)
3. If tied, the organizers will subjectively assess the quality of the results.