top of page

Precision-Recall Analysis of Peripheral Nerve Myelinated Axon Counting Pipeline

Updated: Nov 3, 2020

Trevor Lancon (DRVISION Technologies, LLC)

Iván Coto-Hernández (Surgical Photonics and Engineering Laboratory, Department of Otolaryngology, Massachusetts Eye and Ear Infirmary and Harvard Medical School)


Introduction

Microscopy data, as opposed to tabular data, offers a unique perspective for users of machine learning technology: the results can be inspected visually. Egregiously erroneous results are easily rejected as false. In dense datasets, however, it makes sense to employ statistical analysis to inspect whether results agree with reality. In this application, we perform a precision-recall (PR) analysis to quantify the accuracy of a workflow to count cross-sections of myelinated axons in fluorescence micrographs as shown in Figure 1 [1].


Figure 1. Stain-free micrograph of myelinated axons. Each ring represents one cross-section of one axon. Axons greatly range in size. Scale bar is 100 µm.


Receiver operator characteristic (ROC) curves and PR curves elicit similar information when evaluating classification algorithms. However, ROC curves are criticized for reporting optimistic results in cases where there is a large class imbalance within the results; whereas PR curves are more robust in this respect [2]. For this analysis we see results where the true positives count is over 100, whereas false positives and false negatives are less than 20. Thus, we choose to calculate a PR curve as opposed to an ROC curve. ROC curves also require a count of true negatives, which is not easily obtainable from a voxel-based segmentation mask. Furthermore, precision and recall can be broken down into a single F-score for each data point, making our decision-making process easier.


An overview of precision, recall, and the F-score are presented here in brief, but more information can be found at [3].


Precision is the ability of our algorithm to count only the relevant nerves: