Validation Methods

Validation is the process of establishing the validity of an automated algorithm. Closely related topics are performance assessment and quality control. In fact, these topics are closely related and must be considered together.

Validation and performance are of utmost importance since a carefully validated algorithm with known (and acceptable) performance can be deployed with confidence in biological studies, and vice versa. In the FARSIGHT project, we are interested in validation and performance assessment methods for segmentation, classification, and change analysis algorithms. We are also interested in methods that are practical to use, scalable, and requiring minimal manpower compared to traditional methods.

Classical Validation MethodsThe classical approach to validation is to compare automated results to “ground truth data” that is known in advance to be an accurate result. The notion of ground truth is appealing in principle, but elusive in practice. One approach to ground truth generation is to create a phantom image (or image sequence) with known parameters. This is helpful for evaluating several aspects of image analysis algorithms in a tightly controlled manner, e.g., performance as a function of signal-to-noise ratios, and specific morphological characteristics of objects.