NucleusEditor

From FarsightWiki
Revision as of 15:56, 9 September 2009 by Isaac (Talk | contribs)
Jump to: navigation, search

The main purpose of the NucleusEditor program is to view and segment images of nuclei, and then efficiently edit the segmentation using pattern analysis tools (PACE).

Contents

Image Viewer

NucleusEditor allows users to view image data in a variety of file formats (tif, pic, lsm, etc). An image may be opened by selecting File|Load Image...

Load Image Example

Segment Cell Nuclei

To begin segmentation, an image must already be loaded and visible in NucleusEditor. The nuclear segmentation algorithm is triggered by selecting File|Start Segmentation... The program will use the visible image for the segmentation algorithm. If the image has multiple channels, the program will search for the channel that contains the string "Nuclei" in its name. Before the segmentation algorithm can begin it must be initialized with a few parameters. These parameters can be chosen automatically using an innovative algorithm developed by Yousef Al-Kofahi, or they can be supplied in the form of a parameters file. The user must choose an option from the dialog box that will appear. When the segmentation begins, a new toolbar will be added to the NucleusEditor that shows the progress of the segmentation algorithm and provides a STOP button to cancel the segmentation.

Nuclear Segmentation is achieved in 3 major steps:

  • 1. Initial Segmentation
  • 2. Seed Editing
  • 3. Final Segmentation
Segment.JPG

Upon completion of the initial segmentation, the program will pause and allow the user to inspect the result and edit the seeds [not available]. At this point the user may either continue segmentation (by pressing the "Continue" button), abort segmentation (by pressing the "Stop" button), or modify the parameters and start over (by changing the parameter file, pressing the "Stop" button, and then re-starting the segmentation using the saved parameter file).

After the final segmentation, the program will compute features and show this metadata (mainly object features) using multiple data visualization tools that are all actively linked, allowing the data to be viewed in multiple spaces simultaneously. A cluster of objects to be selected based on any combination of operations in any of the multiple spaces. We term this method actively linked multiple spaces architecture (ALISA). NucleusEditor provides this linked-view architecture where objects can be visualized in image or geometric views, table views, plot views, and histogram views. Operations performed in one view are immediately visible in all of the views.

To save s segmentation result, select File|Save Result

Final Result Example


In the ALISA mode a number of operations become active in each view:

  • Segmentation Result
    • Show/Hide Object Boundaries {View|Show Boundaries or Ctrl+B)
    • Select/Deselect Objects {by clicking on them)
    • Add Scatter Plot (View|New Scatter)
    • Clear Selections (Editing|Clear Selections or Ctrl+C)
    • Edit Functions (Edit Menu - see below)
  • Scatter Plots
    • Select/Deselect Objects
      • Select/deselect by clicking on single object
      • Clear all selections (Options|Clear Selections or Ctrl+C)
      • Select group by holding CTRL key and creating a lasso via a series of clicks
    • Options|Normalize - Ctrl+N (show features in normalized space)
    • Tools|Find Outliers (ALISA - see below)
    • Options (can change columns for X,Y and coloring)
  • Table
    • View|Sort table by... (To select column to sort by - in ascending order)
    • Show/Hide table columns (View|Visible Columns)

Using the PACE Concept

Since automated segmentation algorithms are not perfect, there is a compelling need to develop efficient methods to identify and correct the automated segmentation errors. Due to the large size of the datasets, the identification and correction of the cells to be edited have to be done very fast. In order to achieve this, NucleusEditor implements kernel based pattern analysis algorithms where we can do group or cluster editing of multiple errors simultaneously. This dramatically reduces the amount of manual effort required compared to unassisted edit-based methods. We term this methodology PACE (Pattern Assisted Cluster Editing). By observing the nature of the errors, it will be possible to asses the performance of the segmentation algorithm used in NucleusEditor and modify it to reduce similar errors in the future.

ALISA is designed to allow group editing of objects. The idea is for the user to select a group of objects based on their features. This can be done manually, or using automated pattern analysis algorithms. Manual selection of objects is aided by scatter plots of features. The user can choose the features displayed along the x and y axes of scatter plots. To visualize more than 2 features at a time, ALISA allows the user to display an unlimited number of scatter plots. Each point on the scatter plot corresponds to a specific object in the image (with a unique identifier (ID)). Clicking the mouse on a point selects and highlights the corresponding object in all other views. The user can also draw a polygon on a scatter plot to select a group of objects visually.

Computer-assisted selection of objects can also be performed. Features that highlight specific types of errors in segmentation (we call them diagnostic features) can also be defined and these objects selected and edited in their feature-space. Outlier detection tools are provided to assist in finding unusual objects to be inspected and/or edited. Outliers may correspond to segmentation errors (typical) or unusual cells (rare but possible). For outlier detection, the one-class Support Vector Machine (SVM) is employed [libSVM]. The one-class SVM proves to be a very useful tool for outlier detection, often highlighting a variety of quirky objects that should be inspected more closely, and edited when necessary. The outliers are shown in the segmentation, and scatter views, by a simple color coding scheme. The user can quickly locate, inspect and edit the outliers due to the linked nature of the views.

Clues

Two simple features have been shown to be particularly useful in detecting segmentation errors: volume and percent shared boundary. The following edit operations tend to correlate to these features in this way:

  • Delete: small volume & zero (or small) percent shared boundary
  • Merge: small volume & large percent shared boundary
  • Split: high volume &| high surface area

The Process

Two main techniques can be used to locate objects that need to be edited: sorting by feature and outlier detection. In some cases the scatter plot showing outliers may be too crowded to select specific objects (ex. delete candidates), and it may be more useful to sort the table by a specific feature and select objects from this view.

  • Sorting By Feature: The table view provides two very useful functions: View|Visible Columns and View|Sort By. The NucleusEditor provides a large wealth of feature data, most of which is noise to the editing process. Change the Visible Columns to only include those features that are useful while editing (ex. id, volume, shared boundary, surface area).
    Modify the visible columns
    Next we may want to find all of the objects with very small volume. On the scatter plot it is very hard to select just one of these objects and view it because they are tightly clustered together. By sorting the table by volume, it becomes very easy to find all of the objects with small volumes.
    Sort by a particular column
  • Finding Outliers:


Validation Methods.

Editing Segmentation Results

Existing nuclear segmentation results can be loaded by selecting File|Load Result...

NucleusEditor provides the following three operations to edit cells, which will become active when a valid segmentation result is loaded:

  • Delete Cells
  • Merge Cells
  • Splitting (Cells)

Delete Cells

In order to "Delete a cell", we first pick the cell and click on "Delete Cells" menu item. Alternatively, we can press "D" after clicking on the cell. In the following example, we want to delete the cell with id-13. As soon as this cell is deleted all the views will be updated accordingly. In this example, the cell 13 and all the corresponding data presented in that row will be deleted.

Editing1-delete.JPG

Merge Cells

Merging Cells requires the user to select two neighboring cells first. After clicking on the "Merge Cells" menu item, the cells will be merged and the new feature values will be computed and added into the table with a new id. The feature values associated with the old cells will be removed. In the following example, we merge the cells with id 18 and 37. The last id that we have in the table is 54. After the merge operation, the new cell will have id 55 in the table.

Similar to the "Delete Cell" operation, the user can use the keyboard and press "M" to merge the cells. Note that if the selected cells are not neighbors, they will not be merged.

Editing2-merge.JPG

Splitting (Cells)

Cells can be split after we are in the splitting mode. The splitting menu has a submenu that has two items:

  • Start Splitting
  • End Splitting

We start splitting operation with "Start Splitting" submenu item and select two seed points inside the cell. When we click on points in a cell these points will be colored with red. Once the seed points are ready we can split the selected cell by clicking on "End Splitting".

In the following example, We first choose "Start Splitting" to split the cell with id-1.

Editing3-split0.JPG

We first zoom in using the mouse and click on two seed points.

Editing3-split1.JPG

After splitting the cell with id-1, the split cell will be removed and two new cells (id-56 and 57) with new features will be added into the table and other views.

Editing3-split2.JPG

Logging Editing History

A typical XML result file consists of a set of objects whose structure is given in the following figure. Each object is defined with a set of features. In the following result file, an object corresponds to a cell nuclei which is identified by a unique id. A cell can be either valid or invalid depending on the editing operation. Initially all cells are valid and will be loaded. If a "delete" operation is applied to the cell , it will no longer be valid. But we still keep this cell data in the result file for logging purposes.

Farsight-object.JPG

Every editing operation (delete, merge, or split) is recorded by creating an editing history log for each cell in the result file. We define an <EditHistory> element for each cell and list the editing operations with a proper timestamp and cell ids. There are three cases:

- Delete: We just note that this cell has been deleted (valid=0)

Farsight-object-delete.JPG

- Merge : In this case, two cells (id1, and id2) will be merged and a new cell will be created. Therefore, we first delete the first two cells and update their edit histories with this information, and then we update the edit history of the new cell noting that it is created by merging the cells with id1 and id2.

Farsight-object-merge.JPG

- Split : We note that two new cells will be created (with their ids) after splitting the cell of interest. In addition, the edit histories of each new cell will be updated with the id of the cell they are splitted from.

Farsight-object-split.JPG

Summary

Once the corrected segmentation results are obtained with this way, hypothesis-driven and discovery driven research can be conducted by utilizing object features, associative object measurements, and the other tools that are part of the FARSIGHT_Toolkit.


This page was prepared by Isaac Abbott & Aytekin Vargun

Personal tools