NucleusEditor
The NucleusEditor program provides users with the following functionalities:
Contents |
Image Viewer
The main purpose of the NucleusEditor program is to view and segment images of nuclei, and then efficiently edit the segmentation using pattern analysis tools (PACE). NucleusEditor allows users to view image data from a variety of file formats (tif, pic, lsm, etc). An image can be opened by selecting File|Load Image...
Segment Cell Nuclei
To begin segmentation, an image must already be loaded and visible in NucleusEditor. The nuclear segmentation algorithm is triggered by selecting File|Start Segmentation... The program will use the visible image for the segmentation algorithm. If the image has multiple channels, the program will search for the channel that contains the string "Nuclei" in its name. Before the segmentation algorithm can begin it must be initialized with a few parameters. These parameters can be chosen automatically by the algorithm using an innovative technique designed by Yousef Al-Kofahi, or they can be supplied in the form of a parameters file. The user must chose an option from the dialog box that will appear. When the segmentation begins a new toolbar will be added to the NucleusEditor that shows the progress of the segmentation algorithm and provides a STOP button to cancel the segmentation.
Nuclear Segmentation is achieved in 3 major steps:
- 1. Initial Segmentation
- 2. Seed Editing
- 3. Final Segmentation
Upon completion of the initial segmentation, the program will pause and allow the user to inspect the result and edit the seeds [not available]. At this point the user may either continue segmentation (by pressing the "Continue" button), abort segmentation (by pressing the "Stop" button), or modify the parameters and start over (by changing the parameter file, pressing the "Stop" button, and then re-starting the segmentation with the saved parameter file).
After the final segmentation, the program will compute features, and show this metadata (mainly object features) using multiple data visualization tools that are all actively linked. This linkage is important, and allows the data to be viewed in multiple spaces simultaneously, and a cluster of objects to be selected based on any combination of operations in any of the multiple spaces. We term this method Actively linked multiple spaces architecture (ALISA). NucleusEditor provides this linked-view architecture where objects can be visualized in image or geometric views, table views, plot views, and histogram views. Operations performed in one view are immidiately visible in all of the views.
To save this segmentation result select File|Save Result
Editing Segmentation Results
Existing nuclear segmentation results can be loaded by selecting File|Load Result...
NucleusEditor provides the following three operations to edit cells, which will become active when a valid segmentation result is loaded:
- Delete Cells
- Merge Cells
- Splitting (Cells)
Delete Cells
In order to "Delete a cell", we first pick the cell and click on "Delete Cells" menu item. Alternatively, we can press "D" after clicking on the cell. In the following example, we want to delete the cell with id-13. As soon as this cell is deleted all the views will be updated accordingly. In this example, the cell 13 and all the corresponding data presented in that row will be deleted.
Merge Cells
Merging Cells requires the user to select two neighboring cells first. After clicking on the "Merge Cells" menu item, the cells will be merged and the new feature values will be computed and added into the table with a new id. The feature values associated with the old cells will be removed. In the following example, we merge the cells with id 18 and 37. The last id that we have in the table is 54. After the merge operation, the new cell will have id 55 in the table.
Similar to the "Delete Cell" operation, the user can use the keyboard and press "M" to merge the cells. Note that if the selected cells are not neighbors, they will not be merged.
Splitting (Cells)
Cells can be split after we are in the splitting mode. The splitting menu has a submenu that has two items:
- Start Splitting
- End Splitting
We start splitting operation with "Start Splitting" submenu item and select two seed points inside the cell. When we click on points in a cell these points will be colored with red. Once the seed points are ready we can split the selected cell by clicking on "End Splitting".
In the following example, We first choose "Start Splitting" to split the cell with id-1.
We first zoom in using the mouse and click on two seed points.
After splitting the cell with id-1, the split cell will be removed and two new cells (id-56 and 57) with new features will be added into the table and other views.
Logging Editing History
A typical XML result file consists of a set of objects whose structure is given in the following figure. Each object is defined with a set of features. In the following result file, an object corresponds to a cell nuclei which is identified by a unique id. A cell can be either valid or invalid depending on the editing operation. Initially all cells are valid and will be loaded. If a "delete" operation is applied to the cell , it will no longer be valid. But we still keep this cell data in the result file for logging purposes.
Every editing operation (delete, merge, or split) is recorded by creating an editing history log for each cell in the result file. We define an <EditHistory> element for each cell and list the editing operations with a proper timestamp and cell ids. There are three cases:
- Delete: We just note that this cell has been deleted (valid=0)
- Merge : In this case, two cells (id1, and id2) will be merged and a new cell will be created. Therefore, we first delete the first two cells and update their edit histories with this information, and then we update the edit history of the new cell noting that it is created by merging the cells with id1 and id2.
- Split : We note that two new cells will be created (with their ids) after splitting the cell of interest. In addition, the edit histories of each new cell will be updated with the id of the cell they are splitted from.
Edit-Based Validation - Group Editing
Since automated segmentation algorithms are not perfect, there is a compelling need to develop efficient methods to identify and correct the automated segmentation errors. Due to the large size of the datasets, the identification and correction of the cells to be edited have to be done very fast. In order to achieve this NucleusEditor implements kernel based pattern analysis algorithms where we can do group or cluster editing of multiple errors simultaneously. This dramatically reduces the amount of manual effort required compared to unassisted edit-based methods. We term this methodology PACE (Pattern Assisted Cluster Editing). By observing the nature of the errors and analyzing them, it can be possible to asses the performance of the algorithms used in NucleusEditor for this purpose and modify them to prevent such errors in the future. We explain ALISA and Group Editing in more detail at EVS and Validation Methods.
Once the corrected segmentation results are obtained with this way, hypothesis-driven and discovery driven research can be conducted by utilizing object features, associative object measurements, and the other tools that are part of the FARSIGHT_Toolkit.
This page was prepared by Aytekin Vargun (with Isaac Abbott's contributions)
