Worm Analysis System

From FarsightWiki
(Difference between revisions)
Jump to: navigation, search
(→Tracking Result)
 
(105 intermediate revisions by 2 users not shown)
Line 1: Line 1:
−
There is a compelling need for improved software systems for delineating the C.''elegans'' worms in time-lapse image sequences and tracking their movements. Segmentation and tracking provide the basis for making quantitative measurements of worm morphology, and various aspects of their dynamic behaviors. To facilitate the quantitative studies of worm behaviors, we developed a automated worm analysis system which is open-source and could be adopted by all researchers in the c.''elegans'' community.
+
'''We are now working on applying our 3-D tracing algorithm for the Worm tracking problem. Some preliminary results produced by the [http://www.farsight-toolkit.org/wiki/Open_Snake_Tracing_System tracing system] are shown in figures below.'''
 +
 
 +
 
 +
[[Image:TracingWorm2.png|thumb|center|500px| "Tracing" the worms]]
 +
[[Image:TracingWorm1.png|thumb|center|500px| "Tracing" the worms]]
 +
 
 +
 
 +
 
 +
 
 +
= The Worm Tracking System (Old) =
 +
There is a compelling need for improved software systems for delineating the ''C.elegans'' worms in time-lapse image sequences and tracking their movements. Segmentation and tracking provide the basis for making quantitative measurements of worm morphology, and various aspects of their dynamic behaviors. To facilitate the quantitative studies of worm behaviors, we developed an automated worm analysis system which is open-source and could be adopted by all researchers in the ''C.elegans'' community.
  
−
== News ==
 
−
*'''2010-02-14''': Wiki Page for Worm Analysis System Created.
 
  
 
== System Overview==
 
== System Overview==
Line 8: Line 16:
 
===Main Window===
 
===Main Window===
 
Figure1 shows a screenshot of the main window of the system.  
 
Figure1 shows a screenshot of the main window of the system.  
−
*Menu: the menu provides operations like file operations and [[http://farsight-toolkit.org/wiki/Worm_Analysis_System#Edit Editing Operations]] which allow user to edit (reverse, delete and correct)selected worms in the tracking window.
+
*Menu: the menu provides operations like file operations and [http://farsight-toolkit.org/wiki/Worm_Analysis_System#Edit Editing Operations] which allow user to edit (reverse, delete and correct) selected worms in the tracking window.
−
*Toolbar: the toolbr provides three buttons (figure2: pause tracking, start/resume tracking, start tracking from a saved point) for controlling the tracking process. Users can pause tracking at any timepoint and do some observation or interactive editing when necessary. The thrid button make it possible for the user to start tracking from any saved tracking result which stores worm shapes, features and identities. Thus the user don't need to start over the tracking if the program was closed normally or abnormally.
+
*Toolbar: the toolbar provides three buttons (figure2: suspend tracking, start/resume tracking, start tracking from a saved point) for controlling the tracking process. Users can suspend the tracking process at any time point to do observation or interactive editing when necessary. The third button makes it possible for the user to start the tracking process from any saved tracking result which stores worm shapes, features and identities. Thus the user doesn't need to start over the tracking if the program was closed normally or abnormally.
 
[[Image:Toolbar.png|thumb|center|100px| Figure 2: Toolbar]]
 
[[Image:Toolbar.png|thumb|center|100px| Figure 2: Toolbar]]
−
*Input and Output: the input and output section allows users to specify the directory of image sequence and the directory for storing tracking result. For each frame, the tracking result of it includes a txt file storing the worm structure(centerline and contour points), a excel file storing extracted features(motion state, shape and motion features, events, etc.) and a png image saved from the tracking window. The results could be explored at anytime during or after the tracking [[http://farsight-toolkit.org/wiki/Worm_Analysis_System#Explore_Results Explore Results]].
+
*Input and Output: the input and output sections allow users to specify directory of input image sequence and the directory for storing tracking result. For each frame, the tracking result includes a text file storing worm structure (centerline and contour points), a excel file storing extracted features (locomotion states, shape and motion features, etc.) and a png image took from the tracking window. The results could be explored at anytime during or after the tracking process [[http://farsight-toolkit.org/wiki/Worm_Analysis_System#Explore_Result Explore Result]].
−
*Settings: the settings section allows users to specify settings for core processing algorithms. Please refer to [[http://farsight-toolkit.org/wiki/Worm_Analysis_System#Settings Settings]] for detailed information about these settings.
+
*Settings: the settings section allows users to specify settings for core processing algorithms. Please refer to [http://farsight-toolkit.org/wiki/Worm_Analysis_System#Settings Settings] for detailed information about these settings.
−
*Tracking Window: the tracking window includes the "tracking view" and "foreground view". The tracking view displays the centerline and contour of tracked worms, and also their identity and motion states(F=Forward, R=Reversal, P=Pausing, O=Omega Bend, C=Physical Contact). The foreground view shows the label image for foreground objects obtained by background subtraction method, it could help the user choose the right binarization parameters.
+
*Tracking Window: the tracking window provides both "tracking view" and "foreground view". Tracking view displays the centerline (head is shown as a yellow circle) and contour of tracked worms, and also their identity and motion state (F=Forward, R=Reversal, P=Pausing, O=Omega Bend, C=Physical Contact). Worms are highlighted by a transparent Minimum Enclosing Rectangle once selected. The foreground view shows the label image of foreground objects obtained by background subtraction method, and could help the user choose correct binarization parameters. A good binarization result means worm boundaries are well preserved in the binary image, and the foreground objects have no hole or dent.
−
*Log Window: the log window displays the log of the tracking process. More detailed information about the tracking process is displayed in another terminal window.
+
*Log Window: log window displays a log for the tracking process. More detailed information about the tracking process is displayed in another terminal window.
  
 
===Multiple Linked Views===
 
===Multiple Linked Views===
−
Worm Analysis System provides a linked-view architecture where worm objects can be visualized in different views like image, table and scatter plot. Worm objects selected in one view are immediately highlighted in all the views(figure3).
+
Worm Analysis System provides a linked-view architecture where worm objects can be visualized in different views like image, table and scatter plot. Worm objects selected in one view are immediately highlighted in all the views (figure3).
 
[[Image:Multi-View.png|thumb|center|1000px| Figure 3: Multiple Linked Views]]
 
[[Image:Multi-View.png|thumb|center|1000px| Figure 3: Multiple Linked Views]]
−
[[Image:ForegroundView.png|thumb|center|300px| Figure 4: Foreground Views]]
+
[[Image:ForegroundView.png|thumb|center|300px| Figure 4: Foreground View]]
  
 
===Algorithms===
 
===Algorithms===
  
 
*Background Subtraction: the system provides two background subtraction methods to handle images with different backgrounds.  
 
*Background Subtraction: the system provides two background subtraction methods to handle images with different backgrounds.  
−
**Background Subtraction Method I: A background image is computed first based on all available images, incoming image is subtracted with the background image and the subtraction result is fed to a Graph-Cut algorithm to output a smoothed binary image. This method applies for static background.
+
**Background Subtraction Method I: A background image is computed first from all available images. Incoming image is subtracted with the obtained background image and the subtraction result is fed to Graph-Cut algorithm to output a smoothed binary image. This method applies only for static background.
−
**Background Subtraction Method II: Motion-based and intensity-based models are used to detect moving foreground objects, the obtained foreground confidence map is fed to a Spatio-Temporal Graph-Cut algorithm to output a spatially and temporally smoothed binary foreground image.
+
**Background Subtraction Method II: Motion-based and intensity-based models are used to detect moving foreground objects. The obtained foreground confidence map is fed to a Spatio-Temporal Graph-Cut algorithm to output spatially and temporally smoothed binary foreground image.
−
*Tracking: the system incorporates our latest worm tracking algorithm[1], which could track multiple and overlapping worms and recognize worm locomotion states (Forward, Reversal, Pausing and Omega Bend) simultaneously. The tracker is initialized by the first two image frames on which worm detection is performed and head and tail are automatically determined(we assume worms are moving forward at the first two frames. If worm's head is not correctly identified, users need to manually reverse the worm).
+
*Tracking: the system incorporates our latest worm tracking algorithm [1], which could track multiple worms and recognize their locomotion states (Forward, Reversal, Pausing and Omega Bend) simultaneously. The tracker is initialized by the first two image frames on which worm detection is performed and head and tail are automatically determined (we assume worms are moving forward at the first two frames. If worm's head is not correctly identified, the user need to manually reverse the worm).
−
*Feature Computation: the system computes a subset of worm features in real-time during tracking. More features could be computed from the saved worm structures(centerline and contour points). For a complete list of our worm features, please refer to [[http://farsight-toolkit.org/wiki/Worm_Features%26Events Worm Features and Events]].
+
*Feature Computation: the system computes a subset of worm features in real-time during tracking. More features could be computed from the saved worm structures (centerline and contour points). For a complete list of our worm features, please refer to [[http://farsight-toolkit.org/wiki/Worm_Features%26Events Worm Features and Events]].
  
 
==System Operations==
 
==System Operations==
−
===Edit===
+
===Edit Worms===
−
To edit the worms when necessary, user can pause the tracking process and then use operations provided in the "Edit" menu(figure4). The worms also could be edited directly in the "tracking view". A worm can be deleted with double right-click or reversed with double left-click. When the worm is not in physical contact or doing omega bend, it could be corrected by "Ctrl+C" in case the tracking is not correct.  
+
To edit the worms when necessary, user can suspend the tracking process and then use the "Edit" menu (figure4) which provides three group editing operations. Deleting or reversing single worm could be done by double right-click or double left-click. Worm correction is only effective when selected worms are not in physical contact or doing omega bend.
 
[[Image:Worm_Editing.png|thumb|center|800px| Figure 5: Editing Worm: selected worms could be corrected by detection(if the tracking is not correct, and worm is not doing omega bend or in physical contact), reversed or deleted]]
 
[[Image:Worm_Editing.png|thumb|center|800px| Figure 5: Editing Worm: selected worms could be corrected by detection(if the tracking is not correct, and worm is not doing omega bend or in physical contact), reversed or deleted]]
  
−
===Settings===
 
−
Settings could be saved into a txt file or read back from the txt file. Most settings doesn't need to be change, however, the binarization settings needs to be carefully selected since the tracking performance greatly dependent on the binarization result. The binarization result in the foreground view window can help the user to choose the right binarization settings. Once tracking is started, most settings are unable to be changed.
 
  
−
*General Settings:
 
−
**frame sampling rate: process only one frame from every ''N'' frames.
 
−
**shrink factor: shrink image ''N'' times.
 
−
**explore result: if checked, the user could explore the result by sliding the "Frame Number" slider.
 
−
*Binarization Settings:
 
−
**Method I
 
−
***alpha: the binary foreground image will be oversmoothed with small alpha value. The default value is 1.00.
 
−
***threshold: thresholding value for image after subtraction with the background image. Intensity value below this value will be set to 0. The default value is 40.
 
−
**Method II
 
−
***alpha: smoothing parameter in the graph-cut algorithm. Small alpha value can eliminate noise in the background and holes in the foreground objects, but also can result in oversmoothed binary image. The default value is 1.00.
 
−
***threshold: the default value is 1.00. Large threshold value can eliminate most noise, but also will exclude some parts of foreground objects.
 
−
***min sigma: the default value is 0. It is set to 10 in Demo2 and Demo3, since the sigma of background distribution is very small.
 
−
***init frame: the default value is 10, which means the motion-based model used in this background method is initialized by the first 10 frames. Usually this parameter doesn't need to be changed.
 
−
*Detection Settings:
 
−
**min area and max area: foreground objects with size smaller than "min area" or larger than "max area" will be excluded in the binarized image.
 
−
**min LW ration and max LW ratio: these two values are used to identify and exclude worms with abnormal Length/Width ratio.
 
−
**center points: number of centerline points in the worm model. Suggested value is 31 or 21.
 
−
*Tracking Settings:
 
−
**sigma_PROG: parameter of worm progression movement.
 
−
**sigma_RH: parameter of worm head displacement.
 
−
**sigma_RT: parameter of worm tail displacement.
 
−
**particles: number of particles or hypothesis for each worm. Suggested value is 50 or 100.
 
−
**adjustment: times of centerline adjustment.
 
−
**correction: when the percentage of centerline points with value 1 is below specified value, the worm will be automatically corrected.
 
−
**varying worms: if checked, the tracker will detect and add the worms entering the field.
 
−
**tracking overlapping: if checked, the tracker will track the contour of overlapping worms, if unchecked, the tracker simply track overlapping worms with a bounding box(figure6 and figure7). If the user is not interested in studying worm behavior during physical contact, unchecking this box could speed up the processing.
 
−
**speed tracking: when worms are not in physical contact or doing omega bend, the processing could be sped up by using detection(different from the detector used for initializing tracker, detector here also guarantee that worm identities are correctly maintained) instead of tracking.
 
 
[[Image:TrackingOverlapping.png|thumb|center|400px| Figure 6: Contour tracking for worms in physical contact]]
 
[[Image:TrackingOverlapping.png|thumb|center|400px| Figure 6: Contour tracking for worms in physical contact]]
 
[[Image:TrackingOverlapping1.png|thumb|center|400px| Figure 7: Bounding Box tracking for worms in physical contact]]
 
[[Image:TrackingOverlapping1.png|thumb|center|400px| Figure 7: Bounding Box tracking for worms in physical contact]]
−
 
−
===Explore Results===
 
−
If the directory for saving tracking results is provided, users could explore the result after tracking, or during the tracking process simply by pausing the tracking first, checking the "explore result" box and sliding the "frame number" slider. The "explore result" box needs to be unchecked before resume the tracking.
 
−
[[Image:ExploreResult.png|thumb|center|389px| Figure 8: Explore the results]]
 
−
 
−
===Start Tracking from Saved File===
 
−
To start the tracking from saved tracking result, the user needs to specify the directory storing the tracking result and click the third button in the toolbar and choose the txt file from which the tracking will start(figure9). 
 
−
[[Image:StartFromSavedTxt.png|thumb|center|389px| Figure 9: Start the tracking from saved tracking result]]
 
  
 
===Tracking Result===
 
===Tracking Result===
−
For each frame, the system outputs a image displaying the tracking result, an Excel file storing the worm features and a txt file storing worm shape information. In the txt file, each worm item contains four rows: the feature row which is the same as the information stored in the Excel file, two rows storing worm centerline points and one row storing the width at each centerline point. Features stored in the Excel file include:
+
The user needs to create a folder for storing the result first, and then specify the folder in the worm system. For each frame, the system outputs a image displaying the tracking result, an excel file storing the worm features and a txt file storing worm shape information. In the txt file, each worm item contains four rows: the feature row which is the same as the information stored in the excel file, two rows storing worm centerline points and one row storing the width at each centerline point. Features stored in the excel file include:
  
 
   1.Length // length of the worm
 
   1.Length // length of the worm
Line 89: Line 60:
 
   10.MER.Length //length of the minimum enclosing rectangle
 
   10.MER.Length //length of the minimum enclosing rectangle
 
   11.Speed //axial progression speed
 
   11.Speed //axial progression speed
−
   12.Centroid Velocity // velocity of the worm controid
+
   12.Centroid Velocity // velocity of the worm centroid
 
   13.Track Length //length of the track of the worm
 
   13.Track Length //length of the track of the worm
 
   14.Reversal Duraction //duration of reversal sessions this worm performed so far
 
   14.Reversal Duraction //duration of reversal sessions this worm performed so far
Line 97: Line 68:
 
   18.State //"FORD" (forward), "REVE" (reversal), "OMEB" (omega bend), "PAUS" (pausing), "LEFT" (left)  
 
   18.State //"FORD" (forward), "REVE" (reversal), "OMEB" (omega bend), "PAUS" (pausing), "LEFT" (left)  
  
−
In the "output menu", the user could also generate a summary of worm motion labels in a Excel file, in which each worm has a row storing all its motion labels (0=not active, 1=forward, 2=reversal, 3=Pasuing, 4=Omega Bend, 5=Physical Contact) and a row storing its speed values.
+
In the "output menu", the user can generate a summary of worm motion labels into an excel file, in which each worm has a row storing all its motion labels (0=not active, 1=forward, 2=reversal, 3=Pausing, 4=Omega Bend, 5=Physical Contact) and a row storing all its speed values.
 +
 
 +
===Explore Result===
 +
If the directory for saving tracking results is provided, users could explore the result during the tracking process simply by pausing the tracking first, checking the "explore result" box and sliding the "frame number" slider. The "explore result" box needs to be unchecked before resuming the tracking.
 +
[[Image:ExploreResult.png|thumb|center|389px| Figure 8: Explore the results]]
 +
 
 +
===Start Tracking from Saved File===
 +
To start the tracking from saved tracking result, the user needs to specify the directory storing the tracking result, click the third button in the toolbar and choose the txt file from which the tracking will start(figure9). 
 +
[[Image:StartFromSavedTxt.png|thumb|center|389px| Figure 9: Start the tracking from saved tracking result]]
 +
 
 +
===Test and Choose Settings===
 +
To test the current settings, the user could run the tracking process for several frames. If changes need to be made according to the tracking and binarization results, the user could pause the tracking process, make necessary changes to the settings and then start over the tracking process.
 +
Restarting the tracking from original starting point could be done with the "Start frame" spinbox. For example, after changes to settings have been made, the user could change current "Start frame" spinbox value from 1 to any other different number (which cancels the current tracking process, since tracking will be reset once change in the "Start frame" spinbox is detected), and then change the number back to 1. The "Start frame" spinbox also allows user to start tracking from any time point/frame.
  
 
===Keyboard and Mouse Shortcuts===
 
===Keyboard and Mouse Shortcuts===
Line 108: Line 91:
 
   Ctrl+D: Delete selected worms
 
   Ctrl+D: Delete selected worms
 
   Ctrl+M: Output Summary of motion labels
 
   Ctrl+M: Output Summary of motion labels
−
    
+
   Ctrl+G: Start/Resume Tracking
 +
  Ctrl+P: Pause Tracking
  
−
   Left Click on anyviews: select the clicked worm object
+
   Left Click on any view: select the clicked worm object
−
   Ctrl+Left Click on anyviews: toggle the selection of the clicked worm object
+
   Ctrl+Left Click on any view: toggle the selection of the clicked worm object
 
   Double Left-Click on the tracking view: reverse selected worm
 
   Double Left-Click on the tracking view: reverse selected worm
 
   Double Right-Click on the tracking view: delete selected worm
 
   Double Right-Click on the tracking view: delete selected worm
  
−
==Download Worm System==
+
==Demos==
−
Worm System's source code is contained in a [http://subversion.tigris.org/ subversion repository] that has been configured to allow anonymous read-only access. You can check out a copy from the following location:
+
−
 
+
−
svn://www.openworld.rpi.edu/repos/farsight/branches/NewWormSystem
+
−
 
+
−
Installable versions of the system could be downloaded from:
+
−
 
+
−
 
+
−
Contributors to the New Worm System:
+
−
*Yu Wang
+
−
*Isaac Abbott
+
  
−
For information of our old worm project, please refer to [[http://farsight-toolkit.org/wiki/The_Worm_Project Old Worm Project]].
+
*Demos: four demos are provided for testing the system and helping users familiarize themselves with system operations and settings, some images are from [6] and the others are generously provided by Meghan Jobson at University of Utah. There is a setting file for each demo, and it could be read into the system by "Ctrl+O". The user needs to create a folder for storing the result first, and then specify the folder in the system.
 +
  [[Image:Demo1.png|left|45px]]
 +
  [http://censsis.ecse.rpi.edu/public_downloads/WormAnalysisSystem/Demos/Demo1.zip Demo1]: Multi-Worm.
 +
  [[Image:Demo2.png|left|50px]]
 +
  [http://censsis.ecse.rpi.edu/public_downloads/WormAnalysisSystem/Demos/Demo2.zip Demo2]: 2-Worm physical contact[6].
 +
  [[Image:Demo3.png|left|45px]]
 +
  [http://censsis.ecse.rpi.edu/public_downloads/WormAnalysisSystem/Demos/Demo3.zip Demo3]: 2-Worm physical contact[6].
 +
  [[Image:Demo4.png|left|50px]]
 +
  [http://censsis.ecse.rpi.edu/public_downloads/WormAnalysisSystem/Demos/Demo4.zip Demo4]: Multi-Worm.
  
−
== References ==  
+
==Summary==
−
*[1] Yu Wang, Roysam Badrinath, “Joint Tracking and Locomotion State Recognition of C.elegans from Time-Lapse Image Sequences,” IEEE International Symposium on Biomedical Imaging (ISBI) 2010 (oral presentation).
+
The current system includes the background subtraction, detection, tracking, feature computation, display and interactive editing modules. We are working on adapting the current system into FARSIGHT, and on linking the system with other utilities of the FARSIGHT. We are also going to develop other worm analysis modules like the phenotype classification module.
−
*[2] Roussel Nicolas, Morton Christine A, Finger Fern P, Roysam Badrinath, “A computational model for C. elegans locomotory behavior : Application to multiworm tracking,” IEEE transactions on biomedical engineering, pp. 1786-1797, 2007.
+
−
*[3] Katsunori Hoshi, Ryuzo Shingai, “Computer-driven automatic identification of locomotion states in Caenorhabditis elegans,” J Neurosci Methods, vol. 157(2), pp.355-363, 2006.
+
−
*[4] Huang KM, Cosman P, Schafer WR, “Machine vision based detection of omega bends and reversals in C. elegans,” Journal of Neuroscience Methods, vol. 158, Issue 2, pp. 323-336, 2006.
+
−
*[5] Nicolas Roussel, "A Computational Model for C.elegans locomotory behavior: Application to Multi-Worm tracking", Phd Thesis, 2007.
+
−
*[6] K.M. Huang, P.C. Cosman, and W. Schafer, “Automated tracking of multiple C. elegans with articulated models,” IEEE International Symposium on Biomedical Imaging (ISBI) 2007, pp. 1240-1243, 2007.
+
−
*[7] Fontaine, E., Barr, A., Burdick, J. W., “Model-based tracking of multiple worms and fish,” ICCV Workshop on Dynamical Vision, 2007.
+
−
*[8] Wei Geng; Cosman, P.; Berry, C.C.; Zhaoyang Feng; Schafer, W.R., “Automatic tracking, feature extraction and classification of C. elegans phenotypes,” Biomedical Engineering, IEEE Transactions on, vol.51, no.10, pp.1811-1820, 2004.
+
−
*[9] Isard, M.; Blake, A., “A mixed-state condensation tracker with automatic model-switching,” Computer Vision, 1998. Sixth International Conference on, pp.107-112,1998.
+
−
*[10] G.J. Stephens et al., "Dimensionality and Dynamics in the Behavior of C. elegans", PLoS Comp. Biol., 2008.
+

Latest revision as of 17:22, 7 June 2011

We are now working on applying our 3-D tracing algorithm for the Worm tracking problem. Some preliminary results produced by the tracing system are shown in figures below.


"Tracing" the worms
"Tracing" the worms



Contents

The Worm Tracking System (Old)

There is a compelling need for improved software systems for delineating the C.elegans worms in time-lapse image sequences and tracking their movements. Segmentation and tracking provide the basis for making quantitative measurements of worm morphology, and various aspects of their dynamic behaviors. To facilitate the quantitative studies of worm behaviors, we developed an automated worm analysis system which is open-source and could be adopted by all researchers in the C.elegans community.


System Overview

Figure 1: Main Window of the System

Main Window

Figure1 shows a screenshot of the main window of the system.

  • Menu: the menu provides operations like file operations and Editing Operations which allow user to edit (reverse, delete and correct) selected worms in the tracking window.
  • Toolbar: the toolbar provides three buttons (figure2: suspend tracking, start/resume tracking, start tracking from a saved point) for controlling the tracking process. Users can suspend the tracking process at any time point to do observation or interactive editing when necessary. The third button makes it possible for the user to start the tracking process from any saved tracking result which stores worm shapes, features and identities. Thus the user doesn't need to start over the tracking if the program was closed normally or abnormally.
Figure 2: Toolbar
  • Input and Output: the input and output sections allow users to specify directory of input image sequence and the directory for storing tracking result. For each frame, the tracking result includes a text file storing worm structure (centerline and contour points), a excel file storing extracted features (locomotion states, shape and motion features, etc.) and a png image took from the tracking window. The results could be explored at anytime during or after the tracking process [Explore Result].
  • Settings: the settings section allows users to specify settings for core processing algorithms. Please refer to Settings for detailed information about these settings.
  • Tracking Window: the tracking window provides both "tracking view" and "foreground view". Tracking view displays the centerline (head is shown as a yellow circle) and contour of tracked worms, and also their identity and motion state (F=Forward, R=Reversal, P=Pausing, O=Omega Bend, C=Physical Contact). Worms are highlighted by a transparent Minimum Enclosing Rectangle once selected. The foreground view shows the label image of foreground objects obtained by background subtraction method, and could help the user choose correct binarization parameters. A good binarization result means worm boundaries are well preserved in the binary image, and the foreground objects have no hole or dent.
  • Log Window: log window displays a log for the tracking process. More detailed information about the tracking process is displayed in another terminal window.

Multiple Linked Views

Worm Analysis System provides a linked-view architecture where worm objects can be visualized in different views like image, table and scatter plot. Worm objects selected in one view are immediately highlighted in all the views (figure3).

Figure 3: Multiple Linked Views
Figure 4: Foreground View

Algorithms

  • Background Subtraction: the system provides two background subtraction methods to handle images with different backgrounds.
    • Background Subtraction Method I: A background image is computed first from all available images. Incoming image is subtracted with the obtained background image and the subtraction result is fed to Graph-Cut algorithm to output a smoothed binary image. This method applies only for static background.
    • Background Subtraction Method II: Motion-based and intensity-based models are used to detect moving foreground objects. The obtained foreground confidence map is fed to a Spatio-Temporal Graph-Cut algorithm to output spatially and temporally smoothed binary foreground image.
  • Tracking: the system incorporates our latest worm tracking algorithm [1], which could track multiple worms and recognize their locomotion states (Forward, Reversal, Pausing and Omega Bend) simultaneously. The tracker is initialized by the first two image frames on which worm detection is performed and head and tail are automatically determined (we assume worms are moving forward at the first two frames. If worm's head is not correctly identified, the user need to manually reverse the worm).
  • Feature Computation: the system computes a subset of worm features in real-time during tracking. More features could be computed from the saved worm structures (centerline and contour points). For a complete list of our worm features, please refer to [Worm Features and Events].

System Operations

Edit Worms

To edit the worms when necessary, user can suspend the tracking process and then use the "Edit" menu (figure4) which provides three group editing operations. Deleting or reversing single worm could be done by double right-click or double left-click. Worm correction is only effective when selected worms are not in physical contact or doing omega bend.

Figure 5: Editing Worm: selected worms could be corrected by detection(if the tracking is not correct, and worm is not doing omega bend or in physical contact), reversed or deleted


Figure 6: Contour tracking for worms in physical contact
Figure 7: Bounding Box tracking for worms in physical contact

Tracking Result

The user needs to create a folder for storing the result first, and then specify the folder in the worm system. For each frame, the system outputs a image displaying the tracking result, an excel file storing the worm features and a txt file storing worm shape information. In the txt file, each worm item contains four rows: the feature row which is the same as the information stored in the excel file, two rows storing worm centerline points and one row storing the width at each centerline point. Features stored in the excel file include: 

 1.Length // length of the worm
 2.Width //width of the worm
 3.Curvature //curvature of the worm centerline
 4.Area //foreground object size underlying the worm
 5.Orientation // orientation of the line linking head and tail
 6.Head-Body Dist //distance between head and body (worm centroid)
 7.Head-Tail Dist //distance between head and tail
 8.HCT Angle //angle between line linking centroid and head and the line linking centroid and tail
 9.MER.Amp  //height of the minimum enclosing rectangle
 10.MER.Length //length of the minimum enclosing rectangle
 11.Speed //axial progression speed
 12.Centroid Velocity // velocity of the worm centroid
 13.Track Length //length of the track of the worm
 14.Reversal Duraction //duration of reversal sessions this worm performed so far
 15.Reversal Count //times of reversal sessions this worm performed so far
 16.Contact Duration //duration of physical contact with other worms
 17.Worm Duration //duration of the worm in the field
 18.State //"FORD" (forward), "REVE" (reversal), "OMEB" (omega bend), "PAUS" (pausing), "LEFT" (left)

In the "output menu", the user can generate a summary of worm motion labels into an excel file, in which each worm has a row storing all its motion labels (0=not active, 1=forward, 2=reversal, 3=Pausing, 4=Omega Bend, 5=Physical Contact) and a row storing all its speed values.

Explore Result

If the directory for saving tracking results is provided, users could explore the result during the tracking process simply by pausing the tracking first, checking the "explore result" box and sliding the "frame number" slider. The "explore result" box needs to be unchecked before resuming the tracking.

Figure 8: Explore the results

Start Tracking from Saved File

To start the tracking from saved tracking result, the user needs to specify the directory storing the tracking result, click the third button in the toolbar and choose the txt file from which the tracking will start(figure9).

Figure 9: Start the tracking from saved tracking result

Test and Choose Settings

To test the current settings, the user could run the tracking process for several frames. If changes need to be made according to the tracking and binarization results, the user could pause the tracking process, make necessary changes to the settings and then start over the tracking process. Restarting the tracking from original starting point could be done with the "Start frame" spinbox. For example, after changes to settings have been made, the user could change current "Start frame" spinbox value from 1 to any other different number (which cancels the current tracking process, since tracking will be reset once change in the "Start frame" spinbox is detected), and then change the number back to 1. The "Start frame" spinbox also allows user to start tracking from any time point/frame.

Keyboard and Mouse Shortcuts

 Ctrl+O: Open Settings
 Ctrl+S: Save Settings
 Ctrl+Q: Quit the program
 Ctrl+C: Correct selected worms
 Ctrl+R: Reverse selected worms
 Ctrl+D: Delete selected worms
 Ctrl+M: Output Summary of motion labels
 Ctrl+G: Start/Resume Tracking
 Ctrl+P: Pause Tracking

 Left Click on any view: select the clicked worm object
 Ctrl+Left Click on any view: toggle the selection of the clicked worm object
 Double Left-Click on the tracking view: reverse selected worm
 Double Right-Click on the tracking view: delete selected worm

Demos

  • Demos: four demos are provided for testing the system and helping users familiarize themselves with system operations and settings, some images are from [6] and the others are generously provided by Meghan Jobson at University of Utah. There is a setting file for each demo, and it could be read into the system by "Ctrl+O". The user needs to create a folder for storing the result first, and then specify the folder in the system.
Demo1.png
 Demo1: Multi-Worm.
Demo2.png
 Demo2: 2-Worm physical contact[6].
Demo3.png
 Demo3: 2-Worm physical contact[6].
Demo4.png
 Demo4: Multi-Worm.

Summary

The current system includes the background subtraction, detection, tracking, feature computation, display and interactive editing modules. We are working on adapting the current system into FARSIGHT, and on linking the system with other utilities of the FARSIGHT. We are also going to develop other worm analysis modules like the phenotype classification module.

Retrieved from "http://farsight-toolkit.ee.uh.edu/wiki/Worm_Analysis_System"
Personal tools