Revision as of 20:21, 27 September 2011

The FARSIGHT 3-D Trace Editor

From a biological image, tubular objects such as blood vessels and neurites can be represented by a series of cylinders. The centerlines of the cylinders can be represented by a series of 3-D points. By providing information about the connections between points the tree structure can be traced and stored in a data structure. The FARSIGHT 3-D Trace Editor is designed to provide a streamlined platform for multi-view visualization and Edit-based Validation System(EVS) for automatic neurite and vessel tracing algorithms. The Actively Linked Integrated Spaces Architecture (ALISA) manipulates multiple views and allows for the simplification of the display and subset selection of data, such as a 3D rendering, tables, or scatter plots. Group and automated editing of the trace structure is possible using ALISA, as pattern analysis and clustering can aid in selection. The primary view in the editor is the 3D rendering which has the capability to display the traces as color coded lines with an overlay of a volume rendering from original image. The rendering can be viewed from any angle and at any desired magnification. The secondary display is the linked space where computed features can be displayed simultaneously in table and scatter plot form. The table and plot views can filter and sort the features simplifying selections which are automatically updated in all views. The group and automated functions provide an efficient method for validating complex data sets.

This is an example of how Linked Space Editing can use a query of the features to detect errors in the Data. The specific error being detected is an error in branching where a parallel trace is attached to the trunk.

Since the traces are represented as a sequence of points in 3-D space, they have a sense (directionality). Many automatic tracing algorithms produce a collection of trace fragments (segments) that don't have a consistent set of directions (sense). The trace editor is designed to cope with this challenge. It is also designed to allow "group editing". Using the linked space, you can select multiple segments at once, and perform an edit operation over the selected set of segments.

The Trace Editor allows use of multiple linked spaces for different 'levels' of operations. The Merge operation has its own object type to characterize gaps between trace lines; this linked space is only active and visible when you are working on the merge operations.

This screen shot illustrates the tool's ability to display secondary neuronal structures such as spines. The light blue traces correspond to the dendritic backbone, and the dark blue lines are the dendritic splines. The traces highlighted in orange are the selected traces. The small spheres indicate the spine bases.	This is another view of the linked space of the spines showing the features in a scatter plot allowing comparisons of traces using two features at a time.
This view in the linked space shows the spines' features in table form. Additional features loaded in through the RPIxml file can also be displayed.

Getting Started:

When you double click on the Trace Editor Icon, a dialog will appear asking for user information and initial files to open. The files include an acceptable tracing format and an image. Trace Editor can load additional files through the file menu while running. There are two rendering options for the input images: a raycaster and a contour filter. The Raycaster is used for a more detailed image such as the initial image. The Contour Filter is used for displaying segmented images, such as somas, as a volume. Currently, Trace Editor accepts the following file types as input:

1. Trace Files:
   • .swc
   • xml (RPI format)
   • .vtk^* (this is only the 3D structure)
2. Image Files( 8-bit gray-scale or binary):
   • tif
   • pic
   • mhd
3. Trace Project
   • .xml

Basic Usage

• Navigate 3D Space
  • Rotating View
    • hold LMB while dragging
    • hold ctrl + LMB for spin
  • Pan/Scroll
    • hold MMB while dragging
    • hold shift+ LMB (this is required on Macs and most laptops)
  • Zoom
    • hold RMB while dragging
    • scroll MMB
• Selections
  • Select a trace
    • 3D view: RMB click on the line (note: a 3D cursor will appear if enabled)
    • Linked space plots: LMB click (to make a group selection, hold shift and multiple clicks to make a lasso outline)
• Picking a point in 3D
  • A 3D cursor will be used in the main window to mark where you selected
  • Enable the 3D Cursor dock from the toolbar menu for readout of coordinates and tool options
  • The cursor can be moved by:
    • Typing in a new coordinate into the fields in the 3D Cursor dock
    • Drag the cross-hairs to a new location (press update location if you dragged it)
    • To move only along one axis (x, y, or z) hold shift while dragging cross-hairs

RMB=Right Mouse Button
LMB=Left Mouse Button
MMB=Middle Mouse Button (scroll wheel if you have one)
*Important: 
If you load a .vtk file you must set all necessary root nodes of the trees before you do ANY other operations for branching to work. 
This is necessary because A .swc file already has direction information but the .vtk file is an unordered graph. 
Select your traces and use the 'Set Root' button or hot key 'R' 

This is how the Trace Editor Should look when loaded. The panel on the right is a controller to set up the Trace Editor. User information and the options to load files are displayed on the right. On the left side is the log file. When all files are loaded the start button will start the render window. Be patient as it may take a few moments to finish loading.

Trace Project

The Trace Project is a xml document that tells the trace editor where to find multiple input files. The Project file loads information such as file type, and translation coordinates to tile multiple files into a montage. The following code is an example of an image translated so that it registers to the trace loaded.

 <?xml version="1.0" ?>
 <Source>
  <File FileName="C:\Lab\Traces\NC\Neocortical_Axons_01.tif" Type="Image" tX="73" tY="507" tZ="-5"/>
  <File FileName="C:\Lab\Traces\NC\NC_01.swc" Type="Trace" tX="0" tY="0" tZ="0"/>
 </Source>

Trace Editor will search in the folder the project file is in and load as many of the files as it can find

• Some Information on the project file
  • File Name:
    • If know you can include the Path such as C:\Lab\Traces\NC\
    • For portability between computers just use the filename
  • Type:
    • "Image" Original Image Data
    • "Soma" Segmented Image of the Cell Body
    • "Trace" Any Valid Trace File
  • tX, tY, Tz,
    • Values of the transform to be applied along the respective axis

Basic View

The results can show the Traces displayed as lines, and the original image as a 3D volume with adjustable brightness and opacity. This allows for validation of the traces, and a more intuitive way to visualization of the data than 2D slices or table formats. In addition to being able to display the original image, the Trace Editor can also display segmented soma images. This data is also rendered in 3D and then overlaid on the traces. Rendering the somas along with the traces allows for more accurate validation. The increased accuracy enables greater ease in associating processes with the proper soma and also in correcting and validating the directionality of the processes.

This Image has been annotated to Label the Trace Editor's indicators. The Important markers shown are the 3D Cursor, Branch Point Markers, and Direction Arrows. The Selected Line is highlighted in Yellow, while the other lines are colored Blue. The original image is displayed with the RayCaster. The colors in the RayCast Rendering indicate intensity Red Being highest and Green lower Intensity. The Image Rendering properties can be adjusted using the toolbar on the bottom of the screen.

Cell Analysis Tools

The Cell Analysis is a reimplemented version of the L-Measure toolkit. The Cell Analysis allows comparison of cells based on cellular morphology features. Selecting a row of the cell features linked space highlights all corresponding trace segments.

Editing Tools

Once the Trace Editor has successfully launched, the render window will have interactive 3-D viewing and editing capabilities. Several operations have automated and semi-automated functionality depending upon the input. For manual selective editing, you can point the computer mouse and highlight a series trace segments of by right clicking them(shown by selectable color). The editing operations are grouped by functions in the following categories of selection, modification, and settings.

Menu Bar

The menubar, found in the top-left of the trace editor, is outlined in red. From the menu bar, you can access tools to manipulate the image or retrieve information about the image.

1 File

• Load Traces
• Load Image
• Load Somas
• Save as
• Save Computed Cell Features Table
• Screen Shot
• Export Cells
• Remove Image Actors

2 Tool Bars

• Edit Toolbar
• 3D Cursor
• Segment Vessels
• Branch Toolbar
• Automated Edits
• Project Image Files

3 Visualization

• Editor Settings
• Renderer Mode
• Color by Trees
• RayCast Tools

4 Analysis

• Edit Log Information
• Show Plot
• Show Statistics Toolbar
• Cell Analysis

1 File Menu

Load Traces:

Opens Valid tracing files.

Load Image:

Opens Image into a ray-caster to visualize the original data.

Load Somas:

Opens a 3D Image into the contour rendering. Works best with segmented images.

Save As:

Saves all current Traces into a single .swc or .vtk file.

Save Selected Tree:

Saves entire tree structures selected into a .swc or .vtk file.

Screen Shot:

Saves current screen output into a .jpg or .jpeg file and allows you to select an output resolution.

Cell Export:

Saves selected cells into separate .swc and/or .jpg files.

Remove Image Actors:

Removes images but leaves the traces if there are traces.

2 Tool Bars

Edit Toolbar

Tools for Selecting and Editing traces

Selection Operations

This set of tools is for controlling which traces are modified. The Selections are highlighted int the 3D Viewer window and in the appropriate plots/tables in the linked space.

Small Lines:

Selects lines based on size criteria, selecting only small lines that are not part of a tree and displaying a message box allowing for group deletion.
(Hot Key "a")

List Selection:

Displays information about the currently selected trace lines.
(Hot Key "l") (lowercase L)

Clear Selection:

Deselect all currently highlighted lines. This operation also clears Gap and clears all computations for Gaps(see Merging) and Branch Points.
(Hot Key "c")

Select Tree:

This Operation will start with all selected lines, find the root and select all connected traces.
(Hot Key "t")

Trace Modification Operations

Tools for modifying the trace structure for changing trace structure:

Delete Trace:

Removes all selected trace segments.
(Hot Key "d")

Merge Traces:

operating in one of three modes it will attempt to merge traces based on selection inputs. The first time pressed it will determine and save all the comparisons for selection, and on second press will merge all selected comparisons.
(Hot Key "m")

• Two Traces selected: If a merge is possible under the current tolerance it will be merged.
• Multiple Traces Selected: All probable merges will be made based on selected lines and current tolerance.
• Null Selection: If there are no currently selected lines all possible merges will be made from the entire trace structure.

Split Trace:

Split each selected trace line into two parts, separating them at the point that they were selected.
(Hot Key "s")

Flip Trace Direction:

Reverses the direction of the selected trace lines. If a leaf node of a tree is selected it is the equivalent of calling Set Root, as it will become the new root when flipped and update the rest of the tree consistently. This command will not operate on a continuing segment(one that has both a parent and children).

Trace Type:

This command resets the type of the traces selected. The Types are defined as standard for an SWC file.

• 0 = undefined
• 1 = soma
• 2 = axon
• 3 = dendrite
• 4 = apical dendrite
• 5 = fork point
• 6 = end point
• 7 = custom

Intensity:

For setting the intensity values for every Trace Bit from the loaded image. This command currently only works when one image is loaded.

3D Cursor:

The 3D Cursor Toolbox for point selection

This toolbox allows coordinates to be specified in 3D space. New Trace Bits can be created at the cursor location.

Cursor Location:

Lists the 3D coordinates of the cursor last picked or specified

• The spin box can be used to precisely move the cursor
• Use 3D cursor: toggles the cross-hairs in the view if they are in the way of picking
• Update Location: Reads the coordinates of the 3D cursor

Create Soma:

Specific Create New Trace Bit function that takes two or more selected trees to attach to a new root trace that is a soma

Create New Trace Bit:

Create a new point in the trace structure

• If no traces selected it will create a new Trace Line
• If one trace is selected it will extend to either the front or back of the line (whichever is closest)
• If two or more traces are selected it will call the Create Soma function

Segment Vessels

Non-functional. In development.

Branch Toolbar

Allows user to edit on the tree level.

Tools used to control branching functions

Break:

Removes a branch from the tree. If the resulting branch becomes a fragment it is automatically deleted.
(Hot Key "shift + b")

Explode:

This command is the 'explode' command that breaks all of the branch points connections. The connections are not lost as they are stored in a graph structure. If the set root command is called on a trace the tree will be rebuilt using the specified root.
(Hot Key "e")

Branch Point:

Adds a children to a parent trace, creating a new branch point. The tree structure will be modified to reflect the change.
(Hot Key "b") Defines a parent as the first Trace selected and children as all following selections.

Set Root:

This will reorder the graph structure to define a root. If the tree is not already broken into a graph that will be done automatically.
(Hot Key "r")

Project Image Files:

The 3D Cursor Toolbox for point selection

When a project file (.xml) is loaded, a table is shown that list all the images opened from the project file.

Renderstatus:

Allows for on/off switching of individual images.

2d/3d:

Allows switching between the 2D mode (projection view) and 3D mode for individual images.

3 Visualization

Editor Settings

Allows user to adjust defaults and global parameters. The toolbox can be opened from the menu option Visualization->Editor Settings

Toolbox to Adjust Settings for rendering and global parameters

Selection Settings

Maximum Gap Length:

The maximum distance between endpoints allowed for merging.

Gap length tolerance:

The proportion of the length that a trace can be extended. The fraction of the length of the lines allowable for merging.

Small Line Length:

Threshold for what is considered a fragment, or small line to be deleted. Fragments are unconnected to any tree structure.

Display settings

Line Color RGB:

Change the highlight color for selected lines. Values range from 0-1, with reference values(R:0 - G:.5 - B:1).

Line width:

Adjustment for the width of rendered lines to help with visibility when displaying on screen.

Interactor style:

There are four interactor style options:

• Track Ball - rotation, panning, and zooming.
• Image - panning and zooming.
• RubberBandZoom - select an area to zoom into.
• Slicer - slide through slices.

Projection style:

There are three projection styles using raycasting:

• Maximum - show maximum values
• Mean - show average values
• Minimum - show minimum values

Projection plane:

There are three projection planes: X-Y, X-Z, Y-Z.

Rotation:

Setting the three angles defines where the camera is placed in relation to the image.

• Roll - spin image
• Elevation - up-down rotate
• Azimuth - left-right rotate

Background RGB Color

Setting the three values defines what background color the render window is.

• Sample RGB colors
  • (0,0,0) = black
  • (.6,.6,.6) = %60 gray
  • (1,1,1) = white

Renderer Mode

Set Slicer

(2D mode) View individual slices of a 3-D image. The slider bar allows for the selection of a specific slice, and below the slider bar is a spinbox that sets the slice thickness (multiple slices form one slice). The slices can be viewed in the x-y plane, the x-z plane, or the y-z plane.

Set Projection

(2D mode) Projection view. The styles can be manipulated from the Editor Settings.

Set Raycast

(3D mode)

Color By Trees

Randomly assigns a color to each trace to distinguish separate trees.

RayCast Tools

Adjust opacity threshold and brightness of the image.

4 Analysis

Edit Log Information

Provides information about when the Trace Editor was started.

Show Plots

Provides a Features Plot and a Features Table...

Show Statistics Toolbar

Cell Analysis

Provides a list of measurements using the L-measure functions.

Editing Operations Information

Note that the functions listed above are designed to handle multiple segments at a time. The Merge function determines the closest distances between trace endpoints, distances greater than the threshold are rejected. For detailed information on the features used in computation please see Trace Editor/Features. Conflicts between two lines attempting to connect to one endpoint are resolved by finding the closest endpoints. The merge command can be invoked on a set of traces, and this function will automatically connect all the possible segments. Multiple tracelines can be orginized and merged into a single linear trace.

Generalized work flow for how to edit an automated tracing output file.

Crossovers

Crossovers occur when two traces become close enough to be considered connected; usually the traces are parallel or apparently intersect. If there is separation between the traces the algorithm will either jump traces or form a bridge between them. For the case where the tracer jumped between the neurons the two traces need to be split apart and might require manual tracing. If there is not enough separation between centerlines, typically in the ‘z’ axis, a trifurcation forms where one trace has three children. The artificial branch point needs to be broken and the traces merged so that the two traces continue in their original directions. If there was a bridge formed it will present as either a full or half bridge. A full bridge is two traces with a connecting trace in between creating two false branch points. A half bridge has one false branch point and a child with a large separation from the parent.

Detection of crossings in the traces presented as a flowchart. This is the decision process to individually identify errors.

Crossovers detection can be accelerated using the features computed in the linked space to automatically highlight suspected errors. Crossings can be detected by identifying which branch points have three children. The trifurcation may also indicate false spines in a real branch point but they can be checked as the child trace would contain less than three points. A half bridge can be detected by identifying the separation distance of the child trace from the parent trace. Half bridges usually have high fragmentation of the trace and are usually leaf nodes. A Full bridge would be a continuing segment (neither root nor leaf) and have a low tracing density.

• For more information:
  • Trace Editor FAQ
  • Feature computation and Algorithms

References

• Luisi, Jonathan; Narayanaswamy, Arunachalam; Galbreath, Zachary; Roysam, Badrinath (2011). |"The FARSIGHT Trace Editor: An Open Source Tool for 3-D Inspection and Efficient Pattern Analysis Aided Editing of Automated Neuronal Reconstructions." Neuroinformatics.