FARSIGHT Tutorials/Bio-Formats

From FarsightWiki
Jump to: navigation, search

Contents

Overview

The Bio-Formats command line tools provide a simple yet powerful suite of utilities for working with microscopy file formats. FARSIGHT provides access to the Bio-Formats command line tools through its Python command line.

Using the tools, you can read and display pixels and metadata, convert between formats, manipulate OME-XML, and more. This document provides an overview of the available commands, and describes some useful techniques for working with microscopy data from the command line.


Getting the tools

To enable the command line tools from a command shell:

  1. Download loci_tools.jar and bftools.zip from the Bio-Formats download page.
  2. Unzip the bftools.zip archive into a new folder.
  3. Place your downloaded loci_tools.jar into that same folder.
  4. Optionally, add this folder to your PATH environment variable, for easier access to the commands.

To enable the tools from the FARSIGHT Python interpreter:

  1.  ???


Available commands

The following commands are included as part of the Bio-Formats command line tools:

  • bfconvert – Converts an image file from one format to another. Bio-Formats must support writing to the output file (determined by extension; see the supported formats table).
  • bfview – Launches the Bio-Formats image viewer, displaying the given file (if any).
  • formatlist – Displays a list of supported file formats in HTML, plaintext or XML.
  • showinf – Prints information about a given image file to the console, and displays the image itself in the Bio-Formats image viewer.
  • tiffcomment – Dumps the comment from the given TIFF file's first IFD entry; useful for examining the OME-XML block in an OME-TIFF file.
  • xmlindent – A simple XML prettifier similer to xmllint --format but more robust in that it attempts to produce output regardless of syntax errors in the XML.
  • xmlvalid – A command-line XML validation tool, useful for checking an OME-XML document for compliance with the OME-XML schema.


Supported file formats

You can generate a list of supported file formats using the formatlist command. The command allows for three different output formats—plain text, HTML, and XML—using the -txt, -html and -xml flags, respectively. If no arguments are provided, plain text output is used: 

$ formatlist

Zip: can read (zip)
Animated PNG: can read, can write, can write multiple (png)
JPEG: can read, can write (jpg, jpeg, jpe)
Portable Gray Map: can read (pgm)
[etc.]

Details listed include the name of the format, common filename extensions, and indications of three levels of support:

  1. read – if Bio-Formats provides a reader for opening data from the format
  2. write – if Bio-Formats provides a writer for saving data into the format
  3. write multiple – if the writer supports saving more than one image plane

You can also see a list of supported formats on the Bio-Formats web site.


Displaying pixels

The bfview command launches the Bio-Formats Image Viewer, a simple tool for visualizing images. Optionally, you can specify a filename as an argument, to load it into the viewer. Or you can use the Open command on the File menu to load an image file.

The showinf command is similar, but has many more features from the command line. Type showinf with no arguments for a complete list of options. The following are some examples of showinf in action, using sample data from the LOCI web site.

Probing an image file

The showinf command makes it easy to probe an image file: 

$ showinf 2chZT.lsm
Checking file format [Zeiss Laser-Scanning Microscopy]
Initializing reader
        Removing thumbnails
        Reading LSM metadata for series #0
Initialization took 22.601s

Reading core metadata
Filename = 2chZT.lsm
Series count = 1
Series #0:
        Image count = 399
        RGB = true (2)
[etc.]
        Plane #201 <=> Z 12, C 0, T 9
        Plane #398 <=> Z 20, C 0, T 18

Reading pixel data (0-398) ...[etc.]... [done]
16.03s elapsed (40.175438ms per image)

Launching image viewer
Warning: OME reader not available
Warning: OME writer not available
Warning: OMERO reader not available
Warning: OMERO writer not available

Reading global metadata
BeamSplitter #1 Acquire: true
DataChannel #1 Acquire: true
[etc.]
Track #2 Acquire: true
Track #3 Acquire: true

Reading metadata
BeamSplitter #1 Filter: Plate
BeamSplitter #1 Filter Set: NT3
[etc.]
VoxelSizeY: 0.38434245251816246
VoxelSizeZ: 0.75

The default showinf invocation performs several tasks:

  1. It initializes the file, parsing file headers, offsets and metadata.
  2. It displays core metadata values necessary for understanding the structure of the pixels. You can suppress this output with the "-nocore" option.
  3. It reads the pixels and displays them in the Bio-Formats Image Viewer. You can skip this step with the "-nopix" option.
  4. It displays the original metadata tables containing information specific to the file format implementation. You can suppress this output with the "-nometa" option.


Grouping files

If you have a collection of files numbered across one or more dimensional axes, you can instruct Bio-Formats to group them as a single dataset: 

$ showinf -stitch dub01.pic        
Checking file format [Bio-Rad PIC]
Initializing reader
        Reading image dimensions
        Reading notes
        Reading color table
        Populating metadata
Initialization took 0.166s

Reading core metadata
File pattern = /Volumes/DATA/samples/dub/dub<01-85>.pic
Used files:
        /Volumes/DATA/samples/dub/dub01.pic
        /Volumes/DATA/samples/dub/dub02.pic
[etc.]
        /Volumes/DATA/samples/dub/dub84.pic
        /Volumes/DATA/samples/dub/dub85.pic
Series count = 1
Series #0:
        Image count = 2805
        RGB = false (1)
        Interleaved = false
        Indexed = false (false color)
        Width = 768
        Height = 512
        SizeZ = 33
        SizeT = 85
        SizeC = 1
[etc.]

In the example above, each Bio-Rad PIC file contains 33 image planes corresponding to the focal planes, with each file representing a single time point in an 85 point time series.

Bio-Formats is pretty smart about detecting the "file pattern" of a collection of files. The result is expressed with a notation using angle brackets indicating the first and last numbers of the collection (in the example above: "dub<01-85>.pic"). More complex patterns are possible as well—e.g.: "tubhiswt_C<1-2>_TP<1-43>.ome.tif" or "nd40<01-60>0<2-3>.pic" or "<00-36:3>m.tiff" (00m.tiff, 03m.tiff, 06.tiff, etc., through 36m.tiff).


Other options

There are several other options that affect how the pixel data is read or displayed: 

   -thumbs: read thumbnails instead of normal pixels
    -merge: combine separate channels into RGB image
 -separate: split RGB image into separate channels
   -expand: expand indexed color to RGB
-normalize: normalize floating point images
     -fast: paint RGB images as quickly as possible
    -range: specify range of planes to read (inclusive)
   -series: specify which image series to read
     -swap: override the default input dimension order
  -shuffle: override the default output dimension order
     -crop: crop images before displaying; argument is 'x,y,w,h'
-autoscale: used in combination with '-fast' to automatically adjust brightness and contrast


Displaying metadata

Bio-Formats provides several means of displaying a file's metadata.

Standardizing metadata between formats

The showinf command is also capable of standardizing format-specific metadata into an OME-XML structure. You can invoke this feature using the "-omexml" command line option: 

$ showinf -nocore -nopix -nometa -omexml embryo2
Checking file format [Openlab LIFF]
Initializing reader
        Verifying Openlab LIFF format
        Finding image offsets
Initialization took 12.908s

Generating OME-XML (schema version 2008-09)
<?xml version="1.0" encoding="UTF-8"?>
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2008-09" [etc.]>
   <Image DefaultPixels="Pixels:0:0" ID="Image:0" Name="embryo2">
      <Pixels BigEndian="true" DimensionOrder="XYCZT" ID="Pixels:0:0"
              PhysicalSizeX="0.0" PhysicalSizeY="0.0" PixelType="uint16"
              SizeC="1" SizeT="1" SizeX="672" SizeY="512" SizeZ="99"/>
      <DisplayOptions Display="Grey" ID="DisplayOptions:0"/>
      <LogicalChannel ID="LogicalChannel:0" SamplesPerPixel="1">
         <ChannelComponent Index="0" Pixels="Pixels:0:0"/>
         <DetectorRef Gain="65.0" ID="Detector:0:0" Offset="0.0"/>
      </LogicalChannel>
      <CreationDate>2004-08-26T14:30:26</CreationDate>
      <InstrumentRef ID="Instrument:0"/>
   </Image>
   <Image DefaultPixels="Pixels:1:0" ID="Image:1" Name="embryo2">
      <Pixels BigEndian="true" DimensionOrder="XYCZT" ID="Pixels:1:0"
              PixelType="uint8"
              SizeC="3" SizeT="1" SizeX="672" SizeY="512" SizeZ="33"/>
      <DisplayOptions Display="RGB" ID="DisplayOptions:1"/>
      <LogicalChannel ID="LogicalChannel:1" SamplesPerPixel="3">
         <ChannelComponent Index="0" Pixels="Pixels:1:0"/>
         <ChannelComponent Index="1" Pixels="Pixels:1:0"/>
         <ChannelComponent Index="2" Pixels="Pixels:1:0"/>
      </LogicalChannel>
   </Image>
   <Instrument ID="Instrument:0">
      <Detector ID="Detector:0:0" Type="Unknown"/>
   </Instrument>
</OME>

Parsing schema path
http://www.openmicroscopy.org/Schemas/OME/2008-09/ome.xsd
Validating OME-XML
[etc.]

You can read more about OME-XML on the OME-XML web site.

Extracting OME-XML from an OME-TIFF file

If you have an OME-TIFF file, you can extract its embedded OME-XML comment using the tiffcomment command (piped to the xmlindent command for ease of reading): 

$ tiffcomment tubhiswt_C1.ome.tif | xmlindent -valid
<?xml version="1.0"?>
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2008-02" [etc.]
     UUID="urn:uuid:1fdb0a1a-5374-46da-bd24-d04e8cbf3175">
   <Experiment ID="urn:lsid:loci.wisc.edu:Experiment:OWS348" Type="PGIDocumentation">
      <Description>4 Cell Embryo</Description>
      <ExperimenterRef ID="urn:lsid:loci.wisc.edu:Experimenter:116"/>
   </Experiment>
   <Experimenter ID="urn:lsid:loci.wisc.edu:Experimenter:116">
      <FirstName>Maimoon</FirstName>
      <LastName>Nasim</LastName>
      <Email>mnasim@wisc.edu</Email>
      <Institution>
      </Institution>
      <GroupRef ID="urn:lsid:loci.wisc.edu:Group:3"/>
   </Experimenter>
   <Instrument ID="urn:lsid:loci.wisc.edu:Instrument:OWS1">
      <Microscope Manufacturer="Nikon" Model="Eclipse TE300"
                  SerialNumber="U629762" Type="Inverted"/>
      <LightSource ID="urn:lsid:loci.wisc.edu:LightSource:OWS1"
                   Manufacturer="Spectral Physics" Model="Tsunami 5W" SerialNumber="2123">
         <Laser Type="SolidState" LaserMedium="TiSapphire"/>
      </LightSource>
      <Detector ID="urn:lsid:loci.wisc.edu:Detector:OWS1"
                Type="PMT" Manufacturer="Hamamatzu" Model="H7422"/>
      <Detector ID="urn:lsid:loci.wisc.edu:Detector:OWS2"
                Type="Photodiode" Manufacturer="Bio-Rad" Model="1024TLD"/>
      <Objective ID="urn:lsid:loci.wisc.edu:Objective:OWS2"
                 Manufacturer="Nikon" Model="S Fluor" SerialNumber="044989">
         <Correction>PlanApo</Correction>
         <Immersion>Oil</Immersion>
         <LensNA>1.40</LensNA>
         <NominalMagnification>100</NominalMagnification>
         <CalibratedMagnification>100.0</CalibratedMagnification>
         <WorkingDistance>0.13</WorkingDistance>
      </Objective>
   </Instrument>
   <Image Name="tubhiswt" ID="urn:lsid:loci.wisc.edu:Image:OWS348"
          DefaultPixels="urn:lsid:loci.wisc.edu:Pixels:OWS348">
      <CreationDate>2005-12-09T17:16:09</CreationDate>
      <ExperimenterRef ID="urn:lsid:loci.wisc.edu:Experimenter:116"/>
      <LogicalChannel ID="urn:lsid:loci.wisc.edu:LogicalChannel:OWS348-1">
         <LightSourceRef ID="urn:lsid:loci.wisc.edu:LightSource:OWS1"/>
         <DetectorRef ID="urn:lsid:loci.wisc.edu:Detector:OWS1"/>
         <ChannelComponent Pixels="urn:lsid:loci.wisc.edu:Pixels:OWS348" Index="1"/>
      </LogicalChannel>
      <LogicalChannel ID="urn:lsid:loci.wisc.edu:LogicalChannel:OWS348-2">
         <LightSourceRef ID="urn:lsid:loci.wisc.edu:LightSource:OWS1"/>
         <DetectorRef ID="urn:lsid:loci.wisc.edu:Detector:OWS2"/>
         <ChannelComponent Pixels="urn:lsid:loci.wisc.edu:Pixels:OWS348" Index="0"/>
      </LogicalChannel>
      <Pixels PixelType="uint8" SizeX="512" SizeY="512" SizeZ="1" DimensionOrder="XYZTC"
              ID="urn:lsid:loci.wisc.edu:Pixels:OWS348" BigEndian="false" SizeC="2" SizeT="1">
         <TiffData FirstC="0">
            <UUID FileName="tubhiswt_C1.ome.tif">urn:uuid:1fdb0a1a-5374-46da-bd24-d04e8cbf3175</UUID>
         </TiffData>
         <TiffData FirstC="1">
            <UUID FileName="tubhiswt_C2.ome.tif">urn:uuid:583e4cdd-e828-41d1-a75e-b53caa8876ee</UUID>
         </TiffData>
      </Pixels>
   </Image>
</OME>

You can read more about OME-TIFF on the OME-TIFF web site.


Converting data between formats

Bio-Formats allows conversion between file formats. Of course, it must support reading from the input format, and writing to the output format (see "Supported file formats" above).

The bfconvert command handles such conversions, and supports some of the same options as showinf. E.g.: 

$ bfconvert -stitch TAABA1.PIC taaba.mov
TAABA1.PIC [Bio-Rad PIC] -> taaba.mov [QuickTime] ..[etc.].. [done]
9.758s elapsed (7396.0+1795.0ms per image, 501ms overhead)

Run bfconvert with no arguments to see the complete usage instructions.


Validating metadata

If you have an OME-XML metadata block, you can validate it (check it for syntax errors) using the xmlvalid tool. For example, you can validate an OME-TIFF's metadata block using the command: 

$ tiffcomment myFile.ome.tif | xmlvalid

You will see a list of errors indicating where the OME-XML string does not follow the rules of the OME-XML schemas.

The showinf command also automatically attempts OME-XML validation whenever the "-omexml" option is given.


Questions

Please direct questions to the Bio-Formats team.

Retrieved from "http://farsight-toolkit.ee.uh.edu/wiki/FARSIGHT_Tutorials/Bio-Formats"
Personal tools