REQUEST ACCESS
Lattice Light Sheet FAQs 2017-02-08T20:51:54+00:00

For a more guided introduction to working with lattice light sheet data on the CBMF server, see the post-processing workflow tutorial.

Why is it necessary to deskew lattice light sheet data? 2017-03-03T22:44:55+00:00

The fact that the detection objective lens is tilted at an angle with respect to the axis of stage movement results in an unconventional geometry for the acquired image stack.  The following video attempts to explain the need for post-processing (deskewing) of data acquired in stage-scanning mode on the lattice light sheet microscope.

The video below from the Betzig group is also useful for visualizing the three-dimensional geometry of the sample (gray) as it moves through the light sheet plane (blue).  The red/orange highlighted part of the sample is the optically sectioned image at each position.

Preview images remotely 2017-02-14T11:08:48+00:00

Using the llsinfo over an ssh connection, you can also preview lattice light sheet data remotely.  This is useful when you want to verify the data contained in an LLS folder, perhaps before downloading/processing, etc…  You can view either the MIP file, or the Z stack for a specific channel/timepoint.

  1. Log in to the CBMF server using X11 forwarding.
    • Note: remotely previewing data requires X11 forwarding. You must add the -X flag to your ssh connection and install a local Xserver:
    • ssh -X user@cbmf-latwork.med.harvard.edu

Show MIP file

  1. Use the --showmip or -s flags with the llsinfo command to get a preview of the maximum-intensity-projection (assuming a MIP exists).
    • llsinfo /path/to/LLS/folder --showmip
  2. A window should appear on your local computer with a preview of the dataset.  You can click and drag on the sliders at the bottom to change the channel, or the timepoint.
preview lattice light sheet data

Previewing an image with the llsinfo --show command

Note: this function can usually be used to preview the MIP file for data that has been compressed for long term storage.

Show Z stack for specific channel/timepoint

  1. Use the --showz or -z flags with the llsinfo command to get a preview of a Zstack for a specific channel/timepoint (assuming the file exists).  This flag accepts up to three additional numbers, seperated by spaces representing: [channel, timepoint, deskewed(0)/deconvolved(1)].   Remember, the filenames are zero-indexed, which means the first channel is number “0” and the second channel is number “1”.  The --showz flag should come at the end of the command.  Here are a couple examples:
    • show channel 1, timepoint 1, deskewed:
      • llsinfo /path/to/LLS/folder --showz
    • show channel 1, timepoint 6, deskewed:
      • llsinfo /path/to/LLS/folder --showz 0 5
    • show channel 2, timepoint 1, deconvolved:
      • llsinfo /path/to/LLS/folder -z 1 0 1
  2. A window should appear on your local computer with a preview of the dataset (or an alert may appear saying the file cannot be found).  You can click and drag on the slider at the bottom to change the z position.
Getting info on an LLS data folder 2017-02-08T09:30:05+00:00

When logged in to the CBMF server via ssh, you can use the llsinfo command to get metadata about LLS folders in a given directory (recursively). For instance, if there is a folder 20170120_Lattice in the current directory that contains a number of LLS datasets, type llsinfo 20170120_Lattice to get information:

To get more detailed information, use the -v or --verbose flag:

Transferring files from the CBMF server 2017-02-08T22:56:29+00:00

There are multiple way to transfer files from the CBMF server.  Note all of these methods require being on the HMS local network, or using a VPN to connect to the HMS network.

Note: Connecting to the server using any of these methods in requires that you have an account on the CBMF server.  If it is your first time connecting and you are unable to connect using the instructions below, try first connecting to cbmf-nas.med.harvard.edu with your ecommons credentials using the instructions here.  If you are still unable to log in, contact us.

All user data is stored in a folder with your name at the path /lattice/data_share

Select a method for details:

Mac

  1. From your computer’s desktop choose Finder > Go > Connect to Server
  2. In the Server Address box type one of the two following addresses:
    • For the data folder:   smb://cbmf-nas.med.harvard.edu/data_share
    • For your home folder:   smb://cbmf-nas.med.harvard.edu/home/
  3. Click Connect
  4. Select Registered User
  5. Enter your eCommons user name and password and click Connect

PC

  1. Choose Start > Run from the Desktop taskbar.
  2. Type one of the two following addresses:
    • For the data folder:   \\cbmf-nas.med.harvard.edu\data_share
    • For your home folder:   \\cbmf-nas.med.harvard.edu\home\
  3. Enter your eCommons user name and password when prompted and hit enter.
You may use an SFTP client (such as FileZilla or Cyberduck) to log in to the CBMF server using the following connection settings:

Protocol: SFTP
Server: cbmf-nas.med.harvard.edu
Port: 22
UserName: ecommons username
Password: ecommons password

Advanced Linux or Mac users may use bbcp to transfer files over an SSH connection
Combining multiple channels/timepoints into a hyperstack in Fiji 2017-02-05T23:29:18+00:00

By default, data acquired on the lattice is saved as individual Z-stack TIFF files for each timepoint, for each channel.  Viewing an entire 5D dataset can be very challenging, and demands a computer with enough RAM to hold the full dataset (or the use of virtual stacks, which have drawbacks).  We strongly encourage contacting IDAC for assistance with visualization and analysis of datasets acquired on the lattice light sheet.  If you wish to view the full 5D dataset, the following procedure will arrange the data into a single hyperstack in ImageJ/Fiji.

  1. In Fiji, select File → Import → Image Sequence...
  2. Navigate to the folder containing the data (such as the Deskewed folder in your LLS folder), and click Open.
  3. In the Sequence Options window that pops up, you can either just click OK to select the full dataset, or change the options to select a subset of the data.
  4. Separate the resulting stack into the appropriate dimensions using Image → Hyperstacks → Stack to Hyperstack...
    • Order: typically ‘xyzct’ (but confirm that)
    • Enter the actual number of channels, slices, and frames in the data
      • Channels is easy to figure out
      • Number of frames can be quickly determined from the number of files that you opened (divided by the number of channels).
      • Number of slices can be determined by opening a single Z stack, or by dividing the number of planes in your combined stack by the number of channels multiplied by the number of frames.
      • channels xslices xframes must equal the number of planes in the imported combined stack.
    • Hit OK
  5. The resulting 5D Hyperstack is useful for extracting an ROI from a subset of the data in XYZC or T.
Camera pixel correction 2017-02-05T23:06:31+00:00

Understanding the Correction

There is a phenomenon in the camera used on the Lattice (Hamamatsu, ORCA Flash4.0) that causes a small amount of the electrons that accumulate in a given pixel during an exposure to “carryover” into the following exposure (they “fail” to be cleared during readout), in a non-linear and pixel dependent fashion. This means that a bright object in one channel can leave a noisy “ghost image” in the same part of the field of view in the immediately following image (e.g. in the next channel). This results in a very annoying artifact in the deskewed and max-projected images

lattice light sheet orca flash artifact

Channel 2 demonstrates the residual electron artifact seen in the Hamamatsu ORCA Flash 4.0 in Synchronous Trigger mode (as used on the Lattice Light Sheet Microscope), after deskewing and max projection.

We have characterized this phenomenon for our camera and can “correct” this issue, predicting the carryover charge expected in any given pixel in the image as a function of the intensity of that pixel in the immediately preceding image.

ORCA flash artifact after correction

Hamamatsu ORCA Flash 4.0 residual electron artifact before and after correction.

Applying the Correction

In order to apply this correction to your dataset during post-processing, use the -c or -C flag with the lls command:

lls -C /path/to/LLSfolder or lls -c /path/to/LLSfolder

The capital -C flag additionally applies a selective median filter, after Flash pixel correction, styled after Phillip Keller’s code published here.

The code for this correction exists for both MATLAB and Python and will be provided on request. Please contact us.

Explanation of LLS data structure/format 2017-02-14T11:23:42+00:00

Top level LLS folder

A typical lattice light sheet experiment will yield a single folder that will contain some or all of the following files/folders:

  • Top level folder (e.g. “cell1”)
    • many “raw” (pre-processed) TIFF files.  See below for the naming convention.
    • a Settings.txt file with metadata about acquisition settings.
    • a Deskewed folder that contains the deskewed data.
    • a GPUdecon or CPPdecon folder that contains any deconvolved data. This directory may not include full deconvolved Z-stacks depending post-processing parameters, but will likely include a max intensity projection “MIP” file for all channels & time points for easy viewing (scroll to the bottom of the folder),  the MIP files may also be separated inside a MIP folder.  To retain or retrieve the complete deconvolved data during processing, see here.
    • a ProcessingLog.txt file with JSON-formatted data about the data and post-processing

To combine separate timepoint and/or channel stacks/MIPS into a single 4-5D hyperstack, see here.

File naming convention

All TIFF files follow the following naming convention:

name_channel#_stack#_laser_relativeTime_absoluteTime.tif

The name may additionally include one or more of the following flags:

  • COR means that the image has been corrected for bad/residual pixels
  • deskewed means the image has been deskewd
  • decon means the image has been deconvolved
  • MIP_axis means the image is a maximum intensity projection along the specified axis
Compressing and decompressing data for short and long-term storage 2017-02-05T17:24:35+00:00

Short term storage

We recommend compressing just the raw data after deskewing/deconvolving.  This can be done automatically during the initial processing by adding the --compress or -z flag to the lls command:

lls --compress /path/to/LLSfolder

With data that has already been processed, the raw data can be quickly compressed without affecting the Deskewed or GPUdecon folders by using the llscompress command:

llscompress /path/to/LLSfolder

To undo/decompress the above command, use the -z flag:

llscompress -z /path/to/LLSfolder

Long term storage

If you are not going to analyze the deskewed data anytime soon, we recommend deleting the deskewed/deconvolved data and compressing just the raw data (i.e. “deep freeze”).

llscompress --freeze /path/to/LLSfolder

This can reduce data size as much as 10-40 fold (depending on the data-set). To regenerate deskewed data, a folder that has been “frozen” can be easily reprocessed just like a normal LLS folder:

lls --reprocess /path/to/LLSfolder

bzip2 compression is used by default, but this can be changed in the options.

For a complete list of options in the llscompress program, type the following at the command prompt:

llscompress -h

Post-processing without auto-cropping 2017-02-05T16:43:01+00:00

By default, deskewed/deconvolved images will be “auto-cropped” to exclude regions that do not contain sample (so as to reduce file sizes). To prevent auto-cropping and force the full deskewed image size, add -w 0 to the lls command:

lls -w 0 /path/to/LLSfolder

Retrieving deconvolved Z-stacks 2017-02-14T14:00:18+00:00

By default, the lls command will generate a Deskewed folder that contains the complete deskewed data and a GPUdecon folder with a single file that contains the combined maximum-intensity projections for all time points and all channels that can be easily opened in ImageJ/Fiji to view the dataset.  But the complete deconvolved Z-stacks will not be retained, in order to minimize unnecessary data size inflation.

To keep the complete deconvolved Z-stacks during (re-)processing, add the -i flag to the command and specify the number of desired iterations (5-10 is reasonable), including the -r or --reprocess flag if the dataset has already been processed before:

lls -r -i 8 /path/to/LLSfolder 

Basic post-processing & deskewing 2017-02-05T16:04:39+00:00

For post-processing (such as deskewing and deconvolving) use the program lls. To get the full help menu on this program, type the following at a cbmf-latwork command prompt:

lls -h

For instance, for basic deskewing of a dataset, type the following at a cbmf-latwork command prompt:

lls /path/to/LLSfolder

…where /path/to/LLSfolder is replaced with the absolute or relative path to the experiment folder that you want to process (the folder that has the Settings.txt file in it). Don’t forget, you can use . to represent the current working directory, so you can process the current working directory by typing lls .

If the folder specified has already been processed, you will need to add the --reprocess or -r flag to the command:

lls -r /path/to/LLSfolder

By default, the program will generate a Deskewed folder that contains the deskewed data and a GPUdecon folder with a single file that contains the combined maximum-intensity projections for all time points and all channels that can be easily opened in ImageJ/Fiji to view the dataset.  To keep the complete deconvolved Z-stacks, add the -i flag to the command and specify the number of desired iterations (5-10 is reasonable):

lls -i 8 /path/to/LLSfolder

By default, the images will be “auto-cropped” to exclude regions that do not contain sample (so as to reduce file size), to prevent auto-cropping, add -w 0 to the command:

lls -w 0 /path/to/LLSfolder

For complete details and source code, see the LLSpy GitHub repository.

Logging in to the CBMF server 2017-02-13T17:19:53+00:00

You must either be on the HMS network, or use a VPN client to log in to the HMS network if you are off campus. 

Note: Logging in requires that you have an account on the CBMF workstation (“latwork”).  If you are unable to login at cbmf-latwork.med.harvard.edu following the instructions below, try first logging in to cbmf-nas.med.harvard.edu with your ecommons credentials to create an account, then logging out (type exit), then logging in to cbmf-latwork.med.harvard.edu.  Contact us if you are still unable to log in.

On Mac or Linux computers:

  1. Open the Terminal application.
  2. Enter one of the following commands, substituting your own ecommons username:
    • For processing data: ssh [ecommons_username]@cbmf-latwork.med.harvard.edu
    • For transferring data: ssh [ecommons_username]@cbmf-nas.med.harvard.edu
  3. Enter your ecommons password when prompted

On Windows computers:

  1. Open an ssh client such as Putty
  2. Open a new session with the following connection settings:
    • Host Name: cbmf-latwork.med.harvard.edu
    • Port: 22 (leave as default)
    • Connection Type: SSH (leave as default)
  3. Once the connection is open you should see a prompt asking for username and password.  Use your ecommons credentials to connect.

Using X11 forwarding for previewing data remotely:

If you’d like to use the llsinfo --show command to preview data remotely, you will need to log in using X11 forwarding.

  1. Install an Xserver
  2. Connect with X11 forwarding
    • On Mac, add the -X flag to your ssh command:
      • ssh -X [ecommons_username]@cbmf-latwork.med.harvard.edu
    • If using Putty on Windows, check the “Enable X11 Forwarding” box as shown here: