Compressing and decompressing data for short and long-term storage

Short term storage

We recommend compressing just the raw data after deskewing/deconvolving.  This can be done automatically during the initial processing by adding the --compress or -z flag to the lls command:

lls --compress /path/to/LLSfolder

With data that has already been processed, the raw data can be quickly compressed without affecting the Deskewed or GPUdecon folders by using the llscompress command:

llscompress /path/to/LLSfolder

To undo/decompress the above command, use the -z flag:

llscompress -z /path/to/LLSfolder

Long term storage

If you are not going to analyze the deskewed data anytime soon, we recommend deleting the deskewed/deconvolved data and compressing just the raw data (i.e. “deep freeze”).

llscompress --freeze /path/to/LLSfolder

This can reduce data size as much as 10-40 fold (depending on the data-set). To regenerate deskewed data, a folder that has been “frozen” can be easily reprocessed just like a normal LLS folder:

lls --reprocess /path/to/LLSfolder

bzip2 compression is used by default, but this can be changed in the options.

For a complete list of options in the llscompress program, type the following at the command prompt:

llscompress -h