Scmos noise-correction algorithm for microscopy images

Access through your institution Buy or subscribe To the Editor: Scientific complementary metal-oxide semiconductor (sCMOS) cameras are rapidly gaining popularity in the biological sciences.

The sCMOS sensor provides significant advances in imaging speed, sensitivity and field of view over traditional detectors such as charge-coupled devices (CCD) or electron-multiplying CCDs

(EMCCD)1,2. However, this sensor introduces pixel-dependent noise; each pixel has its own noise statistics—primarily offset, gain and variance. Left uncorrected, this sCMOS-specific noise

generates imaging artifacts and biases in quantification3. A suite of algorithms was developed to characterize this noise in each pixel and incorporate the noise statistics in the likelihood

function for single-molecule localization3. However, these algorithms work exclusively on images with point objects such as in single-particle tracking or single-molecule-switching

nanoscopy. No general algorithm that works on conventional microscopy images exists. We developed such an algorithm that dramatically reduces sCMOS noise from microscopy images with

arbitrary structures. We show that our new method corrects pixel-dependent noise in fluorescence microscopy using an sCMOS sensor, and this allows the sensor's performance to approach

that of an ideal camera. This is a preview of subscription content, access via your institution ACCESS OPTIONS Access through your institution Access Nature and 54 other Nature Portfolio

journals Get Nature+, our best-value online-access subscription $29.99 / 30 days cancel any time Learn more Subscribe to this journal Receive 12 print issues and online access $259.00 per

year only $21.58 per issue Learn more Buy this article * Purchase on SpringerLink * Instant access to full article PDF Buy now Prices may be subject to local taxes which are calculated

during checkout ADDITIONAL ACCESS OPTIONS: * Log in * Learn about institutional subscriptions * Read our FAQs * Contact customer support REFERENCES * von Diezmann, A., Shechtman, Y. &

Moerner, W.E. _Chem. Rev._ 117, 7244–7275 (2017). Article  CAS  PubMed  PubMed Central  Google Scholar  * Huang, Z.-L. et al. _Opt. Express_ 19, 19156–19168 (2011). Article  PubMed  Google

Scholar  * Huang, F. et al. _Nat. Methods_ 10, 653–658 (2013). Article  CAS  PubMed  PubMed Central  Google Scholar  * Goodman, J.W. _Introduction to Fourier Optics_ (Roberts & Company,

2005). Google Scholar  * Liu, S., Kromann, E.B., Krueger, W.D., Bewersdorf, J. & Lidke, K.A. _Opt. Express_ 21, 29462–29487 (2013). Article  PubMed  PubMed Central  Google Scholar 

Download references ACKNOWLEDGEMENTS We thank C. Pellizzari for discussion on algorithm development; D.A. Miller, K.F. Ziegler and P.M. Ivey for helping with the project and for their

suggestions on the manuscript. D.M.S. was supported by an NSF grant (1146944-IOS). S.L., M.J.M. and F.H. were supported by grants from the NIH (R35 GM119785) and DARPA (D16AP00093). AUTHOR

INFORMATION AUTHORS AND AFFILIATIONS * Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA Sheng Liu, Michael J Mlodzianoski & Fang Huang * School of

Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana, USA Zhenhua Hu * Department of Biological Sciences, Purdue University, West Lafayette, Indiana, USA Yuan

Ren, Kristi McElmurry & Daniel M Suter * Purdue Institute for Integrative Neuroscience, Purdue University, West Lafayette, Indiana, USA Daniel M Suter & Fang Huang * Bindley

Bioscience Center, Purdue University, West Lafayette, Indiana, USA Daniel M Suter * Birck Nanotechnology Center, Purdue University, West Lafayette, Indiana, USA Daniel M Suter * Purdue

Institute of Inflammation, Immunology and Infectious Disease, Purdue University, West Lafayette, Indiana, USA Fang Huang Authors * Sheng Liu View author publications You can also search for

this author inPubMed Google Scholar * Michael J Mlodzianoski View author publications You can also search for this author inPubMed Google Scholar * Zhenhua Hu View author publications You

can also search for this author inPubMed Google Scholar * Yuan Ren View author publications You can also search for this author inPubMed Google Scholar * Kristi McElmurry View author

publications You can also search for this author inPubMed Google Scholar * Daniel M Suter View author publications You can also search for this author inPubMed Google Scholar * Fang Huang

View author publications You can also search for this author inPubMed Google Scholar CORRESPONDING AUTHOR Correspondence to Fang Huang. ETHICS DECLARATIONS COMPETING INTERESTS S.L. and F.H.

are co-inventors on a patent application related in part to the material presented here. INTEGRATED SUPPLEMENTARY INFORMATION SUPPLEMENTARY FIGURE 1 DIAGRAM OF THE NOISE CORRECTION ALGORITHM

Starting with a raw sCMOS frame, sequential steps include, pre-correction using offset and gain pixel maps, calculating negative log-likelihood using variance and gain pixel maps and noise

contribution in Fourier space and iterative update to minimize the pixel-wise sum of the two quantities (See Supplementary Notes 2–7) SUPPLEMENTARY FIGURE 2 TEMPORAL FLUCTUATION COMPARISON

OF FLUORESCENCE MICROSCOPY IMAGES. Peroxisome membrane proteins in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. (A) Temporal standard

deviation (STD) map over 400 sCMOS frames (pre-corrected by gain and offset). The colormap scale is from min (STD of 2.3, black) to max (STD of 12.3) in units of effective photon count. (B)

Temporal STD map over 400 NCS (noise correction for sCMOS camera) frames. The terms sCMOS and NCS frame will be used throughout the supplemental figures. (C) Zoom in regions i and ii from A

and B show pixels with high variance are effectively removed after NCS. (D) The pixel intensity traces of selected pixels from cropped regions i and ii over 50 frames. For the pixels with

high readout noise (pixel 1 and 3), the value fluctuation decreases significantly after NCS, while for the pixels with low readout noise (pixel 2 and 4), the pixel value fluctuation remains

the same. And the mean pixel values stay the same in both high and low readout noise cases before and after noise correction. SUPPLEMENTARY FIGURE 3 PIXEL FLUCTUATION COMPARISON BEFORE AND

AFTER NOISE CORRECTION AT LOW PHOTON LEVELS. End-binding protein 3 in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. (A) A single sCMOS frame

pre-corrected for gain and offset for comparison purpose with an exposure time of 10 ms and at time point t = 0 s. (B) Time series of selected regions in A from sCMOS frames and the

corresponding NCS frames showing the significant reduction of sCMOS noise while maintaining the underlying signal. SUPPLEMENTARY FIGURE 4 RESOLUTION COMPARISON USING BOTH EXPERIMENTAL DATA

AND SIMULATED DATA. (A) 100 nm yellow-green fluorescent bead images from sCMOS camera and NCS corrected images. To cancel readout noise in sCMOS frames for a fair comparison between the

sCMOS frames and NCS frames, images were averaged over 200 frames for both cases. The intensity profiles were generated by averaging over the vertical dimension of each bead image and fitted

with a Gaussian function to extract their widths, ΣsCMOS and ΣNCS. (B) Simulated bead images based on the parameters in Supplementary Note 14. The simulated bead images were averaged over

20 frames from sCMOS and NCS frames. From both experimental data and simulated data, the ΣNCS is slightly larger than ΣsCMOS, resulting in 5.5 nm and 4 nm decrease in resolution, a small

decrease is potentially negligible compared with the diffraction limit of approximately 250 nm. SUPPLEMENTARY FIGURE 5 COMPARISON OF NCS RESULT USING OTF WEIGHTED AND NOISE ONLY MASKS.

Peroxisome membrane proteins in COS-7 cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. (A) Temporal standard deviation map over 400 sCMOS frames, NCS

frames with OTF weighted mask and noise only mask respectively. The colormap scale is from min (STD of 2.3, dark red) to max (STD of 12.3, white) in units of effective photon count. (B)

Average of a sequence of sCMOS and NCS frames as in A and their corresponding amplitudes in Fourier space. Average images were used to cancel pixel dependent noise for fair comparison of

sCMOS and NCS frames. (C) Radial average of the amplitude in Fourier space from the sCMOS and NCS frames. SUPPLEMENTARY FIGURE 6 COMPARISON OF NCS ALGORITHM WITH LOW PASS FILTER. In order to

illustrate the fundamental differences between low pass filters and the NCS algorithm, the simulated bead data uses a simulated high variance map (3000∼6000 ADU2). (A) sCMOS frame. (B) NCS

frame. (C) The sCMOS frame blurred by a 2D Gaussian kernel with a sigma equal to 1 pixel. (D) The sCMOS frame after a low pass filter with a cutoff frequency equal to the OTF radius. The

cutout region in each image is the 2× zoom of the region above the white box. It shows that both the Gaussian blur and the OTF filter cannot effectively remove the sCMOS noise (yellow boxes

and yellow arrows), while the NCS algorithm can significantly reduce the sCMOS noise fluctuations. Furthermore, the Gaussian blur method also reduces the resolution of the original image

(red circles) (Supplementary Note 12). SUPPLEMENTARY INFORMATION SUPPLEMENTARY TEXT AND FIGURES Supplementary Figures 1–6, Supplementary Methods and Supplementary Notes 1–15 (PDF 8395 kb)

SUPPLEMENTARY SOFTWARE NCS software package (ZIP 15641 kb) SUPPLEMENTARY VIDEO 1 Time series and pixel fluctuation of sCMOS and NCS frames - EB3 in COS-7 cells End-binding protein 3 in COS-7

cells tagged with tdEos were imaged on a conventional wide-field fluorescence microscope. The top panel shows the time series of the sCMOS and NCS frames of the experimental data. The

bottom traces show the pixel value fluctuations of the circled regions in sCMOS and NCS frames. (AVI 78450 kb) SUPPLEMENTARY VIDEO 2 Time series of raw sCMOS and NCS frames – SiR-actin in

Aplysia bag neuron cell F-actin in peripheral domain and transition zone of Aplysia bag cell neuronal growth cones tagged with SiR-actin were imaged on a conventional wide-field fluorescence

microscope. The movie shows the time series of the raw sCMOS (without offset and gain correction) and NCS frames of the experimental data. (AVI 65351 kb) LIFE SCIENCES REPORTING SUMMARY

Reporting Summary (PDF 67 kb) RIGHTS AND PERMISSIONS Reprints and permissions ABOUT THIS ARTICLE CITE THIS ARTICLE Liu, S., Mlodzianoski, M., Hu, Z. _et al._ sCMOS noise-correction algorithm

for microscopy images. _Nat Methods_ 14, 760–761 (2017). https://doi.org/10.1038/nmeth.4379 Download citation * Published: 01 August 2017 * Issue Date: 01 August 2017 * DOI:

https://doi.org/10.1038/nmeth.4379 SHARE THIS ARTICLE Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently

available for this article. Copy to clipboard Provided by the Springer Nature SharedIt content-sharing initiative