Deep learning-enhanced light-field imaging with continuous validation

Nat Methods. 2021 May;18(5):557-563. doi: 10.1038/s41592-021-01136-0. Epub 2021 May 7.

Abstract

Visualizing dynamic processes over large, three-dimensional fields of view at high speed is essential for many applications in the life sciences. Light-field microscopy (LFM) has emerged as a tool for fast volumetric image acquisition, but its effective throughput and widespread use in biology has been hampered by a computationally demanding and artifact-prone image reconstruction process. Here, we present a framework for artificial intelligence-enhanced microscopy, integrating a hybrid light-field light-sheet microscope and deep learning-based volume reconstruction. In our approach, concomitantly acquired, high-resolution two-dimensional light-sheet images continuously serve as training data and validation for the convolutional neural network reconstructing the raw LFM data during extended volumetric time-lapse imaging experiments. Our network delivers high-quality three-dimensional reconstructions at video-rate throughput, which can be further refined based on the high-resolution light-sheet images. We demonstrate the capabilities of our approach by imaging medaka heart dynamics and zebrafish neural activity with volumetric imaging rates up to 100 Hz.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Animals
  • Biomechanical Phenomena
  • Calcium / chemistry
  • Deep Learning*
  • Heart / physiology*
  • Image Processing, Computer-Assisted / methods*
  • Larva / physiology
  • Microscopy / methods*
  • Oryzias / physiology
  • Reproducibility of Results
  • Zebrafish / physiology

Substances

  • Calcium