Neo 0.8.0 release notes¶
27th September 2019
Lazy loading¶
Neo 0.8 sees a major new feature, the ability to selectively load only parts of a data file (for supported file formats) into memory, for example only a subset of the signals in a segment, a subset of the channels in a signal, or even only a certain time slice of a given signal.
This can lead to major savings in time and memory consumption, or can allow files that are too large to be loaded into memory in their entirety to be processed one section at a time.
Here is an example, loading only certain sections of a signal:
lim0, lim1 = -500*pq.ms, +1500*pq.ms
seg = reader.read_segment(lazy=True) # this loads only the segment structure and metadata
# but all data objects are replaced by proxies
triggers = seg.events[0].load() # this loads all triggers in memory
sigproxy = seg.analogsignals[0] # this is a proxy object
all_sig_chunks = []
for t in triggers.times:
t0, t1 = (t + lim0), (t + lim1)
sig_chunk = sigproxy.load(time_slice=(t0, t1)) # here the actual data are loaded
all_sig_chunks.append(sig_chunk)
Not all IO modules support lazy loading (but many do). To know whether a given IO class supports lazy mode,
use SomeIO.support_lazy
.
For more details, see Lazy option and proxy objects.
Image sequence data¶
Another new feature, although one that is more experimental, is support for image sequence data, coming from calcium imaging of neuronal activity, voltage-sensitive dye imaging, etc.
The new ImageSequence
object contains a sequence of image frames as a 3D array.
As with other Neo data objects, the object also holds metadata such as the sampling rate/frame duration
and the spatial scale (physical size represented by one pixel).
Three new IO modules, TiffIO
, AsciiImageIO
and BlkIO
, allow
reading such data from file, e.g.:
from quantities import Hz, mm, dimensionless
from neo.io import TiffIO
data = TiffIO(data_path).read(units=dimensionless, sampling_rate=25 * Hz,
spatial_scale=0.05 * mm)
images = data[0].segments[0].imagesequences[0]
ImageSequence
is a subclass of the NumPy ndarray
, and
so can be manipulated in the same ways, e.g.:
images /= images.max()
background = np.mean(images, axis=0)
preprocessed_images = images - background
Since a common operation with image sequences is to extract time series from regions of interest,
Neo also provides various region-of-interest classes which perform this operation,
returning an AnalogSignal
object:
roi = CircularRegionOfInterest(x=50, y=50, radius=10)
signal = preprocessed_images.signal_from_region(roi)[0]
Other new features¶
- new neo.utils module
- Numpy 1.16+ compatibility
time_shift()
method forEpoch
/Event
/AnalogSignal
time_slice()
method is now more robust- dropped support for Python 3.4
See all pull requests included in this release and the list of closed issues.
Bug fixes and improvements in IO modules¶
- Blackrock
- Neuroshare
- NixIOFr
- NixIO (array annotation + 1d coordinates)
- AsciiSignal (fix + json metadata + IrregularlySampledSignals + write proxy)
- Spike2 (group same sampling rate)
- Brainvision
- NeuralynxIO
Warning
Some IOs (based on rawio) when loading can choose to split each
channel into its own 1-channel AnalogSignal
or to group them
in a multi-channel AnalogSignal
.
The default behavior (either signal_group_mode='split-all'
or 'group-same-units'
) is not the same for all IOs for backwards
compatibility reasons. In the next release, all IOs will have the default
signal_group_mode='group-same-units'
Acknowledgements¶
Thanks to Achileas Koutsou, Chek Yin Choi, Richard C. Gerkin, Hugo van Kemenade, Alexander Kleinjohann, Björn Müller, Jeffrey Gill, Christian Kothe, Mike Sintsov, @rishidhingra, Michael Denker, Julia Sprenger, Corentin Fragnaud, Andrew Davison and Samuel Garcia for their contributions to this release.