(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Blik is an extensible 3D visualisation tool for the annotation and analysis of cryo-electron tomography data [1]

['Lorenzo Gaifas', 'Institut De Biologie Structurale', 'Université Grenoble Alpes', 'Cea', 'Cnrs', 'Ibs', 'Grenoble', 'Moritz A. Kirchner', 'Joanna Timmins', 'Irina Gutsche']

Date: 2024-05 

Abstract Powerful, workflow-agnostic and interactive visualisation is essential for the ad hoc, human-in-the-loop workflows typical of cryo-electron tomography (cryo-ET). While several tools exist for visualisation and annotation of cryo-ET data, they are often integrated as part of monolithic processing pipelines, or focused on a specific task and offering limited reusability and extensibility. With each software suite presenting its own pros and cons and tools tailored to address specific challenges, seamless integration between available pipelines is often a difficult task. As part of the effort to enable such flexibility and move the software ecosystem towards a more collaborative and modular approach, we developed blik, an open-source napari plugin for visualisation and annotation of cryo-ET data (source code: https://github.com/brisvag/blik). blik offers fast, interactive, and user-friendly 3D visualisation thanks to napari, and is built with extensibility and modularity at the core. Data is handled and exposed through well-established scientific Python libraries such as numpy arrays and pandas dataframes. Reusable components (such as data structures, file read/write, and annotation tools) are developed as independent Python libraries to encourage reuse and community contribution. By easily integrating with established image analysis tools—even outside of the cryo-ET world—blik provides a versatile platform for interacting with cryo-ET data. On top of core visualisation features—interactive and simultaneous visualisation of tomograms, particle picks, and segmentations—blik provides an interface for interactive tools such as manual, surface-based and filament-based particle picking, and image segmentation, as well as simple filtering tools. Additional self-contained napari plugins developed as part of this work also implement interactive plotting and selection based on particle features, and label interpolation for easier segmentation. Finally, we highlight the differences with existing software and showcase blik’s applicability in biological research.

Citation: Gaifas L, Kirchner MA, Timmins J, Gutsche I (2024) Blik is an extensible 3D visualisation tool for the annotation and analysis of cryo-electron tomography data. PLoS Biol 22(4): e3002447. https://doi.org/10.1371/journal.pbio.3002447 Academic Editor: David Bhella, University of Glasgow, UNITED KINGDOM Received: November 8, 2023; Accepted: April 2, 2024; Published: April 30, 2024 Copyright: © 2024 Gaifas et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability: The current version of blik at time of publication is available permanently at Zenodo (https://doi.org/10.5281/zenodo.10894490). The source code is maintained at https://github.com/brisvag/blik. Funding: This project received funding from GRAL, the Grenoble Alliance for Integrated Structural and Cell Biology, a programme of the Chemistry Biology Health Graduate School of Université Grenoble Alpes (ANR-17-EURE-0003), and from the Agence Nationale de la Recherche (grant ANR DecRisp ANR-19-CE11-0017-01 to I.G.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: The authors have declared that no competing interests exist. Abbreviations: 3D, three-dimensional; cryo-EM, cryo-electron microscopy; cryo-ET, cryo-electron tomography; GUI, graphical user interface; IO, input/output; ML, machine-learning; SDF, signed distance field

Introduction Cryo-electron tomography (cryo-ET) is a powerful three-dimensional (3D) cryo-electron microscopy (cryo-EM) imaging technique for visualisation and structural analysis of biological samples in situ [1]. In recent years, rapid development of software for cryo-ET data processing and analysis has brought great advances in tools for tilt series alignment [2], particle picking [3,4], averaging and classification routines [5–7], denoising [8,9], and more [10]. Thanks to subtomogram averaging, cryo-ET is now routinely used to determine the structure of biological macromolecules in situ, achieving in the most favorable cases sub-nanometer resolutions [11,12]. While more and more powerful, existing workflows still rely on extensive human intervention due to the high heterogeneity of requirements and samples [13,14]. Powerful and user-friendly visualisation tools are needed for effective human-in-the-loop pipelines to minimise the friction at the human–machine interface, and should be composed of modular and extensible software, to maximise reusability and simplify integration of different existing toolkits. A common practical challenge encountered by scientists working on cryo-ET data is indeed the (in)compatibility between different software tools. Some of the most widespread cryo-ET software suites (such as IMOD [15], PEET [16,17], Relion [18], Dynamo [19], emClarity [20], and PyTom [21,22]) all use different file formats for particle poses and tilt series metadata. This constitutes a barrier for users who need to use features from different tools on the same data: at best, users might miss out on important features from other software; at worst, integration may silently go wrong and cause issues in later steps. Entire software suites such as Scipion [23] are devoted to integrating normally incompatible cryo-EM and cryo-ET software into pipelines. With useful features scattered among different programs (e.g., AreTomo’s unsupervised alignment [2], Topaz’s denoising [8], CrYOLO’s filament picking [4], EMAN2’s trainable segmentation [10]), and numerous small custom scripts developed by researchers tailored to a specific project’s needs (distributing or selecting particles, improving alignments, etc.), software integration is a real and common concern. Even when a compatible tool is available or conversion possible, it is often hard to tell when it worked properly due to lack of generalised visualisation tools for inspecting and validating data throughout the pipeline. Due to the aforementioned compatibility restrictions, popular software suites (such as IMOD and Dynamo) often provide built-in visualisation tools, duplicating development efforts and further deepening the separation between pipelines. Finally, many existing visualisation tools are not easily hackable by users to extend them with custom functionality. Even those that offer ways to extend their functionality (such as ChimeraX [24] through its Python API or Dynamo with MATLAB [25] code), provide limited interface to data and rendering code, or require considerable programming skills to do so. To address these issues, we present blik, a new software for interactive visualisation, manipulation, and analysis of cryo-ET data. The code is open-source and welcomes community contributions at https://github.com/brisvag/blik. blik is a plugin for napari [26,27] (https://napari.org/), a visualisation software focused on scientific imaging, with data segmentation and annotation available as core features. It has both a programmatic and a graphical interface, allowing for seamless integration of interactive visualisation and scripted pipelines. napari offers great customisation options, powerful built-in tools, and a growing plugin ecosystem, which allows blik to focus on specific cryo-ET needs. To address the challenges listed above, blik’s design choices, features, and architecture reflect the following primary goals: Compatibility: blik can read and write data in file formats from a variety of different software suites, including IMOD , Relion , and Dynamo . This makes it easy to switch between tools or to integrate custom scripts into a workflow.

can read and write data in file formats from a variety of different software suites, including , , and . This makes it easy to switch between tools or to integrate custom scripts into a workflow. Interactivity: blik provides interactive visualisation that allows users to explore data programmatically and visually at the same time. Data is always accessible through a standard Python console and in simple, well-established formats such as numpy arrays and pandas dataframe. This makes it easy to validate data during processing and to identify problems as soon as they arise, as well as to provide a framework for quicker prototyping and debugging of new workflows.

provides interactive visualisation that allows users to explore data programmatically and visually at the same time. Data is always accessible through a standard Python console and in simple, well-established formats such as arrays and dataframe. This makes it easy to validate data during processing and to identify problems as soon as they arise, as well as to provide a framework for quicker prototyping and debugging of new workflows. Hackability: blik is open-source and easy to extend with custom functionality. This allows users to tailor the software to their specific needs. The napari plugin ecosystem also allows taking advantage of many existing analysis tools, even from different imaging fields. Additionally, blik ’s input/output (IO) capabilities are easily extensible by users to include new custom formats.

is open-source and easy to extend with custom functionality. This allows users to tailor the software to their specific needs. The plugin ecosystem also allows taking advantage of many existing analysis tools, even from different imaging fields. Additionally, ’s input/output (IO) capabilities are easily extensible by users to include new custom formats. User-friendliness : blik’s, and most of napari’s, functionality are also exposed in the graphical user interface (GUI), making it also easy to use for non-programmers.

: blik’s, and most of napari’s, functionality are also exposed in the graphical user interface (GUI), making it also easy to use for non-programmers. Performance: Thanks to napari ’s visualisation backend vispy , blik has performant rendering which can handle large 3D (and more) datasets, even larger-than-memory thanks to dask .

Thanks to ’s visualisation backend , has performant rendering which can handle large 3D (and more) datasets, even larger-than-memory thanks to . Community: To foster community contribution, code reuse, and jumpstart other projects, many contributions were upstreamed to napari , vispy , or extracted into simple single-use libraries usable by other projects ( teamtomo ). The use of Python for the development of blik and napari is crucial to further these goals. In the last years, the integration of scientific Python in high school and university educational curricula played a pivotal role in its democratisation. The surge of popularity of the scientific Python ecosystem with modular and reusable tools is largely due to its versatility, readability, simplicity, extensive documentation, and a wide community support. This makes blik a convenient entry point for cryo-ET-interested newcomers and lowers the barrier for the creative leveraging of basic programming skills for their everyday research and applications, enabling individuals with varying levels of programming experience to quickly grasp and implement solutions. Moreover, the scientific Python ecosystem is well-established in many imaging fields, and is becoming a player in the cryo-EM and cryo-ET world. By adhering to the practices and conventions of this ecosystem, blik allows to easily integrate many available field-agnostic tools (e.g., scikit-image and scipy for image and annotation processing, pytorch and the plethora of Python machine-learning (ML) tools for several types of analysis), take advantage of existing solutions and avoid the tendency to “reinvent the wheel” that scientific software can often be prone to. Finally, to help users to seamlessly integrate blik into their existing workflow, blik is currently being integrated into Scipion as a plugin (https://github.com/scipion-em/scipion-em-blik).

Discussion With blik, we add to the cryo-ET software ecosystem an integratable and interactive data viewer, radicated in the Python ecosystem, and a reusable and extensible set of libraries and tools for annotation and picking. There are several existing tools with similar goals and functionality to blik; in this section, we discuss the most widely used options known to us, examine their compatibility with blik, and compare their features with our tool. This should provide a starting point for the reader to choose the appropriate tool for their project. blik was tested primarily to work within the Relion&Warp pipeline [7,13,18], but was designed to be workflow-independent and thanks to cryohub easily extensible with new data formats. Following this workflow, other than the ubiquitous.mrc image format and Relion’s .star particle format, one of the first compatibilities to be developed was with image and particle data from the MATLAB software suite Dynamo [19] and its processing pipeline. Dynamo provides its own visualisation and picking tools, which were also the main inspiration for the geometry-based filament and surface particle generation in blik. While particle picking in blik has currently a smaller scope compared to Dynamo, its implementation in Python is intended to be more easily maintained and extended in the future by users, while leveraging the full-featured napari visualisation. IMOD [15] is arguably the giant in the field, with a long history of development and a wealth of features for image processing and annotation. Given its extensive capabilities and the constant development, it is usually the first choice when it comes to cryo-EM/ET processing, visualisation, and annotation. Many of these features are very useful, and we plan to add support for them in blik or napari in the future. Integration between blik and IMOD is seamless for most common operations, such as working with data processed through IMOD‘s tomography pipeline etomo. IMOD‘s 3d viewer 3dmod is full-featured and fast, and has several modes for 2D projection and slicing. It has however a more limited 3D renderer compared to napari (no volumetric projections, slicing planes in 3D view, or native particle pick viewer), which limits its applicability for the inherently 3D work of tomography. Being written in C, it is also nontrivial to extend for users with limited programming knowledge. Thermo Fisher’s Amira [31] is also a popular choice for image visualisation and annotation. It is particularly appreciated for its easy-to-use labeling tools to improve manual annotation and tracing, such as label interpolation, and image-guided picking. Amira’s label interpolation was the main inspiration behind napari-label-interpolator, and image-guided picking is in the future plans for blik development. Being an image-focused tool, Amira is more limited when it comes to particle coordinate generation although it has some model-based filament picking that can be repurposed for particle generation with the help of user scripts. However, Amira’s closed-source proprietary nature is a big downside for open science practices, making it hard or impossible to extend, contribute to, and freely share within the scientific community. EMAN2 [10] is a full software suite with an entire tomography pipeline from raw data to reconstruction. I/O compatibility between EMAN2 and blik is partially implemented, allowing to read particles and tomograms. EMAN2 has some tools for visualisation and picking, and is especially powerful for automated picking and segmentation thanks to ML tools. Its 3D visualisation is similar to IMOD in features. Tomviz [32] is an open-source application focusing on tomogram reconstruction and visualisation, providing also a few segmentation and analysis tools. When it comes specifically to particle picking and visualisation, a powerful tool recently developed is ArtiaX [33], a ChimeraX [24] extension for cryo-ET. Thanks to ChimeraX’s beautiful ray-tracing renderer, ArtiaX is ideal to make figures for publications. Particle visualisation is also very convenient and allows even for visualising subtomogram averaging results (map isosurfaces) distributed on the tomograms thanks to a performant implementation with instanced rendering. ChimeraX also provides a Python API to control its visualisations, but does not offer the same level of two-way and direct access to the visualised data as napari with its IPython console. ChimeraX‘s features and ecosystem are also more focused on protein structure visualisation and analysis, whereas napari is first and foremost an imaging tool. Another tool that blik already integrates with is CrYOLO [4], which provides powerful ML picking and segmentation routines. CrYOLO itself has recently adopted napari as its visualisation front-end. A similar tool to the surface resampling widget in blik exists in Membranorama [34,35], which allows for visualising surfaces in a tomogram with the surrounding volume projected perpendicularly onto the surface, as well as extracting individual surface patches which can be “planarised” for easier inspection. This tool is very useful for interactive visualisation and exploration of the projected surface in situ. blik, on the other hand, does not allow in situ projection, but instead focuses on generating a square-grid resampling which—differently from Membranorama—can be exported as an ordinary volume or reused immediately for further processing with blik or other tools. Given the popularity and fast growth of Cryo-ET, the field offers many other tools and software suites with features and goals compatible with blik. Some of of them offer excellent opportunities of integration with the napari and blik ecosystem, such as TomoSegMemTV [36] and MemBrain [37,38], which offer pixel-based and automated membrane segmentation (as opposed to the manual, surface-based annotation provided by blik) or pycurv [39] and surface-morphometrics [40], which can compute meshes, measurements, and statistics about pre-annotated membranes. With a collaborative and modular approach to software development, we strive for blik and teamtomo to become a starting point to enable such integrations in the future.

Conclusions The work presented in this paper aims to reduce the friction of working with cryo-ET data and to enable developers in the field to share, reuse, and contribute as a community. The development of blik and its components is a stepping stone towards these goals. Working within napari allows us to delegate (and share with other fields) many non-cryo-ET-specific components, while retaining interactivity and extensibility; napari is in rapid development, and direct contributions to the community-developed project are always welcome. Even where direct contributions are unfeasible, developers can take advantage of the plugin ecosystem (such as blik does) or simple scripting. Now that blik’s core features are established, we aim to reach out to other developers and cryo-ET software users and encourage reusing, adopting, or contributing to the work here presented. Future planned features for blik include: Exploit napari ’s nD visualisation, for example, to easily view the progression of a particle refinement.

’s nD visualisation, for example, to easily view the progression of a particle refinement. Conclude the work on napari multicanvas, allowing multiple views on the data (e.g., picking in orthoviews).

multicanvas, allowing multiple views on the data (e.g., picking in orthoviews). Implement instanced rendering in vispy to allow rendering full particle maps in the tomogram at high performance (like ArtiaX ).

). Offer more geometric models for picking (e.g., spheres, 3D lattices).

Materials and methods Not all the code contributions from this work live in the same place. Tools and implementations were split into standalone libraries or contributed to core napari when possible, in the interest of sharing and avoiding code duplication. Contributions from work in this paper are summarised below, and readers are encouraged to take advantage of all these open-source components for their own work. Refer to the individual repositories for the most up-to-date and in-depth documentation. Cryotypes and cryohub The data structures and IO functions used by blik are extracted into 2 usage-agnostic libraries: no assumptions from blik are carried over, which makes these libraries suitable for adoption by any Python software working with cryo-EM and cryo-ET data. Both libraries live in the github community project teamtomo. cryotypes : Defines simple and extensible data structures for cryo-EM data types and metadata, and provides simple validation and checking functions to ensure a given object conforms to the specification.

: Defines simple and extensible data structures for cryo-EM data types and metadata, and provides simple validation and checking functions to ensure a given object conforms to the specification. cryohub : Provides reading and writing functions for popular image formats and particle data, with both fine-grained controls and a higher level “magic” interface ( cryohub.open(<anything>) ). Data is read to and from cryotypes data structures, easily allowing for conversion between formats and integration in any third-party Python tool. cryohub provides granular I/O functions such as read_star and read_mrc, which will all return objects following the cryotypes specification. read_star poseset = read_star( ) A higher-level function called read adds some magic to the IO procedure, guessing file formats and returning a list of cryotypes. read data = read( , , lazy = , name_regex = ) See the help for each function for more info. Similarly to the read_* functions, cryohub provides a series of write_* functions and a magic higher-level write function. write write([poseset1, poseset2], ) Morphosamplers Surface and filament picking and particle generation code are not specific to cryo-ET. They were developed as part of a field-agnostic library called morphosamplers, which collects several tools for sampling image data with morphological objects. As part of this work, models for spline filaments and spline-based surfaces were developed, with their relative tools for particle generation and image resampling. Napari plugins Some of the napari-specific functionalities developed for blik were also not cryo-EM-specific and could instead be useful for many other applications in the napari ecosystem. These were extracted into their own napari plugins, which can be installed separately. napari-label-interpolator : A simple utility to interpolate n-dimensional labels along a specified dimension. Its main use in the context of cryo-ET is to reduce the manual annotation necessary to fully segment a volume. However, such functionality can also be used for example to track objects such as cells over a 2D (or 3D) time series.

: A simple utility to interpolate n-dimensional labels along a specified dimension. Its main use in the context of cryo-ET is to reduce the manual annotation necessary to fully segment a volume. However, such functionality can also be used for example to track objects such as cells over a 2D (or 3D) time series. napari-properties-plotter : Several napari layers such as Points and Labels can hold features data for each of their items. This plugin allows to display any of such feature combinations in an interactive plot widget; users can then select a subset of the data items based on a selected section of the plot. Napari and vispy Where possible, napari-specific code was contributed upstream to the napari core repository (or vispy for rendering-related code). Much of this work was distributed and collaborative in nature; here are listed some highlights that were crucial for the proper development of blik and to which significant contributions were made as part of this work. Improvements to quality, performance, and interactivity of 3D rendering for volumes and labels, including work such as proper depth buffer and blending usage, arbitrary plane slicing and clipping, additional 2D and 3D interpolation modes.

Visualisation of points as spheres for more intuitive 3D visualisation.

Improvements to surface mesh visualisation (shading).

Projection of n-dimensional bounding box instead of simple point-slicing. Blik Any remaining functionality specific to napari and cryo-EM was implemented directly in blik by often wrapping the aforementioned tools. Manual particle picking makes use of the simple point picker in napari, while automatically adding orientations and metadata needed for writing to file. Surface and filament picking are mainly wrappers around morphosamplers, but add a GUI for setting parameters and use napari layers for picking. The manual picks can then be used to generate visualisations such as filaments and meshes, for particle picking for subtomogram averaging, and for volume resampling. A few image filtering and processing tools are also provided with blik for ease of visualisation, such as bandpass filtering and a power spectrum generator.

Supporting information S1 Tutorial blik tutorial. A detailed tutorial for blik, describing the basics of the user interface and how to use essential features for practical applications. https://doi.org/10.1371/journal.pbio.3002447.s001 (PDF)

Acknowledgments We are particularly grateful to Alister Burt for stimulating discussions, suggestions, and initial guidance.

[END]
---
[1] Url: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3002447

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/