DatView: A graphical user interface for visualizing and querying large data sets in serial femtosecond crystallography

Natasha Stander, Petra Fromme, Nadia Zatsepin

Research output: Contribution to journalArticle

Abstract

DatView is a new graphical user interface (GUI) for plotting parameters to explore correlations, identify outliers and export subsets of data. It was designed to simplify and expedite analysis of very large unmerged serial femtosecond crystallography (SFX) data sets composed of indexing results from hundreds of thousands of microcrystal diffraction patterns. However, DatView works with any tabulated data, offering its functionality to many applications outside serial crystallography. In DatView's user-friendly GUI, selections are drawn onto plots and synchronized across all other plots, so correlations between multiple parameters in large multi-parameter data sets can be rapidly identified. It also includes an item viewer for displaying images in the current selection alongside the associated metadata. For serial crystallography data processed by indexamajig from CrystFEL [White, Kirian, Martin, Aquila, Nass, Barty & Chapman (2012). J. Appl. Cryst. 45, 335-341], DatView generates a table of parameters and metadata from stream files and, optionally, the associated HDF5 files. By combining the functionality of several commonly needed tools for SFX in a single GUI that operates on tabulated data, the time needed to load and calculate statistics from large data sets is reduced. This paper describes how DatView facilitates (i) efficient feedback during data collection by examining trends in time, sample position or any parameter, (ii) determination of optimal indexing and integration parameters via the comparison mode, (iii) identification of systematic errors in unmerged SFX data sets, and (iv) sorting and highly flexible data filtering (plot selections, Boolean filters and more), including direct export of subset CrystFEL stream files for further processing.

Original languageEnglish
Pages (from-to)1440-1448
Number of pages9
JournalJournal of Applied Crystallography
Volume52
DOIs
Publication statusPublished - Dec 1 2019

Fingerprint

Crystallography
Graphical user interfaces
Metadata
Microcrystals
Systematic errors
Sorting
Diffraction patterns
Eagles
Statistics
Feedback
Processing
Datasets

Keywords

  • data analysis
  • graphical user interfaces
  • serial femtosecond crystallography
  • visualization
  • X-ray free-electron lasers

ASJC Scopus subject areas

  • Biochemistry, Genetics and Molecular Biology(all)

Cite this

DatView : A graphical user interface for visualizing and querying large data sets in serial femtosecond crystallography. / Stander, Natasha; Fromme, Petra; Zatsepin, Nadia.

In: Journal of Applied Crystallography, Vol. 52, 01.12.2019, p. 1440-1448.

Research output: Contribution to journalArticle

@article{7b7360f17a3948848fe3ddf0d02249d6,
title = "DatView: A graphical user interface for visualizing and querying large data sets in serial femtosecond crystallography",
abstract = "DatView is a new graphical user interface (GUI) for plotting parameters to explore correlations, identify outliers and export subsets of data. It was designed to simplify and expedite analysis of very large unmerged serial femtosecond crystallography (SFX) data sets composed of indexing results from hundreds of thousands of microcrystal diffraction patterns. However, DatView works with any tabulated data, offering its functionality to many applications outside serial crystallography. In DatView's user-friendly GUI, selections are drawn onto plots and synchronized across all other plots, so correlations between multiple parameters in large multi-parameter data sets can be rapidly identified. It also includes an item viewer for displaying images in the current selection alongside the associated metadata. For serial crystallography data processed by indexamajig from CrystFEL [White, Kirian, Martin, Aquila, Nass, Barty & Chapman (2012). J. Appl. Cryst. 45, 335-341], DatView generates a table of parameters and metadata from stream files and, optionally, the associated HDF5 files. By combining the functionality of several commonly needed tools for SFX in a single GUI that operates on tabulated data, the time needed to load and calculate statistics from large data sets is reduced. This paper describes how DatView facilitates (i) efficient feedback during data collection by examining trends in time, sample position or any parameter, (ii) determination of optimal indexing and integration parameters via the comparison mode, (iii) identification of systematic errors in unmerged SFX data sets, and (iv) sorting and highly flexible data filtering (plot selections, Boolean filters and more), including direct export of subset CrystFEL stream files for further processing.",
keywords = "data analysis, graphical user interfaces, serial femtosecond crystallography, visualization, X-ray free-electron lasers",
author = "Natasha Stander and Petra Fromme and Nadia Zatsepin",
year = "2019",
month = "12",
day = "1",
doi = "10.1107/S1600576719012044",
language = "English",
volume = "52",
pages = "1440--1448",
journal = "Journal of Applied Crystallography",
issn = "0021-8898",
publisher = "International Union of Crystallography",

}

TY - JOUR

T1 - DatView

T2 - A graphical user interface for visualizing and querying large data sets in serial femtosecond crystallography

AU - Stander, Natasha

AU - Fromme, Petra

AU - Zatsepin, Nadia

PY - 2019/12/1

Y1 - 2019/12/1

N2 - DatView is a new graphical user interface (GUI) for plotting parameters to explore correlations, identify outliers and export subsets of data. It was designed to simplify and expedite analysis of very large unmerged serial femtosecond crystallography (SFX) data sets composed of indexing results from hundreds of thousands of microcrystal diffraction patterns. However, DatView works with any tabulated data, offering its functionality to many applications outside serial crystallography. In DatView's user-friendly GUI, selections are drawn onto plots and synchronized across all other plots, so correlations between multiple parameters in large multi-parameter data sets can be rapidly identified. It also includes an item viewer for displaying images in the current selection alongside the associated metadata. For serial crystallography data processed by indexamajig from CrystFEL [White, Kirian, Martin, Aquila, Nass, Barty & Chapman (2012). J. Appl. Cryst. 45, 335-341], DatView generates a table of parameters and metadata from stream files and, optionally, the associated HDF5 files. By combining the functionality of several commonly needed tools for SFX in a single GUI that operates on tabulated data, the time needed to load and calculate statistics from large data sets is reduced. This paper describes how DatView facilitates (i) efficient feedback during data collection by examining trends in time, sample position or any parameter, (ii) determination of optimal indexing and integration parameters via the comparison mode, (iii) identification of systematic errors in unmerged SFX data sets, and (iv) sorting and highly flexible data filtering (plot selections, Boolean filters and more), including direct export of subset CrystFEL stream files for further processing.

AB - DatView is a new graphical user interface (GUI) for plotting parameters to explore correlations, identify outliers and export subsets of data. It was designed to simplify and expedite analysis of very large unmerged serial femtosecond crystallography (SFX) data sets composed of indexing results from hundreds of thousands of microcrystal diffraction patterns. However, DatView works with any tabulated data, offering its functionality to many applications outside serial crystallography. In DatView's user-friendly GUI, selections are drawn onto plots and synchronized across all other plots, so correlations between multiple parameters in large multi-parameter data sets can be rapidly identified. It also includes an item viewer for displaying images in the current selection alongside the associated metadata. For serial crystallography data processed by indexamajig from CrystFEL [White, Kirian, Martin, Aquila, Nass, Barty & Chapman (2012). J. Appl. Cryst. 45, 335-341], DatView generates a table of parameters and metadata from stream files and, optionally, the associated HDF5 files. By combining the functionality of several commonly needed tools for SFX in a single GUI that operates on tabulated data, the time needed to load and calculate statistics from large data sets is reduced. This paper describes how DatView facilitates (i) efficient feedback during data collection by examining trends in time, sample position or any parameter, (ii) determination of optimal indexing and integration parameters via the comparison mode, (iii) identification of systematic errors in unmerged SFX data sets, and (iv) sorting and highly flexible data filtering (plot selections, Boolean filters and more), including direct export of subset CrystFEL stream files for further processing.

KW - data analysis

KW - graphical user interfaces

KW - serial femtosecond crystallography

KW - visualization

KW - X-ray free-electron lasers

UR - http://www.scopus.com/inward/record.url?scp=85075790572&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075790572&partnerID=8YFLogxK

U2 - 10.1107/S1600576719012044

DO - 10.1107/S1600576719012044

M3 - Article

AN - SCOPUS:85075790572

VL - 52

SP - 1440

EP - 1448

JO - Journal of Applied Crystallography

JF - Journal of Applied Crystallography

SN - 0021-8898

ER -