QIIME 2 release 2017.2: paired-end analysis support, 100x faster denoising with DADA2

We're excited to announce the 2017.2 release of the QIIME 2 framework and supporting plugins. This is a synchronized release, meaning that we intend all interfaces and plugins versioned 2017.2.* to work with QIIME 2 2017.2.*. For information about QIIME 2, see https://qiime2.org.

The 2017.2 synchronized release includes the following new features and interface changes. We aren’t listing all changes here, just the ones we are most excited about and those that will affect end users. For future releases, as the QIIME 2 project is now beginning to stabilize, we are committed to documenting all user-facing changes in changelogs instead of the summary below.

New and improved features:

• Support for paired-end read analysis with the new methods demux emp-paired and dada2 denoise-paired.
• Improved performance for DADA2 denoising using multi-threaded processing (see the n-threads parameter) and by training error models on only a subset of the sequences (see the n-reads-learn parameter). When denoising a full MiSeq run on 32 processors, this reduced the runtime by over 100x relative to QIIME 2 2.0.6.
• Improved error reporting from q2cli command line interface. When an error occurs, users are now presented with concise error messages, but not full Python tracebacks (unless the command is run with the --verbose flag, in which case the full Python traceback is presented). This should help users identify and correct errors.
• Sample metadata can now be integrated in feature table summary plots, allowing users to interactively explore rarefaction depth parameters in the context of sample metadata. This is available through the new Interactive Sample Detail tab in the visualizations generated by feature-table summarize if sample metadata is provided.
• The runtime and memory requirements for training and running feature classifiers with q2-feature-classifier has been greatly decreased.
• All methods and visualizers now have parameter-specific help text.
• A new method, diversity filter-distance-matrix, has been added to support metadata-based filtering of distance matrices.
• The QIIME 2 VirtualBox install instructions have been expanded and improved based on user feedback, and now includes screenshots to help users with installation.

Interface changes:

• demux emp has been renamed demux emp-single, to indicate that is it intended to be used for single-end read processing.
• dada2 denoise has been renamed dada2 denoise-single, to indicate that is it intended to be used for single-end read processing.

In this release cycle, we removed all CHANGELOG.md files as these haven't been well-maintained yet. We're working on a new solution for presenting this information which will be in place by the time we transition to our beta release. In the meantime, we'll present the most important changes in these release notes, and we are available to help users and developers transition between releases on Slack, the QIIME 2 Forum, or repository issue trackers. Get in touch through one of those mechanisms if you have any questions.

We have also updated our version numbering scheme with this release. Our release numbers now take the format year.month.patch-number. For example, this release is 2017.2.0 and we're currently planning another release for March, 2017, which will be version 2017.3.0. If any bug fix releases are necessary between these releases, those would be numbered by incrementing the patch number so that, for example, the hypothetical next patch release would be 2017.2.1.

We're very grateful to all of the plugin developers who have contributed functionality to this pipeline.

Because our microbiome analysis functionality is still currently very new, QIIME 2 is not yet recommended as a replacement for QIIME 1, but we are very interested in having users test QIIME 2 and give us feedback.

In addition to user testing, we're also interested in working with developers to contribute new QIIME 2 plugins or interfaces. You should read our draft plugin developer documentation at https://qiime2.org to get started. While QIIME 2 is in its alpha release stage, backward incompatible interface changes that impact plugins and interfaces can and will occur.

As always, thanks for your interest in QIIME 2!

qiime 2.0.6: data provenance, simplified viewing, and new forum

We're excited to announce the 2.0.6 minor release of the QIIME 2 framework and supporting plugins. This is the second synchronized release, meaning that we intend all interfaces and plugins versioned 0.0.6 to work with QIIME 2.0.6. For information about QIIME 2, see https://qiime2.org.

The 2.0.6 synchronized release includes the following new features and interface changes. We aren’t listing all changes here, just the ones we are most excited about and those that will affect end users. For future releases, as the QIIME 2 project is now beginning to stabilize, we are committed to documenting all user-facing changes in changelogs instead of the summary below.

New features:

• New .qza/.qzv file format supporting decentralized data provenance tracking. Provenance can be visualized and inspected by uploading a .qza/.qzv file to https://view.qiime2.org (more details about this service below). Other provenance visualization/inspection will be available in the future. Note: data generated with QIIME 2.0.5 will continue to be supported in the current and future releases, though the data will not contain provenance information. All new data generated with QIIME 2.0.6 will contain provenance information.
• https://view.qiime2.org is a new service for viewing .qza/.qzv files in the browser without requiring QIIME 2 to be installed. This service is great for sharing QIIME data with collaborators who may not have QIIME 2 installed. In addition to file uploads, the service supports viewing .qza/.qzv files accessible via URLs (e.g. Dropbox, Google Drive, GitHub links).
• https://docs.qiime2.org is the new site for accessing QIIME 2 user documentation. Many improvements have been made to the site, including links to all data files generated in tutorials and other documentation. These precomputed .qza/.qzv files can be easily viewed at https://view.qiime2.org or downloaded directly. By providing precomputed results, it is possible to read the tutorials and view results without installing QIIME 2 or additional software.
• New QIIME 2 forum (separate from the QIIME 1 forum) for user and developer support, community discussion, announcements, etc: https://forum.qiime2.org
• Support for exporting data from .qza/.qzv files (new CLI command qiime tools export and API qiime.sdk.Result.export_data).
• qiime taxa barplot has more discrete color palettes and can remove sorting categories, among several other bug fixes and usability enhancements.
• Most visualizations have been updated to have a consistent look-and-feel, and many have improved usability and functionality.

Interface changes:

• Removed HELP_URL, CITATION, and CONDA_CHANNEL from qiime.sdk API.
• qiime feature-table filter has been removed in place of qiime feature-table filter-samples and filter-features, which provide many more table filtering operations. The previous filtering functionality provided by qiime feature-table filter is now available in qiime phylogeny filter-table.
• qiime feature-table view-seq-data is now qiime feature-table tabulate-seqs.
• qiime feature-table view-taxa-data is now qiime taxa tabulate.
• qiime diversity core-metrics --p-counts-per-sample is now --p-sampling-depth.

We're very grateful to all of the plugin developers who have contributed functionality to this pipeline.

Because our microbiome analysis functionality is still currently very new, QIIME 2 is not yet recommended as a replacement for QIIME 1, but we are very interested in having users test QIIME 2 and give us feedback.

In addition to user testing, we're also interested in working with developers to contribute new QIIME 2 plugins or interfaces. You should read our draft plugin developer documentation at https://qiime2.org to get started. While QIIME 2 is in its alpha release stage, backward incompatible interface changes that impact plugins and interfaces can and will occur.

As always, thanks for your interest in QIIME!

qiime 2.0.5: Initial End-to-End Microbiome Analysis Pipeline

We're excited to announce the 2.0.5 minor release of the QIIME 2 framework and supporting plugins. This is the first synchronized release, meaning that we intend all interfaces and plugins versioned 0.0.5 to work with QIIME 2.0.5. As always you can see the QIIME 2 ChangeLog for additional information.

The new QIIME 2 website is also now live, and contains tutorials and example data that you can use to learn QIIME 2. (This site will eventually transition to replace the current qiime.org.)

Highlights of this release include:

• Our first attempt to stabilize the .qza/.qzv archival formats, which means that we intend to support artifacts and visualizations generated by this release (2.0.5) into the future. While we expect the format to continue to evolve, we now have machinery to support future changes in backwards compatible ways.
• Significant improvements to QIIME 2's command line interface (q2cli). These include a faster response time, tab-completion of command and parameter names, per-command configuration via an .ini file, support for default values, and more.
• Default values are now supported by the framework, allowing plugin developers to indicate what parameters are not required from the user.
• Improved importing of data. It is now possible for QIIME to consume data in more than one format via the use of transformers. This means we can expand the variety of file formats that QIIME 2 will be able to consume (and existing plugins will automatically benefit from it).

This is also the first release to include an end-to-end microbiome analysis pipeline (from raw sequence data through diversity analysis statistics and interactive visualizations). The plugins that make this possible are:

• q2-demux : Demultiplex Illumina data
• q2-dada2 : Apply DADA2 to denoise sequence data, including chimera filtering and phiX filtering
• q2-feature-table : Work with feature tables (QIIME 2's equivalent of OTU Tables)
• q2-feature-classifier : Assign taxonomy to sequence reads using a variety of machine learning classifiers
• q2-taxa : Generate interactive taxonomic composition plots
• q2-alignment : Perform sequence alignment to support diversity analyses
• q2-phylogeny : Perform phylogenetic reconstruction to support diversity analyses
• q2-diversity : Compute alpha and beta diversity analysis, and generate statistics and interactive visualizations
• q2-emperor : Generate interactive principal coordinate plots
• q2-composition : Apply differential abundance testing using ANCOM

We're very grateful to all of the plugin developers who have contributed functionality to this pipeline.

Because our microbiome analysis functionality is still currently very new, QIIME 2 is not yet recommended as a replacement for QIIME 1, but we are very interested in having users test QIIME 2 and give us feedback.

In addition to user testing, we're also interested in working with developers who are interested in contributing new QIIME 2 plugins or interfaces. You should read our draft plugin developer documentation at https://2.qiime.org to get started. While QIIME 2 is in its alpha release stage, backward incompatible interface changes that impact plugins and interfaces can and will occur. We will make every effort to minimize these, and we will always document these in the QIIME 2 ChangeLog.

Finally, we're wrapping up our initial version of the QIIME 2 documentation and tutorials. We expect the new site to go live by this Tuesday, 11 October 2016. That documentation will be at http://2.qiime.org, and you'll see a placeholder at that page until the new site goes live.

As always, thanks for your interest in QIIME!

qiime 2.0.2: Minor alpha release

We're excited to announce the 2.0.2 minor release of the QIIME 2 framework. This release has a few new features that will be useful for plugin developers, and which are detailed in the QIIME 2 ChangeLog.

One exciting new feature is the inclusion of a templating framework for creating new plugins based on cookiecutter (#4). This allows a new plugin developer to rapidly create a working QIIME 2 plugin, which can then be adapted to add new functionality to QIIME. This should make it even easier to create new plugins. See Initializing a QIIME 2 plugin in the QIIME 2 wiki to learn how to use this (or just call qiime tools plugin-init after installing this release and updating q2cli).

Another important addition is the citation_text and user_support_text properties to Plugin objects. These allow plugin developers to communicate to users how they should cite the plugin they're using, and how to request help with it. This information can then be presented to users by interfaces. For example, this is accessible through q2cli by calling --help on the plugin:

In our initial QIIME 2 development phase, most of our focus has been on building the underlying framework, and the microbiome analysis functionality is limited. During our alpha release phase (which we are currently in), we'll be rapidly expanding the available microbiome analysis functionality through the development of new plugins. Because our microbiome analysis functionality is currently limited, QIIME 2 is not yet recommended as a replacement for QIIME 1, but we are very interested in having users test QIIME 2 and give us feedback. To see what QIIME 2 plugins are currently available or in development, see the QIIME 2 Plugins page.

In addition to user testing, we're also interested in working with developers who are interested in contributing new QIIME 2 plugins or interfaces. You should read our draft plugin developer documentation to get started. While QIIME 2 is in its alpha release stage, backward incompatible interface changes that impact plugins and interfaces can and will occur. We will make every effort to minimize these, and we will always document these in the QIIME 2 ChangeLog.

To get started with installing and using QIIME 2, see our wiki at 2.qiime.org. The wiki is a temporary location for our documentation while we're in alpha. Our documentation will move to www.qiime.org when we transition to our beta release phase.

As always, thanks for your interest in QIIME!

First QIIME 2 alpha release at SciPy 2016!

We're extremely excited to announce the first alpha release of QIIME 2, occurring during SciPy 2016, and approximately 6.5 years after the initial release of QIIME (version 0.9.0)! A lot has changed in that time, both in scientific computing and in microbiome bioinformatics, and as a result QIIME 2 is a complete rewrite of the QIIME platform.

In our initial QIIME 2 development phase, most of our focus has been on building the underlying framework, and the microbiome analysis functionality is limited. During our alpha release phase (beginning now), we'll be rapidly expanding the available microbiome analysis functionality. Because our microbiome analysis functionality is currently limited, QIIME 2 is not yet recommended as a replacement for QIIME 1, but we are very interested in having users test QIIME 2 and give us feedback.

The alpha version of the framework provides exciting new functionality that we think will revolutionize microbiome bioinformatics, including:

• a plugin system for microbiome bioinformatics developers to add support for new methods and visualizations to QIIME,
• decentralized provenance tracking of all data generated by QIIME to simplify methods reporting and reproducibility,
• a semantic type system to help users use QIIME correctly,
• and a software development kit (SDK) for interface developers that enables development of different types of interfaces, including QIIME Studio, our Electron-based graphical user interface.

To get started with installing and using QIIME 2, see our wiki at 2.qiime.org. The wiki is a temporary location for our documentation while we're in alpha. Our documentation will move to www.qiime.org when we transition to our beta release phase.

In addition to user testing, we're also interested in working with developers who are interested in contributing new QIIME 2 plugins or interfaces. You should read our draft plugin developer documentation to get started. While QIIME 2 is in its alpha release stage, backward incompatible interface changes that impact plugins and interfaces can and will occur. We will make every effort to minimize these, and we will always document these in the QIIME 2 ChangeLog.

We now have a dedicated QIIME twitter account, which you can follow for updates on QIIME 2. We'll have a more official system for communication with our users and developers timed with an upcoming release. In the meantime, if you have questions, bug reports, or feature requests, please post them to the QIIME 2 issue tracker.

As always, thanks for your interest in QIIME! We are very excited to work with our user community as we transition to QIIME 2, and to expand our development community through decentralized plugin and interface development.

Happy QIIME-ing!