Suppose the task were to identify in a lake all the different types of fish as well as their relative abundance. Somewhat analogously 16S rRNA () and metagenomic sequencing ( ) try to do likewise for microbial communities in various habitats. 16S rRNA is akin to doing it with a bunch of fishing rods, i.e., pool of PCR ( ) specific for defined bacterial 16S rRNA genes, while metagenomic sequencing is akin to a dragnet, high throughput sequencing of entire microbial genes, even genomes. Both start with DNA extracted from a sample followed by mapping to reference databases and annotation (see below from ).
In that light, 16S rRNA is more a ‘biological fingerprint‘ () for bacterial species while metagenomics analyzes a given biological (human poop, skin swab, etc.) or environmental (soil, aquifer, etc.) sample more thoroughly and comprehensively and thus generates a more accurate snapshot of not just the bacteria ( ) but also archaea, eukaryotes, fungi, protists and viruses ( ) present in it.
Other problems with 16S rRNA analysis include
- Greater scope for introducing bias since
- Certain 16S rRNA sequences may be amplified more efficiently compared to others as different bacterial species carry different numbers of 16S rRNA genes ( ) while copy numbers aren’t even known for many species, especially those found during human microbiome analyses since many of these species are novel, haven’t been cultured and may likely be ‘non-culturable’ using standard bacteriological techniques.
- Chimeric amplification products can skew results ( ) but the microbiome field is plagued with weak adherence to relatively easy technical fixes that could prevent such artifacts from occurring in the first place ( ).
- Can’t be used to identify to species level since many bacterial species have identical 16S rRNA genes, for example some Anthrax species ( ).
Major problem with metagenomics is cost as in still rather prohibitively expensive.
16S rRNA Sequencing & Metagenomics Share Many Critical Bias-introducing Steps
However, debating the relative merits of 16S rRNA versus metagenomics risks missing the forest for the trees since both approaches share many critical bias-introducing steps (see below from).
- These include study design (
), sample collection, processing, storage, DNA extraction and sequencing, all of which can introduce a variety of biases and errors over and above those inherent to the two techniques.
- Differences in sample collection (8, ), processing ( ), transport, storage ( , , ) and DNA extraction ( , ) can change microbiome results.
- Even different sequencing platforms can yield different results from the same samples (16).
- Not only are reference databases incomplete ( ), commonly used computational models vary in sensitivity and precision ( ).
- Both 16S rRNA and metagenomics also share another error common to any rapidly expanding scientific field, namely problematic assumptions that underlie the complex statistical approaches necessary for analysis of the colossal amount of data generated by such techniques. For example, a commonly used approach, , has the potential to dramatically skew false positive and false negative rates ( ).
- In other words, even as the microbiome field explodes and is therefore subject to constant technological innovation, critical quality control measures such as development of standardized protocols haven’t kept pace ( , 21). Since different published studies use different methods, results can’t be compared across studies either ( , ).
- Most microbiome studies consist of small numbers of subjects. Such studies have inherently poor statistical power meaning difficult to be conclusive about a particular hypothesis. After all, when diet ( ) and location ( ) dramatically change snapshot reads of microbiome composition of a single individual, stands to reason that each microbiome study be considered a stand-alone piece of data and yet it’s become standard to wildly extrapolate and exaggerate results from one small-scale study to an entire field or even to human physiology itself ( ).
- Currently many microbiome studies also do an extremely poor job of accounting for confounding factors when comparing microbiome differences between two groups of individuals, regardless of the method used to assess it in the first place.
- An illustrative example of such errors, one study ( ) reported that rugby players (n=40) have more diverse microbiome compared to age-matched controls (n=23, 23). Authors as well as popular press interpreted this apparent difference as proof that ‘exercise increases gut microbial diversity in humans‘ ( ). Really? Exercise was the only difference between those two groups? Not diet, not more extensive contact with soil, to mention just two plausible differences between rugby players and age-matched controls ( )?
- In other words, many microbiome studies continue making the fatal error of conflating correlation with causation.
- Sample replicates are vital in attesting to the robustness of experimental approaches, especially so for extremely sensitive molecular biological techniques such as 16S rRNA sequencing or metagenomics. Vastly expanded sensitivity greatly increases the burden of separating signal from noise ( ). A most crucial quality-control step, replication, i.e., test the same sample multiple times, thus becomes critical in helping to better discriminate signal from noise. Currently, microbiome research tends to woefully neglect sample replicates (see table below from ).
Prosser rightfully nails the critical problem of sample numbers and replicates on its head thus (),
‘At the moment, if I measured the height of one Englishmen, one American, one Australian and one African and then used the data to postulate ways in which height was determined by continent I would, correctly, be ridiculed. If I were to take a single faecal sample from each of the same individuals and performed massive, parallel, high throughput, 454-sequencing to generate sequence lists, relative abundances of different phylogenetic groups and pie-charts, and used these data to postulate continent associated differences in intestinal microbial communities, it is likely that the paper would be published in one of the highest profile, general science journals. There are already several precedents for this and, as a community, we should be embarrassed.’
Seven years on from the above comment about this predicament, little has changed regarding this status quo even as microbiome research has vastly expanded in scope, entering any and all areas of research on human physiology and disease.
1. Jonsson, Viktor. Statistical analysis and modelling of gene count data in metagenomics. Diss. Göteborgs universitet, 2017.
2. Carlos, Nossa, Yi-Wei Tang, and Zhiheng Pei. “Pearls and pitfalls of genomics-based microbiome analysis.” Emerging Microbes and Infections 1 (2012): e45.
3. Meisel, Jacquelyn S., et al. “Skin microbiome surveys are strongly influenced by experimental design.” Journal of Investigative Dermatology 136.5 (2016): 947-956.
4. Pei, Anna Y., et al. “Diversity of 16S rRNA genes within individual prokaryotic genomes.” Applied and environmental microbiology 76.12 (2010): 3886-3897.
5. Boers, Stefan A., John P. Hays, and Ruud Jansen. “Micelle PCR reduces chimera formation in 16S rRNA profiling of complex microbial DNA mixtures.” Scientific reports 5 (2015).
6. Gohl, Daryl M., et al. “Systematic improvement of amplicon marker gene methods for increased accuracy in microbiome studies.” Nature Biotechnology 34.9 (2016): 942-949.
7. Bik, Elisabeth M. “Focus: Microbiome: The Hoops, Hopes, and Hypes of Human Microbiome Research.” The Yale Journal of Biology and Medicine 89.3 (2016): 363.
8. Dominianni, Christine, et al. “Comparison of methods for fecal microbiome biospecimen collection.” BMC microbiology 14.1 (2014): 103.
9. Sinha, Rashmi, et al. “Collecting fecal samples for microbiome analyses in epidemiology studies.” Cancer Epidemiology and Prevention Biomarkers 25.2 (2016): 407-416.
10. Hsieh, Yu-Hsin, et al. “Impact of Different Fecal Processing Methods on Assessments of Bacterial Diversity in the Human Intestine.” Frontiers in Microbiology 7 (2016).
11. Choo, Jocelyn M., Lex EX Leong, and Geraint B. Rogers. “Sample storage conditions significantly influence faecal microbiome profiles.” Scientific reports 5 (2015): 16350.
12. Klymiuk, Ingeborg, et al. “16S based microbiome analysis from healthy subjects’ skin swabs stored for different storage periods reveal phylum to genus level changes.” Frontiers in Microbiology 7 (2016).
13. Song, Se Jin, et al. “Preservation methods differ in fecal microbiome stability, affecting suitability for field studies.” mSystems 1.3 (2016): e00021-16.
14. Kennedy, Nicholas A., et al. “The impact of different DNA extraction kits and laboratories upon the assessment of human gut microbiota composition by 16S rRNA gene sequencing.” PloS one 9.2 (2014): e88982.
15. Wesolowska-Andersen, Agata, et al. “Choice of bacterial DNA extraction method from fecal material influences community structure as evaluated by metagenomic analysis.” Microbiome 2.1 (2014): 19.
16. Hahn, Andrea, et al. “Different next generation sequencing platforms produce different microbial profiles and diversity in cystic fibrosis sputum.” Journal of Microbiological Methods 130 (2016): 95-99.
17. Amato, Katherine R. “An introduction to microbiome analysis for human biology applications.” American Journal of Human Biology (2016).
18. Weiss, Sophie, et al. “Correlation detection strategies in microbial data sets vary widely in sensitivity and precision.” The ISME journal (2016).
19. McMurdie, Paul J., and Susan Holmes. “Waste not, want not: why rarefying microbiome data is inadmissible.” PLoS Comput Biol 10.4 (2014): e1003531.
20. Salter, Susannah J., et al. “Reagent and laboratory contamination can critically impact sequence-based microbiome analyses.” BMC biology 12.1 (2014): 87.
21. Clavel, Thomas, Ilias Lagkouvardos, and Andreas Hiergeist. “Microbiome sequencing: challenges and opportunities for molecular medicine.” Expert review of molecular diagnostics 16.7 (2016): 795-805.
22. Lozupone, Catherine A., et al. “Meta-analyses of studies of the human microbiota.” Genome research 23.10 (2013): 1704-1714.
23. David, Lawrence A., et al. “Diet rapidly and reproducibly alters the human gut microbiome.” Nature 505.7484 (2014): 559-563.
25. Clarke, Siobhan F., et al. “Exercise and associated dietary extremes impact on gut microbial diversity.” Gut (2014): gutjnl-2013.
26. Knight, Rob, et al. “Unlocking the potential of metagenomics through replicated experimental design.” Nature biotechnology 30.6 (2012): 513-520.
27. Prosser, James I. “Replicate or lie.” Environmental microbiology 12.7 (2010): 1806-1810.