Contents

Acknowledgements

Members of the SCANDNAnet steering committee included:

Researchers associated with SCANDNAnet:

We like to thank Anne Hill Mikkelsen and Sigga Jacobsen for their patience and support throughout the project. Additionally, we like to thank Jani Heino for comments on an earlier draft of the manuscript and NORCE for the identification of Norwegian macroinvertebrate samples.

1 Preface

Most European rivers and lakes are found in the Nordic countries; hence it is not surprising that freshwater assessments are an essential and mandatory part of Nordic environmental monitoring programs. For inland and coastal waters, monitoring requirements are largely determined by requirements of the European Water Framework Directive (WFD) (2000/60/EC), adopted by all Nordic countries. The WFD aims to protect and enhance the status of aquatic ecosystems and to promote sustainable water use in rivers, lakes, transitional and coastal waters and groundwaters. Given the large amount of water bodies in the Nordic countries, the demands of the WFD for biological monitoring and subsequent taxonomical identification requires considerable expertise and financial resources. To deal with these financial burdens, agencies and environmental managers are currently considering options that could replace costly and time-consuming procedures such as morphological identification. Depending on the organisms, different molecular techniques have recently been suggested as viable alternatives to current identification methods.

Aquatic macroinvertebrates are frequently used in WFD monitoring of both rivers and lakes as their responses to environmental pressures such as eutrophication, acidification and hydro-morphological alterations are well known and predictable (Johnson et al. 1993). However, identification of specimens to species-level by morphology is often both time consuming and difficult due to challenging taxonomy and high diversity. As an alternative to morphological identification, a molecular technique termed “DNA metabarcoding” can be used to potentially identify all species in a bulk sample using standardized stretches of their DNA (so-called “DNA barcodes”). DNA metabarcoding has proven to be very effective for species detection and identification in many studies, if a well-developed reference library can be used in the identification process. More work is needed on reference library development as well as laboratory and bioinformatic methods to improve estimates of biomass and abundance. While individual case studies indicate the efficacy of metabarcoding in monitoring and assessment programs using macroinvertebrates, no concerted, large-scale tests on suitability and accuracy have thus far been performed. Moreover, no information is available concerning changes needed before adopting this approach in routine Nordic aquatic monitoring.

In this Nordic validation study, we assessed if DNA metabarcoding has the potential to produce the same level of taxonomic information as current methods based on traditional morphological identification. Specifically, we tested whether these new methods result in improved processing speeds, comparability and cost efficiency, and assessed whether the implementation of these new methods will affect ongoing assessment processes, i.e. the current metrics or classifications of ongoing assessment systems.

It is noteworthy, that the rapidly advancing field of molecular research currently lacks guidance documents or international standards for implementing molecular methods in routine monitoring. The Nordic members of the SCANDNAnet consortium consists of researchers involved with national WFD monitoring and/or international method standardization in their respective countries. This uniquely positions the Nordic agencies participating in this validation effort as auditors of the metabarcoding approach also allowing them to advance the creation of joint, standardized protocols under the umbrella of the European Committee for Standardization (CEN). The need for guidance on molecular methods for use in aquatic biomonitoring has been identified by CEN. In 2019, the first dedicated workgroup on “DNA and eDNA methods” was established by the technical committee for Water Quality standards.

2 Executive summary

Over 75% of all lakes and 40% of all rivers in the EU are found in the Nordic countries. Freshwater biomonitoring according to the EU Water Framework Directive (WFD) has been adopted by all Nordic countries and forms an integral part of management efforts to preserve and restore the ecological quality of freshwaters and their ecosystem services.

Current identifications of organisms used in WFD biomonitoring are based on expert morphological identification; an approach that is time consuming and prone to errors. Molecular identification methods could alleviate many problems but thus far have not been rigorously tested for use in routine monitoring. Using a questionnaire, we consulted Nordic experts to assess opinions on the applicability of these methods in routine biomonitoring. Further, we validated the practical use of molecular metabarcoding identification for use in freshwater aquatic macroinvertebrate monitoring in all Nordic countries. A total of 297 waterbodies including both lakes and streams were sampled which constitutes the single largest validation test of the method so far. Experts in each country sampled macroinvertebrates according to their national protocols with only minor modifications and samples were analyzed using a standardized laboratory protocol.

The results of this study indicate that in most cases identifications using molecular DNA-based methods were highly congruent with traditional expert-based identifications. However, the study also revealed the importance of using unified guidance documents. Several samples showed signs of DNA degradation, affecting reliability of the method. Nordic experts generally recognized molecular identification methods as playing a major role in future biomonitoring, if issues associated with the current lack of unified methodology are resolved.

Based on this study, we suggest that concerted Nordic or European efforts towards implementation and standardization of DNA-based methodology should be undertaken to swiftly ensure the use of this promising tool into WFD compliant monitoring.

3 Introduction

3.1 Macroinvertebrate biomonitoring in the Nordic countries

Freshwater ecosystems are threatened by various anthropogenic activities, increasing the need for monitoring of their ecological status (e.g. Heino et at. 2020a, 2020b). Given the relatively high number of lakes and rivers in the Nordic countries novel approaches and methods are needed to implement robust and cost-effective monitoring programs All Nordic countries monitor benthic macroinvertebrates and other biological quality elements as well as supporting variables in lakes, rivers and coastal waters in accordance with the European Water Framework Directive, WFD, and the Marine Strategy Framework Directive, MSFD (Andersen et al. 2016). The timing of sampling varies between Nordic countries but is often conducted in autumn (note Norway). The status of rivers and lakes is assessed using over ten different macroinvertebrate-based indices in the Nordic countries. Similarly, around ten different indices are used to assess the status of coastal waters using macroinvertebrates. The information obtained through different macroinvertebrate indices, often combined into multimetric indices, is used in the assessment of status, and when combined with other quality elements it is used to infer the ecological status of a water body. Most macroinvertebrate indices used in Nordic countries do not require information on taxon abundance when estimating status at quality element level.

3.2. Taxonomic identification methods

3.2.1 Morphological identification

In all Nordic countries, morphological identifications of macroinvertebrates in biomonitoring are done by taxonomic experts. After sample collection, the macroinvertebrates and detritus are preserved in the field according to national guidelines, transported to the laboratory and sorted from debris (see 4.1.1). Macroinvertebrates are identified by microscopy to national minimum levels of taxonomic resolution. All Nordic experts keying macroinvertebrates use nationally defined minimum levels of identification which often include different levels taxonomic resolution, and which are specifically optimized for manual identification. To shorten processing times as well as due to limited taxonomic expertise, the standardized identification protocols often do not include the identification of certain groups to the species level, such as chironomids or oligochaetes in lake littoral or river habitats. Taxon groups usually keyed to species level include EPT-taxa (Ephemeroptera (mayflies), Plecoptera (stoneflies), Trichoptera (caddisflies)) as well as dragonflies and molluscs. EU Water Framework Directive compliant biomonitoring in the Nordic countries involves sampling many water body types, with each sample comprised of many specimens that need to be identified. For example, even in relatively species-poor regions like northern Finland, the calculation of the EU WFD related indices involves the identification of 44-113 lotic benthic macroinvertebrate taxa, depending on the water body type (Aroviita et al. 2019). Due to the inherent time-consuming identification process, traditional manual identification constitutes a bottleneck in bioassessments often resulting in significant time lags between sampling and the data availability. The growing need for more biomonitoring combined with declines in funding and in the number of taxonomic experts necessitates the exploration of alternative processes for reliable and comparable species identification (e.g. Elbrecht et al. 2017, Ärje et al. 2020). In addition, the high abundance of specimens in a sample poses a bottleneck and financial constraint to the identification of taxa to the most precise taxonomic level. Hering et al. (2004) identified three commonly used approaches to address the issues when samples contain high numbers of specimens: (1) analysing only a subset of the sample (e.g. Norway), or (2) the use of a coarser than genus or species identification or (3) a combination of both approaches.

3.2.2 DNA metabarcoding

DNA metabarcoding potentially allows for the identification of most macroinvertebrate species in a bulk sample without the added cost of sorting (Taberlet et al., 2012, Hajibabaei et al. 2019, Pereira-da-Conceicoa et al. 2020). By sequencing a short, standardized gene fragment (i.e. DNA barcode, Hebert et al. 2003) of DNA extracted and amplified from a homogenized macroinvertebrate sample and comparing the resulting sequences to DNA reference sequences of morphologically identified species, species lists of the complete sample can be obtained. In this process, millions of sequences are generated by high throughput sequencing, which are typically quality filtered and clustered into Operational Taxonomic Units (OTUs) by similarity. One OTU roughly corresponds to one phylogenetic species that is associated with a Linnean name if the species is present in the reference library. However, the cluster can match several species if genetic divergence between species is low (rarely the case for the marker used here or for the WFD macroinvertebrate species identified), or conversely there can be multiple OTUs for the same species identification if there is high genetic divergence within a species. These OTUs, however, are normally assigned to only a single species. DNA metabarcoding has the advantage that it allows for rapid identification of hundreds of samples, when employing laboratory automatization like processing samples in parallel (96-well plate format) or utilising robotics. Additionally, as the same methods are used across samples, the results are more objective and thus more comparable than morphological identification by different taxonomists (Haase et al. 2010, Sweeney et al. 2011). With reliable reference databases, specimens in the sample are often identified to species level. Sometimes even population genetic analysis is possible (Elbrecht et al. 2018).

However, metabarcoding also has some methodological drawbacks and there are several variables that can potentially influence the results. These include laboratory methods and protocols used, for instance in how DNA is extracted from the sample. Several studies have shown that DNA metabarcoding using DNA isolated from homogenized samples provides a better estimate of the species composition in the sample than DNA isolated from the preservative (Erdozain et al. 2019; Zizka et al. 2018); although other studies have shown that the differences between the two methods are not substantial (Gauthier et al. 2019; Hajibabaei et al. 2012). Organisms that have a hard exoskeletons or exterior cases, such as some freshwater beetles, insects and mussels, leak less DNA into the preservative and therefore remain undetected or produce less sequences than expected due to their biomass (Carew et al. 2018; Zizka et al. 2018; Martins et al. 2020). Likewise, terrestrial arthropods that have evolved hard exoskeletons to conserve water have also been shown to leak less DNA into the preservative (Marquina et al. 2019). However, in some cases, it may be important and necessary to keep vouchers for morphological identification as references for recorded biodiversity. In such instances, where homogenization of the collected samples is not possible, non-destructive DNA extraction of bulk samples using extraction buffer and the enzyme Proteinase K can be a viable alternative (Nielsen et al. 2019), as has been shown for freshwater macroinvertebrates (Carew et al. 2018). It is also worth noting that different types of DNA-extractions from homogenized samples can generate different results, especially if there are PCR-inhibiting substances originating from plants or sediments in the sample (Majaneva et al. 2018).

Sensitivity of the PCR DNA polymerases (Nichols et al. 2018; Taberlet et al. 2018), primer and marker choice (Elbrecht et al. 2018; Elbrecht & Leese 2017; Elbrecht et al. 2016), and even sequencing instrument (Braukmann et al. 2019) can influence the results of DNA metabarcoding. Studies have shown that metabarcoding cannot deliver precise specimen counts, due to variation in specimen biomass (Elbrecht et al. 2017) as well as inefficiencies in copying of the barcoding gene for some species (Elbrecht and Leese, 2015; Krehenwinkel et al., 2017; Piñol et al., 2015). There are, however, indications that read counts correlate to some extent with biomass (Elbrecht and Leese, 2017; Piñol et al. 2019; Schenk et al., 2020). Thus, read abundance can be used to compare the relative species abundance between samples, but this approach is currently not able to quantify the absolute abundance.

Many bioinformatic pipelines are used in analysing the output from high throughput sequencing instruments, both with regard to quality filtering and taxonomic assignments (e.g. Boyer et al. 2016; Callahan et al. 2019). Regardless of the methods chosen, it is important to have knowledge of your input data as naïve settings are likely to produce erroneous results (Majaneva et al. 2015).

3.3 Stakeholder survey

During the kickoff meeting on 27 June 2018, Steering committee members were asked to list key national stakeholders relevant to SCANDNAnet objectives. Stakeholders were divided into several categories: organization (Sector (11 levels), country, evaluation of opinion (4 levels) and stakeholder dissemination level (4 levels). Using this approach, 66 stakeholders belonging to over 30 different organizations were identified by September 2018.

3.3.1 Stakeholders

A questionnaire was created with a total of 6 background questions regarding the respondent and nine thematic questions designed to probe key stakeholder opinion on using genetic methods to identify aquatic macroinvertebrates in routine monitoring.

The questionnaire was distributed to all previously identified Nordic key stakeholders of the SCANDNAnet –project in September 2018. The link to the questionnaire was also distributed to non-Nordic experts. Altogether 20 respondents answered the questionnaire. Most replies were received from the Nordic countries (n = 16), but we also received replies from several different network institutions from a total of seven countries (AU, CZ, FI, GE, GR, NO, SE):

• Federal Agency for Water Management - Institute for Water Ecology, Fisheries and Lake Research, Austria
• T.G. Masaryk Water Research Institution, p.r. i., Czech Republic
• Centre for Economic Development, Transport and Environment of South Eastern Finland
• Centre for Economic Development, Transport and Environment of South Savo, Finland
• Finnish Environment Institute, Finland
• KVVY Tutkimus Oy, Finland
• Bundesanstalt für Gewässerkunde, Germany
• Hellenic Centre for Marine Research, Greece
• Norwegian Environment Agency, Norway
• Norwegian Institute for Water Research, Norway
• County Administration Board of Jönköping, Sweden
• Lake Vättern Society of Water Conservation, Sweden
• Medins Havs och Vattenkonsulter AB, Sweden
• Stockholm university, Sweden
• Swedish University of Agricultural Sciences, Sweden

3.3.2 Design and results of the stakeholder analysis

The first four questions were used to collect general information about the respondents and their interest towards getting updates from the SCANDNAnet project and its activities. Replies were mainly provided by people familiar with macroinvertebrate taxonomy, with 65% (13 out of 20) of the respondents having a self-assessed intermediate level of expertise concerning traditional identification methods of aquatic macroinvertebrates (question 9, Appendix 1).  Most of the participants (80%) had low or no experience with genetic methods for identification (question 10, Appendix 1).

Opinions on the use of genetic methods for identification of macroinvertebrate species were all highly positive. All except one respondent (95%) considered the implementation of genetic identification methods to be of intermediate or high necessity in future biomonitoring and assessment programs (question 11, Appendix 1), and to be of intermediate to superior reliability (95%) (question 12, Appendix 1).

There was larger spread of replies concerning opinions of the cost efficiency of genetic methods (question 13, Appendix 1). Half of the respondents see genetic methods as cost neutral or of higher cost efficiency when compared to traditional morphological methods. Two respondents view the method as more expensive than traditional methods and five respondents did not express an opinion.

Almost all respondents (95%) opined that the use of genetic methods will increase in taxa identification in the next 5–10 years (question 14, Appendix 1). Most respondents (75%) went so far as to view this as the definitive path, but respondents also commented that methodology still requires modifications (40%), if not totally new methodologies (50%) to assess ecological status of aquatic environments. The use of genetic methods for ecological assessments is welcomed by most respondents, although with some reservations by a few (30%) (question 15, Appendix 1).

We also collected views on the major factors limiting the shift from traditional assessment methods to the use of genetic methods (question 16, Appendix 1). The main reasons (85% of the votes) concerned lack of standardized methods and of comparability to traditional methods. Half of the respondents rated also quality issues of accuracy and reliability as the third most important factor. In addition to answering pre-defined questions, respondents provided three alternative reasons that might limit or act as barriers to method changes, these were:

• the current need to also monitor size and age of fish, therefore any novel genetic methods will need to be complemented by other sampling and laboratory methods;
• the question of taxonomical resolution i.e. “the problem of species” identified based on morphological traits vs. genetic identification and;
• the necessity of well-informed policy makers who know the method well enough to be able to design projects that are tailored for use in ecosystem management.

4 A validation of metabarcoding for aquatic monitoring in the Nordic countries

4.1 Sample collection

4.1.1 Sample collection protocols

Iceland – Samples were collected from five streams which are monitored annually. Two of the streams are situated in the west of Iceland and three in the north-east. One to three sites were sampled within each stream, in total eight sites. For the sites in the north-east, a protocol that was established in the mid-1990s was followed, where rocks were removed from the streambed and rinsed to collect macroinvertebrates. The material retained was sieved through 125 µm sieve and preserved in 96% ethanol. For the sites in the west of Iceland, Surber samples were collected and processed in the same way as the stone samples. The ethanol was decanted within 24 h and replaced with fresh 96% ethanol. Samples were sorted and macroinvertebrates were identified to the lowest taxonomic unit feasible using a dissecting microscope. For each site, four to five replicate samples were processed.

Denmark – A total of 60 samples were collected from streams covering gradients in stream size and level of anthropogenic impact. Sampling was undertaken according to the guidelines of the Danish Stream Fauna Index (DSFI; Skriver, Friberg & Kirkegaard, 2000). The sampling procedure is standardized with a total of 12 samples collected along three transects (ca. 10 m apart) across the stream. Samples were taken using a hand net (mesh size 0.5 mm) supplemented with a 5-min qualitative sample taken by hand-picking from submerged stones and large woody debris. The samples were preserved in 80% ethanol and the sample was shaken to speed up the displacement of water with ethanol within the animals.  The ethanol was decanted and replaced with 96% ethanol usually within two to three hours after sampling but always within a maximum of 24 hours. The samples were sorted in the lab using x 10 magnification and macroinvertebrates were identified to the lowest taxonomic unit feasible.

Sweden – Fifty benthic invertebrate samples were collected from littoral habitats of lakes (n=25 lakes) and riffle habitats of streams (n=25 streams) in the autumn (October–November) of 2017 according to Swedish guidelines. All lakes and streams are part of the national monitoring program. The study sites were chosen to reflect gradients in latitude and taxon diversity. Five kick samples were taken using a hand net (mesh size 0.5 mm) from hard-bottom (stony) substratum.  Each replicate sample consisted of disturbing the substratum along a 1-m segment of lakeshore or stream reach for 20 s or 60 s, respectively. Samples were immediately preserved in 99% ethanol. On arrival to the lab (within 24-48 h) the ethanol was decanted and replaced with 99% ethanol. The samples were processed in the laboratory by sorting using x 10 magnification and macroinvertebrates were identified to the lowest taxonomic unit feasible (usually species) and counted using light and dissecting microscopes.

Norway – 139 benthic invertebrate samples were collected from littoral habitats/outlets of lakes (n=31 lakes) and riffle habitats of streams (n=36 streams) in the autumn (September–October) of 2018. From some of the lakes, additional samples were taken in early summer (May–June). All lakes and streams are part of the national monitoring program. The study objects were chosen to reflect different lake- and river types (covering gradients in latitude, alkalinity and humic content) and taxon diversity. For lakes one sample was taken from the lakeshore and one from the outlet. For rivers, a single sample was taken. All samples were taken using a hand net (mesh size 0.25 mm) from hard-bottom (stony) substratum by disturbing the substratum along a segment of lakeshore or stream reach for 60 s (kick-sampling). Samples were immediately preserved in 96% ethanol. The ethanol was decanted and replaced with 96% ethanol within 24 h, either in the field or upon arrival to the lab. The samples were processed in the laboratory by sorting using x 10 magnification and macroinvertebrates were identified to the lowest taxonomic unit feasible (usually species) and counted using light and dissecting microscopes.

Finland – A total of 48 benthic samples were collected in the autumn (September–November) of 2017, consisting of 23 stream samples, 9 lake profundal samples and 16 lake littoral samples. Sampling sites, distributed across most of Finland, are part of the national monitoring program for effects of forestry and agricultural practices (Aroviita et al. 2014) and sampling followed national WFD biomonitoring guidelines. Each sample was a composite of 6 subsamples taken at each stream or site lake. In streams and lake littorals each subsample was taken with a hand net by moving upstream or upwind for 1m and for a duration of 30 s (littorals 20 s) while disturbing the substrate by kicking. For lake littoral habitats, depending on the presence of stony littoral habitat in each lake, sampling was conducted in two lakes at one littoral site (six subsamples from each site), in five lakes at two littoral sites (three subsamples from both sites) and in nine lakes at three littoral sites (two subsamples from each site). For lake profundal habitats, the lake sample consisted of a total of six Ekman grab subsamples taken at a depth of at least 90% of the lake’s maximum depth. Benthic samples were sieved using a 0.5 mm mesh and preserved in the field using denatured 96% ethanol. The ethanol was decanted and replaced with 96% ethanol within 24 h. The samples were sorted in the laboratory with no magnification and the macroinvertebrates were identified and counted according to the national requirements mostly to species or genus (excluding identification of dipteran families and Oligochaeta) using light and dissecting microscopes.

4.1.2 Sample preparation

After morphological identification all samples were returned to vials, stored in 96% ethanol and sent to NTNU for laboratory processing. Samples were stored at 6°C on arrival to the laboratory until processed for DNA extraction explained in the next session.

4.2 Laboratory methods

To remove ethanol, samples were dried 24–48 hours in sterile disposable grinding chambers (MT 40.10, IKA®-Werke GmbH & Co. KG, Staufen, Germany). The dried specimens were homogenized by grinding at 10000 rpm for 3 min. in an IKA TUBE-MILL control. Small samples that did not homogenize well, were further grinded with a sterile pestle in the same grinding chamber. From each sample, three subsamples were taken with a sterile spoon. The subsamples contained on average 14.93 mg (SD = 5.69 mg) of homogenized tissues. Total DNA was extracted from each subsample, using DNeasy 96 Blood & Tissue Kit (Qiagen Hilden, Germany) following the manufacturer’s instructions. Some of the samples were so small that there was only a sufficient amount (ca. 15 mg) of homogenized tissue for only one or two subsamples. In these cases, the DNA extract was divided into two or three aliquots. DNA was diluted with a 1:10 ratio to decrease the concentration of possible inhibitors. DNA quality was checked using a 1% agarose gel. The molecular protocol initially used is provided in Appendix 3. However, use of this protocol failed to provide (for undetermined reasons) a DNA library that could be successfully sequenced.

The metabarcoding process was therefore repeated. DNA was amplified using a two-step PCR protocol. All of the following steps were carried out on a Biomek FXP (Beckmann Coulter, Bread, CA, USA) liquid handling station. In the first PCR, all samples coming from the same plate were tagged with an inline tag (Elbrecht and Steinke 2019), while each individual well was tagged with unique 8 bp Illumina indexes at both ends (i5 and i7 index), allowing to multiplex all samples in one HiSeq Rapid sequencing run. For the first step, DNA was amplified using the Qiagen Multiplex PCR Plus Kit (Qiagen, Hilden, Germany) with a final concentration of 1x Multiplex PCR Master Mix, 1x CoralLoad Dye, 100 nM of each primer (BF3/BR2, (Elbrecht et al. 2019; Elbrecht and Leese 2017)), 2.5 μL of DNA filled up to a total volume of 25 μL using a touchdown PCR protocol with: 95 °C for 5 min initial denaturation, 10 cycles of 95 °C for 30 s denaturation, 60 - 51 °C for 90 s annealing (1 °C decrease per cycle) and 72 °C for 36 s elongation followed by 20 cycles with 50 °C as annealing temperature ending with 68 °C for 10 min as a final elongation step. In the second step, DNA was amplified 1x Multiplex PCR Master Mix, 1x CoralLoad Dye, 100 nM of each primer, 1 μL of PCR product filled up to a total volume of 25 μL. The PCR protocol was 95 °C for 5 min initial denaturation followed by 15 cycles of 95 °C for 30 s denaturation and 72 °C for 135 s for annealing and elongation followed by 68 °C for 10 min as a final elongation step. PCR success was checked on a 1% agarose gel. DNA concentrations were normalized using the SequalPrep 96-Well-Kit (Applied Biosystems, Foster City, CA, USA) with an elution volume of 20 μL. The full library volume of each 96 well plate was pooled in the final library. DNA was concentrated using the NucleoSpin Gel and PCR Clean-up kit (Macherey-Nagel, Düren, Germany) and checked for unspecific amplification (e.g. primer-dimers) using the Fragment Analyzer (Standard Sensitivity NGS Fragment Analysis Kit; Advanced Analytical, Ankeny, USA). To remove primer-dimers the library was size-selected using the NucleoMag NGS Clean-up and Size Select Kit (Macherey Nagel, Düren, Germany). Concentration of the cleaned-up library was measured using the Qubit 2.0 (Broad Range Kit; Thermo Fisher Scientific, Beverly, USA). Sequencing carried out by Macrogen (Seoul, South Korea) using both lanes of two Illumina HiSeq PE rapid runs with 250 bp paired end reads with 5% PhiX spike in to improve sequence diversity and therefore sequencing quality.

4.3 Bioinformatic analysis

Samples were demultiplexed by the sequencing provider using dual Illumina indexing on a total of 88 wells (8 wells per plate were empty, and thus not included). Additionally, each of the demultiplexed fastq files were split into the 12 individual samples based on inline tags used for each of the 12 plates, respectively.  Demultiplexing was done for both HiSeq run A and B. Using a small subset of samples from both runs, it was verified that both sequencing runs returned the same results. Therefore, and also because sequencing depth of individual samples was very similar across both runs (Figure 1A), each respective sample from Run A and B were combined into a single file and processed together.

The 1056 samples (297 samples times three replicates, 33 positive controls and 132 negative controls) were processed using the JAMP v0.78 pipeline (https://github.com/VascoElbrecht/JAMP/) relying on Cutadapt v2.8 (Martin 2011) and Vsearch v2.14.2 (Rognes et al. 2016). Samples were paired-end merged using Vsearch, with fastq_maxdiffs = 99, and fastq_maxdiffpct = 25 to maximize the amount of sequences merged. Primers were removed using Cutadapt on default settings, and sequences below 408 bp and above 428 bp (+/- 10 bp of the target length) were discarded. Sequences with expected errors above 1 were discarded as well (Edgar and Flyvbjerg 2015). The remaining filtered sequences were denoised using Vsearch on a per sample basis, using the unoise3 module with a minimum size of 5 after dereliction. There was no additional filtering applied to the raw ESV table and ESVs grouped into OTUs using 3% clustering. The generated OTU table was filtered for spurious OTUs using LULU on default settings (Frøslev et al. 2017), with merging affected OTUs. The resulting OTU table was further filtered, by only keeping reads in each sample that were present in two out of the three replicates. After these filtering steps, all three replicates were merged into one sample for further processing. Taxonomy was assigned using https://www.gbif.org/tools/sequence-id, as well as BOLDigger (Buchner and Leese 2020) v1.1.10 using the JAMP setting. Resulting taxonomic assignment was manually consolidated. Unfortunately, there was not sufficient time to also taxonomically assign the OTUs against the NCBI database when writing this report. A sample was deemed to be metabarcoded successfully, when it had a minimum of 100,000 reads in the final OTU table. Samples were not rarefied to the same sequencing depth, to maximize the number of taxa detected. For taxonomic analysis, only OTUs matching to Annelida, Arthropoda, Mollusca, Nemertea and Platyhelminthes with at least 90% identity were considered.

5 Results

5.1 Sample and DNA extraction

A total of 297 samples were collected, covering all of the Nordic countries (Figure 1). For Denmark 60 samples were processed, 48 for Finland, eight for Iceland, 131 for Norway and 50 for Sweden. Amounts of DNA varied between samples (based on gel imaging). 92% of the samples showed clear DNA bands and only 8% of samples showed no visible DNA or only degraded DNA across all three replicates, with Denmark and Finland being most affected (12 or 8 samples, respectively). Especially Danish samples lacked a visible DNA band (Figure 1D). DNA quality across samples was only moderately consistent, with 45% of replicate samples showing deviating DNA quality across extraction replicates.

The whole laboratory analysis of 11 plates, i.e. from DNA to final library for sequencing took three working days with the Biomek FxP Dual hybrid, in comparison to approximately ten working days that one person initially used for the failed libraries. Unfortunately, a label mix-up that occurred when transferring the sample positions from the spreadsheet to the demultiplexing file affected two rows of one microplate in the demultiplexing-file. This issue was noticed and resolved by changing the layout map for that 96 well plate retrospectively. We noticed that one of the 11 negative controls showed a clearly visible band for this plate. This might be caused by cross contamination, although 97% of the 132 negative controls across all 12 plates did not show visible bands in the DNA gels, indicating that contamination between samples was not a serious issue. Due to the extensive use of negative controls (11 per plate) and replication the human error in plate one was detected.

Figure 1. Map depicting the geographical distribution of the sampling sites in the Nordic countries.

5.2 Metabarcoding success

The initial metabarcoding libraries did not sequence successfully, despite no dimers being visible on an agarose gel or fragment analyzer, and the bands having an expected size of around 600 bp and no visible dimers over 90% of the obtained reads were dimers (SRA: SRR12615112, SRR12615111). We cleaned up the library several times with SpriSelect but the problem remained, as was evident by two iSeq runs (SRA: SRR12615016, SRR12615005). As the problem could not be resolved, all PCRs and library preparation steps were repeated using a different library preparation method, and then sequenced successfully. Samples on all 12 plates were uniquely tagged on both sides using Illumina indexing for 96 wells and inline tagging for each plate and pooled into the same Library, which was sequenced on two HiSeq rapid runs (2x250 bp). The first run A yielded 296,210,273 PE reads, and the second run B 289,096,640 PE reads in total. The raw data is available under the SRA accession PRJNA662474 and the information on inline tags available as supporting information (Scripts S1). As both runs had a similar amount of reads per sample (Figure 2A) and generated the similar metabarcoding results (tested on a small subset, data not shown), reads from both runs were pooled for each sample and analyzed together. Additionally, 10 sample pairs from both sequencing runs were chosen at random, and processed with the full metabarcoding pipeline, to validate that they produced similar results. Results between both runs were consistent, except for sequences of lower abundance which are more affected by stochastic effects (Leray & Knowlton 2017).

An average of 370,794 reads were obtained per replicate (Figure 2B). Except for samples from Denmark, over 95% of samples were successfully metabarcoded for all three 3 replicates (over 33.000 reads remained after bioinformatic processing, Figure 1F). Six samples from Denmark (DK24, DK25, DK26, DK29, DK31, DK37) and one sample each from Norway (NO138) and Sweden (SE25) had under 100,000 sequences after bioinformatic analysis and pooling of all three replicates.

The number of taxa recovered using morphology and DNA metabarcoding differed by country (Figure 3). For Norway, Iceland and Finland, DNA metabarcoding recovered more taxa than morphology in almost all cases. For Denmark and Sweden, results where more mixed, with morphological methods detecting more taxa than metabarcoding in 28% or 38% of the samples.

When looking at the 200 most abundant taxa determined with morphological identification, metabarcoding was able to recover 133 of them based on our preliminary data. However, merging the DNA-based data is difficult due to synonyms and the incomplete DNA reference database, and therefore will require more manual work and curation. Thus, the exact number will likely increase with a more detailed analysis that includes taxonomic expert curation. When looking at the 200 most abundant taxa detected with metabarcoding, 63 were not detected with morphological identification and mostly belong to groups that have larval stages or are difficult if not impossible to identify for use in routine monitoring. These include taxa belonging to Diptera, EPT (Ephemeroptera, Plecoptera, Trichoptera), Haplotaxida, Trombidiformes and Rhynchobdellida.

Figure 2: Summary statistics of sequencing depth and DNA extraction, PCR and metabarcoding success across replicates. A Comparison of sequencing depth per sample in the two HiSeq sequencing runs (A and B). B Histogram showing sequencing depth across samples (both HiSeq runs combined). C Boxplot showing the sequencing depth of samples by country. D Assessment of DNA quality by country. E PCR success by country. F Number of replicates per sample containing more than 33,000 reads in the final Exact Sequence Variant (ESV) table. NC = negative control, PC = positive control.

Figure 3: Comparison of number of taxa detected with morphological determination and DNA metabarcoding (OTUs) across all 297 samples. The letters stand for the respective countries: I = Iceland, N = Norway, F = Finland, S = Sweden, D = Denmark. Note that this plot is made with preliminary data, which has not yet been manually checked for taxonomic validity.

5.3 Iceland

All the eight sites from Icelandic streams were all spread well above the 1:1 line (Figure 3), which means that there are considerably more taxa identified by the DNA-based method than by traditional morphological methods. Number of taxa identified by morphology were 26–69% of the number of taxa that were identified by DNA-based method. This difference may possibly be explained by a low taxonomic resolution for some of the major taxa in Icelandic streams such as Chironomidae and Oligochaeta. If chironomids, identified by traditional methods, had been added to the matrix the number would have increased by 8–16 species, resulting in a closer fit to the 1:1 line in Figure 3. This demonstrates the need for including chironomid identifications in monitoring freshwaters at higher latitudes where chironomids often predominate.

5.4 Norway

To compare the overlap between morphological and DNA-based identification methods, results from four rivers (samples NO084-NO087) and four lakes (samples NO003-NO004, NO007, NO088-NO090, NO100-NO103) were checked manually. The number of aquatic macroinvertebrate taxa was higher using DNA-based identification than morphology-based identification, averaging 24 and 15 taxa, respectively (paired t-test, p<0.01). On average, 13 (rivers) and 11 (lakes) taxa were found with both identification methods per sample. On average three taxa per sample were found only with morphological identification, while 12 taxa per sample were found only with DNA-based identification in rivers and lakes.

The higher number of taxa with the DNA-based identification is mostly due to a species-level identification of Clitellata, Chironomidae and Simuliidae that were not identified further than class-level or family-level using morphology. In some samples, DNA-based identification also found two or three species of EPT from one genus, while morphology-based identification found only one. These results are as expected since Clitellata and larvae of Chironomidae, Simuliidae and EPT taxa within one genus can be difficult to separate morphologically. The taxa that were only found with morphology are more interesting. Two cases were somewhat expected: hard-shelled bivalves and gastropods as well as small copepods and branchiopods. The same bivalves and gastropods were found with both methods if the number of specimens in the samples was high (5–38% of total specimens for bivalves, 0.7–4% for gastropods) while they were found only with morphology if the number of specimens was low (1.5% for bivalves, 0.4% for gastropods). DNA-based detection of Mollusca has been shown to be lower than detection of other taxonomic groups in unsorted bulk samples (Beentjes et al. 2019).  The number of copepods and branchiopods was very low in all samples, which may explain the lower success rate using DNA-based identification. Lysis of the whole sample may increase the chance of detecting rare species. Another reason may be slight primer mismatch or amplification efficiency, which is reported for this primer (Elbrecht and Leese 2017). More critical to current assessments are the few cases where Clitellata, Simuliidae or some EPT taxa were found with morphology but not with DNA-based identification. In these cases, the lack of taxa discovery in DNA-results cannot be attributed to low numbers of specimens. For example, species of Clitellata were found with both methods in one case when their proportion in the morphologically identified sample was as low as 0.2%; however not in another case when their proportion was 5% of the morphologically identified specimens. Such trends also were visible in sample NO103 for Baetis rhodani despite the fact that this species’ DNA amplifies readily in other samples. It is likely that these patterns indicate considerable DNA degradation (e.g. visible in sample NO103), which would explain the incomplete discovery of taxa.

5.5 Sweden

The Swedish samples were dispersed along the 1:1 line of morphological determinations and DNA metabarcoding. However, one site (stream SE25) deviated markedly: only 18 OTUs were identified compared to 86 taxa by microscopy. This site was also an outlier in terms of yielded sequences (see 5.2) and thus technical reasons (e.g. DNA degradation) likely explain deviating results. Overall, the span in number taxa along the 1:1 line reflects the geographic distribution of lakes and streams and environmental gradients (e.g. latitude) studied here. The relatively high agreement between morphological and OTUs is likely due to the expertise and standardised protocols of the lab identifying samples collected in the national monitoring programs. Samples from the national lake and stream monitoring programs are sorted and taxonomically identified by only a few experts (three persons), resulting in high temporal continuity within the monitoring programs during the past 40 years. Currently, the standardised list of taxa required to be identified by the taxonomists comprises over 500 species, although higher resolution is also used in some projects. Furthermore, in the national monitoring programs chironomid midges collected from lake littoral or stream habitats are not usually identified to species. The lab routinely runs intercalibrations for consultants involved in regional monitoring and participates in international (ICP-Waters) calibrations. These activities likely result in the relatively high level of coherence reflected in Figure 2.

5.6 Denmark

Of all countries, Danish samples had the highest proportion of samples (i.e. 28%) with fewer OTUs assessed by DNA metabarcoding compared to Linnean taxa identified by morphology. This is likely due to the preservation scheme that used 80% ethanol in the initial preservation step of the sample. Given that the time span to changing the ethanol to 96% varied between 3-24h and due to the fact that the use 80% ethanol resulted in a lower than desired initial concentration in the specimen tissue this likely resulted in the degradation of specimens. Further support for this conjecture is the fact that in morphological identification many of these samples had considerable abundances of Chironomidae which likely would have been identified and thus would have increased the number of OTUs beyond the morphologically identified number of taxa.

5.7 Finland

In the preliminary data metabarcoding detected in large majority of cases (96%) more OTUs than were identified by morphology (Figure 3). Across stream and lake littoral samples, the number of taxa detected by morphology was about 2/3 of the number of OTUs detected by metabarcoding. One reason for the disagreement is that for stream and lake littoral samples dipteran families and e.g. Oligochaeta were not identified by morphology which followed the national target taxa lists for WFD assessment. The agreement in the number of taxa between the two detection methods was stronger in the samples from lake produndal (result not shown), where communities consist mainly of Chironomidae and Oligochaeta that indeed were also identified by morphology. Only two Finnish samples (4%) had fewer OTUs identified by metabarcoding (Figure 3). Both of these were from streams and had considerable abundances of morphologically identified specimens belonging to the families Chironomidae and Simuliidae. Possible explanations for this mismatch can be morphological misidentification (higher intraspecific variation than accounted for) or partial DNA-degradation, which is supported by the molecular indicators of degradation in 18 subsamples (Figure 2D).

6. Discussion and recommendations

6.1 Conclusions of the stakeholder analysis

Stakeholder views towards the implementation of molecular identification methods for routine monitoring was positive amongst the respondents, even though most were not experts in the field. A general consensus was that these methods will be increasingly used in monitoring and assessment programs. However, most respondents also felt that some crucial issues need to be resolved before DNA methods can be used for genetic identification in routine monitoring. We conclude that the resoundingly positive response by respondents forms a strong foundation for the future implementation of these methods by the Nordic countries.

Figure 4: Laboratory and bioinformatic workflow. Orange bubbles highlight encountered problems and challenges. (1) Two sample columns where swapped when setting up the sample position sheet DNA extraction. (2) Gel images and image scoring were not consistent (human bias), and some samples showed signs of DNA degradation (issue with sample taking / storage). (3) The sequencing run initially failed due to presence of primer dimers, even though non where visible. (4) Bioinformatic processing took around one week, which is exceptionally long, as no server was available. (5) Sequencing was done by a sequence provider, who had to ship the data as a hard drive because it was too much to download. (6) The raw data (not demultiplexed) provided by the sequencing provider was corrupted and could thus not be uploaded to the NCBI short read archive to make it publicly accessible. (7) The current database accessibility BOLD via API is incomplete, thus sequences had to be compared using the BOLD online taxonomic search, which reduces the verifiability of the results. (8) Comparison of morphology and DNA metabarcoding based taxa lists is not ideal, as synonyms and other potential errors like falsely assigned taxonomy were not manually resolved due to the lack of time.

6.2. General findings of the method validation

Macroinvertebrate indices based on present-absence molecular data have been shown to largely correspond to those values based on traditionally keyed samples (Elbrecht et al. 2018; Beentjes et al. 2018). Due to time constraints and the difficulties with generating a reliable reference database (see 6.3 for details) we were not able to fully explore the congruence between morphological and DNA-based identification, including the patterns that would emerge when calculating national indices using either method, during the span of this project. This work will be conducted in-depth at a later stage.

6.3 Issues encountered in the metabarcoding process

We successfully metabarcoded 289 of 297 samples (97.3%) and in most samples more taxa were detected using metabarcoding than with the morphology-based determinations. However, we also encountered major obstacles and challenges that resulted in substantial delays of the project (see figure 4).

On the laboratory side, sample processing was carried out initially by a team that received the laboratory protocols, but that did not have extensive experience in the particular protocol. This led to errors in the bioinformatic sample assignment step (demultiplexing), where samples are separated based on the attached unique molecular indices. The position of each sample was planned using a spreadsheet, and on the first plate two rows were exchanged due to a copy paste error. This error was detected and mitigated retrospectively, due to use of negative controls on the plate and due to the fact that each sample was processed with three replicates distributed across 12 plates. Despite this, the possibility remains that other samples on plate 1 were erroneously put in the wrong wells or that there was true cross-contamination between samples, which was indicated by a negative control that contained many reads.  Ideally, the entire DNA extraction and metabarcoding of the whole plate would have been repeated. The issue of sample cross-contamination could have been spotted when running a DNA gel of the DNA extraction or measuring the DNA concentration with digital methods. This is a critical step to assess DNA quality and quantity before DNA is used in PCR. In this step, a simple DNA gel picture or detection of a significant amount of DNA would have revealed that one of the negative controls contained a substantial amount of DNA, which would have allowed us to repeat the whole extraction. However, as gel pictures of the DNA extracts where only taken retrospectively, this mistake was not detected early in the process.

To minimize the risk of such errors early in the process chain, we recommend rigorous quality assurance training for the team carrying out sample processing, as well as taking photographs of the samples as they are placed in the extraction plates. This way, if for example a negative control is found to be positive, the photographic documentation can be checked to see if samples were misplaced. Also, the use of a dedicated laboratory information system (LIMS) is advised, as it allows tracking of samples through the entire work process. Further, the use of QR-code labeled vials and sampling tubes in combination with barcode scanners for sample intake, as well as automated processing of samples using robotics, would significantly reduce the risk of human error. Regardless of whether robotics is used, rigorous training and auditing of personnel along with detailed written process documentation is paramount to reduce errors at this stage of the process. Similar certification exists for medical and commercial laboratories and adoption of such routines for environmental metabarcoding are possible.

We assessed DNA quality as well as PCR success using gel pictures. For DNA extractions we repeated this procedure independently twice but results of these assessments varied substantially in some cases. The DNA gel images were only consistent for 55% of the samples among the three replicates available for each sample. This might indicate issues with preparing the DNA extraction replicates from the ground tissue powder. However, it is also very likely that there are inconsistencies between the different gel documentation systems used by both institutes, especially because these laboratories only performed analyses within a very short time frame (3 days) and because initial attempts with library preparation failed by another lab. It should be pointed out that all these methods are usually very reliable. While gel pictures are sufficient for assessing PCR success, we would recommend using digital gel imaging methods (e.g. Fragment Analyzer, Agilent Technologies). Digital gel imaging allows to precisely assess and quantify DNA concentration and degradation by measuring DNA fragment length. This allows for a more precise identification of samples which have been affected by DNA degradation. DNA degradation can occur both when there are a lot of specimens in a sample, or when the ethanol is not replaced with fresh ethanol after collecting the samples. In both cases, the water in the specimens and detritus can dilute the 96% ethanol to 80–70% at which point DNA degradation can already occur. This can be problematic, as it might degrade the DNA from some or even all specimens in the sample, greatly reducing the chance for good amplification of the barcoding fragment in PCR or even leading to complete PCR failure. This might have been the case for some samples from Denmark, where over 30% of replicates had DNA quality issues. As a consequence, six samples did not work at all, with 14 additional samples showing dropouts in at least one replicate. A follow up study will determine if the Danish samples were collected according to protocol, or contained an unusual number of specimens, in which case the sampling protocol might need to be adapted by increasing the amount of ethanol or replacing the ethanol with fresh ethanol more than once.

An additional challenge that led to substantial delays was that the initial metabarcoding laboratory work carried out according to the protocol in appendix 3 did not result in a library that could be sequenced successfully. While we obtained sequences, over 90% of them where primer dimers, and only a small proportion were target sequences. These issues are puzzling, since the metabarcoding laboratory process partly functioned, as a small amount of sequences generated were of the expected length and assigned to macroinvertebrates. Additionally, the primer dimers were not visible in gel pictures or using digital gel imaging methods, but flooded the sequencing run regardless even after repeated cleanups as well as gel checks of that library after repeating the failed run. We were not able to determine or resolve the cause of these issues. Therefore, DNA samples were shipped to a different laboratory, where DNA was quantified and the whole procedure started again with small modification, i.e. the BF3 primer instead of BF2, as well as a touchdown PCR, even though touch down PCRs are usually not recommended for metabarcoding (Aylagas et al. 2016). We have to stress here, that the issues we encountered with sequencing are extremely rare (it has never happened in two of the three labs involved despite having processed over 100 sequencing runs) and could have been caused by a handling issue when preparing the DNA libraries.

Bioinformatic processing (Figure 3) went well but did encountered a few logistical challenges. Data delivery and processing formed a bottleneck, as a server was not available at that time. Thus, processing of the almost 600 million sequences in this data set had to be carried out on a 2019 MacBook pro, which took about one week. Offline data processing of 160 GB compressed data on separate devices is prone to device error or malfunction and also presents challenges to long term data storage and to data up and download. Such issues can be circumvented with properly implemented scalable cloud computing solutions which are currently not readily available. Demonstrating the need for more streamlined workflows was also exemplified that even the external sequencing provider had issues delivering the data online and chose to send the sequenced raw data per mail on an unencrypted hard drive. While this is not only ineffective, it also poses data security risks and the challenge to reupload data back onto a server, whereas server to server transfers are usually fast. Thus, we recommend implementing an appropriate cloud-based bioinformatics pipeline, with dedicated long-term raw data storage strategies. For this project we decided to use the NCBI short read archive for long term storage of the data, which allows us to also make the data publicly accessible (SRA accession PRJNA662474). However, not all of the raw data could be uploaded, some files were delivered containing errors in the sequence order, and thus could not be accepted by SRA. The error in the raw data delivery format was due to an error on the sequencing provider site and should have been checked upon data delivery to us. We are currently working with the sequencing provider to resolve this issue. While there were some issues with the company that did the sequencing, we still recommend using an external sequencing provider. This is because sequencing technology is rapidly improving and evolving, and thus inhouse sequencing and investment costs of up to 1 million USD of sequencing machines which are likely obsolete within years are not economically feasible unless there is a huge throughput. Outsourcing of the sequencing to commercial providers is more cost effective, even though part of the process is not fully controllable.

Further bioinformatics challenges were encountered when assigning taxonomy to the resulting DNA sequences after bioinformatic processing. For the COI marker used there are two main sequence repositories for reference sequences, BOLD and NCBI. Since we are dealing with millions of reference sequences, generating a reliable reference database is not trivial. Some of the sequences uploaded to these databases might be annotated incorrectly, may contain errors or are from a misidentified organism, thus linking to the wrong taxon. An additional complication is that both databases might use synonymous species names and differ in taxonomy. With BOLD there is an additional complication, as around 50% of the data are not publicly accessible, and only can be queried using the website API. Even publicly available BOLD data cannot be easily downloaded. This complicates the data base situation and given the limited available time towards the end of this project taxonomy was assigned using the BOLD website. This has the advantage that it allowed us access private data as well. However, for long-term monitoring, we strongly recommend that Nordic/nation-wide reference databases for macroinvertebrates are generated using strictly quality controlled public data from BOLD and NCBI, as well as specimens collected in each country. In fact, at least Denmark and Norway have already started this process of generating national, curated reference databases (for Denmark see https://dnamark.ku.dk, for Norway see discussion in Åström et al. 2020). An accurate and complete reference database will be key to a future monitoring program and needs to be thoroughly checked for errors and missing taxa with the aid of taxonomic experts. The web based BOLD reference database we used does not fully meet these quality criteria.

6.4. Assessing the applicability of the DNA-based identification

For the assessment of the applicability of any novel method, a number of criteria need to be investigated: method reliability, environmental impact, added value, limitations and required expertise as well as costs (Mack et al. 2020). DNA-metabarcoding scored “high applicability” in a recent assessment of metabarcoding viability in the context of Baltic marine monitoring (Mack et al. 2020) but this assessment did not entail a large-scale validation exercise like the one conducted in this study. Hering et al. (2018) discuss the implementation options of molecular methods in the context of the WFD in detail, dividing the different pathways into option 1 and option 2. In option 1, the “DNA-based identification” simply replaces traditional identification and the molecular information obtained is reverted to the Linnean taxa system. With DNA-based identification, current indices and the subsequent assessment systems could remain the same or adoption would necessitate only minor modifications. The advantages of this approach would primarily be in increased comparability, processing speed and cost efficiency. However, option 2 would make full use of the molecular data and allow for analyses utilizing taxonomy-free approaches and also functional gene assessments that could provide previously unattainable information on trends in response to single and multiple stressors not previously encountered (Beermann et al. 2018, Beermann et al. 2021, Cordier et al. 2020). While this prospect is tempting, option 2 is not likely to become a reality in the next 10 years. However, a switch from traditional identification to DNA-based identification and its extensions (e.g. the inclusion of taxa typically not identified in routine bioassessments, such as Chironomidae) it is possible and would have the potential to provide a much more accurate estimate of environmental conditions and enable the gradual progression towards option 2.

Reliability of a method is an important deciding factor when deciding on its implementation into routine use. Mack et al. (2020) rated reliability of novel methods as ‘high’, either when the resulting data had an improved reliability or when it was comparable but resulted in substantially greater amounts of data. With the design we used, a fully independent assessment of method reliability is not possible. This is because the samples used in metabarcoding were first subjected to manual morphological identification, which could have led to additional degradation of the DNA in samples during the identification process due to diluting the samples with water to avoid excessive vaporization of alcohol during manual keying. Our results strongly suggest that for most samples suffering from degradation, this occurred before samples went through the genetic pipeline, thereby likely reducing the reliability of the DNA-based identification. Levels of degraded samples in Denmark were much higher than what could be tolerated during routine operation. However, as the trends for sample degradation were not equally distributed among countries, we are confident that the poor preservation of DNA in the Danish samples was mainly due to an easily avoidable procedural error before sequencing, given that most other samples had good DNA quality. Similarly, the fact that much of the laboratory work had to be repeated in this study may seem to indicate that metabarcoding is not ready for routine use yet. Indeed, it shows that the method can be sensitive to protocol violations. However, we argue that rather than pointing to a lack of reliability, the observed shortcomings emphasize the need for internationally agreed upon laboratory protocols and staff training. Laboratory automation as performed in this project is one step into this direction. With the Biomek FxP (Fig. 5) the over 1000 individual reactions were processed successfully within only three working days.

Work on developing existing European standards for sample preservation during macroinvertebrate sampling should also be revisited or the drafting of novel guidance considered. Nordic efforts for developing such guidance and providing training to laboratory and field personnel in the context of national reference laboratories must clearly be a future priority.

The need to assess the viability of molecular methods for routine use in identification is not unique to the use of metabarcoding in aquatic biomonitoring. In Norway, an extensive pilot study tested various practical methodologies for implementing a general insect monitoring scheme (Åström et al. 2020). The pilot study used flying insects collected by Malaise traps, later to be identified by DNA-metabarcoding in the lab and validated by traditional morphological identification. In addition to measuring the insect communities and comparing DNA metabarcoding techniques with morphological identifications, the report contains a cost-benefit analysis of a potential future general, nation-wide insect monitoring scheme.

Figure 5: The liquid handling robot enabled standardized processing of samples in shortest time.
Picture: Till Macher

There is no doubt that metabarcoding can provide additional value and a wealth of information that is not available at present. The full value of this method will only materialize over time when the method will move from DNA-based identification towards option 2 (i.e. utilizing also functional gene assessments and taxonomy-free approaches). But it is evident that the application of this metabarcoding will require an investment into recruiting and training until the expertise of staff involved in its application to acceptable and stable levels. Based on our findings we agree with the assessment of Mack et al. (2020), namely, that DNA-based identification using metabarcoding indeed has high applicability but also caution that work on unified guidance is paramount for routine implementation.

The existing momentum and enthusiasm about the prospect of DNA-based identification has led to several separate national initiatives that are developing the methodology. The authors of this report urge these developers to consider larger Nordic and European cooperation and method standardization as a priority. This will not only speed up the uptake of the methods beyond the pure research domain into routine monitoring, but also ensure method comparability and reliability. Like the respondents of our key stakeholder survey we are convinced that DNA-based identification and its future extension are the way forward. The development is now at an important crossroad and only through intense cooperation will we be able to avoid the same fate of method pluralism that is now used in biomonitoring and assessment in implementation of the WFD (Birk et al. 2012). Cooperation on method and index development but also agreement on metadata formats and database structures will be key to increase the overall benefit that can be gained from the adoption of this approach. Especially when working towards using the full potential of molecular methods, (i.e. “option 2”, sensu Hering et al. 2018) such concerted Nordic efforts would set international examples of best practice which would greatly benefit both research and assessments alike.

7 Conclusions

This validation study took an in-depth look at the reliability of DNA-based taxonomic identifications for routine WFD compliant monitoring in the Nordic countries. The results are very encouraging but our study also identified areas that will require addressing before routine implementation into Nordic monitoring programs can be achieved. These are, in particular, quality assurance and training, especially when several labs are involved in the whole assessment process. A number of conditions will need to be met to adopt DNA-based identifications in all Nordic freshwater bioassessments. These include stronger international cooperation in method development as well as the development of standardized guidance documents which currently is lacking almost entirely. 

8 References

Andersen, J.H., Aroviita, J., Carstensen, J., Friberg, N., Johnson, R.K., Kauppila, P., Lindegarth, M., Murray, C. & Norling, K. (2016). Approaches for integrated assessment of ecological and eutrophication status of surface waters in Nordic Countries. Ambio 45: 681–691. http://dx.doi.org/10.1007/s13280-016-0767-8

Aroviita J., Vuori K.-M., Hellsten S., Jyväsjärvi J., Järvinen M., Karjalainen S. M., Kauppila P., Korpinen S., Kuoppala M., Mitikka S., Mykrä H., Olin M., Rask M., Riihimäki J., Räike A., Rääpysjärvi J., Sutela T., Vehanen T. & Vuorio K. (2014). Maa- ja metsätalouden kuormittamien pintavesien ekologinen tila ja sen seuranta. Suomen ympäristökeskuksen raportteja 12/2014.  http://hdl.handle.net/10138/45017

Aroviita J., S. Mitikka & S. Vienonen (eds.) (2019). Status classification and assessment criteria of surface waters in the third river basin management cycle. Reports of the Finnish Environment Institute 37/2019: 1–177. (in Finnish) http://hdl.handle.net/10138/306745

Aylagas E., Borja Á., Irigoien X. & N. Rodríguez-Ezpeleta (2016) Benchmarking DNA Metabarcoding for Biodiversity-Based Monitoring and Assessment, Frontiers in Marine Science 3, https://www.frontiersin.org/articles/10.3389/fmars.2016.00096/full

Ärje J., Raitoharju J., Iosifidis A., Tirronen V., Meissner, K., Gabbouj M., Kiranyaz S. & S. Kärkkäinen (2020). Human experts vs. machines in taxa recognition. Signal Processing: Image Communication.  https://doi.org/10.1016/j.image.2020.115917  

Åström J., Birkemoe T., Dahle S, Davey M., Ekrem T., Endrestøl A., Fossøy F., Nystad Handberg Ø., Hanssen O., Magnussen K., Majaneva M.A.M., Navrud S., Staverløkk A., Sverdrup-Thygeson A. & Ødegaard F. (2020). Proposal for a national insect monitoring program in Norway – Findings from a pilot study with cost-benefit analysis. NINA rapport 1725. Norsk institutt for naturforskning.

Beentjes K.K., Speksnijder A.G.C.L., Schilthuizen M., Schaub B.E.M. &van der Hoorn B.B. (2018) The influence of macroinvertebrate abundance on the assessment of freshwater quality in The Netherlands. Metabarcoding and Metagenomics 2: e26744. https://doi.org/10.3897/mbmg.2.26744

Beentjes K.K., Speksnijder A.G.C.L., Schilthuizen M., Hoogeveen M., Pastoor R. & van der Hoorn B.B. (2019). Increased performance of DNA metabarcoding of macroinvertebrates by taxonomic sorting. PLoS ONE 14: e0226527. https://doi.org/10.1371/journal.pone.0226527

Beermann, A.J., Zizka, V.M.A., Elbrecht, V., Baranov, V. & Leese, F. (2018). DNA metabarcoding reveals the complex and hidden responses of chironomids to multiple stressors. Environmental Sciences Europe 30, 26. https://doi.org/10.1186/s12302-018-0157-x

Beermann, A.J., Werner, M.-T., Elbrecht, V., Zizka, V.M.A. & Leese, F. (2021). DNA metabarcoding improves the detection of multiple stressor responses of stream invertebrates to increased salinity, fine sediment deposition and reduced flow velocity. Science of The Total Environment 750, 141969. https://doi.org/10.1016/j.scitotenv.2020.141969

Birk, S., Bonne, W., Borja, A., Brucet, S., Courrat, S., Poikane, S., Solimini, A., van de Bund, W., Zampoukas, N. & Hering, D. (2012). Three hundred ways to assess Europe’s surface waters: An almost complete overview of biological methods to implement the Water Framework Directive. Ecological Indicators 18, 31-41.  http://dx.doi.org/10.1016/j.ecolind.2011.10.009

Boyer F, Mercier C, Bonin A, Le Bras Y, Taberlet P & Coissac E. (2016). OBITOOLS: a UNIX-inspired software package for DNA metabarcoding. Molecular Ecology Resources 16(1):176-182 https://doi.org/10.1111/1755-0998.12428 

Braukmann T.W.A., Ivanova N.V., Prosser S.W.J., Elbrecht V., Steinke D., Ratnasingham S., de Waard J.R., Sones J.E., Zakharov E.V. & Hebert P.D.N. (2019). Metabarcoding a diverse arthropod mock community. Molecular Ecology Resources 19(3):711-727 https://doi.org/10.1111/1755-0998.13008

Buchner D. & Leese F. (2020). BOLDigger – a Python package to identify and organise sequences with the Barcode of Life Data systems. Metabarcoding and Metagenomics 4. DOI: 10.3897/mbmg.4.53535

Callahan B.J., McMurdie P.J. & Holmes S.P. (2017).  Exact sequence variants should replace operational taxonomic units in marker‐gene data analysis. The ISME Journal, 11, 2639–2643. https://doi.org/10.1038/ismej.2017.119

Carew M.E., Coleman R.A. & Hoffmann A.A. (2018). Can non-destructive DNA extraction of bulk invertebrate samples be used for metabarcoding? Peerj 6:e4980 https://doi.org/10.7717/peerj.4980

Cordier, T., Alonso‐Sáez, L., Apothéloz‐Perret‐Gentil, L., Aylagas, E., Bohan, D.A., Bouchez, A., Chariton, A., Creer, S., Frühe, L., Keck, F., Keeley, N., Laroche, O., Leese, F., Pochon, X., Stoeck, T., Pawlowski, J.& Lanzén, A., (2020). Ecosystems monitoring powered by environmental genomics: a review of current strategies with an implementation roadmap. Molecular Ecology.  https://doi.org/10.1111/mec.15472

Edgar, R.C. & Flyvbjerg, H. (2015). Error filtering, pair assembly and error correction for next-generation sequencing reads. Bioinformatics 31, 3476–3482

Elbrecht V. & Leese F. (2015). Can DNA-Based Ecosystem Assessments Quantify Species Abundance? Testing Primer Bias and Biomass—Sequence Relationships with an Innovative Metabarcoding Protocol. PLOS ONE 10(7): e0130324. https://doi.org/10.1371/journal.pone.0130324

Elbrecht V. & Leese F. (2017). Validation and development of COI metabarcoding primers for freshwater macroinvertebrate bioassessment. Frontiers in Environmental Science 10, https://doi.org/10.3389/fenvs.2017.00011

Elbrecht V., Taberlet P., Dejean T., Valentini A., Usseglio-Polatera P., Beisel J.N., Coissac E., Boyer F. & Leese F. (2016). Testing the potential of a ribosomal 16S marker for DNA metabarcoding of insects. Peerj 4 https://doi.org/10.7717/peerj.1966

Elbrecht V., Peinert B. &  Leese F.  (2017). Sorting things out: Assessing effects of unequal specimen biomass on DNA metabarcoding. Ecology and Evolution 7: 6918– 6926. https://doi.org/10.1002/ece3.3192

Elbrecht V., Vamos E., Meissner K., Aroviita J. & F. Leese (2017). Assessing strengths and weaknesses of DNA metabarcoding based macroinvertebrate identification for routine stream monitoring. Methods in Ecology and Evolution. https://doi.org/10.1111/2041-210X.12789

Elbrecht V, Vamos EE, Steinke D & Leese F. (2018). Estimating intraspecific genetic diversity from community DNA metabarcoding data. PeerJ 6:e4644 https://doi.org/10.7717/peerj.4644

Elbrecht V. & Steinke D. (2019). Scaling up DNA metabarcoding for freshwater macrozoobenthos monitoring. Freshwater Biology 64: 380–387. https://doi.org/10.1111/fwb.13220

Elbrecht V., Braukmann T.W.A., Ivanova N.V., Prosser S.W.J., Hajibabaei M., Wright M., Zakharov E.V., Hebert P.D.N. & Steinke D. (2019). Validation of COI metabarcoding primers for terrestrial arthropods. PeerJ 7:e7745. https://doi.org/10.7717/peerj.7745

Erdozain M., Thompson D.G., Porter T.M., Kidd K.A., Kreutzweiser D.P., Sibley P.K., Swystun T., Chartrand D. & Hajibabaei M. (2019). Metabarcoding of storage ethanol vs. conventional morphometric identification in relation to the use of stream macroinvertebrates as ecological indicators in forest management. Ecological Indicators 101:173-184

Frøslev, T. G., Kjøller, R., Bruun, H. H., Ejrnæs, R., Brunbjerg, A. K., Pietroni, C., & Hansen, A. J. (2017). Algorithm for post‐clustering curation of DNA amplicon data yields reliable biodiversity estimates. Nature Communications, 8(1), 1188. https://doi.org/10.1038/s41467-017-01312-x

Gauthier M, Konecny-Dupré L, Nguyen A, Elbrecht V, Datry T, Douady CJ & Lefébure T. (2019). Enhancing DNA metabarcoding performance and applicability with bait capture enrichment and DNA from conservative ethanol. Molecular Ecology Resources 20(1): 79-96. 

Haase, P., Pauls, S.U., Schindehutte, K. & Sundermann, A., (2010). First audit of macroinvertebrate samples from an EU Water Framework Directive monitoring program: human error greatly lowers precision of assessment results. Journal of the North American Benthological Society 29, 1279–1291.

Hajibabaei, M., Spall J.L., Shokralla S. & van Konynenburg S. (2012). Assessing biodiversity of a freshwater benthic macroinvertebrate community through non-destructive environmental barcoding of DNA from preservative ethanol. BMC Ecology 12 https://doi.org/10.1186/1472-6785-12-28 

Hajibabaei, M., Porter, T.M., Robinson, C.V., Baird, D.J., Shokralla, S. & Wright, M.T.G. (2019). Watered-down biodiversity? A comparison of metabarcoding results from DNA extracted from matched water and bulk tissue biomonitoring samples. PLoS One 14. https://doi.org/10.1371/journal.pone.0225409

Heino, J., Culp, J., Erkinaro, J., Goedkoop, W., Lento, J., Ruhland, K. & Smol, J.P. (2020) Abruptly and irreversibly changing Arctic freshwaters urgently require standardized monitoring. Journal of Applied Ecology 57: 1192-1198. https://doi.org/10.1111/1365-2664.13645

Heino, J., Alahuhta, J., Bini, L.M., Cai, Y., Heiskanen, A.-S., Hellsten, S., Kortelainen, P., Kotamäki, N., Tolonen, K.T., Vihervaara, P., Vilmi, A. & Angeler, D.G. (2020b) Lakes in the era of global change: moving beyond single-lake thinking in maintaining biodiversity and ecosystem services. Biological Reviews, in press. https://doi.org/10.1111/brv.12647

Hebert P.D., Cywinska A., Ball S.L. & deWaard J.R. (2003). Biological identifications through DNA barcodes. Proceedings. Biological sciences, 270, 313–321. https://doi.org/10.1098/rspb.2002.2218 

Hering, D., O. Moog, L. Sandin & P. F. M. Verdonschot (2004). Overview and application of the AQEM assessment system. Hydrobiologia 516: 1–20.

Hering D., Borja A., Jones J. I., Pont D., Boets P., Bouchez A., Bruce K., Drakare S., Hänfling B., Kahlert M., Leese F., Meissner K., Mergen P., Reyjol Y., Segurado P., Vogler A., & M. Kelly (2018) Implementation options for DNA-based identification into ecological status assessment under the European Water Framework Directive. Water research. https://doi.org/10.1016/j.watres.2018.03.003

Johnson R.K., Wiederholm T. & D.M. Rosenberg. (1993). Freshwater biomonitoring using individuals organisms, populations, and species assemblages of benthic macroinvertebrates. In: Freshwater Biomonitoring and Benthic Invertebrates. pp. 40-158, Chapman and Hall.

Krehenwinkel H., Wolf M., Lim J.Y., Rominger A.J., Simison W.B. & Gillespie R.G. (2017). Estimating and mitigating amplification bias in qualitative and quantitative arthropod metabarcoding. Science Reports 7, 17668. https://doi.org/10.1038/s41598-017-17333-x

Leray, M., & Knowlton, N. (2017). Random sampling causes the low reproducibility of rare eukaryotic OTUs in Illumina COI metabarcoding. PeerJ 5, e3006. https://doi.org/10.7717/peerj.3006

Mack L., Attila J., Aylagas E., Beermann A., Borja A., Hering D., Kahlert M., Leese F., Lenz R., Lehtiniemi M.,Liess A., Lips U., Mattila O-P., Meissner K., Setälä O., Strehse J.S., Uusitalo L., Willstrand Wranne A. &  Birk, S. (2020) A synthesis of marine monitoring methods with the potential to enhance the status assessment of the Baltic Sea. Frontiers in Marine Science doi: 10.3389/fmars.2020.552047 (accepted)

Majaneva M., Diserud O.H., Eagle S.C.H., Hajibabaei M. & Ekrem T. (2018). Choice of DNA extraction method affects DNA metabarcoding of unsorted invertebrate bulk samples. Metabarcoding and Metagenetics. 2:e26664. https://doi.org/10.3897/mbmg.2.26664

Majaneva M, Hyytiainen K, Varvio SL, Nagai S & Blomster J. (2015). Bioinformatic Amplicon Read Processing Strategies Strongly Affect Eukaryotic Diversity and the Taxonomic Composition of Communities. PLOS ONE 10(6) https://doi.org/10.1371/journal.pone.0130035

Marquina D., Esparza‐Salas R., Roslin T. & Ronquist F. (2019). Establishing arthropod community composition using metabarcoding: Surprising inconsistencies between soil samples and preservative ethanol and homogenate from Malaise trap catches. Molecular Ecology Resources.  19: 1516 – 1530. https://doi.org/10.1111/1755-0998.13071

Martin, M. (2011) Cutadapt removes adapter sequences from high-throughput sequencing reads. EMBnet Journal 17,10–12.

Martins, F.M.S., Porto, M., Feio, M.J., Egeter, B., Bonin, A., Serra, S.R.Q., Taberlet, P. & Beja, P. (2020). Modelling technical and biological biases in macroinvertebrate community assessment from bulk preservative using multiple metabarcoding markers. Molecular Ecology https://doi.org/10.1111/mec.15620

Nichols R.V., Vollmers C., Newsom L.A., Wang Y., Heintzman P.D., Leighton M., Green R.E. & Shapiro B. (2018). Minimizing polymerase biases in metabarcoding. Molecular Ecology Resources 18(5):927- 939 https://doi.org/10.1111/1755-0998.12895

Nielsen M., Gilbert M.T.P., Pape T. & Bohmann K. (2019). A simplified DNA extraction protocol for unsorted bulk arthropod samples that maintains exoskeletal integrity. Environmental DNA 1(2): 144-154. https://doi.org/10.1002/edn3.16

Pereira-da-Conceicoa L., Elbrecht V., Hall A., Briscoe A., Barber-James H. & B. Price (2020). Metabarcoding unsorted kick-samples facilitates macroinvertebrate-based biomonitoring with increased taxonomic resolution, while outperforming environmental DNA bioRxiv 792333; doi: https://doi.org/10.1101/792333

Piñol J., Mir G., Gomez-Polo P. & Agustí N. (2015). Universal and blocking primer mismatches limit the use of high-throughput DNA sequencing for the quantitative metabarcoding of arthropods. Molecular Ecology Resources. 15(4):819-30. https://doi.org/10.1111/1755-0998.12355

Piñol J., Senar M.A. & Symondson W.O.C. (2019). The choice of universal primers and the characteristics of the species mixture determine when DNA metabarcoding can be quantitative. Molecular Ecology. 28: 407– 419. https://doi.org/10.1111/mec.14776

Rognes, T., Flouri, T., Nichols, B., Quince, C., & Mahé, F. (2016). VSEARCH: a versatile open source tool for metagenomics. PeerJ, 4, e2584. https://doi.org/10.7717/peerj.2584

Schenk J., Kleinbölting N. & Traunspurger W. (2020). Comparison of morphological, DNA barcoding, and metabarcoding characterizations of freshwater nematode communities. Ecology and  Evolution 10: 2885–2899. https://doi.org/10.1002/ece3.6104

Skriver, J., Friberg, N. & Kirkegaard, J. (2000), Biological assessment of running waters in Denmark: introduction of the Danish Stream Fauna Index (DSFI), Verhandlungen der Internationalen Vereinigung fur Limnologie 27, 1822–1830

Sweeney B.W., Battle J.M., Jackson J.K. & Dapkey T. (2011). Can DNA barcodes of stream macroinvertebrates improve descriptions of community structure and water quality? Journal of the North American Benthological Society. 30(1), 195–216. https://doi.org/10.1899/10-016.1

Taberlet P., Bonin A., Zinger L. & Coissac E. (2018). Environmental DNA. For biodiversity research and monitoring. Oxford University Press, Oxford, UK.

Zizka V.M.A., Leese F., Peinert B. & Geiger M.F. (2018). DNA metabarcoding from sample fixative as a quick and voucher-preserving biodiversity assessment method. Genome 62(3):122-136. https://doi.org/10.1139/gen-2018-0048

Appendix 1: Stakeholder survey questions

1. Please provide your full name.
2. Do you agree to the storage of your personal data (name, position and email address)? Yes, No
3. Do you want to be informed of the progress made in the SCANDNAnet project regarding the validation of metabarcoding for taxa identification in the context of aquatic invertebrate bioassessment? Yes, No
4. Do you want to be invited to a webinar discussion of the outcome of the project? Yes, No
5. What is your current level of expertise concerning traditional (morphological) identification of aquatic macroinvertebrates None, Low, Intermediate, High, Expert
6. What is your current level of expertise concerning genetic identification of aquatic macroinvertebrates? None, Low, Intermediate, High, Expert
7. How necessary do you view the use of genetic identification in future aquatic bioassessments?  Unnescessary, Low necessity, Intermediate necessity, High necessity
8. Please provide your view on the reliability of genetic taxa identification compared to traditional identification No opinion, Unreliable, Low reliability, Intermediate reliability, High relialibility, Superior relialibility
9. Please provide your view on the cost effectiveness of genetic taxa identification methods when compared to traditional methods No opinion, Not cost effective, Cost neutral, Highly cost effective
10. How likely do you perceive the increased use of genetic taxa identification in the next 5-10 years?  Will not increase, Unlikely to increase, Likely to increase, Will definitely increase
11. Please provide your understanding on how genetic approaches can be used in current assessment methods of ecological status Directly, With few modifications, With several modifications, New methods will be required
12. Do you welcome the development of new assessment methods to assess ecological status from genetic identification data? Yes, to some extent, No (Why?) 
13. In your opinion what are the major factors most limiting a shift towards an increased use of genetic identification methods?  (Legislative requirements, Quality issues (accuracy and reliability), Cost of genetic methods, Lack of standardized methods, Lack of information for decision makers (i.e. policy briefs, reports), Lack of laboratories capable of conducting analysis, Lack of trained personnel, Lack of comparability to traditional methods, Other)

Appendix 2: Taxa lists