Go to content

Appendix N. Critical Review statement

Review of life cycle assessment of single-use and reuse packaging

Tomas Ekvall, adjunct professor in Environmental Systems Analysis at Chalmers University of Technology, and consultant in Tomas Ekvall Research Review & Assessment
20-12-2023

Summary review statement

This study is a mainly attributional comparative environmental assessment of single-use and reuse packaging for take-away food and for e-commerce. An external expert reviewed the study based on the international standards for LCA. Overall, the reviewer finds that the methodological choices in the study are adequate and justified in the report. The findings and conclusions are sound.
The LCA report complies with the majority of the many reporting requirements posed by ISO 14044 on a comparative assertion disclosed to the public. It includes an adequate assessment of key uncertainties and a full-bodied discussion of the results. However, while the study is transparent in theory, the effective transparency is low, making it difficult to understand how the results are produced.
The numerical results should be used with care because, as stated in the report, other results would have been obtained with a different set of input data or calculation methods.

Introduction

The study
This study compares single-use and reuse packaging for take-away food and for shipping clothes bought on the internet. It was conducted by a team of consultants at Ramboll with 2-3 years each of experience from life cycle assessment (LCA). The work was checked internally by Katja Gradin and coordinated by Janus Kirkeby, both with extensive experience from LCA research.
The study is intended to provide decision support to authorities as well as companies. The funding for the project was provided by the Nordic Working Group for Circular Economy (NCE) under the Nordic Council of Ministers. A steering group with representatives from the national environmental authorities in Denmark, Finland, Norway, and Sweden oversaw the project.
The report states in Chapter 2 that the study is made in accordance with the international standards for LCA: ISO 14040 and ISO 14044. ISO 14040 specifies the structure of the analysis. ISO 14044 gives some guidance on the methods used in the study, but primarily includes requirements on the reporting and review of the LCA. These requirements are more comprehensive in studies, such as this, when the results are used as basis for assertions disclosed to the public about what product or system is environmentally superior.
The review
For LCAs that makes comparative assertions to the public, ISO 14044 requires that a review be conducted by a panel of interested parties and experts. This review deviates from ISO 14044 here, because NCE decided a review by an external expert was sufficient. The reviewer appointed by NCE is Tomas Ekvall, adjunct professor in Environmental Systems Analysis at Chalmers University of Technology, and consultant in Tomas Ekvall Research Review & Assessment. Professor Ekvall has over 30 years of experience from life cycle assessment and is a frequently cited scientific expert on modelling in life cycle inventory analysis (LCI).
The scope of the review is consistent with ISO 14044. The main aim of the review was to ensure that the methods used in the study are scientifically and technically valid, that the input data are appropriate, that the interpretations reflect the goal and limitations of the study, and that the report is transparent, clear, and consistent. In addition, the reviewer investigated to what extent the methods and report are consistent with the international LCA standard ISO 14044.
The reviewer was involved all through the project and gratefully acknowledges a good and constructive collaboration with the authors of the study. The project team at Ramboll hosted five meetings where they presented results and got feedback from the reviewer and the Nordic steering group.  The review also included an early scoping report, with the literature review and the goal and scope definition of the LCA, and a preliminary version of the complete report. This meant several of the perspectives and insights of the reviewer could be integrated in the study, and that most of the initial review comments are addressed in the final version of the report.
Comments
The remaining review comments are listed below. Note that these comments are valid for the manuscript version on which the review report is based. Some or all of them might be amended before the LCA report is published.

General

The report includes a thorough background study to collect information about the systems and give a basis for the choice of cases studied in the LCA. The LCA, in turn, is ambitious in terms of environmental impacts (13 impact categories) and sensitivity analyses. This means it generates many numerical results.
The study is mostly based on generic input data rather than site-specific data. This is justified because the aim is not to investigate a very specific case but to increase the knowledge on the environmental performance of single-use and reusable packaging overall in the Nordic countries.
The report adequately describes the systems investigated and the methodological choices made in the LCA. However, it does not present sufficient input data and calculations to make it clear how the results from the life cycle impact assessment (LCIA) and sensitivity analyses are produced (see also Transparency below).
Numerical results from an LCA depend heavily on subjective methodological choices and uncertain input data. The sensitivity analyses in this study illustrate only part of the range of possible results. For this reason, readers of the report should focus on the qualitative findings rather than the numerical results.
The executive summary is comprehensive, clear, and reflects the content of the study.

Transparency

The report includes sources with references to the input data and the characterization factors. It also presents the methods used in the calculations and outlines the calculation procedure. This makes the study transparent in theory. It also meets most of the many requirements posed by ISO 14044 on a comparative assertion disclosed to the public. An important exception is that input data on important processes, such as material production, are not included in the report. These data are from the Ecoinvent database, which prohibits publication of the data.
Important partial results, such as the LCI results, are missing in the report. Characterization factors and end-pointy conversion factors are also absent. The contribution and sensitivity analyses make it possible for the reader to understand what is important for the results, but the effective transparency of the calculations is low. This makes it difficult for readers to assess the study and its results.

Clarity of the report

The text in the report is in most parts clear but would benefit from editing and a language check.
The results are mainly presented in bar diagrams. Bar diagrams are good in that they show the relationships between different results without indicating that the numerical results are precise. However, the dominance analysis (Figures 11-12 and 14-16) is presented with shades rather than clear colors. However, in the presentation of the sensitivity analysis, the colored square for each analysis is very small. This makes it hard for the reader to grasp what results relate to what sensitivity analysis.

LCI modelling approaches

Many LCA experts distinguish between attributional and consequential LCA. Attributional LCA can be said to assign part of the total human impact on the environment to the product investigated. A consequential LCA, on the other hand, estimates how the production and use of the product affect the total human impact on the environment; this can include increases as well as reductions in emissions and resource use.
This study is presented as a an attributional LCA; however, waste management and recycling are modelled with the Circular Footprint Formula (CFF) from the Environmental Footprint methodology, the EU framework for LCA. The CFF includes aspects that are often associated with consequential LCA. Credits are given to the product investigated for the energy and part of the primary-material production that are avoided when the product is incinerated or recycled after use. The use of recycled material in the product investigated is also burdened with part of the primary-material production that the recycled material substitutes. Hence, the study is not attributional in a strict sense.
When an LCA includes a mix of attributional and consequential methods, the results do not respond to a clear question. This can be considered a weakness in the study. However, the methodological choice was approved by the steering group, which considered the method adequate and relevant.
A purely attributional assessment can probably be obtained by combining variations that are separately tested in the sensitivity analysis: setting both CFF Factors A and B to 1. However, in this particular attributional approach no emissions from waste incineration are assigned to the packaging. The steering group would be likely to object to such an approach.

Functions and functional units

The single-use and reusable packaging solutions are compared based on their primary function: containing takeaway food and transporting clothes bought on the internet to their customer. The packaging might have secondary uses, such as storing food leftovers at home and containing clothes returned to the seller. These secondary functions are not considered in the study.
The packaging solutions can also vary in additional functionality such as printing/advertising and safekeeping of the goods. Any such differences in functionality are not accounted for in this study.
This means the packaging solutions cannot be expected to have identical functionality. The authors are clear about this limitation in the study. This should also be remembered when using the results as basis for policy decisions, as stated in the discussion of the results.

Modelling of recycling

Paper and plastic packaging materials are assumed to be 100% primary materials. This is a conservative assumption, since recycled material might be used at least for shipping clothes. Hence, the assumption might contribute to net results overestimating environmental impacts where primary production is worse than the recycling process.

Modelling of electricity

The electricity supply is modelled using average data for the country or region where the electricity is used. This is a common approach in attributional LCA. Electricity used in the Nordic countries is modelled using Nordic average data rather than national average data. This makes the electricity use reasonably significant in the LCA results. However, the use of average data does not reflect the foreseeable consequences of using electricity, because the electricity system is affected on the margin. This is one of the reasons why attributional LCA does not reflect how the choice of packaging solutions affects the affect the human impacts on the environment.

Life cycle impact assessment methods

The choice of impact categories is ambitious and well justified. The choice of calculation methods in the LCIA is not clearly justified but reasonable.

Calculations and results

Most of the calculations could not be checked, since important input data, LCI results, characterization factors, etc. are not included in the report. The climate impact associated to production and waste incineration of plastics could be compared to other sources to check if they are reasonable: they are reasonable. Other results depend heavily on case-specific factors or on input data with great variability between sites and over time. This makes it difficult to assess them without checking the calculations.

Interpretation

The study includes a dominance analysis and a good number of sensitivity analyses. Besides the significance of CFF Factor B, the sensitivity analyses illustrate how the results are affected by a small change (10%) in the weight of the packaging, but also by extreme assumptions on, for example, consumer behavior (the mode of transportation and the prewashing of used takeaway packaging), and on the market for recycled material – represented by Factor A in the CFF.
The results from the baseline calculations and sensitivity analysis are discussed under the statement or assumption that a 30% difference is significant. This is in line with the general language, where a “significant difference” is understood to mean “large difference”. However, in an LCA, “significant difference” should be used to denote differences that are larger than the uncertainty. The uncertainty varies greatly between, for example, impact categories, as indicated in Section 4.2.4. Hence, a 30% difference might be significant for the climate-change impact, but it will often not be significant for acidification or eutrophication, and rarely for the impact categories ionizing radiation, land use, water use, etc.
The discussion of the results is clear and accounts for the limitations of the study.

Conclusions and recommendations in the report

The report includes several conclusions and recommendations that are both valid and relevant to the broad intended audience: the public, private companies, and authorities.