The study presented in this report has been performed on behalf of the Swedish Energy Agency within the Nordic cooperation Nordsyn, sponsored by the Nordic Council of Ministers. Nordsyn is a cooperation of Nordic agencies responsible for policy and market surveillance of ecodesign and energy labelling. The study was performed by Kasper Mogensen at Big2Great ApS. Any opinions set out in the study are those of Big2Great and do not necessarily reflect the opinions of the Nordsyn members.
This study has been performed in parallel with the study “Nordcrawl3 – A web-based tool to calculate energy savings of ecodesign and energy labelling policies in the Nordic countries”, TemaNord 2021:523. The present study focuses on the impacts from market surveillance activities to ensure compliance with the ecodesign and energy labelling requirements so that the energy savings estimated with Norcrawl3 in the parallel study are realized.
Lovisa Blomqvist, Swedish Energy Agency
This project estimates the savings from market surveillance of ecodesign and energy labelling in the Nordic countries to be about 20 million euro during 2011–2019 (2 million euro per year on average). It is a very conservative estimation, and the actual savings can be up to 147 million euro during 2011–2019. Nevertheless, the study proves that market surveillance is cost-effective on a societal level. It also proves the benefits and improvement potential of cooperation between countries. The benefit calculated in this project is the energy costs saved by consumers due to market surveillance authorities finding and correcting products using too much energy – compared with the cost of the performed market surveillance. The project is an update of the Nordsyn report "The Nordic Ecodesign Effect Project", TemaNord 2015:563, with some adjustments.
The purpose of this project is to make an estimate of the effect of market surveillance activities. The project is an update of the Nordsyn report "The Nordic Ecodesign Effect Project"The Nordic Ecodesign Effect Project - Estimating benefits of Nordic market surveillance of ecodesign and energy labelling - Troels Fjordbak Larsen 2015, https://www.norden.org/en/publication/nordic-ecodesign-effect-project from 2015, TemaNord 2015:563. This project makes one estimate of the benefit from market surveillance. More specifically, it focuses on the energy saved for the consumers due to market surveillance authorities testing and finding products that are non-compliant with the ecodesign and energy labelling regulation. One of the differences between this project and the old project is that this project only focuses on products tested in a laboratory and not document control. The old project had data from 2011 to 2013, where this project has data from 2011 to 2019 and includes more product types regulated during that time.
This project only focuses on products tested in a laboratory, so savings from document control, control of advertising, internet stores etc. is excluded. Also, one of the significant benefits of market surveillance is that when markets are surveilled, and products are controlled, manufacturers and importers are more motivated to comply with the regulations. This effect is, of course, hard to measure.
The calculation approach is as follows: the difference in the average annual consumption between the non-compliant product the annual consumption of that product if it was complying with the regulation. In some cases, we use a standard purchase as an alternative. The difference is multiplied by the non-compliance rate for the particular product group and the target year's annual sales volume. The result is the annual energy savings loss per product group. Multiplying that by product-specific lifespan, the total lifespan loss is calculated. Summing up overall product groups and all Nordic countries, a Nordic estimate for lost savings is calculated.
|E||Estimated lost energy savings|
|CNCij||Average annual consumption of non-compliant appliances, product group i, country j|
|CCij||Average annual consumption of compliant appliances (or standard purchase), product group i, country j|
|Rij||Average non-compliance rate, product group i, country j|
|Sij||Sales in target year, product group i, country j|
|Li||Lifespan, product group i|
|i||Product groups regulated|
This project's primary data source is test reports and market surveillance overview reports for the products tested in a laboratory. We collected all available test reports and market surveillance overview reports from 2012–2019 and then created an overview sheet with all the tested products and whether they were compliant. The compliance was discussed with the market surveillance authorities if there were doubts.
Another data source plus the invoices from the test laboratories with the price of performing the test.
Below is a table with all the available test samples:
|Actual available Lab samples||DK||FI||IS||NO||SE|
Table 1: Actual available laboratory samples
Based on a subset of data, sampling is used to say something about a whole population. E.g., a sample of washing machines is examined to say something about all washing machines on the market. Random sampling is when the sample is selected randomly, and the probability of picking any given sample can be calculated. When applying a non-random, or hand-picked sample, the probability approach is no longer valid (since the sample is pre-determined), and the representativity of the sample for the whole population is destroyed. In many situations, it is still chosen to perform non-random/ judgmental/hand-picked/targeted sampling. This is often the case for market surveillance, where products suspected to be non-compliant with the regulations are selected. This is because a general picture of the market situation in terms of a non-compliance rate is not the primary goal, but instead a specific wish and obligation to monitor and eventually get rid of the illegal products through contact to the producers of the non-compliant products that occur. Still, can this hand-picked sample say something about the whole market situation with regards to compliance rates? The simple answer is no. But in practice, this is the knowledge about the market that is at hand. Assumptions must then be introduced in order to extract any information about the market from the targeted sampling. Also, in some cases, the hand-picked samples are supplemented by a small random sample from the remainder of the market. How can this be included? In the following paragraphs, the cases are described and suggestions to calculation methods are specified.
The sampling can be divided into three different categories:
In the description of the sampling scenarios the following letters is used:
|P||Non-compliance rate for market|
|p||Non-compliance rate for sample|
|N||Market size (number of models)|
The illustrations below show the three different scenarios.
Figure 1: Sampling strategy
Below is assumed a total population (market) of N elements (i.e., different models on the market that all could be relevant to test), a sample size of s (s1 and s2 for the mixed situation) p is the number of elements in the sample found not to be compliant (p1 and p2 in the mixed situation). P is the rate of non-compliance for the whole market, i.e., the targeted estimator we want to be able to calculate. In each sample, the elements are examined with regard to compliance with the regulation. The reason for non-compliance can be different things. Still, to keep it simple, we are only looking at compliant or not in energy use/efficiency (i.e., only how much the energy use/energy efficiency differ from the ecodesign limit or the given energy label, not considering energy loss due to much standby-usage, failing to go into standby/off-mode quickly enough etc.). Other kinds of non-compliance like documentation lacks, high noise levels etc., are not included in this calculation.
Comments to this assumption: if pIn this case, the statistical theory can directly provide a predictor, since we have a sample that follows the Binomial distribution (compliant or not). Hence, the estimate for a non-compliance rate for the whole market N is:
P = p/s, p = number of non-compliant elements in the sample size of s, and the total number of non-compliant elements is N*P.
In this situation, the sample cannot be said to follow a probability distribution. We have to introduce an assumption: the hand-picking is practical and based on specific knowledge, leading to the assumption that all picked elements are non-compliant as default. The rate P for the whole market N is then: P = p/N, p is the number of elements in the sample found not to be compliant. Comments to this assumption: if p < s (i.e., not all hand-picked elements were non-compliant), this could mean that the hand-picking is not entirely successful, i.e., some non-compliant elements have escaped the surveillance and are still to be discovered, OR that there are only p non-compliant elements among the N. The latter is the situation expressed in the formula. If p=s (i.e., all in the sample are non-compliant), the first situation that some could have escaped is emphasized, since all are non-compliant in the sample, and the sample size then could be limiting the picture of how many non-compliant elements there really are. Therefore, if assuming effective hand-picking, getting close to all elements being NC in the sample this somehow weakens the predictor formula's reliability as it is less and less certain that all NC elements are captured. In this situation, supplementary sampling should be conducted (which is often the case in practice).
The total number of non-compliant elements is thus P*N = p.
In the mixed situation, the calculation/estimation formula becomes a bit more complicated.
The market surveillance authorities normally use risk-based sampling. The idea is that the authorities want to have the biggest impact with a limited number of samples, by selecting popular models that are likely to be non-compliant. The risk-based sampling takes a couple of criteria into accountMarket Surveillance Regulation 2019/1020 article 11:
(a) possible hazards and non-compliance associated with the products and, where available, their occurrence on the market;
(b) activities and operations under the control of the economic operator;
(c) the economic operator's past record of non-compliance;
(d) if relevant, the risk profiling performed by the authorities designated under Article 25(1);
(e) consumer complaints and other information received from other authorities, economic operators, media and other sources that might indicate non-compliance.
The spill-over effect is that an action taken in one country affects the other country. in this example, market surveillance activities in one country affect the market in another country. This effect is considered mainly positive in the Nordic countries where products identified as non-compliant in one country are correct in the other countries. the argument is that many of the retailers and manufacturers are the same in the Nordic countries. There have also been examples of negative spill-over effects where banned products in one country are moved and sold in another country.
In the old project, it was assumed that the Nordic countries were one market, and therefore, products taking off the market in one country will be taken off the market in all countries. We know that this is not the case yet, but there is still a spill-over effect. Therefore, we assume that a country that does not test a specific product group will have an effect on the other country's average non-compliance findings. The Nordsyn group set effect factor. An example: an effect factor of 10% means that a country that does not test will have 10% of the effect of a country that performs tests. If it were considered the same market, the effect factor would be 100%.
The non-compliance rate for energy related compliance and expected number of appliances for a specific product group can be estimated using the formulas mentioned above. To estimate the total energy effects of non-compliant appliances, the energy deficit between non-compliant and alternative compliant appliances must be estimated.
The general assumption is the consumer believes the information in the product information or energy label to be accurate. It means that the consumer would have bought a similar product with similar energy consumptions if the non-compliant product was removed from the market.
Therefore, the energy consequence is the difference between the measured consumption and the declared.
In the real world, the consumer will not always have a choice of an exact similar model, but we assume that the number of cases where the consumer will choose a product with a better performance evens out the cases where the consumer picks a less efficient product.
Below are the calculated average non-compliance energy consequences:
|Non-Compliance (E) kWh/y||Avg|
Table 2: Average non-compliance energy consequences
The effect of a non-compliance purchase has an impact in the year of the purchase but as long as the appliance is in use. Therefore, in the formula for the non-compliance effects, each appliance type's lifespan is included to capture the effect for all of the years the specific appliance uses energy.
|Lifespans per product group in years||Avg|
|* average assumption from NordCrawl bottom-up|
Table 3: Lifespans per product group in years
The lifespan used in this report is based on the average assumed used in the NordCrawl effect moduleNordcrawl3 - A web-based tool to calculate energy savings of ecodesign and energy labelling policies in the Nordic countries – Kasper Mogensen 2021 for Denmark and Sweden or from the assumptions used in the Impact Accounting ReportEcodesign Impact Accounting - OVERVIEW REPORT 2018 - Prepared by VHK for the European Commission December 2018 (rev. Jan. 2019) - https://ec.europa.eu/energy/sites/ener/files/documents/eia_overview_report_2017_-_v20171222.pdf
To convert the calculated non-compliance effects in terms of lost energy savings into economic effects, some final assumptions about this are made in this chapter. The cost of purchasing a non-compliant appliance will be the energy price Pend-user multiplied by the identified energy penalty for the end-user. I.e.:
Cend-user = EP * Pend-user
Where the price may vary from sector to sector and in time (depending on different tax levels), an annual average will be used. For society, another price can be calculated. In fact, the marginal extra energy use may cause the need for enlargement of the power supply, infrastructure, etc. These costs are complicated to estimate. A more straightforward approach is to calculate the more marginal extra costs of the primary fuel needed to produce the energy and the costs of the extra CO2 emissions it has led to, depending on the production efficiency. I.e.
Cmarginal = EP * (k*Pfuel + e*PCO2)
Where k is the conversion factor from secondary to primary energy (normally set to 2,1 or 2,5 for electricity), Pfuel is the fuel price, e is the average CO2 emission factor in kg per produced energy, and PCO2 is the price for emitting 1 kg of CO2. All factors can be settled per country. This calculation is, however, not done within this project. On the other hand, if it is assumed that the market surveillance efforts – in time – lead to full compliance, society's costs are only the costs of the market surveillance. I.e.
Csociety = Ʃ Csurveillance i
And the estimate for the achieved benefits would be exactly the avoided end-user costs. Summing up all end-user costs and surveillance costs can give us an indicative benefit/cost ratio of the market surveillance. Only indicative, since the real effect/benefit of market surveillance should be measured as the difference between having surveillance and not having surveillance. But since the latter situation will not be possible (except for other EU-countries), the best estimate is described using previous symbols. This calculation method is used within this project.
|CNCij||Average annual consumption of non-compliant appliances, product group i, country j.|
|CCij||Average annual consumption of standard purchase (compliant appliances), product group i, country j.|
|Rij||Average non-compliance rate, product group i, country j.|
|Sij||Sales in target year, product group i, country j.|
|Li||Lifespan, product group i.|
|i||1..circa 40 product groups regulated.|
|j||Nordic countries (Sweden, Denmark, Norway, Finland, Iceland).|
|Pend-user||Energy price for the end-user.|
|Ck||Total costs of each surveillance effort.|
The sales numbers per year per country comes for three different sources:
The NordCrawl top-down model: EU sales numbers are scaled down to country by using the same scale as for energy savings. The EU sales figures are from the Impact Accounting ReportEcodesign Impact Accounting - OVERVIEW REPORT 2018 - Prepared by VHK for the European Commission December 2018 (rev. Jan. 2019) - https://ec.europa.eu/energy/sites/ener/files/documents/eia_overview_report_2017_-_v20171222.pdf See more in the policy report.Nordcrawl3 - A web-based tool to calculate energy savings of ecodesign and energy labelling policies in the Nordic countries – Kasper Mogensen 2021
The NordCrawl bottom-up model: Sales figures are used. The sales figures mainly come from APPLiA. Some of the figures are Danish scaled to the other countries. See more in the policy report.Nordcrawl3 - A web-based tool to calculate energy savings of ecodesign and energy labelling policies in the Nordic countries – Kasper Mogensen 2021
The Danish stock model, Elmodelbolig, has sales figures for a long-range of large and small appliances in households. Some sales figures are from sales statistics like APPLiA, while others are calculated from product ownership rate (obtained from surveys).
The scales are calculated as follows:
|Dwellings 2016 (in '000)||Scale|
Tabel 4: Sales number scales
Below are the sales numbers:
|Sales per year, est.||DK||FI||IS||NO||SE||Source|
Table 5: Sales per year (estimated)
The electricity prices used are from 2018:Statistista: e.g. Norway https://www.statista.com/statistics/596381/electricity-household-price-norway/
Table 6: Electricity prices per country
The cost for taking a sample consist of three parts:
The laboratory cost is the price paid to a certified laboratory to perform the product's test according to the test standard. The Nordic countries use the same laboratory, so it is assumed that the price all the same independent dentally off which countries order the test. If a test was performed internally, we assume that the price is the same as an external laboratory.
The administrative cost is the cost of having an employee spend time selecting samples, ordering the product, sending it to the laboratory, getting the results, interpreting the results, and acting upon the results like legal activities to correct non-compliance. For Sweden, this cost is calculated based on time taking and an internal cost per hour. In Denmark, the cost is based on invoices sent based on time spent. For Finland, we assume a cost that is between the Danish and Swedish cost.
The product purchase cost is the price of purchasing the product. In Denmark, the market surveillance authorities get the product for free, so it's only included in Sweden and Finland's total cost.
Below are the total costs per test in million EURO:
|Costs per sample (million EURO)||DK||FI||SE|
Table 7: total costs per test in million. EURO
In the following results, these assumptions are used:
The assumptions are considered to be conservative. The results with higher assumption will be discussed in the end of this chapter.
Below are the non-compliance rates for the three countries that have performed laboratory tests. The "R" column shows the non-compliance rate calculated with the random method, the "HP" is hand-picked and the "MX" is the mixed method that is a weights average of the two. The last column shows the average mixed non-compliance rate that is used for calculating the spill-over effect. In the rest of the results the average MX non-compliance rate will be used.
|Non-Compliance (E) pct Alternatives||DK||DK||DK||FI||FI||FI||SE||SE||SE||All|
Table 8: Non-compliance rates
Below are the effects of market surveillance:
|Effects (GWh) full lifespan||DK||FI||IS||NO||SE||Sum|
Table 9: Effects in GWh full lifespan
And the effects in Euro:
|Effects (Million EURO) full lifespan||DK||FI||IS||NO||SE||Sum|
Table 10: Effects (Million EURO) full lifespan
Below are the total costs for all tests. The costs are calculated by multiplying the number of tests by the price per test.
|Total costs (Million EURO)||DK||FI||SE||Sum|
Table 11: Total costs (Million EURO)
Below are the total benefits for all countries. The benefits are the total effects minus the cost. The benefit could also be called the profit of doing market surveillance. Some of the benefits are negative, indicating that the savings from testing that product type were too low to cover the cost of testing or that the product group was compliant. In the table below a spill-over of 10% and a sampling of 10% random 90% hand-picked was used.
|Total benefits (Million EURO) with spill-over||DK||FI||IS||NO||SE||Sum|
Table 12: Total benefits (Million EURO) with spill-over
To show the sensitivity of the assumptions made, below are different results obtained by changing the spill-over effect and the sampling method. By changing the spill-over effect from 10% to 100%, the Nordic countries' total benefit is more than double. It could be viewed as the potential if the Nordic countries share test results and enforce them in all countries.
The second part of the table shows how changing the weights between the random sampling, and hand-picked sampling affects the overall benefit. The table shows that the results are very sensitive to the weight of the sampling.
|Total benefits (Million EURO) various assumptions|
|Spill-over factor||Random||Hand picked||DK||FI||IS||NO||SE||SUM|
Table 13: Total benefits (Million EURO) various assumptions
This project estimates the benefits of market surveillance of ecodesign and energy labelling in the Nordic countries to be about 20 million Euro during 2011–2019 (this translates to about 2 million euro per year on average). These savings can also be seen as coming from energy savings that would not had been realized without market surveillance. This is based on conservative assumptions and the sensitivity analysis shows that the actual savings can be up to 147 million Euro during 2011–2019 depending on the assumptions considered. In any case, the study proves that market surveillance is cost effective on a societal level. It also proves the benefits and the improvement potential of cooperation.
This study has been performed in parallel with the study “Nordcrawl3 – A web-based tool to calculate energy savings of ecodesign and energy labelling policies in the Nordic countries”. The present study focuses on the impacts from market surveillance activities to ensure compliance with the ecodesign and energy labelling requirements so that the energy savings estimated with Nordcrawl3 in the parallel study are realized. The study using Nordcrawl3 includes ex-ante and ex-post estimations of energy savings from ecodesign and energy labelling policies, which assumes full market compliance. The present market surveillance study shows the actual measured savings from conducted laboratory tests for market surveillance (in specific years and countries). The results of the market surveillance study can be seen as having ensured to realize a portion of the savings estimated but that could have been lost. Estimations show that lost energy savings from non-compliance can be 10–20%European Commission review report 2012, https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2012:0765:FIN:EN:PDF and it is therefore important to show market actors that market surveillance is performed. The results show that market surveillance activities are cost-effective. Furthermore, the results show that both the impact and the cost-effectiveness can be further developed through increased cooperation between countries.
Methodology and assumptions. There are several delimitations and assumptions worth commenting on since they affect the outcome significantly. Firstly, this project solely looked at the energy faults found by testing products. So, all other kind of non-compliance was not included – even though it could impact the energy use if the consumer for example get the wrong information about a product and that effects the choice of product. Secondly, as only non-compliance found by actual tests was included, all non-compliance found by other market surveillance actions – like document control, control of advertising, control of internet or physical stores – was not included.
Further, to do this kind of calculation many assumptions are needed, and two things that effect the results significantly is how the sampling and the spill-over effects are calculated. We have in this calculation chosen to be conservative in the lack of evidence – which means that the effects may be considerably underestimated.
Sampling. One example is the treatment of the hand-picked samples. It is done so that the number of NC's are compared with the total market size since the whole market size is the sample size when hand-picking. But it introduces a huge underestimation (actually, the minimum NC rates are estimated this way) since not all NC's may be tested due to practical limits, and therefore the actual NC rate is higher. Additional random sampling should be added to avoid this underestimation. Or even better, if the sampling methods were better documented and investigated, more accurate results could be obtained. Also, adding information on sales for each model would give a better idea of the actual saving from finding each non-compliant model. Until then, the results must be considered conservative. Random samples are, of course, favorable in these kinds of calculations, but not practical. It is actually demanded from EU-commission that market surveillance shall be risk-based to get as large effect as possible by a limited budget.
Spill-over effect. The spill-over effect is how much a non-compliance found and corrected in one country is also affecting other markets – here the Nordic markets. In this report we have used a rather conservative assumption of 10% spill-over effect due to uncertainty. To keep in mind the samples used here are from the period 2011–2019, and Norway and Iceland had not fully incorporated the ecodesign and energy labelling regulations from start – which might have led to even a negative spill-over effect. If a higher spill-over effect is used in these calculations, much higher savings are obtained. Also, as it is from 16 July 2021REGULATION (EU) 2019/1020 OF THE EUROPEAN PARLIAMENT it is obligatory to put all non-compliant products in a specific format in the ICSMS-database, it should be easier in the future to act on each other’s results. So, the potential is 100% spill-over which, if used in this case, would mean saving the double, 40 million Euro instead of 20 million Euro with 10% spill-over. It affects the overall benefit because it's way more cost-efficient to only test in one country and enforce in the other. This shows great potential for sharing more test results than is shared today, which can be facilitated using the ICSMS database.
Lifespan assumptions. It is assumed that lifespans for each product group are equal to estimations used in the NordCrawl3Nordcrawl3 - A web-based tool to calculate energy savings of ecodesign and energy labelling policies in the Nordic countries – Kasper Mogensen 2021 top-down and bottom-up models (cf results in table 1). It is assumed that Denmark's sales figures can be transferred to the other Nordic countries using a scale. If more accurate country specific figures numbers are used, the estimations can be improved.
Within the EU-project EEpliant2Eepliant2 Final Report, Prosafe, 2020 an estimation of savings from market surveillance was made. The calculations are not directly comparable due to differences in how sampling was incorporated, they also incorporated other kind of non-compliance than to high energy consumption, and they also incorporated results from document controls.
Kasper Schäfer Mogensen
ISBN 978-92-893-7029-5 (PDF)
ISBN 978-92-893-7030-1 (ONLINE)
© Nordic Council of Ministers 2021
Cover photo: Cottonbro/ Pexels
This publication was funded by the Nordic Council of Ministers. However, the content does not necessarily reflect the Nordic Council of Ministers’ views, opinions, attitudes or recommendations.
This work is made available under the Creative Commons Attribution 4.0 International license (CC BY 4.0) https://creativecommons.org/licenses/by/4.0.
Translations: If you translate this work, please include the following disclaimer: This translation was not produced by the Nordic Council of Ministers and should not be construed as official. The Nordic Council of Ministers cannot be held responsible for the translation or any errors in it.
Adaptations: If you adapt this work, please include the following disclaimer along with the attribution: This is an adaptation of an original work by the Nordic Council of Ministers. Responsibility for the views and opinions expressed in the adaptation rests solely with its author(s). The views and opinions in this adaptation have not been approved by the Nordic Council of Ministers.
Third-party content: The Nordic Council of Ministers does not necessarily own every single part of this work. The Nordic Council of Ministers cannot, therefore, guarantee that the reuse of third-party content does not infringe the copyright of the third party. If you wish to reuse any third-party content, you bear the risks associated with any such rights violations. You are responsible for determining whether there is a need to obtain permission for the use of third-party content, and if so, for obtaining the relevant permission from the copyright holder. Examples of third-party content may include, but are not limited to, tables, figures or images.
Photo rights (further permission required for reuse):
Any queries regarding rights and licences should be addressed to:
Nordic Council of Ministers/Publication Unit
Ved Stranden 18
Nordic co-operation is one of the world’s most extensive forms of regional collaboration, involving Denmark, Finland, Iceland, Norway, Sweden, and the Faroe Islands, Greenland and Åland.
Nordic co-operation has firm traditions in politics, economics and culture and plays an important role in European and international forums. The Nordic community strives for a strong Nordic Region in a strong Europe.
Nordic co-operation promotes regional interests and values in a global world. The values shared by the Nordic countries help make the region one of the most innovative and competitive in the world.
The Nordic Council of Ministers
Ved Stranden 18
Read more Nordic publications on www.norden.org/publications