Go to content
Photos: iStock and Dreamstime

Refine existing and optionally develop new indicators, conclusion and future work

The existing indicators

Asking users (e.g. personnel at health institutions and citizens / patients) is a well-established means of assessing the value of e-health / digital health systems. The National Usability-Focused HIS-Scale (NUHISS) questionnaire was developed in Finland
Hyppönen H, Kaipio J, Heponiemi T, Lääveri T, Aalto AM, Vänskä J, et al. Developing the National Usability-Focused Health Information System Scale for Physicians: Validation Study. J Med Internet Res. 2019;21(5):e12875.
. Hannele Hyppönen, the first author in the publication describing NuHISS is the founding mother of NeRN. NuHISS has been translated to many languages and used to assess the outcome of EHR procurement and implementation processes as well as to assess the value of EHRs from the perspective of health professional societies. The Finnish representatives in NeRN rank among the most prominent experts in using this questionnaire.
There is no reason to move away from using NuHISS in the forthcoming monitoring activities. We must however consider the possibility to develop indicators that can be collected by tapping into the information systems themselves. Also, indicators suggested in the national policies must be taken into account. Finally, as the rest of Europe aspires to take down barriers that prevent citizens from obtaining healthcare services from any European country and, at the same time, enable biomedical research and innovation by improving access to health and healthcare data, we must also consider the indicators that are suggested as part of EHDS initiatives.   

Indicators suggested in the national policy documents

The Strategy for digitisation and information management from Finland describes indicators for the following areas: h) Evaluating and updating the Kanta information system services, i) Strengthening the benefits of Kanta data, j) National and regional steering and management of the service system by developing secondary use of health and social services data, k) Use of information in research, development and innovation activities, l) Developing a management and steering model for digitalisation and information management, j) Digital security.
Area
Indicator constructs and method of collection
Digital health and social services centre and customer and service counselling
Percentage of digital service use, usage of the digital health and social services centre, customer feedback, percentage of customers for whom a service needs assessment has been made
From digitalisation of information to digital operating models
Job satisfaction, customer satisfaction, number of automated processes
Giving customers access to their own data
Use of wellbeing data in service processes, customer satisfaction
Customer participation in services
Customer satisfaction, number of digitally performed tasks, application usage and usability
Digital wellbeing operating models and service task
Healthcare measures (Avohilmo), number of people doing physical activity, etc. (Healthy Finland)
Ensuring data quality
Data quality monitoring indicators
Interoperability policies and selection of data models
Number of solutions using international/eU standards and interoperable data structures
Evaluating and updating the Kanta information system services
Purchasing the light user interface, number of users, satisfaction of users, percentage of information transferred on the basis of data
Strengthening the benefits of Kanta data
Kanta data transfer volumes, customer satisfaction, analysed register data, etc.
National and regional steering and management of the service system by developing secondary use of health and social services data
Realisation of information base development needs, timeliness and level of automation in information production, customer satisfaction
Use of information in research, development and innovation activities
Development of research and innovation activities, number of data permits, customer satisfaction, process turn-around time (Findata)
Developing a management and steering model for digitalisation and information management
Model and new steering structures in use, utilisation of the results of the evaluation of the benefits generated by development projects
Digital security
No separate indicators.

e-health relevant indicators in supranational policies

In the EHDS legislative framework
The EHDS regulative Section 1.4.4. Indicators of performance. Available at eur-lex.europa.eu.
, we find the following suggestions for indicators:
Areas (abbreviated)
Indicators of performance
Citizens control over and use of their own data (Objective nr 1).
Number of healthcare providers of different types connected to MyHealth@EU calculated a) in absolute terms, b) as share of all healthcare providers and c) as share of natural persons that can use the services provided in MyHealth@EU.
 
Volume of personal electronic health data of different categories shared across borders through MyHealth@EU.
 
Percentage of natural persons having access to their electronic health records.
 
Level of natural persons satisfaction with MyHealth@EU services.
Suppliers and manufacturers of EHR systems will relate to one set of requirements on interoperability and security (Objective nr 2).
Number of certified EHR systems and labelled wellness applications enrolled in the EU database.
 
Number of non-compliance cases with the mandatory requirements.
Natural persons should benefit from a wealth of innovative health products and services that are provided and developed based on health data primary and secondary use, while preserving trust and security (Objective nr 3).
Number of datasets published in the European data catalogue.
The primary and secondary use of health data by researchers, innovators, policymakers and regulators (Objective nr 3).
Number of data access requests, disaggregated in national and multi-country requests, processed, accepted or rejected by health data access bodies.

Conclusion

This chapter ends the report from NeRN for the current mandate period. If we are allowed to continue this work, we will proceed with refining existing and suggesting a bundle of new indicators for testing at the national level in the Nordic countries. By then, the Nordic countries should be equipped with a monitoring tool that both reflects the achievements of the past as well as taking into account the new, emergent and potentially disruptive technologies that soon will pervade all healthcare systems and health ecosystems.

References

Hyppönen H, Kaipio J, Heponiemi T, Lääveri T, Aalto AM, Vänskä J, et al. Developing the National Usability-Focused Health Information System Scale for Physicians: Validation Study. J Med Internet Res. 2019;21(5):e12875.