Infokiri







SARMS output type GUIDES AND TOOLKITS [GAT]

The focus is on building capacity and contributing to continuous professional development.

These outputs are methods guides and toolkits providing methodological guidance to strengthen ECDC and country capabilities including self-assessments for core public health activities and supporting functions such as outbreak investigations, statistics, modelling, evidence synthesis, guideline development etc., but for which no specific standard has been agreed between ECDC and its stakeholders.

In case of agreed standards for mandated activities such as EU surveillance and monitoring, chose the output type 'Protocols and standards'.

The final editing and formatting will be done by the ECDC editors.

Abbreviations

AMR Antimicrobial resistance
ECDC The European Centre for Disease Prevention and Control
EEA European Economic Area
ENP European neighbourhood policy
EQA External quality assessment
EU European Union
EURL EU reference laboratory
SRM Stakeholder relationship management
WHO World Health Organization

Context
Regulation 2022/2370 provides an amended mandate for ECDC, and Regulation 2022/2371 on serious cross-border threats to health reinforces ECDC’s responsibilities for detection, surveillance, and risk assessment of threats to human health from communicable diseases at the EU level [1, 2]. Under these mandates, ECDC is tasked to ensure coordination and operation of EU surveillance networks, encourage cooperation between expert laboratories, as well as foster the development of sufficient capacity within the community for the diagnosis, detection, identification and characterisation of infectious agents which may pose a threat to public health [1].
Within the integrated networks for epidemiological surveillance, ECDC has established disease and/or laboratory (sub-)networks. ECDC operates these networks to enhance capabilities and strengthen capacity for pathogen detection, characterisation and uniform surveillance of specific diseases and antimicrobial resistance (AMR). One of the key activities implemented to support the network laboratories has been the provision of external quality assessment (EQA) schemes. These EU-level EQA schemes aim to strengthen capacity and harmonise laboratory methodologies to ensure comparability of data for EU-level surveillance purposes.
Since 2010, ECDC has supported more than 180 EQA schemes for over 30 pathogens and special health issues including AMR. From external evaluation and feedback from the ECDC disease networks, the EQA schemes have been assessed to be the most effective laboratory capacity building activity supported by ECDC [3]. Until 2024, all EQA activities were provided to disease and/or laboratory (sub-)networks by one or more laboratories contracted by ECDC to set up and execute these EQAs, usually as part of a more comprehensive laboratory support contract. Contracts between ECDC and the laboratories providing these EQAs were established through open procurement procedures or framework partnership agreements.
In November 2022, Regulation 2022/2371 on serious cross-border threats to health was adopted [2]. This regulation provides the legal mandate for the European Commission to designate EU reference laboratories (EURLs) in the area of public health. The network of EURLs for public health will be operated by ECDC and provide support to the disease and/or laboratory (sub-)networks in order to promote good practice and alignment among Member States on diagnostics, testing methods, use of certain tests for harmonised surveillance, notification, and harmonised data reporting by Member States, with the provision of EQA specifically mentioned as one area of responsibility of the EURLs for public health. Given this, much of the laboratory support contracted by ECDC will gradually be transitioned to the EURL model.
Audience
The guide is intended for the following audiences:
• EURLs for public health that will provide EQAs to public health microbiology laboratories.
Objectives of this guide
This guide is intended to provide principles for EURLs for public health on how to plan, coordinate, and implement EQAs for public health microbiology laboratories at EU level.
This document is not intended to be a comprehensive technical guidance on the general implementation of EQAs. For such detailed guidance, please see the WHO manual for organizing a national EQA programme for health laboratories and other testing sites [4], and/or relevant national guidelines.
Definition and description of EU-level public health microbiology laboratory EQAs
External quality assessment (EQA) is a system designed to objectively assess the quality of test results obtained by a laboratory by means of an external actor (i.e. EQA provider). It allows for the performance comparison of a laboratory’s testing to that of a peer group of laboratories and/or a reference laboratory [5]. Participation in EQAs is a fundamental aspect of laboratory quality management.
For the purpose of this guide, EQA is defined as the process when an external provider (in this case, a EURL for public health) sends samples or sequence data for testing and/or analysis to the set of laboratories participating in the EQA, where the results of all laboratories are then compared and analysed by the EQA provider before the results are reported back to the participating laboratories. An EQA scheme may consist of multiple EQA rounds or distributions per year, and its outcomes should be used to monitor laboratory performance, to identify challenges, and provide basis for planning and implementation of corrective actions. to generate recommendations corrective actions within the participating laboratories to improve future performance.
EU-level EQA schemes should complement existing national and international EQA schemes and be focused on standard diagnostic and typing quality of clinical diagnostic laboratory testing under ISO15189 and/or ISO17025. Even though EU-level EQAs are not primarily designed to support accreditation efforts, participation in EU-level EQAs is sometimes used by participating laboratories to meet their national accreditation requirements.
Objectives of EU-level public health microbiology laboratory EQAs

The objectives of implementing EQAs for EU-level public health microbiology laboratories are to assess proficiency in order to:
• Assure high quality and comparability of public health laboratory data at the EU level;
• Assure capability for detection and characterization of pathogens, antimicrobial resistance, timely threat detection and other issues of public health relevance; and
• Identify capacity building needs for the purpose of improving the detection and characterization of pathogens and antimicrobial resistance issues of public health relevance.

Planning and performing EU-level public health microbiology laboratory EQAs
Guiding principles
EU-level public health microbiology EQAs should:
• Be an integral part of the capacity strengthening process for ECDC disease and/or laboratory (sub-)networks;
• Have clear objectives that are defined in relation to ECDC's disease-specific surveillance objectives;
• Strive for participation from as many disease and/or laboratory (sub-)networks countries as possible;
• Adhere to international quality and safety standards (e.g. relevant ISO standards);
• Cover EU or international standard diagnostic methods;
• Monitor compliance with international interpretation criteria, units of measurement, and result reporting formats, i.e. EU case definitions and laboratory guidance for surveillance;
• Be reported back to participants in a timely manner;
• Include technical follow-up and troubleshooting for the participating laboratories when needed;
• Include participating laboratories feedback on the use of the results for quality improvement or accreditation purposes;
• Result in the identification of training and capacity building needs;
• Include proposals of corrective measures if an EQA detects non-proficiency or harmonisation gaps; and
• Be publicised with main EQA findings based on aggregated data, including advice and/or recommendations on future EQA needs.

Design and implementation of EU-level public health microbiology laboratory EQAs
The EURLs for public health are tasked to plan, implement and evaluate all aspects of the EQA cycle (Figure 1), and should guarantee sustainable EQA implementation to ensure continuity and quality as well as the optimised use of the EQA results.

Figure 1. EQA cycle to be planned, implemented, and evaluated by EURLs.

The following aspects should be included in the preparation and execution of the EQAs for public health microbiology laboratories:
Planning
The EURL must prepare an EQA plan including a timeline as per Annex 1, with a detailed description of the different EQA steps described in the sections below.
Since the implementation of EQAs must be well aligned with other network activities coordinated by ECDC, it is imperative that the EQA plan is discussed and agreed with ECDC before any implementation of EQA activities begins. EURLs should therefore start to prepare their plan and initiate discussions with ECDC at least three months before any EQA activities are scheduled to begin.
EU-level public health microbiology laboratory EQAs are primarily implemented for laboratories based in EU/EEA countries. However, the participation of laboratories from non-EU/EEA countries (for example the Western Balkans [6], EU candidate countries [7], and/or European Neighbourhood Policy [ENP] countries [8]) may have a strong public health added value and therefore be of interest both to the EU and to the country in question. The decision whether to involve non-EU/EEA countries shall take into account the following:
• Participation of the countries’ laboratories in the ECDC disease and/or laboratory (sub-)networks as defined in ECDC’s Stakeholder Relationship Management [SRM] system;
• Existing European Commission and ECDC strategies for participation of EU candidate/potential candidate countries in ECDC disease networks and surveillance activities;
• Existing cooperation frameworks (such as a memorandum of understanding, an administrative agreement, or a cooperation project for technical assistance) between the European Commission or ECDC and the countries in question;
• ECDC’s International Relations Policy [9]; and
• Participation of the countries in question in the funding programme for the EURLs for public health.
EQA providers should clearly state the eligibility of non-EU/EEA countries in each EQA plan, consulting with ECDC on this point at an early stage.
Participation in EU-level public health microbiology laboratory EQAs must be free-of-charge for the EQA participants. In addition, the EQA provider should strive for inclusive laboratory participation, i.e. covering as many diseases and/or laboratory (sub-)networks countries as possible, with an expected minimum of 80% of EU Member States participating.1 Within their summary EQA report, the EURL should request, record and analyse the reasons for non-participation of countries.
Communication management
As part of the EQA plan, the EURL should propose a communication strategy presenting what information will be shared with ECDC, participating disease and/or laboratory (sub-)networks laboratories and nominated operational contact points. As a general principle ECDC should be involved in strategic communications with the participating laboratories and nominated operational contact points related to the implementation and operational activities of the EQA. ECDC will provide the EURL for public health with the relevant contact information of the disease and/or laboratory (sub-)networks laboratories that will be invited to participate in the EQA scheme.
EQA preparation, methodology protocol and material distribution
The EQA objectives should be clearly defined in relation to ECDC’s disease-specific surveillance objectives. In addition, they should take into account areas of improvement identified from previous EQAs, the potential introduction of new laboratory methods, additional needs collected from the disease and/or laboratory (sub-)networks etc.
The EURL for public health should provide the participants with an EQA protocol including a clear description of what samples, matrices and/or sequence data that are planned to be included, and what methods and interpretation criteria will be applied. Any samples included in the panel for educational purposes as identified by previous activities should be specified. This EQA protocol should be submitted for review as part of the EQA plan prior to the implementation of any activities.
An invitation letter to the laboratories shall include at least the following:
• The rationale and objectives of the EQA;
• Participation requirements (e.g. being part of the disease and/or laboratory (sub-)network);
• Reporting requirements and timelines (including the minimum requirements for obtaining an EQA certificate);
• Provisions for intellectual property, data ownership and sharing;
• Descriptions of planned post-EQA outputs such as reports and publications; and
• Information about planned post-EQA participant feedback.
The EURL for public health shall also perform appropriate quality controls according to best practise standards, including stability and homogeneity testing of the panels. The EQA may include biological samples and/or specimens, materials, isolates and/or sequence data, with the EURL for public health selecting and preparing EQA panel samples and/or sequence data that are the most relevant given the objectives of the EQA.
The EQA shall be distributed to all participants with instructions for reporting of the results including the participating laboratory methods/kits and/or standards in use. All EQAs involving biological samples must be shipped following international standards and include detailed instructions regarding packaging, biosafety precautions and storage.
Data analysis
The results of participating laboratories shall be collected, compiled, and analysed as per the description in the EQA plan. This includes ensuring data completeness and addressing any anomalies. Statistical analyses such as calculating descriptive statistics, applying comparative methods, and conducting trend analyses over multiple cycles are performed to assess laboratory performance.
Objectives of Data Analysis in EQA should:
• Measure individual laboratory results against predefined criteria or consensus values across other laboratories to identify variations in test outcomes.
• Detect trends indicating systematic errors or biases in the testing process.
• Identify areas for improvement that will aid laboratories in their analytical accuracy, precision, and operational procedures.
• Compare performance across different laboratories to establish performance benchmarks and promote best practices.
Results feedback to participants and relevant country support
The individual laboratory proficiency reports including detailed results analysis shall be prepared by the EURL for public health and shared with each participating laboratory. The EURL for public health shall also assist the laboratories that did not achieve acceptable level of performance after the EQA exercise, by providing troubleshooting assistance and advice as part of the feedback. The EURL for public health may also propose laboratory-specific capability-building or training activities.
EQA participation certificates
EURLs should issue certificates for participating laboratories that have completed an EQA round/scheme. Annex 2 includes a description of the certificate content and a template that can be modified along the elements described below.
The certificate should attest participation and completion of an EQA round/scheme; however, the certificate should not include any results assessment or indicate any level of performance by the laboratory. Information and data on laboratories’ performances should only be provided to the participant in the individual laboratory proficiency reports described above.
A certificate should be issued if the laboratory has returned all results within the given timeframe and achieving the minimum criteria (i.e. all results from the set of tests/methods indicated as minimum requirement; results from other tests included as optional should not affect whether the certificate is given or not to the laboratory in question).
Provide reports to ECDC
EQA data, results and reports for individual countries must be made available to ECDC by the EURL for public health upon request.
Reporting and dissemination
The EURL shall in a timely manner, as detailed in the EQA plan submitted, deliver a technical report based on anonymised and aggregated data summarising the results of the EQA performance of the participating countries. The report must include identified overall areas of training needs or needs for capacity building and recommendations for training or capacity building activities. EQA findings that point to limitations in the methods routinely used for laboratory-based surveillance or event confirmation (e.g. technical inconsistencies, harmonisation gaps) should be particularly highlighted as these technical problems may compromise the accuracy and comparability of surveillance data. Upon agreement with relevant stakeholders and ECDC, the EURL for public health may prepare a scientific publication based on the anonymised and aggregated results. It is also expected that these results are presented at relevant network meetings are the earliest opportunity.
Survey of feedback on usefulness from participants
The EURL must conduct a survey to collect feedback on the EQA from the participating laboratories. This survey should cover at minimum the practical implementation of the EQA itself, corrective actions taken, the perceived usefulness of the EQA, suggestions for future EQAs and whether the participating laboratory feels that additional training activities are required (including if yes which training activities would be most valuable). This survey should be included in the EQA plan and anonymised results of this survey shall be shared with ECDC.
Data management, ownership and sharing
References

1. European Commission. Regulation (EU) 2022/2370 of the European Parliament and of the Council of 23 November 2022 amending Regulation (EC) No 851/2004 establishing a European centre for disease prevention and control. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32022R2370
2. European Commission. Regulation (EU) 2022/2371 of the European Parliament and of the Council of 23 November 2022 on serious cross-border threats to health and repealing Decision No 1082/2013/EU. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32022R2371
3. Economisti Associati srl, Coffey International Development Ltd., Libero Istituto Universitario Carlo Cattaneo – CREMS, CEMKA-EVAL, and IBF International Consulting. The second independent evaluation of the ECDC in accordance with its Founding Regulation (European Parliament and Council Regulation (EC) no. 851/2004); Bologna: Economisti Associati; 2014.; Available from: https://ecdc.europa.eu/sites/portal/files/media/en/aboutus/Key%20Documents/ECDC-external-evaluation-2014.pdf.
4. World Health Organization. WHO manual for organizing a national external quality assessment programme for health laboratories and other testing sites.; Available from: https://www.who.int/publications/i/item/9789241549677.
5. World Health Organization. Overview of External Quality Assessment (EQA). Available from: https://www.who.int/publications/m/item/overview-of-external-quality-assessment-eqa.
6. European Union. Western Balkans.; Available from: https://www.eeas.europa.eu/eeas/western-balkans_en.
7. European Commission. EU enlargement. Available from: https://commission.europa.eu/strategy-and-policy/policies/eu-enlargement_en.
8. European Union. European Neighbourhood Policy.; Available from: https://www.eeas.europa.eu/eeas/european-neighbourhood-policy_en.
9. European Center for Disease Prevention and Control. International cooperation.; Available from: https://www.ecdc.europa.eu/en/about-us/who-we-work/international-activities.


Annex 1 – EQA plan outline
The EQA plan submitted by EURLs for ECDC approval should contain at least the following sub-headers, clearly specifying the overall timeline of the EQA to be implemented.
Objectives
The EQA objectives should be clearly defined in relation to ECDC’s disease-specific surveillance objectives.
Participants
Eligibility: Detail criteria for which laboratories can participate in the EQA.
Registration: Outline the process for registration, including deadlines and necessary documentation.
Preparation
An invitation letter to disease and/or laboratory (sub-)networks laboratories shall include at least what is outlined in the guide above.
Methodology
EQA Design: Describe the overall design of the EQA and timeline, including types of samples, testing parameters, and expected outcomes.
EQA protocol: An EQA protocol describing the methodology should be attached to this EQA plan. Details outlined in the EQA protocol do not need to be repeated in the EQA plan.
Distribution
Schedule: Provide the timeline for the distribution of EQA materials.
Logistics: Outline logistical arrangements, including transportation methods of shipments to ensure secure and timely delivery.
Instructions for Completion (Minimum Requirements)
Testing Procedures: Define specific testing procedures that must be followed.
Documentation: List the documentation to be completed and submitted along with the EQA results.
Submission Deadline: Specify the deadline for the submission of completed testing results.
Data Analysis
Statistical Methods: Detail the statistical methods used for analysing EQA data.
Performance Indicators: Define performance indicators like accuracy, precision, and comparison against benchmarks.
Reporting of Results
Reports to Participants
Content: Outline what the report will include, e.g., individual laboratory performance, comparative analyses.
Delivery Method: Specify how reports will be delivered to participants.
Publications
Criteria for Publication: Specify criteria for deciding which EQA data or findings are suitable for public dissemination.
Process: Describe the process for preparing and submitting publications based on EQA data.
Reports to ECDC
Requirements: Specify the format and content required by ECDC.
Timeline: Provide timelines for report submission following EQA completion.
Participant Survey
Purpose: Describe the objectives of conducting a survey among participants.
Topics Covered: List key topics to be covered in the survey, including satisfaction, perceived value, and suggestions for improvement.
Methodology: Outline how the survey will be conducted and the timeline for its execution and analysis.
Data Management, Ownership, and Sharing
Data Handling: Specify how data collected through EQA will be managed, stored, and protected.
Ownership: Clarify ownership of EQA data.
Data Sharing: Detail the conditions under which EQA data may be shared with third parties, including other researchers and public health authorities.




Annex 2 – Template Laboratory EQA certificate

Issuing Certificates of Participation in EQA schemes

Certificate content
• Logos – the certificate must carry the ECDC logo. The certificate may have two additional logos:
◦ Logo of the EQA provider – adding the logo of the EQA provider to your certificate could increase the value of the certificate for the participating laboratories, especially if for the provision of the scheme in question, the EQA provider is ISO certified (e.g. ISO 17043). Therefore, below the logo of the EQA provider, please indicate the ISO certification for the service provided.
◦ Logo of a co-organiser – if the EQA is contracted out and jointly organised with another public health institution, e.g. WHO, you could include the logo of this institution/organisation. This should be followed by some specification, e.g. “In cooperation with ….” And if such cooperation is covered by a special agreement/contract, reference should be name to this “under agreement number …”.
• The name of the laboratory network – where applicable, the name of the ECDC/EU network via which the EQA is conducted and to which the laboratory participates should be indicated.
• “Certificate of participation” – the certificate clearly indicates that it is awarded for participation without specifying level of performance.
• “Awarded to” - Name of the laboratory to which the certificate is awarded, including the full name of the hosting institution and the country.
• “to attest participation in” – the certificate clearly indicates that it is awarded upon completion of a set of tests (so called minimum criteria needed to meet in order for the participation to be considered successful). This means that if a laboratory does not return results from tests which are considered as part of the minimum, it would not qualify for receiving a certificate.
• “External Quality Assessment scheme for…” – the name of the specific EQA scheme should follow and the date or time period during which the scheme was conducted should be indicated; note that the term assessment should be used and not assurance.
• “Covering the following: …” (list of laboratory tests/methods) – this is followed by a list of laboratory tests which the laboratory returned results for, including specific pathogen strains, etc... The template has a provision for several lines; more can be added and those not needed could be deleted.







Network name

CERTIFICATE OF PARTICIPATION
awarded to
Laboratory name, hosting institution, country



to attest participation in
External Quality Assessment scheme for name of the assessment scheme, date
Covering the following:

• Laboratory test, method 1
• Laboratory test, method 1
• Laboratory test, method 1
•
Laboratory test, method 1
• Laboratory test, method 1
• Laboratory test, method 1
•
Laboratory test, method 1
• Laboratory test, method 1
• Laboratory test, method







SARMS output type GUIDES AND TOOLKITS [GAT]

The focus is on building capacity and contributing to continuous professional development.

These outputs are methods guides and toolkits providing methodological guidance to strengthen ECDC and country capabilities including self-assessments for core public health activities and supporting functions such as outbreak investigations, statistics, modelling, evidence synthesis, guideline development etc., but for which no specific standard has been agreed between ECDC and its stakeholders.

In case of agreed standards for mandated activities such as EU surveillance and monitoring, chose the output type 'Protocols and standards'.

The final editing and formatting will be done by the ECDC editors.

Abbreviations

AMR Antimicrobial resistance
ECDC The European Centre for Disease Prevention and Control
EEA European Economic Area
ENP European neighbourhood policy
EQA External quality assessment
EU European Union
EURL EU reference laboratory
SRM Stakeholder relationship management
WHO World Health Organization

Context
Regulation 2022/2370 provides an amended mandate for ECDC, and Regulation 2022/2371 on serious cross-border threats to health reinforces ECDC’s responsibilities for detection, surveillance, and risk assessment of threats to human health from communicable diseases at the EU level [1, 2]. Under these mandates, ECDC is tasked to ensure coordination and operation of EU surveillance networks, encourage cooperation between expert laboratories, as well as foster the development of sufficient capacity within the community for the diagnosis, detection, identification and characterisation of infectious agents which may pose a threat to public health [1].
Within the integrated networks for epidemiological surveillance, ECDC has established disease and/or laboratory (sub-)networks. ECDC operates these networks to enhance capabilities and strengthen capacity for pathogen detection, characterisation and uniform surveillance of specific diseases and antimicrobial resistance (AMR). One of the key activities implemented to support the network laboratories has been the provision of external quality assessment (EQA) schemes. These EU-level EQA schemes aim to strengthen capacity and harmonise laboratory methodologies to ensure comparability of data for EU-level surveillance purposes.
Since 2010, ECDC has supported more than 180 EQA schemes for over 30 pathogens and special health issues including AMR. From external evaluation and feedback from the ECDC disease networks, the EQA schemes have been assessed to be the most effective laboratory capacity building activity supported by ECDC [3]. Until 2024, all EQA activities were provided to disease and/or laboratory (sub-)networks by one or more laboratories contracted by ECDC to set up and execute these EQAs, usually as part of a more comprehensive laboratory support contract. Contracts between ECDC and the laboratories providing these EQAs were established through open procurement procedures or framework partnership agreements.
In November 2022, Regulation 2022/2371 on serious cross-border threats to health was adopted [2]. This regulation provides the legal mandate for the European Commission to designate EU reference laboratories (EURLs) in the area of public health. The network of EURLs for public health will be operated by ECDC and provide support to the disease and/or laboratory (sub-)networks in order to promote good practice and alignment among Member States on diagnostics, testing methods, use of certain tests for harmonised surveillance, notification, and harmonised data reporting by Member States, with the provision of EQA specifically mentioned as one area of responsibility of the EURLs for public health. Given this, much of the laboratory support contracted by ECDC will gradually be transitioned to the EURL model.
Audience
The guide is intended for the following audiences:
• EURLs for public health that will provide EQAs to public health microbiology laboratories.
Objectives of this guide
This guide is intended to provide principles for EURLs for public health on how to plan, coordinate, and implement EQAs for public health microbiology laboratories at EU level.
This document is not intended to be a comprehensive technical guidance on the general implementation of EQAs. For such detailed guidance, please see the WHO manual for organizing a national EQA programme for health laboratories and other testing sites [4], and/or relevant national guidelines.
Definition and description of EU-level public health microbiology laboratory EQAs
External quality assessment (EQA) is a system designed to objectively assess the quality of test results obtained by a laboratory by means of an external actor (i.e. EQA provider). It allows for the performance comparison of a laboratory’s testing to that of a peer group of laboratories and/or a reference laboratory [5]. Participation in EQAs is a fundamental aspect of laboratory quality management.
For the purpose of this guide, EQA is defined as the process when an external provider (in this case, a EURL for public health) sends samples or sequence data for testing and/or analysis to the set of laboratories participating in the EQA, where the results of all laboratories are then compared and analysed by the EQA provider before the results are reported back to the participating laboratories. An EQA scheme may consist of multiple EQA rounds or distributions per year, and its outcomes should be used to monitor laboratory performance, to identify challenges, and provide basis for planning and implementation of corrective actions. to generate recommendations corrective actions within the participating laboratories to improve future performance.
EU-level EQA schemes should complement existing national and international EQA schemes and be focused on standard diagnostic and typing quality of clinical diagnostic laboratory testing under ISO15189 and/or ISO17025. Even though EU-level EQAs are not primarily designed to support accreditation efforts, participation in EU-level EQAs is sometimes used by participating laboratories to meet their national accreditation requirements.
Objectives of EU-level public health microbiology laboratory EQAs

The objectives of implementing EQAs for EU-level public health microbiology laboratories are to assess proficiency in order to:
• Assure high quality and comparability of public health laboratory data at the EU level;
• Assure capability for detection and characterization of pathogens, antimicrobial resistance, timely threat detection and other issues of public health relevance; and
• Identify capacity building needs for the purpose of improving the detection and characterization of pathogens and antimicrobial resistance issues of public health relevance.

Planning and performing EU-level public health microbiology laboratory EQAs
Guiding principles
EU-level public health microbiology EQAs should:
• Be an integral part of the capacity strengthening process for ECDC disease and/or laboratory (sub-)networks;
• Have clear objectives that are defined in relation to ECDC's disease-specific surveillance objectives;
• Strive for participation from as many disease and/or laboratory (sub-)networks countries as possible;
• Adhere to international quality and safety standards (e.g. relevant ISO standards);
• Cover EU or international standard diagnostic methods;
• Monitor compliance with international interpretation criteria, units of measurement, and result reporting formats, i.e. EU case definitions and laboratory guidance for surveillance;
• Be reported back to participants in a timely manner;
• Include technical follow-up and troubleshooting for the participating laboratories when needed;
• Include participating laboratories feedback on the use of the results for quality improvement or accreditation purposes;
• Result in the identification of training and capacity building needs;
• Include proposals of corrective measures if an EQA detects non-proficiency or harmonisation gaps; and
• Be publicised with main EQA findings based on aggregated data, including advice and/or recommendations on future EQA needs.

Design and implementation of EU-level public health microbiology laboratory EQAs
The EURLs for public health are tasked to plan, implement and evaluate all aspects of the EQA cycle (Figure 1), and should guarantee sustainable EQA implementation to ensure continuity and quality as well as the optimised use of the EQA results.

Figure 1. EQA cycle to be planned, implemented, and evaluated by EURLs.

The following aspects should be included in the preparation and execution of the EQAs for public health microbiology laboratories:
Planning
The EURL must prepare an EQA plan including a timeline as per Annex 1, with a detailed description of the different EQA steps described in the sections below.
Since the implementation of EQAs must be well aligned with other network activities coordinated by ECDC, it is imperative that the EQA plan is discussed and agreed with ECDC before any implementation of EQA activities begins. EURLs should therefore start to prepare their plan and initiate discussions with ECDC at least three months before any EQA activities are scheduled to begin.
EU-level public health microbiology laboratory EQAs are primarily implemented for laboratories based in EU/EEA countries. However, the participation of laboratories from non-EU/EEA countries (for example the Western Balkans [6], EU candidate countries [7], and/or European Neighbourhood Policy [ENP] countries [8]) may have a strong public health added value and therefore be of interest both to the EU and to the country in question. The decision whether to involve non-EU/EEA countries shall take into account the following:
• Participation of the countries’ laboratories in the ECDC disease and/or laboratory (sub-)networks as defined in ECDC’s Stakeholder Relationship Management [SRM] system;
• Existing European Commission and ECDC strategies for participation of EU candidate/potential candidate countries in ECDC disease networks and surveillance activities;
• Existing cooperation frameworks (such as a memorandum of understanding, an administrative agreement, or a cooperation project for technical assistance) between the European Commission or ECDC and the countries in question;
• ECDC’s International Relations Policy [9]; and
• Participation of the countries in question in the funding programme for the EURLs for public health.
EQA providers should clearly state the eligibility of non-EU/EEA countries in each EQA plan, consulting with ECDC on this point at an early stage.
Participation in EU-level public health microbiology laboratory EQAs must be free-of-charge for the EQA participants. In addition, the EQA provider should strive for inclusive laboratory participation, i.e. covering as many diseases and/or laboratory (sub-)networks countries as possible, with an expected minimum of 80% of EU Member States participating.1 Within their summary EQA report, the EURL should request, record and analyse the reasons for non-participation of countries.
Communication management
As part of the EQA plan, the EURL should propose a communication strategy presenting what information will be shared with ECDC, participating disease and/or laboratory (sub-)networks laboratories and nominated operational contact points. As a general principle ECDC should be involved in strategic communications with the participating laboratories and nominated operational contact points related to the implementation and operational activities of the EQA. ECDC will provide the EURL for public health with the relevant contact information of the disease and/or laboratory (sub-)networks laboratories that will be invited to participate in the EQA scheme.
EQA preparation, methodology protocol and material distribution
The EQA objectives should be clearly defined in relation to ECDC’s disease-specific surveillance objectives. In addition, they should take into account areas of improvement identified from previous EQAs, the potential introduction of new laboratory methods, additional needs collected from the disease and/or laboratory (sub-)networks etc.
The EURL for public health should provide the participants with an EQA protocol including a clear description of what samples, matrices and/or sequence data that are planned to be included, and what methods and interpretation criteria will be applied. Any samples included in the panel for educational purposes as identified by previous activities should be specified. This EQA protocol should be submitted for review as part of the EQA plan prior to the implementation of any activities.
An invitation letter to the laboratories shall include at least the following:
• The rationale and objectives of the EQA;
• Participation requirements (e.g. being part of the disease and/or laboratory (sub-)network);
• Reporting requirements and timelines (including the minimum requirements for obtaining an EQA certificate);
• Provisions for intellectual property, data ownership and sharing;
• Descriptions of planned post-EQA outputs such as reports and publications; and
• Information about planned post-EQA participant feedback.
The EURL for public health shall also perform appropriate quality controls according to best practise standards, including stability and homogeneity testing of the panels. The EQA may include biological samples and/or specimens, materials, isolates and/or sequence data, with the EURL for public health selecting and preparing EQA panel samples and/or sequence data that are the most relevant given the objectives of the EQA.
The EQA shall be distributed to all participants with instructions for reporting of the results including the participating laboratory methods/kits and/or standards in use. All EQAs involving biological samples must be shipped following international standards and include detailed instructions regarding packaging, biosafety precautions and storage.
Data analysis
The results of participating laboratories shall be collected, compiled, and analysed as per the description in the EQA plan. This includes ensuring data completeness and addressing any anomalies. Statistical analyses such as calculating descriptive statistics, applying comparative methods, and conducting trend analyses over multiple cycles are performed to assess laboratory performance.
Objectives of Data Analysis in EQA should:
• Measure individual laboratory results against predefined criteria or consensus values across other laboratories to identify variations in test outcomes.
• Detect trends indicating systematic errors or biases in the testing process.
• Identify areas for improvement that will aid laboratories in their analytical accuracy, precision, and operational procedures.
• Compare performance across different laboratories to establish performance benchmarks and promote best practices.
Results feedback to participants and relevant country support
The individual laboratory proficiency reports including detailed results analysis shall be prepared by the EURL for public health and shared with each participating laboratory. The EURL for public health shall also assist the laboratories that did not achieve acceptable level of performance after the EQA exercise, by providing troubleshooting assistance and advice as part of the feedback. The EURL for public health may also propose laboratory-specific capability-building or training activities.
EQA participation certificates
EURLs should issue certificates for participating laboratories that have completed an EQA round/scheme. Annex 2 includes a description of the certificate content and a template that can be modified along the elements described below.
The certificate should attest participation and completion of an EQA round/scheme; however, the certificate should not include any results assessment or indicate any level of performance by the laboratory. Information and data on laboratories’ performances should only be provided to the participant in the individual laboratory proficiency reports described above.
A certificate should be issued if the laboratory has returned all results within the given timeframe and achieving the minimum criteria (i.e. all results from the set of tests/methods indicated as minimum requirement; results from other tests included as optional should not affect whether the certificate is given or not to the laboratory in question).
Provide reports to ECDC
EQA data, results and reports for individual countries must be made available to ECDC by the EURL for public health upon request.
Reporting and dissemination
The EURL shall in a timely manner, as detailed in the EQA plan submitted, deliver a technical report based on anonymised and aggregated data summarising the results of the EQA performance of the participating countries. The report must include identified overall areas of training needs or needs for capacity building and recommendations for training or capacity building activities. EQA findings that point to limitations in the methods routinely used for laboratory-based surveillance or event confirmation (e.g. technical inconsistencies, harmonisation gaps) should be particularly highlighted as these technical problems may compromise the accuracy and comparability of surveillance data. Upon agreement with relevant stakeholders and ECDC, the EURL for public health may prepare a scientific publication based on the anonymised and aggregated results. It is also expected that these results are presented at relevant network meetings are the earliest opportunity.
Survey of feedback on usefulness from participants
The EURL must conduct a survey to collect feedback on the EQA from the participating laboratories. This survey should cover at minimum the practical implementation of the EQA itself, corrective actions taken, the perceived usefulness of the EQA, suggestions for future EQAs and whether the participating laboratory feels that additional training activities are required (including if yes which training activities would be most valuable). This survey should be included in the EQA plan and anonymised results of this survey shall be shared with ECDC.
Data management, ownership and sharing
References

1. European Commission. Regulation (EU) 2022/2370 of the European Parliament and of the Council of 23 November 2022 amending Regulation (EC) No 851/2004 establishing a European centre for disease prevention and control. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex%3A32022R2370
2. European Commission. Regulation (EU) 2022/2371 of the European Parliament and of the Council of 23 November 2022 on serious cross-border threats to health and repealing Decision No 1082/2013/EU. Available from: https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32022R2371
3. Economisti Associati srl, Coffey International Development Ltd., Libero Istituto Universitario Carlo Cattaneo – CREMS, CEMKA-EVAL, and IBF International Consulting. The second independent evaluation of the ECDC in accordance with its Founding Regulation (European Parliament and Council Regulation (EC) no. 851/2004); Bologna: Economisti Associati; 2014.; Available from: https://ecdc.europa.eu/sites/portal/files/media/en/aboutus/Key%20Documents/ECDC-external-evaluation-2014.pdf.
4. World Health Organization. WHO manual for organizing a national external quality assessment programme for health laboratories and other testing sites.; Available from: https://www.who.int/publications/i/item/9789241549677.
5. World Health Organization. Overview of External Quality Assessment (EQA). Available from: https://www.who.int/publications/m/item/overview-of-external-quality-assessment-eqa.
6. European Union. Western Balkans.; Available from: https://www.eeas.europa.eu/eeas/western-balkans_en.
7. European Commission. EU enlargement. Available from: https://commission.europa.eu/strategy-and-policy/policies/eu-enlargement_en.
8. European Union. European Neighbourhood Policy.; Available from: https://www.eeas.europa.eu/eeas/european-neighbourhood-policy_en.
9. European Center for Disease Prevention and Control. International cooperation.; Available from: https://www.ecdc.europa.eu/en/about-us/who-we-work/international-activities.


Annex 1 – EQA plan outline
The EQA plan submitted by EURLs for ECDC approval should contain at least the following sub-headers, clearly specifying the overall timeline of the EQA to be implemented.
Objectives
The EQA objectives should be clearly defined in relation to ECDC’s disease-specific surveillance objectives.
Participants
Eligibility: Detail criteria for which laboratories can participate in the EQA.
Registration: Outline the process for registration, including deadlines and necessary documentation.
Preparation
An invitation letter to disease and/or laboratory (sub-)networks laboratories shall include at least what is outlined in the guide above.
Methodology
EQA Design: Describe the overall design of the EQA and timeline, including types of samples, testing parameters, and expected outcomes.
EQA protocol: An EQA protocol describing the methodology should be attached to this EQA plan. Details outlined in the EQA protocol do not need to be repeated in the EQA plan.
Distribution
Schedule: Provide the timeline for the distribution of EQA materials.
Logistics: Outline logistical arrangements, including transportation methods of shipments to ensure secure and timely delivery.
Instructions for Completion (Minimum Requirements)
Testing Procedures: Define specific testing procedures that must be followed.
Documentation: List the documentation to be completed and submitted along with the EQA results.
Submission Deadline: Specify the deadline for the submission of completed testing results.
Data Analysis
Statistical Methods: Detail the statistical methods used for analysing EQA data.
Performance Indicators: Define performance indicators like accuracy, precision, and comparison against benchmarks.
Reporting of Results
Reports to Participants
Content: Outline what the report will include, e.g., individual laboratory performance, comparative analyses.
Delivery Method: Specify how reports will be delivered to participants.
Publications
Criteria for Publication: Specify criteria for deciding which EQA data or findings are suitable for public dissemination.
Process: Describe the process for preparing and submitting publications based on EQA data.
Reports to ECDC
Requirements: Specify the format and content required by ECDC.
Timeline: Provide timelines for report submission following EQA completion.
Participant Survey
Purpose: Describe the objectives of conducting a survey among participants.
Topics Covered: List key topics to be covered in the survey, including satisfaction, perceived value, and suggestions for improvement.
Methodology: Outline how the survey will be conducted and the timeline for its execution and analysis.
Data Management, Ownership, and Sharing
Data Handling: Specify how data collected through EQA will be managed, stored, and protected.
Ownership: Clarify ownership of EQA data.
Data Sharing: Detail the conditions under which EQA data may be shared with third parties, including other researchers and public health authorities.




Annex 2 – Template Laboratory EQA certificate

Issuing Certificates of Participation in EQA schemes

Certificate content
• Logos – the certificate must carry the ECDC logo. The certificate may have two additional logos:
◦ Logo of the EQA provider – adding the logo of the EQA provider to your certificate could increase the value of the certificate for the participating laboratories, especially if for the provision of the scheme in question, the EQA provider is ISO certified (e.g. ISO 17043). Therefore, below the logo of the EQA provider, please indicate the ISO certification for the service provided.
◦ Logo of a co-organiser – if the EQA is contracted out and jointly organised with another public health institution, e.g. WHO, you could include the logo of this institution/organisation. This should be followed by some specification, e.g. “In cooperation with ….” And if such cooperation is covered by a special agreement/contract, reference should be name to this “under agreement number …”.
• The name of the laboratory network – where applicable, the name of the ECDC/EU network via which the EQA is conducted and to which the laboratory participates should be indicated.
• “Certificate of participation” – the certificate clearly indicates that it is awarded for participation without specifying level of performance.
• “Awarded to” - Name of the laboratory to which the certificate is awarded, including the full name of the hosting institution and the country.
• “to attest participation in” – the certificate clearly indicates that it is awarded upon completion of a set of tests (so called minimum criteria needed to meet in order for the participation to be considered successful). This means that if a laboratory does not return results from tests which are considered as part of the minimum, it would not qualify for receiving a certificate.
• “External Quality Assessment scheme for…” – the name of the specific EQA scheme should follow and the date or time period during which the scheme was conducted should be indicated; note that the term assessment should be used and not assurance.
• “Covering the following: …” (list of laboratory tests/methods) – this is followed by a list of laboratory tests which the laboratory returned results for, including specific pathogen strains, etc... The template has a provision for several lines; more can be added and those not needed could be deleted.







Network name

CERTIFICATE OF PARTICIPATION
awarded to
Laboratory name, hosting institution, country



to attest participation in
External Quality Assessment scheme for name of the assessment scheme, date
Covering the following:

• Laboratory test, method 1
• Laboratory test, method 1
• Laboratory test, method 1
•
Laboratory test, method 1
• Laboratory test, method 1
• Laboratory test, method 1
•
Laboratory test, method 1
• Laboratory test, method 1
• Laboratory test, method