Pöördumine

1

Enhancing the Impact of European Regional Development Fund and Cohesion Fund Operations through Trial-and-Adopt Impact

Evaluations and Data Ecosystem Enhancements

Call for Expression of Interest for ERDF and CF Projects

European Commission – Directorate General for Regional and Urban Policy (DG REGIO)

World Bank – Development Impact Evaluation Department (WBG-DIME)

1. Background

The Directorate-General for Regional and Urban Policy (DG REGIO), oversees the implementation of regional policy across the European Union (EU). This policy is delivered through three of the EU’s structural and investment funds, the European Social Fund+ (ESF+), the European Regional Development Fund (ERDF), and the Cohesion Fund (CF). The ERDF focuses on making Europe and its regions more competitive, smarter, greener, connected, and socially inclusive, while a significant portion of CF’s allocation is directed towards climate objectives. The ESF+ is managed by the EC Directorate-General for Employment, Social Affairs and Inclusion (DG EMPL) and supports initiatives in employment, social policies, education, skills development, structural reforms, and socio-economic recovery. DG REGIO encourages evidence-based programming, promotes innovation and experimentation, and supports actions that strengthen the monitoring and impacts of its programs. DG REGIO has established a strategic partnership with the World Bank’s Development Impact Department (DIME, see Annex 1) to build capacity and identify opportunities to bolster the effectiveness of EU cohesion policy investments through improving the use and integration of data, scientifically trialing alternative program design options and implementation modalities to test innovations and learn what works best in practice, and rigorously documenting impacts. As part of this partnership, some ERDF and CF projects will be supported on a pilot basis to embed trial-

and-adopt impact evaluations and facilitate data system integration to better track outcomes and

maximize impacts. The pilot will focus on the initial training and design stages of the “Trial-and-Adopt”

process (see Annex 1 and 2).

2

2. Purpose of the call DG REGIO and the World Bank are looking for projects funded by ERDF and CF that are interested in collaborating to embed trial-and-adopt impact evaluation pilots in their roll-out and enhance their monitoring data systems. The call is open to all ERDF and CF projects targeting, for example, individuals or households and addressing priority policy areas such as green transition (energy efficiency), mobility, digitalization/data/AI for service delivery, and services for people (including health, education, and skills). Each Member State is welcome to propose one or several projects for the workshop. Projects targeting firms can refer to the call under the complementary private sector engagement The European Regional Competitiveness Policy Lab Here.1 Following the call for expression of interest, up to 8 projects will be invited to a one-week workshop to design a trial-and-adopt impact evaluation strategy. The workshop is expected to take place from May 19 to May 23, 2025. The workshop’s objective will be to train selected ERDF /CF program management and monitoring and evaluation teams on methodologies to design and implement impact evaluations, provide the latest actionable evidence in the priority policy area, and develop a prospective trial-and-adopt impact evaluation strategy to be embedded in future program implementation. The impact evaluations may test multiple interventions, delivery modalities, design features, and targeting mechanisms to identify effective strategies that increase project impact. The Impact evaluations will include a measurement framework that captures outcomes along the theory of change, using existing data systems and proposed enhancements. For instance, results on short-term impacts or intermediate outcomes can help steer policy decisions and encourage program adjustments to boost medium to long-term effects. Additionally, during the workshop, teams will discuss constraints and inefficiencies in their current data systems and brainstorm options for automating and streamlining data systems to generate just-in-time evidence to monitor project outcomes and measure impact. For each project, the delegation for the workshop is expected to include 4-5 people, including a

representative from the managing authority, the project manager/coordinator or equivalent, 1 technical

staff member with in-depth knowledge of the project's design and implementation, and 1 technical staff

member responsible for monitoring data systems. Partners (such as DG EMPL, DG EAC or DG REFORM),

World Bank staff, researchers, and other technical experts will also participate.

Following the workshop, up to 4 projects will receive dedicated technical support through March 2026 to

prepare and generate full-fledged design concept notes that outline the modalities through which

Member States could embed impact evaluations in the project cycle. For those selected pilots, the WB-

DIME will assemble an impact evaluation team led by a principal investigator and with a field coordinator

and a research assistant. The impact evaluation research teams will work closely with Member States to

provide technical support, co-produce the impact evaluation designs, and ensure that the approach

defined through the collaboration is of high quality and policy relevance.

1 Please note that the two calls have different opening and closing dates.

3

In addition, these trial-and-adopt impact evaluations may be complemented by data ecosystem

assessments where projects will be selected for dedicated technical to identify potential enhancements

to the data used to monitor project impacts (See Annex 2). This would include a review of existing data

with potential linkages to project impacts, such as (i) micro-level administrative data (e.g. employment,

social security, tax, primary care, housing, etc.), (ii) geo-referenced data, (iii) micro-level survey data, and

(iv) data collected from digital and mobile technologies. The assessment will explore how existing data

could be leveraged to track outcomes over time and efficiently document intervention impacts, in

accordance with the applicable personal data protection regulations under the General Data Protection

Regulation (GDPR).

3. Selection criteria

Projects will be selected to participate in the one-week workshop based on the following criteria:

relevance, technical quality and maturity, scope, innovation and equity.

1. Relevance: The project addresses critical policy challenges and knowledge gaps in relation to key

priority policy areas such as green transition (energy efficiency), mobility, and services for people

(including health, education, and skills) and targets primarily individuals or households.

2. Technical quality and maturity: The project interventions and implementation modalities are

well-defined, with clear timelines and targets.

3. Scope: The project has the potential to enhance the effectiveness of a program that represents a

significant portion of the ERDF budget.

4. Innovation: The project tackles a relevant policy challenge through a new or untested

intervention.

5. Equity: The project benefits disadvantaged populations, promoting inclusivity.

6. Engagement and interest: The project is interested in and willing to facilitate essential processes

for designing and implementing the impact evaluation, such as granting or supporting access to existing information, and administrative data, and exploring data system integration options.

After the workshop, proposals for pilot project proposals will be selected for dedicated technical support,

based on the following additional criteria: impact evaluation design quality, implementation feasibility,

data access and availability, and scalability for broader application. Support budget availability and needs

per operation may also be considered and will co-determine how many projects can ultimately be

supported.

4

4. Submission details

The call for expression of interest is open from the 10th of February 2025, 18:00 CET. Interested managing and implementing authorities or beneficiaries (in agreement with the Managing authority) can submit their expression of interest (EoI) through the application form on this THIS LINK 2 by the deadline on Tuesday the 25th of March 2025, 18:00 CET.

Representatives of interested public authorities/agencies can request further information by contacting

Tillmann Heidelk ([email protected]) and Ghida Karbala

([email protected]).

2 https://survey.wb.surveycto.com/collect/eoi_dg_regio_final?caseid=

5

Please complete file this form electronically here: THIS LINK

1. Information about the Applicant

1.1 Member State

1.2 Name of managing and implementing authorities or beneficiaries submitting project

1.3 If beneficiary, indicate contact person at the Managing authority

Name, Last Name

Job title

Phone (e.g. +32-000000000)

Email

1.4 Contact person for this application Name, Last Name

Job title

Phone (e.g. +32-000000000)

Email

2. Project description

2.1 Name of project

2.2 Brief description of the project (Objectives, target population, intervention components, implementation modalities, etc., …)

(no word limit; suggested length – 300-500 words)

2.3 (optional) Additional documents with project information

(optional; you can upload additional project documents here)

2.4 Amount of ERDF funding allocated to this project

2.5 What is the planned implementation period for this project and its current status?

(please describe the planned project timeline, and if

the project is at the design stage, approved,

whether it has already started, …)

2.6 Please provide any available information on the project's geographic scope (covered regions, municipalities, etc.) and target

6

number of participants (for example number of individuals, households)?

2.7 What are the main potential impact evaluation questions you would like to answer for this project?

2.8 Is there another agency involved in the implementation of the program?

(select one answer)

Yes

No

2.9 What are the main other agencies involved in the implementation of the program

2.10 Main contact person at the agency leading the implementation of the program?

Name, Last Name

Job title

Phone (e.g. +32-000000000)

Email

3. Data Linkages

3.1 Administrative data collected by government information systems can enable low-cost evaluation of program impacts in real-time without requiring additional survey data collection. Do administrative datasets exist for tracking program participants such as individuals or households in your program(s)?

If yes, please name and in as far as possible briefly describe potential data sources. Do you already have access to these datasets?

If not, are you willing and able to facilitate access for the purpose of the impact evaluation?

(select one answer)

Yes

No

(please briefly describe potential data sources that could be explored, max 300 words)

Please describe any potential obstacles/hindrances to accessing these datasets.

7

Annex 1: WB Development Impact Group (DIME)

In the last twenty years, the World Bank’s Development Impact Group (DIME) has demonstrated that investing in, and building local capacities for, data, impact evaluation and artificial intelligence can help double the impact of public policy investments. DIME works with governments and international organizations to deliver data products and impact evaluations that increase the effectiveness of public policies. Its mission is to foster evidence-based and data-driven planning, programming, and implementation throughout the policy cycle. DIME’s approach to establishing and delivering joint impact programs delivers at the highest standards of transparency, quality, and rigor in generating evidence actionable for policy.

Figure 1 provides an overview of DIME’s “Trial-and-Adopt” Impact Evaluation approach, broken down into an iterative workflow that continuously supports evidence-based decision making. Each step in the workflow involves close coordination with policymakers and program implementers to directly shape public policy based on rigorous evidence. The partnership between WB-DIME and DG REGIO will tailor this approach to embed it into a subset of ERDF operations on a pilot basis.

Figure 1. DIME’s Approach to Inform Policy

Collaboratively, DG REGIO and the WB-DIME seek to transform ERDF design and implementation to become more evidence-based and results-driven. The partnership involves a pilot focused on the initial training and design stages of the “Trial-and-Adopt” process. The pilot phase will provide opportunities for capacity-building, and joint learning, and will inform the preparation of the framework that will help mainstream the use of experimentation in future funding cycles (2028- 2034). It will also better outline how to link improvements in the monitoring data ecosystems and complementary data to enhancements in policy effectiveness. The Trial-and-Adopt impact evaluation designs will outline what it would take to implement the designs, both in terms of operational implications and resources, for the Member States to assess how to move to implementation. By emphasizing continuous learning, administrative efficiency, and a learning-centric culture, this partnership strives to identify opportunities to strengthen the effectiveness of the ERDF and its innovation components. The initiative will be built on co-creation with Member States opting to participate in the pilot. External researchers will be engaged as partners in the delivery of activities in collaboration and coordination with World Bank and DG REGIO staff.

8

ANNEX 2: Data Integration With the goal of enhancing and automating the process of project monitoring, DIME will analyze

opportunities to strengthen linkages between the DG REGIO monitoring system and

disaggregated data from Member States such as (i) administrative data (e.g. employment, social

security, tax, primary care, housing, etc.), (ii) geo-referenced data, (iii) micro-level (e.g. survey)

data, and (iv) data collected from digital and mobile technologies.

During the workshop, teams will discuss constraints and inefficiencies in their current data systems and brainstorm options for automating and streamlining data systems to generate just- in-time evidence to monitor project outcomes and measure impact. Following initial discussions with Member States and DG REGIO at the workshop in May (19-23), 2-4 Member States will be selected for a detailed case study. The cases studies will illustrate the types of data linkages that can be established and how these linkages can highlight regional variation for key convergence indicators and improve the monitoring of developments in regional convergence. Additionally, they will show how these linkages can be used to document the effectiveness of EU investments when combined with policy experimentation.

1

Enhancing the Impact of European Regional Development Fund and Cohesion Fund Operations through Trial-and-Adopt Impact

Evaluations and Data Ecosystem Enhancements

Call for Expression of Interest for ERDF and CF Projects

European Commission – Directorate General for Regional and Urban Policy (DG REGIO)

World Bank – Development Impact Evaluation Department (WBG-DIME)

1. Background

The Directorate-General for Regional and Urban Policy (DG REGIO), oversees the implementation of regional policy across the European Union (EU). This policy is delivered through three of the EU’s structural and investment funds, the European Social Fund+ (ESF+), the European Regional Development Fund (ERDF), and the Cohesion Fund (CF). The ERDF focuses on making Europe and its regions more competitive, smarter, greener, connected, and socially inclusive, while a significant portion of CF’s allocation is directed towards climate objectives. The ESF+ is managed by the EC Directorate-General for Employment, Social Affairs and Inclusion (DG EMPL) and supports initiatives in employment, social policies, education, skills development, structural reforms, and socio-economic recovery. DG REGIO encourages evidence-based programming, promotes innovation and experimentation, and supports actions that strengthen the monitoring and impacts of its programs. DG REGIO has established a strategic partnership with the World Bank’s Development Impact Department (DIME, see Annex 1) to build capacity and identify opportunities to bolster the effectiveness of EU cohesion policy investments through improving the use and integration of data, scientifically trialing alternative program design options and implementation modalities to test innovations and learn what works best in practice, and rigorously documenting impacts. As part of this partnership, some ERDF and CF projects will be supported on a pilot basis to embed trial-

and-adopt impact evaluations and facilitate data system integration to better track outcomes and

maximize impacts. The pilot will focus on the initial training and design stages of the “Trial-and-Adopt”

process (see Annex 1 and 2).

2

2. Purpose of the call DG REGIO and the World Bank are looking for projects funded by ERDF and CF that are interested in collaborating to embed trial-and-adopt impact evaluation pilots in their roll-out and enhance their monitoring data systems. The call is open to all ERDF and CF projects targeting, for example, individuals or households and addressing priority policy areas such as green transition (energy efficiency), mobility, digitalization/data/AI for service delivery, and services for people (including health, education, and skills). Each Member State is welcome to propose one or several projects for the workshop. Projects targeting firms can refer to the call under the complementary private sector engagement The European Regional Competitiveness Policy Lab Here.1 Following the call for expression of interest, up to 8 projects will be invited to a one-week workshop to design a trial-and-adopt impact evaluation strategy. The workshop is expected to take place from May 19 to May 23, 2025. The workshop’s objective will be to train selected ERDF /CF program management and monitoring and evaluation teams on methodologies to design and implement impact evaluations, provide the latest actionable evidence in the priority policy area, and develop a prospective trial-and-adopt impact evaluation strategy to be embedded in future program implementation. The impact evaluations may test multiple interventions, delivery modalities, design features, and targeting mechanisms to identify effective strategies that increase project impact. The Impact evaluations will include a measurement framework that captures outcomes along the theory of change, using existing data systems and proposed enhancements. For instance, results on short-term impacts or intermediate outcomes can help steer policy decisions and encourage program adjustments to boost medium to long-term effects. Additionally, during the workshop, teams will discuss constraints and inefficiencies in their current data systems and brainstorm options for automating and streamlining data systems to generate just-in-time evidence to monitor project outcomes and measure impact. For each project, the delegation for the workshop is expected to include 4-5 people, including a

representative from the managing authority, the project manager/coordinator or equivalent, 1 technical

staff member with in-depth knowledge of the project's design and implementation, and 1 technical staff

member responsible for monitoring data systems. Partners (such as DG EMPL, DG EAC or DG REFORM),

World Bank staff, researchers, and other technical experts will also participate.

Following the workshop, up to 4 projects will receive dedicated technical support through March 2026 to

prepare and generate full-fledged design concept notes that outline the modalities through which

Member States could embed impact evaluations in the project cycle. For those selected pilots, the WB-

DIME will assemble an impact evaluation team led by a principal investigator and with a field coordinator

and a research assistant. The impact evaluation research teams will work closely with Member States to

provide technical support, co-produce the impact evaluation designs, and ensure that the approach

defined through the collaboration is of high quality and policy relevance.

1 Please note that the two calls have different opening and closing dates.

3

In addition, these trial-and-adopt impact evaluations may be complemented by data ecosystem

assessments where projects will be selected for dedicated technical to identify potential enhancements

to the data used to monitor project impacts (See Annex 2). This would include a review of existing data

with potential linkages to project impacts, such as (i) micro-level administrative data (e.g. employment,

social security, tax, primary care, housing, etc.), (ii) geo-referenced data, (iii) micro-level survey data, and

(iv) data collected from digital and mobile technologies. The assessment will explore how existing data

could be leveraged to track outcomes over time and efficiently document intervention impacts, in

accordance with the applicable personal data protection regulations under the General Data Protection

Regulation (GDPR).

3. Selection criteria

Projects will be selected to participate in the one-week workshop based on the following criteria:

relevance, technical quality and maturity, scope, innovation and equity.

1. Relevance: The project addresses critical policy challenges and knowledge gaps in relation to key

priority policy areas such as green transition (energy efficiency), mobility, and services for people

(including health, education, and skills) and targets primarily individuals or households.

2. Technical quality and maturity: The project interventions and implementation modalities are

well-defined, with clear timelines and targets.

3. Scope: The project has the potential to enhance the effectiveness of a program that represents a

significant portion of the ERDF budget.

4. Innovation: The project tackles a relevant policy challenge through a new or untested

intervention.

5. Equity: The project benefits disadvantaged populations, promoting inclusivity.

6. Engagement and interest: The project is interested in and willing to facilitate essential processes

for designing and implementing the impact evaluation, such as granting or supporting access to existing information, and administrative data, and exploring data system integration options.

After the workshop, proposals for pilot project proposals will be selected for dedicated technical support,

based on the following additional criteria: impact evaluation design quality, implementation feasibility,

data access and availability, and scalability for broader application. Support budget availability and needs

per operation may also be considered and will co-determine how many projects can ultimately be

supported.

4

4. Submission details

The call for expression of interest is open from the 10th of February 2025, 18:00 CET. Interested managing and implementing authorities or beneficiaries (in agreement with the Managing authority) can submit their expression of interest (EoI) through the application form on this THIS LINK 2 by the deadline on Tuesday the 25th of March 2025, 18:00 CET.

Representatives of interested public authorities/agencies can request further information by contacting

Tillmann Heidelk ([email protected]) and Ghida Karbala

([email protected]).

2 https://survey.wb.surveycto.com/collect/eoi_dg_regio_final?caseid=

5

Please complete file this form electronically here: THIS LINK

1. Information about the Applicant

1.1 Member State

1.2 Name of managing and implementing authorities or beneficiaries submitting project

1.3 If beneficiary, indicate contact person at the Managing authority

Name, Last Name

Job title

Phone (e.g. +32-000000000)

Email

1.4 Contact person for this application Name, Last Name

Job title

Phone (e.g. +32-000000000)

Email

2. Project description

2.1 Name of project

2.2 Brief description of the project (Objectives, target population, intervention components, implementation modalities, etc., …)

(no word limit; suggested length – 300-500 words)

2.3 (optional) Additional documents with project information

(optional; you can upload additional project documents here)

2.4 Amount of ERDF funding allocated to this project

2.5 What is the planned implementation period for this project and its current status?

(please describe the planned project timeline, and if

the project is at the design stage, approved,

whether it has already started, …)

2.6 Please provide any available information on the project's geographic scope (covered regions, municipalities, etc.) and target

6

number of participants (for example number of individuals, households)?

2.7 What are the main potential impact evaluation questions you would like to answer for this project?

2.8 Is there another agency involved in the implementation of the program?

(select one answer)

Yes

No

2.9 What are the main other agencies involved in the implementation of the program

2.10 Main contact person at the agency leading the implementation of the program?

Name, Last Name

Job title

Phone (e.g. +32-000000000)

Email

3. Data Linkages

3.1 Administrative data collected by government information systems can enable low-cost evaluation of program impacts in real-time without requiring additional survey data collection. Do administrative datasets exist for tracking program participants such as individuals or households in your program(s)?

If yes, please name and in as far as possible briefly describe potential data sources. Do you already have access to these datasets?

If not, are you willing and able to facilitate access for the purpose of the impact evaluation?

(select one answer)

Yes

No

(please briefly describe potential data sources that could be explored, max 300 words)

Please describe any potential obstacles/hindrances to accessing these datasets.

7

Annex 1: WB Development Impact Group (DIME)

In the last twenty years, the World Bank’s Development Impact Group (DIME) has demonstrated that investing in, and building local capacities for, data, impact evaluation and artificial intelligence can help double the impact of public policy investments. DIME works with governments and international organizations to deliver data products and impact evaluations that increase the effectiveness of public policies. Its mission is to foster evidence-based and data-driven planning, programming, and implementation throughout the policy cycle. DIME’s approach to establishing and delivering joint impact programs delivers at the highest standards of transparency, quality, and rigor in generating evidence actionable for policy.

Figure 1 provides an overview of DIME’s “Trial-and-Adopt” Impact Evaluation approach, broken down into an iterative workflow that continuously supports evidence-based decision making. Each step in the workflow involves close coordination with policymakers and program implementers to directly shape public policy based on rigorous evidence. The partnership between WB-DIME and DG REGIO will tailor this approach to embed it into a subset of ERDF operations on a pilot basis.

Figure 1. DIME’s Approach to Inform Policy

Collaboratively, DG REGIO and the WB-DIME seek to transform ERDF design and implementation to become more evidence-based and results-driven. The partnership involves a pilot focused on the initial training and design stages of the “Trial-and-Adopt” process. The pilot phase will provide opportunities for capacity-building, and joint learning, and will inform the preparation of the framework that will help mainstream the use of experimentation in future funding cycles (2028- 2034). It will also better outline how to link improvements in the monitoring data ecosystems and complementary data to enhancements in policy effectiveness. The Trial-and-Adopt impact evaluation designs will outline what it would take to implement the designs, both in terms of operational implications and resources, for the Member States to assess how to move to implementation. By emphasizing continuous learning, administrative efficiency, and a learning-centric culture, this partnership strives to identify opportunities to strengthen the effectiveness of the ERDF and its innovation components. The initiative will be built on co-creation with Member States opting to participate in the pilot. External researchers will be engaged as partners in the delivery of activities in collaboration and coordination with World Bank and DG REGIO staff.

8

ANNEX 2: Data Integration With the goal of enhancing and automating the process of project monitoring, DIME will analyze

opportunities to strengthen linkages between the DG REGIO monitoring system and

disaggregated data from Member States such as (i) administrative data (e.g. employment, social

security, tax, primary care, housing, etc.), (ii) geo-referenced data, (iii) micro-level (e.g. survey)

data, and (iv) data collected from digital and mobile technologies.

During the workshop, teams will discuss constraints and inefficiencies in their current data systems and brainstorm options for automating and streamlining data systems to generate just- in-time evidence to monitor project outcomes and measure impact. Following initial discussions with Member States and DG REGIO at the workshop in May (19-23), 2-4 Member States will be selected for a detailed case study. The cases studies will illustrate the types of data linkages that can be established and how these linkages can highlight regional variation for key convergence indicators and improve the monitoring of developments in regional convergence. Additionally, they will show how these linkages can be used to document the effectiveness of EU investments when combined with policy experimentation.