IMF Working Papers

Enhancing IMF Economics Training: AI-Powered Analysis of Qualitative Learner Feedback

By Andras Komaromi, Xiaomin Wu, Ran Pan, Yang Liu, Pablo Cisneros, Anchal Manocha, Hiba El Oirghi

August 2, 2024

Download PDF Order a Print Copy
Order a Print Copy

Preview Citation

Format: Chicago

Andras Komaromi, Xiaomin Wu, Ran Pan, Yang Liu, Pablo Cisneros, Anchal Manocha, and Hiba El Oirghi. Enhancing IMF Economics Training: AI-Powered Analysis of Qualitative Learner Feedback, (USA: International Monetary Fund, 2024) accessed November 21, 2024

Disclaimer: IMF Working Papers describe research in progress by the author(s) and are published to elicit comments and to encourage debate. The views expressed in IMF Working Papers are those of the author(s) and do not necessarily represent the views of the IMF, its Executive Board, or IMF management.

Summary

The International Monetary Fund (IMF) has expanded its online learning program, offering over 100 Massive Open Online Courses (MOOCs) to support economic and financial policymaking worldwide. This paper explores the application of Artificial Intelligence (AI), specifically Large Language Models (LLMs), to analyze qualitative feedback from participants in these courses. By fine-tuning a pre-trained LLM on expert-annotated text data, we develop models that efficiently classify open-ended survey responses with accuracy comparable to human coders. The models’ robust performance across multiple languages, including English, French, and Spanish, demonstrates its versatility. Key insights from the analysis include a preference for shorter, modular content, with variations across genders, and the significant impact of language barriers on learning outcomes. These and other findings from unstructured learner feedback inform the continuous improvement of the IMF's online courses, aligning with its capacity development goals to enhance economic and financial expertise globally.

Subject: Artificial intelligence, Automation, Information systems and platforms, Machine learning, Technology

Keywords: Application of artificial intelligence, Artificial intelligence, Asia and Pacific, Automation, Europe, IMF Economics training, Information systems and platforms, Learner feedback, Learner satisfaction, Machine learning, Measuring training impact, Middle East and Central Asia, Sub-Saharan Africa, Western Hemisphere

Publication Details

  • Pages:

    37

  • Volume:

    ---

  • DOI:

    ---

  • Issue:

    ---

  • Series:

    Working Paper No. 2024/166

  • Stock No:

    WPIEA2024166

  • ISBN:

    9798400286124

  • ISSN:

    1018-5941

You May Also Be Interested In...

IMF

The Managing Director's Global Policy Agenda, Annual Meetings 2024: Secure A Soft Landing And Break From The Low Growth–High Debt Path

October 24, 2024

Development Committee: The Managing Director's Written Statement

October 22, 2024

What Can Artificial Intelligence Do for Stagnant Productivity in Latin America and the Caribbean?

October 11, 2024

Implications of Central Bank Digital Currency for Monetary Operations

October 4, 2024