1. Introduction

2. The Data Quality Assessment Framework

3. Refinements to the DQAF

4. The DQAF at the Center of the DQP

5. Priorities for Future Work

Text Boxes

1. Data Quality Program (DQP)
2. Helping to Define a Road Map for Statistical Development

Appendix

I. Introduction

1. The Data Quality Assessment Framework (DQAF) was developed to address the Executive Board's interest in data quality as expressed during the December 1997 discussion of the Progress Report on the Provision of Information to the Fund for Surveillance.¹ This interest was reaffirmed at the Third Review of the Fund's Data Standards Initiatives in March 2000 and during the discussion of Data Provision to the Fund for Surveillance Purposes in June of that year. A paper outlining the DQAF was presented to the Executive Board in July 2001, as background to the discussion on the Fourth Review of the Fund's Data Standards Initiatives. The Executive Board welcomed the DQAF and supported its integration into the data module of the Report on the Observance of Standards and Codes (ROSC). The Board also endorsed the integration of the various applications of the DQAF in an overall data quality assessment program.

Data Quality Assessment Framework-Generic Framework

2. This Supplement presents an overview of the DQAF and its role in the data quality program (DQP).² Section II summarizes the DQAF; Section III briefly describes a set of refinements to the DQAF. Section IV outlines the role of the DQAF in the DQP, and Section V sets out the priorities for the work ahead under the DQP.

II. The Data Quality Assessment Framework

The DQAF provides a structure for assessing data quality by comparing country statistical practices with best practices, including internationally accepted methodologies. Rooted in the United Nations Fundamental Principles of Official Statistics,³ it is the product of an intensive consultation with national and international statistical authorities and data users inside and outside the Fund. It focuses on the quality-related features of governance of statistical systems, core statistical processes, and statistical products. Under the DQAF, assessments have a six-part structure starting with a review of the legal and institutional environment (prerequisites of quality) and followed by an analysis of five dimensions of quality. The DQAF has a cascading structure, moving from the dimensions common to all datasets, as captured in the Generic Framework, to the more detailed aspects appropriate to individual datasets in the dataset-specific DQAFs. 3.

4. As intended, the DQAF has proved to be valuable for at least three groups of users:

• the Fund-strengthening its operational work through assessments of data quality included in data ROSCs. These assessments help strengthen IMF surveillance and program design, e.g., by raising the profile of institutional weaknesses in discussions with country authorities. As well, they provide a sound basis for consideration of technical assistance (TA) needs and design of TA programs;

• country authorities-enabling the self-assessment of statistical systems. Such assessments encourage preparation of well-targeted improvement plans that could help garner donor support as needed; and

• private and public data users-providing an overview of the dimensions that make up data quality and equipping users to gauge data quality for their own purposes.

III. Refinements to the DQAF

5. The DQAF has been refined to reflect experience and international statistical developments since the Fourth Review of the Fund's Data Standards Initiatives. These experiences and developments include the following:

• good statistical practices identified through data ROSCs and developments in the Special Data Dissemination Standard and the General Data Dissemination System;

• methodological improvements made at both the international level (e.g., Government Finance Statistics Manual 2001) and the regional level (e.g., European Union guidelines); and

• work with other international organizations on harmonizing approaches to data quality.

6. The refinements have been made to fill some gaps at the most detailed level and to address overlapping coverage in a few areas. Most refinements entailed introducing greater precision in the description of good statistical practices (e.g., separate assessment of the adequacy of staff resources, facilities, and computer resources). Drawing from the range of countries' experiences, more examples were cited. Greater attention was given to the effectiveness and efficiency of the management of statistical processes. In addition, some elements of the DQAF were reordered (e.g., "relevance" shifted from serviceability to prerequisites of data quality) or renamed ("integrity" renamed "assurance of integrity"). These refinements have been incorporated in the Generic Framework presented in the Appendix.

IV. The DQAF at the Center of the DQP

7. The DQP comprises a set of well-integrated initiatives centered on the DQAF.⁴ As shown in Box 1, it comprises applications of the DQAF, projects to support and promote good statistical practices identified in the DQAF, and the maintenance and development of dataset-specific DQAFs.

8. In addition to its use in data ROSCs, the DQAF has been applied in the Fund's statistical TA program and in the statistical capacity building indicators developed under the auspices of the Partnership in Statistics for Development in the 21^st Century (PARIS21).⁵ The statistical capacity building indicators provide a snapshot view of the statistical system of a country with a set of 18 DQAF-based qualitative indicators and 16 quantitative indicators. For Fund TA, recent experience points to the key role of the DQAF in enhancing the prioritization and effectiveness of TA. In this setting, some data ROSC missions have provided diagnostics leading to TA, and others have validated statistical improvements resulting from earlier TA (Box 2). Also, data ROSC/TA missions have facilitated countries' subscription to the SDDS or participation in the GDDS.

Box 1: Data Quality Program (DQP) The DQP comprises a set of well-integrated initiatives, centered on the DQAF, that permit the identification, promotion, and assessment of good statistical practices. Applications of the DQAF Data ROSCs TA programs and their evaluation PARIS21 statistical capacity building indicators Countries' self-assessment and peer review Users' guide to DQAF applications (continuing) Support for good statistical practices the DQAF Data Quality Reference Site (http://0-dsbb-imf-org.library.svsu.edu/Applications/web/dqrs/dqrshome/) Development of guidance on specific practices Compendium of good statistical practices (future work) Maintenance and development of dataset-specific DQAFs Currently used National accounts Consumer price indices Producer price indices Government finance statistics Monetary statistics Balance of payments Jointly prepared with the World Bank Consumer expenditures under poverty Under development with other international organizations External debt statistics (with the Inter-Agency Task Force on Finance Statistics) Merchandise trade (with the Task Force on International Trade Statistics) Education statistics (with the World Bank and the United Nations Educational, Scientific and Cultural Organization) Labor statistics (with the International Labour Office)

Box 2: Helping to Define a Road Map for Statistical Development Two key elements of the Data Quality Program-data quality assessments included in data ROSCs and provision of TA-reinforce each other in helping to define a road map for statistical development. Turkey is an example of a country where a data ROSC helped direct and prioritize TA. A data ROSC in October 2001 confirmed that data dissemination was in observance of the SDDS (Turkey had subscribed in 1996), but identified shortcomings. In particular, the ROSC mission saw scope for updating the methodological basis of the national accounts, improving the accuracy and reliability of the price indices, and redressing the fragmentation of responsibility for fiscal accounting and enhancing the coverage and timeliness of the fiscal statistics. Following up on the ROSC recommendations, in April 2002, a TA mission assisted the State Institute of Statistics in revising the national accounts, including a rebasing of the constant price estimates through 1996. In July 2002, a TA mission reviewed progress on the implementation of new accounting and reporting arrangements for fiscal data, and also provided input on the draft law on public financial management and financial control. In September 2002, a TA mission assisted in preparation for the planned revision of the price indices. Kazakhstan is an example of a country where extensive TA helped establish sound statistical practices, which were subsequently validated by a data ROSC mission. Prior to independence (1991), the statistical system supported central planning. In this context, relatively few data expressed in monetary terms were collected. Data on the balance of payments, external debt, and national accounts were not available, and shortcomings affected external trade, price, monetary, and fiscal statistics. With TA from the Fund and others, the new authorities addressed the challenge of transiting to a statistical system consistent with the data needs of a market economy. From the early 1990s, a sustained TA program focused on developing the institutional framework appropriate to the needs of a market economy and establishing procedures for collecting and compiling macroeconomic statistics. In assessing Kazakhstan's statistical system, a data ROSC in 2002 concluded that the quality of its macroeconomic statistics had improved significantly. The ROSC mission noted that the authorities had a good track record of implementing TA recommendations. In March 2003, Kazakhstan became the first GDDS participant to graduate to the SDDS.

9. Data ROSCs and related TA have identified areas for priority work to provide the international infrastructure to support countries' efforts towards strengthening their statistical capacity. For example, ROSCs confirmed that, despite the critical importance of revisions of data, statistical agencies are often less than transparent in providing information about causes of revisions and the schedule of revisions. The May 2002 Executive Board discussion of Data Provision to the Fund for Surveillance encouraged national authorities to articulate their policies on revisions, and revision practices were also an issue in the June 2003 Board discussion on Strengthening the Effectiveness of Article VIII, Section 5. STA staff is nearing completion of a paper that, drawing on country experiences, proposes a set of good practices for revision policies. STA will invite international comment, including at an upcoming meeting of the heads of statistical units of international organizations, as a step towards an internationally agreed set of practices to guide countries.

10. Dataset-specific DQAFs are used in ROSCs for six datasets: national accounts, consumer price indices, producer price indices, government finance statistics, monetary statistics, and balance of payments. The success of the approach is evidenced by other international agencies' interest in developing additional dataset-specific DQAFs. In collaboration with the World Bank, a DQAF has been prepared for consumer expenditures under poverty. Four more are underway in collaboration with other international agencies (see Box 1).

V. Priorities for Future Work

11. The DQAF-based work within the DQP will proceed on three fronts: TA and statistical capacity building efforts, promotion of good statistical practices, and collaboration with international organizations.

TA and capacity building:

• guiding countries' statistical plans and related TA requests, including coverage of legal and organizational matters; and

• promoting the PARIS21 statistical capacity building indicators to serve for use by countries for self-assessment and peer review.

• developing a users' guide to DQAF applications

Good statistical practices:

• identifying and promoting good statistical practices in key areas, including completion of the work on revision policies; and

• producing a compendium of good statistical practices drawing from data ROSCs.

Collaboration with international organizations:

• harmonizing the DQAF and the related glossary with the quality frameworks emerging in other international organizations (e.g., Eurostat and the OECD);

• pursuing work on methodological guidelines with supranational and regional bodies, including harmonization of guidelines, notably for the more recent guidelines such as the Government Finance Statistics Manual 2001 and the Monetary and Financial Statistics Manual (2000); and

• extending the DQAF to socio-demographic datasets to buttress the GDDS' socio-demographic component and to support statistical development in areas relevant to countries' poverty reduction strategies.

APPENDIX

Data Quality Assessment Framework-Generic Framework
(July 2003 Framework)