August 16, 1999
Prepared by IMF staff

Contents

1. Introduction and Key Issues
2. Preparation of Second Round Case Studies
3. The Outreach Program
4. Issues Arising from the Case Studies and the Outreach Program
1. Approach to preparing reports on the implementation of standards
2. Content of assessments - disclosure versus quality
3. Response of participating Fund members to the case studies
4. Collaboration with other institutions
5. Range of standards covered
6. Resource implications
5. Further Areas for Guidance
1. Phased introduction for reports on the implementation of standards?
2. Should reports on the implementation of standards be published?
6. Issues for Discussion

Text boxes

1. Executive Board Reaction to First Round Case Studies
2. The SDDS and the GDDS as Benchmarks for Preparation of Reports on Standards
3. Reports on Standards and Other Architecture Initiatives
4. Disclosure, Definition and Quality
5. Peer Review as an Approach to the Assessment of Data Quality

Text Table

1. Second Round Reports on Standards

1.  Developments in the world economy over the last two years have led to a consensus on the importance of strengthening the architecture of the international financial system. Efforts to address vulnerabilities in domestic and international financial systems have taken a number of concrete forms. Many national supervisory agencies have enhanced regulatory oversight of their financial sectors; the Basle Committee has reviewed gaps in the existing set of supervisory standards, developed standards for banks' interactions with Highly Leveraged Institutions (HLIs), and released a discussion paper on updating the 1988 Capital Accord; agreement has been reached on the OECD Principles of Corporate Governance, and the World Bank and the OECD have decided to establish the Global Forum on Corporate Governance; the Financial Stability Forum has been established and has commenced consideration of the impact on financial system stability of capital flows, offshore centers and HLIs; and the World Bank and the Fund have taken important steps to enhance their collaboration and coordination, including through jointly conducted financial sector assessments (the Financial Sector Assessment Program - FSAP).

2.  A theme common to much of the discussion on strengthening the architecture of the international monetary system has been the need to develop and implement internationally recognized standards and codes of good practice.¹ The development and adoption of standards in areas central to the effective operation of domestic and international financial systems holds the promise of better informed lending and investment decisions, increased accountability of policy makers and better policy making and, ultimately, improved economic performance.

3.  The Fund has played a major role in developing and disseminating standards and codes in areas central to its direct operational focus — the core areas of data dissemination, transparency in fiscal policy, monetary and financial policy transparency, and banking supervision: the Special Data Dissemination Standard has been strengthened, notably in the area of external debt and international reserves; efforts are underway to assist members to implement the Code of Good Practices on Fiscal Transparency; the Code of Good Practices on Transparency in Monetary and Financial Policies was approved by the Executive Board on July 9, 1999 for transmittal to the Interim Committee, and a supporting guide to its implementation is under preparation; and work to assess specific standards is also proceeding in a number of new initiatives as part of broader assessments of vulnerabilities and reform needs, including Financial Sector Stability Assessments (FSSA) and technical cooperation action plans (TCAP). Progress also continues to be made by other standard-setting bodies in areas which, although critical for the effective operation of economic and financial systems, lie outside the Fund's direct operational focus. Standards for the so-called "non-core" areas of securities and insurance regulation, corporate governance, and accounting and auditing have all been developed further in recent years.

4.  While progress in developing and disseminating standards is welcome, standards need to be implemented to be effective. In discussing International Standards and Fund Surveillance: Further Issues, hereafter referred to as Further Issues, the Executive Board indicated that "although the main incentive to implement standards must come from the private sector, the official sector could help strengthen the incentives to adopt standards."² Most Directors considered that some form of monitoring of countries' observance of recognized standards could play a useful role to that end (see Box 1).

5.  The focus of this paper is on progress in developing the modalities of producing what the G-22 and the G-7 termed "transparency reports". These reports were intended to summarize "the degree to which an economy meets internationally recognized disclosure standards".³ With the support of the Executive Board and the involved member countries, staff have been preparing a second round of experimental case studies to help highlight the practical and conceptual issues involved in preparing such reports. One premise of this work, based on the reaction by the Executive Board to the first round of case studies, is that disclosure is not synonymous with transparency. In other words, for information to be useful, there must be some appreciation of the integrity and underlying quality of what is being released.

6.  In this respect, it has become increasingly apparent that the term 'transparency report' fails to adequately capture the essence of these reports. Given their ultimate status as an assessment of countries' disclosure practices as well as their observance of standards, something along the lines of 'Report on the Implementation of Standards and Codes' (RISC) would appear to be a more accurate title and this name is used for the purposes of this paper. For ease of reference, this will be abbreviated to 'Reports on Standards' or 'Reports', and reference to 'standards' will cover both standards and codes.

7.  While the case studies were being prepared, thinking in the international community has also continued to evolve. In April 1999, the Interim Committee called on the Fund to develop its surveillance "process to encompass the codes and standards relevant to international financial stability," and "encouraged the Fund to use transparency reports on a trial basis as part of its surveillance."⁴ Finance Ministers of the G-7 countries endorsed this position, calling for "a system for surveillance of implementation of the codes and standards" and the "systematic incorporation of information on a country's observance of transparency standards in the Fund's regular Article IV surveillance reports as well as in special reports on country transparency practices prepared by staff."⁵

8.  To some extent, the recommendation of the G-7 recognizes developments that have been underway for some time. Assessments of the implementation of standards have begun to play an increasingly significant role in surveillance (and program design), a view shared by Executive Directors in March, 1999 when they indicated that "it is important for the Fund, as part of its surveillance activities, to understand thoroughly country practices in areas of direct operational concern ... the core areas." However, a number of Directors also noted that countries' "practices in the areas of accounting, auditing, bankruptcy, corporate governance and the like have had an increasing bearing on issues arising in Fund surveillance and Fund-supported programs ... and that, in order to conduct effective surveillance ... it was important for the Fund to have a sufficient understanding of practices in these areas ..."⁶

9.  This evolution in thinking about reporting on standards and transparency practices has raised a number of important questions, including: 1) the uses to which judgements on countries' implementation of various standards are to be put; 2) the scope of assessments, namely the coverage of non-core areas; 3) the role of the Fund and other institutions in the establishment, assessment and monitoring of implementation of standards; 4) the role of standards in the Fund's surveillance process; 5) the form which assessments of implementation of standards might take. These issues are discussed in this paper.

10.  Amplifying on the first question, the proposed use of assessments has not stopped at simply being part of the surveillance process. Recent developments indicate that judgements regarding implementation of certain standards may be used increasingly to categorize countries, or institutions within countries, for various purposes. For example, the criteria for access to the Fund's own Contingent Credit Lines (CCL) include:

"taking into account the extent of the member's adherence to relevant internationally- accepted standards; in particular, the member would have subscribed to the Special Data Dissemination Standard (SDDS) and be judged to be making satisfactory progress toward meeting its requirements."⁷

11.  Similarly, the Basle Committee's discussion paper on revising the capital adequacy framework to replace the 1988 Accord proposes that subscription to the SDDS be a requirement for a sovereign borrower to be eligible for a risk weighting below 100 percent, and that no bank could have a risk weight less than that applying to its sovereign.⁸ The Basle Committee is also considering recommendations aimed at putting pressure on jurisdictions which are not making adequate efforts to implement international standards on the information exchange required between home and host supervisors.

12.  Finance Ministers from the G-7 went further, indicating that G-7 regulators should consider whether a foreign bank's home country adheres to international standards as part of prudential criteria considered when evaluating if the foreign bank should be allowed entry to their market. In addition, the G-7 recommended that the International Organization of Securities Commissions (IOSCO) and the Basle-sponsored working groups "make membership in their bodies contingent on progress towards implementation of international standards."⁹

13.  These proposals suggest not only that incentives for adoption of standards are increasing, but also that the Fund is seen at the center of such efforts. This raises difficult issues for the Fund (and for other institutions associated with standards). In particular, it involves an operational focus, and a specificity in judgement on the implementation of standards, that were not anticipated at the time the Fund considered its role in monitoring standards in
July, 1998¹⁰ , and were still not anticipated when the Executive Board considered Further Issues in March, 1999. It also implies that judgements regarding the degree of implementation of certain standards by a particular country may have direct and tangible implications, not only in affecting financial market spreads, but also if used to categorize countries in terms of access to markets, Fund resources, and membership in international groups.

14.  Regarding the publication of reports on standards, the G-22 and others have made a strong case that publication be mandatory, and staff believe that publication would be in the best long-term interests of members. However, it can be anticipated that some members would disagree with the merits of publication of such documents. Moreover, publication may raise the same issue that arises in a number of other contexts: whether the Fund can act as confidential policy advisor while simultaneously publishing its judgements of members' policies.

15.  Difficult issues are also raised by the suggestion that reports cover standards beyond the Fund's areas of direct operational concern and expertise. To be most effective, the range of standards covered should be relatively comprehensive. In practice, this would face a number of complex hurdles. First, standards in the non-core areas are at very different stages of development, and some do not lend themselves easily to assessment. Second, ways would need to be found to ensure that other bodies, particularly the World Bank, participate in a meaningful fashion and in a manner which fosters genuine ownership of elements of reports they co-produce.

16.  The evolving nature of reports on standards, and the uses to which they might be put, have posed challenges for staff in determining the resource implications that may be associated with work on standards as well as preparing reports on standards. However, with guidance from the Board on the issues raised in this paper, it should be possible to proceed with further work using a relatively common approach across countries, but allowing for some flexibility in approach across standards, to more accurately calculate the resource implications associated with assessing implementation of standards, and to better integrate the preparation of assessments with other work underway at the Fund.

17.  Finally, in preparing the case studies, it has been necessary to consider what sort of assessments should be made. Each standard consists of a number of different elements, each of which requires a separate judgement as to whether a country is in observance. These elements effectively provide a benchmark from which an assessment can be made, although the ease of making an assessment varies across the range of standards and codes. In preparing assessments on a member's implementation of a given standard, staff need to be prepared to make qualitative judgements where desirable and feasible.

18.  In view of the different nature and dimensions of each standard, it would be difficult, if not impossible, to assign a member a "pass or fail" with respect to implementation of a given standard. The notion of assigning a grade could also be counter-productive if it led to resistance by members to being assessed. In addition, some of these standards were designed with the understanding that they would be voluntary, and a grading system might violate the spirit in which these standards were established. The most promising approach would be to characterize a country's effort by describing those elements that have been met and those that have not, combined with a description of the progress that has been achieved over time. It would provide the users of the assessments with a better perception of strengths and weaknesses and of the member's commitment to achieving further improvement. It would also reflect the call from the Board for assessments of the implementation of standards to be set in an appropriate context for individual countries. In other words, it would allow for consideration of the different stages of economic development, the range of administrative capacities, and the different cultural and legal traditions across the membership.

19.  The first series of experimental case studies exploring the modalities of preparing and publishing reports on standards were completed and circulated as background documents for the Executive Board's consideration of Further Issues in March, 1999. The present paper addresses Directors' requests emanating from that discussion. Specifically, the paper:

• draws lessons from the preparation of the first and second rounds of case studies; and

• describes the results so far of a small outreach program to selected potential users in the private sector, and to other standard setting bodies, on the value, form and content of these reports.

20.  The paper draws on the experience so far of the case studies and the outreach to:

• suggest that there are potentially both large benefits and costs associated with the preparation of reports on standards, although these are difficult to quantify with precision given the still-evolving nature of the exercise;

• propose that further work on assessing implementation of standards proceed using modules in a selective manner and aim to better define the concept, form and content of reports; refine the linkages with other architecture initiatives; consider further how to bring other standard-setting bodies and international organizations into the process; and better assess the organizational issues for the Fund; and

• outline a series of issues on which the guidance of the Executive Board is needed in the event the Board decides to continue with work on assessing the implementation of standards.

21.  Staff plan to circulate a supplement to the paper closer to the time of the Board discussion with detailed estimates of the resource costs associated with the preparation of the case studies and estimated costs of preparing reports on a more general basis.

22.  The second round of experimental case studies has continued to explore how reports on standards should be prepared. The selection of countries for the second round has also sought to accommodate the Board's request to broaden the sample of countries from the first round (Argentina, Australia and the United Kingdom) to include economies at various stages of development and with varied experience in the implementation of standards.

23.  Experimentation in the approach to preparing reports has also continued. An effort to prepare large numbers of reports would result in significant logistical and coordination difficulties if all standards had to be assessed simultaneously. These difficulties would be exacerbated if other agencies needed to be involved and their activities coordinated with those of the Fund. In addition, the second round studies suggest that some members may also confront serious difficulties freeing sufficient resources to deal with the simultaneous assessment of many standards.

24.  The most logical response to these issues is to consider the assessment of the implementation of standards as conceptually akin to a loose-leaf binder comprised of modules, or individual chapters, for each standard. While the Argentine and United Kingdom reports comprised both core and non-core modules, it may be equally effective to prepare the modules over a period of time, focussing in the early phase on key standards relevant to each case. The preparation of any given module could serve as a step towards the compilation of assessments of all relevant standards, if that was so desired.

25.  The most recent case studies consist of five reports covering all core and some non-core standards for Hong Kong SAR¹¹ , Bulgaria, the Czech Republic, Tunisia and Uganda, and individual modules for Bangladesh, Ukraine, Cameroon, and Colombia (Table 1 provides more detail).

26.  The five reports consist of staff assessments in at least the four core areas of data dissemination, fiscal transparency, monetary and financial policy transparency (including payments systems and deposit insurance) and banking supervision. In addition, an assessment of the transparency of the securities market was prepared for the Czech Republic, while the banking supervision sections for Bulgaria and the Czech Republic assess actual observance of the Basle Core Principles in addition to the disclosure aspects. Descriptive information on country practices in non-core areas was also provided in the case of Hong Kong SAR, Uganda and Bulgaria.

27.  In light of time and resource constraints, the countries approached to participate in the preparation of individual modules were those where the functional departments in the Fund had pre-existing work programs in place covering SDDS/GDDS issues, FSSAs, and case studies on the implementation of the fiscal code. As a result, staff prepared a data dissemination module for Bangladesh, fiscal transparency modules for Cameroon and the Ukraine, and both banking supervision and monetary and financial policy transparency modules for Colombia. The banking supervision module in the latter case also assesses observance of the Basle Core Principles.

28.  Area departments coordinated the preparation of the individual second round country studies, which were prepared jointly by the area and functional departments. Reports were subject to review by all concerned departments. As with the first round case studies, all studies were prepared with the cooperation of the relevant country authorities, who had the opportunity to comment on the draft reports.

29.  The second round of case studies also provided an opportunity to experiment with World Bank involvement in the preparation of reports on standards. The role of World Bank staff has varied across countries. In Tunisia, in view of the World Bank's extensive involvement in banking restructuring, Bank staff prepared the first draft of the modules on the Transparency of Monetary and Financial Policies, and the transparency elements of the Basle Core Principles. In the Czech Republic, the Fund worked closely with the Bank in preparing the chapter on transparency of the securities market. The World Bank was also involved in the preparation of the joint Bank-Fund FSAP report for Colombia, which subsequently formed the basis of the modules on banking supervision standards and monetary and financial policy transparency standards for Columbia.

30.  The modules contain a description of country practices prepared by Fund staff on the basis of questionnaire results and a dialogue with the authorities, followed by an assessment of these practices against the relevant international standards. For the Czech report, there was a significant element of self assessment in the fiscal transparency, monetary and financial policy transparency, and banking supervision sections. However, while the questionnaire responses provide elements of a self-assessment, no country prepared a self-assessment report along the lines of the one produced for Australia. This issue is addressed further in Section IV.

31.  During the March Board discussion of Further Issues, Directors encouraged staff to undertake an outreach program to solicit feedback from the private sector and regulatory authorities on the value and form of the experimental case studies. Consequently, the staff made first round reports available on the Fund website and contacted a small number of potential private sector users of reports on standards in the financial markets and the investment industry. After consulting with the World Bank, staff also wrote to the OECD and the following standard-setting bodies: IOSCO, IASC, IFAC, and IAIS.¹²

32.  These groups were asked whether, in general, they found the information presented in the case studies useful and, if so, whether the reports should provide information on the definition and quality of standards in addition to the disclosure aspects. In addition, staff sought the views of private sector participants on which standards should be given priority, while the institutions were asked if they had made any progress in determining how assessments of their own standards might be carried out and incorporated into the reports.

33.  The response so far of both private sector participants and regulatory authorities has been generally favorable, although the reaction of each group should be considered separately in view of the different perspective each brings to the table. Considering first the private sector's views, these should be tempered by the fact that market agents are, in general, likely to respond positively to any offer of free information, regardless of its strengths or weaknesses. Keeping this in mind, most private sector respondents indicated that the reports provided them with useful information and served as a good reference document. They particularly liked the IMF Staff Commentary and felt strongly about the need for an independent assessment for purposes of credibility and objectivity. Some indicated that the Staff Commentary should go deeper, including, for example, more information on areas needing improvement. All respondents liked the emphasis on the IMF's core areas, while a few, including those interested in equity investment, expressed an interest in the non-core areas. They also considered that the reports should assess the quality of information provided and the basis on which it is compiled, not simply the disclosure practices.¹³

34.  A few private sector respondents were less enthusiastic about the value of the case studies in contributing to their daily task of assessing risk, in part because the studies did not analyze sufficiently the definition and quality of the information being provided, particularly as related to economic and financial data. Nevertheless, these respondents viewed the general promulgation of standards and the advancement of policies which encouraged countries to improve transparency and implement standards as important steps to improving the quantity and reliability of information. In this regard, they appreciated that an international framework to promote and assess the implementation of standards was being considered and viewed this as a positive contribution to strengthening financial systems.

35.  Most of the other standard-setting bodies and the OECD reacted positively to the studies, confirming the need to devote more attention to standards. For example, the IAIS found the reports very useful as a means of disseminating information about, and monitoring observance of, their standards and requested that future reports undertake a deeper assessment of observance of their standards. Most of these bodies have developed, or are in the process of developing, a set of core principles and assessment methodologies which they would like to be used in future assessments of standards. IOSCO indicated that, while it was keen to participate in continuing discussions on the issue of standards, it was too early to respond definitively to IMF reports on standards in light of array of initiatives underway (e.g. FSAP, Basle Committee proposals) and also due to the fact that IOSCO is currently considering how to encourage and assess implementation of its own Objectives and Principles of Securities Regulation.¹⁴ While these institutions reaffirmed the need to strengthen the establishment and monitoring of standards, they were either not ready or not able to suggest how they could contribute to the exercise.

36.  Given the time constraints and the small number of experimental reports available for use as examples, the outreach program was deliberately limited in scope. Further efforts to canvas the private sector and other institutions should be undertaken in order to obtain a better appreciation for the interest in, and potential usefulness of, these reports, particularly in light of their evolving form and content.

37.  At the time of the Board discussion of Further Issues, a number of tentative lessons had been drawn from the first round of reports on standards. These included the importance of ensuring that sufficient numbers of skilled staff were available to conduct assessments; the dependence of staff on the availability and cooperation of counterparts in the respective countries; and the need to limit assessments by Fund staff to core areas.¹⁵

38.  While these lessons remain pertinent, additional experience has highlighted a number of other issues. For some of these, solutions have emerged readily from the experience preparing the case studies and staff are working towards developing ways to address the more difficult issues. Other issues have arisen, such as the reaction of members that participated in the case studies, which are simply tabled for information at this stage; more experience will be required to determine their significance. There is also a set of issues that are fundamental to the future direction of assessments of implementation of standards, but which are independent of the lessons learned from the case studies and the outreach program. This latter group is addressed in detail in Section V: Further Areas for Guidance.

39.  No country participating in the second round case studies sought to follow the two-track approach favored by many Directors in March, i.e., preparing a self assessment report of their country's implementation of all standards, along the lines of the one prepared for Australia, accompanied by a staff commentary on the appropriateness of that self-assessment (although the report for the Czech Republic was based in significant ways on their self-assessments.) While the tight deadlines imposed on the production of these reports are likely to have played a role in deterring members from this route, the novelty of the exercise may also have played a role. Members, particularly developing countries, may require more assistance and more examples of good self-assessment reports before they are willing to consider this option.¹⁶

40.  While the merits of the two-track approach — in terms of maximizing countries' ownership of the report and ensuring its credibility — are clear, at least initially the most common approach for the preparation of reports on standards is likely to be the one first used for Argentina and the United Kingdom. This approach involves a description of practice prepared by staff on the basis of the dialogue with the authorities¹⁷ followed by a staff assessment. To date, the participating authorities have also provided comments on the description and assessment prepared by staff. However, participation by some industrial country members in the two-track approach may encourage more widespread use of this approach by the membership. Accordingly, staff believe it would be desirable to continue to allow some further experimentation in the event that further case studies were to be prepared.

41.  At the time of the preparation of the Australian report, there was some sense that the preferred approach would be to marshall a small team to conduct simultaneous assessments in a wide range of areas. However, the experience of preparing the second round of studies has led staff instead to strongly favor a step-wise construction of reports using the modular approach. The modular approach would assist staff in the internal coordination of both specialist expertise and the other architecture initiatives currently under development (see Boxes 2 and 3). It would allow for greater consistency with the other work plans of the technical assistance and cooperation departments involved, including by allowing for better integration of the work on standards with the technical assistance work program. In addition, a modular approach would allow the Fund to focus its attention and resources more effectively given the differing priorities across countries e.g., at any particular point in time, transparency of monetary and financial policies may be the priority in one country while fiscal transparency may be more critical in another. The modular approach allows such priorities to be addressed.

42.  Moreover, the construction of the different standards was initiated for different reasons and has evolved over different time frames. Consequently, an assessment of each standard produces a different measure of observance in terms of its definition and quality elements. In this regard, by not imposing a unified framework on all the standards, the modular approach highlights the individuality of each standard and can thereby foster a more meaningful assessment of each standard. Indeed, one observation from the experience with preparing the case studies was that the format followed for the full reports entailed a fairly homogeneous treatment of different standards which, if not carefully handled, has the potential to produce more superficial, repetitive documents. At the same time, it should be noted that there is a need to ensure comparability of treatment for a given standard across countries.

43.  Finally, by staggering the workload associated with the preparation of a report covering a collection of standards, a modular approach is also likely to ease resource demands and permit better prioritization by both the Fund and member countries, particularly those with limited managerial and institutional capacity. It also has advantages in facilitating coordination between the Fund and other organizations should they co-produce transparency reports.

44.  A number of issues remain outstanding under the modular approach, including the appropriate time frame over which the various modules should be prepared in order to deliver a relatively comprehensive assessment of progress across the range of standards; how often modules should be updated; how to incorporate the assessments of others and how to ensure they receive appropriate recognition and visibility; and how to integrate the modular assessments into the Fund's surveillance process. These issues notwithstanding, staff believe the modular approach offers the most efficient and flexible means of assessing standards, and a potentially useful vehicle for incorporating the work of other institutions and agencies.

45.  Central to the issues confronting the Fund in monitoring standards is whether assessments should be restricted to disclosure practices per se or whether some attempt should be made to consider the quality of the information released and the basis on which it is compiled (see Box 4).

46.  If the objective of assessing standards is to contribute to better informed decisions, then monitoring must go beyond disclosure practices to address, to some extent, the substance of the information being released. Indeed, disclosure is not synonymous with transparency,¹⁸ a view shared by most Directors in March when they indicated that:

"monitoring needed to go beyond the disclosure elements for the information to be most useful; an understanding of the basis or definition on which the information was compiled was needed and, where feasible, this should also be complemented by an understanding of what mechanisms are in place to ensure the quality of the information being released".¹⁹

47.  Preparation of a report which accurately describes an economy's disclosure practices but which fails to acknowledge problems with the underlying data would be misleading and potentially harmful. The experience with preparing the second round of reports confirmed that it is difficult to sidestep the quality aspects when making judgments about the implementation of standards. In a few cases, members received positive assessments of particular disclosure practices although the data or information being released was in need of improvement. In response, staff commented on the underlying quality of the information being released wherever possible. However, treatment of quality issues is not consistent across the case studies for a number of reasons. In part, the realization of the need to address quality in the case studies came relatively late in the process. More importantly, and as noted earlier, the treatment of quality of information is addressed to different degrees in each standard.

48.  Given the different dimensions of each standard with respect to information quality, staff favor addressing the quality issue systematically and explicitly. However, this broader focus brings with it additional difficulties. In particular, it will be significantly more resource and time intensive since assessing quality aspects of standards puts a premium on information which is thoroughly understood and up-to-date.²⁰ Moreover, it needs to be made clear that addressing quality issues is not the same as auditing the information released. In other words, countries' compliance with the various codes, as written, would not give adequate assurances that the quality of data disclosed is reliable.

49.  While staff would not themselves set out to analyze the data or information released for quality issues in the course of assessing the implementation of standards, they should be able to assess whether there are systems in place to provide assurances of quality. Systems in this case could mean either institutional arrangements or alternative, publicly available, information sources that allow the user to undertake a cross-check on the quality of the data. One approach already reflected in the Code of Good Practices on Fiscal Transparency is to verify the existence of independent assurances of integrity, such as an independent national audit office with a requirement to report findings publicly to the legislature. Another alternative would be to report efforts of authorities to improve the information being released (see Box 5).²¹

50.  Another possibility would be to introduce a system of self-declaration into each standard whereby the producer of official information would provide publicly accessible statements that describe their systems and practices and how data are produced. Such an approach is built into the SDDS. Self-declaration via a well-articulated framework could provide a viable alternative for how to go about gauging quality. This is an area where further work would be desirable in order to define better what might be used as quality criteria for individual standards. Quite apart from formal assessments of quality, if staff are aware of any specific problems with the quality of information being released, this should be highlighted in the report on standards.

51.  Reactions of members invited to participate in the case studies have been mixed. The experience of preparing the first three reports was viewed positively by both the participating national authorities and Fund staff. However, a number of national authorities approached by staff for the second round case studies declined to participate. Members generally cited resource constraints as the reason for not participating, although staff also detected a greater degree of wariness among these members about the value of the exercise.

52.  This raises a serious issue about how the results of the case studies should be interpreted. The case studies likely suffer from sample selection bias, with those members opting to participate being among those most confident of receiving a generally positive assessment. There may also have been an element of selection bias on the part of staff where those members perceived as least likely to participate were not approached.

53.  Staff are interested in obtaining members' views on their experience in participating in the second round case studies and hope that Executive Directors would be able to comment on this at the forthcoming Board discussion. From staff's perspective, a number of observations are worth highlighting.

54.  First, there was resistance by some country authorities to the staff's commentary in certain areas, particularly the more critical the commentary and the more it related to the quality of information being disclosed. It is unclear whether the sensitivity related to the substance of the staff assessment or to the possibility that the assessment could be published.

55.   Second, in some cases coordination among the agencies of the participating member did not appear to run smoothly. This seems to have been most prevalent on data related issues, where anecdotal reports suggest that some of the statistical agencies most closely involved with the technical issues under consideration were not always fully involved in the preparation of the report. In other cases, national statistical agencies appear to have been involved as an afterthought. While these may have been isolated incidents associated with the novelty of the process, they highlight the need for ensuring good coordination among all participants — national authorities as well as Fund staff and other institutions conducting such assessments.

56.  Third, the extent of "ownership" of the reports on the part of the various national authorities and their officials was mixed. Most participating members saw the reports as an opportunity to improve transparency. However, in one case, the staff's counterparts were sometimes reluctant to reveal shortcomings in observance of standards for fear of superiors' reactions to such disclosures, even though the government welcomed the opportunity to have the report prepared. In two instances, there appeared to have been little understanding of the process at the officials' level. In one case, while officials cooperated in the preparation of the report, they took little interest in the results, indicating that they believed they had been "volunteered" into the case study at the behest of the Fund area department. These reactions, together with the evidence of coordination problems, suggest there is a need for an outreach program to explain what is being done, and why, in order to improve organization and foster greater ownership on the part of the authorities.

57.  Activities related to the assessment of implementation of standards are also evolving steadily at the World Bank in relation to the non-core areas. The challenge is to find effective vehicles to coordinate the work of the two institutions.

58.  As mentioned in Section II, World Bank staff participated in the preparation of a number of assessments. The Bank's input was of high quality, timely and extremely valuable, and the experience overall was generally very positive. Nonetheless, it has highlighted a number of issues. First, unless handled carefully, the involvement of several organizations in the assessment of implementation of standards could run the risk of blurring the mandates of the respective organizations, resulting in complications for both the national authorities and the staff of the institutions involved. Given the Fund's inability to free sufficient expert resources to consider monetary and financial policy transparency and banking supervision in Tunisia, Bank staff prepared the initial drafts of these modules. However, the involvement of Bank staff in issues generally perceived as the core responsibility of the Fund, particularly in discussing the institutional arrangements underpinning monetary policy decisions²² , may have caused some confusion as to the assignment of responsibilities between the two institutions.

59.  A related issue is that of "ownership" of the assessment. If reports on standards are to be co-produced, all institutions need to feel a sense of ownership of the product. This raises the question whether these reports should be seen as the product of the Fund or of the international community more broadly.

60.  An advantage of having the Fund solely responsible for the organization and coordination of reports on standards lies in the accountability this delivers. One approach would be to leave the collaborative process for producing assessments up to staff at the Fund and the outside agency at the operational level, and have the Fund take responsibility for whatever is presented to the Executive Board. However, this raises a series of difficult questions, such as how the Fund would ensure the appropriate acknowledgment of the contribution of other organizations; whether the Fund would need to retain the capacity to review the judgements reached on assessments in non-core areas by outside organizations; if so, with what expertise the Fund would make those judgements and what incentive effects this would create for other organizations; and, if the Fund failed to retain ultimate control over every judgement, whether it would be seen as giving its imprimatur to a judgement in which it had no involvement.

61.  In contrast, a model that saw an assessment of implementation of standards as having shared ownership has certain attractions. The ultimate responsibility for judgements reached on the Fund's core areas could lie with the Fund, even if staff of other institutions assist in preparing those assessments. Analogously, the World Bank and other relevant bodies could be responsible for the judgements reached on implementation of standards in areas outside the Fund's core areas, with or without Fund assistance in the preparation of those assessments. However, it is not always easy or necessarily desirable to define each institution's responsibilities in rigid terms. For those areas where the Fund and the Bank have overlapping mandates, some mechanism to reach a consensus judgement would need to be developed, perhaps along the lines of that used in the FSAPs. In any event, decisions in this regard should be based on a clear understanding of the relevant responsibilities of the Fund vis à vis the Bank and other organizations.

62.  The modular approach to assessing implementation of standards provides a good mechanism for each institution to receive recognition for its contribution and to be held accountable for its work. World Bank staff have reacted positively to the idea of a modular approach whereby assessments of implementation of standards are seen as a joint product and where the Bank had clear responsibility for particular areas. Indeed, Bank staff view the assessment of implementation of standards as a potentially powerful tool to delivering a virtuous cycle of increasing disclosure leading to strengthened implementation of standards.

63.  The Bank has advised that it has initiatives underway to assess implementation of standards in the areas of bankruptcy, accounting and auditing. In addition, the World Bank and the OECD have decided to establish a Global Corporate Forum to broaden the dialogue on corporate governance beyond the OECD and to respond to the growing needs of individual countries seeking to strengthen their systems. The Forum will bring together the range of relevant international institutions as well as the private sector. It will help countries develop their reform agenda through assessments of corporate governance and periodic investor surveys, and provide a mechanism for coordinating and channeling technical assistance. While it is too early to determine whether these initiatives could provide stand-alone modules, the efforts underway within the Bank offer the potential for a more comprehensive coverage of standards than could be achieved by the Fund alone.

64.  It remains to be determined, though, how to extend the coverage of non-core areas to countries with which the Bank is not involved. There are currently around 26 such Part 1 (non-borrowing) countries, and further consideration will need to be given to developing mechanisms to prepare assessments in non-core areas for these Fund members.

65.  At its simplest, a report on standards is nothing more than an organizing framework for the presentation of a series of assessments of implementation of standards in specific functional areas.

66.  These assessments would be carried out using the various instruments discussed in Boxes 2 and 3. The FSSA will generate — as a by-product to its main focus of assessing financial vulnerabilities and development needs — a report on the financial system standards deemed important given the country's prospective vulnerabilities and reform needs. These could contribute a significant part of a report on standards. The involvement of the Fund, the World Bank, other international organizations and collaborating national central banks and supervisory agencies through the FSAP process thereby allows for implementation of standards to be assessed in areas such as banking supervision, insurance and securities regulation, deposit insurance and payments systems for the purposes of reports on standards.

67.  In part because of the resource intensity of FSSAs, reflecting the number of standards in the financial system, it is possible that the demand for reports on standards in the areas covered by an FSSA will outstrip the number of FSSAs produced in the near term. Consequently, modules covering monetary and financial sector standards may also be required. Fund and Bank staff have been preparing a series of Basle Core Principles assessments which provide an output that can be utilized in preparing the banking supervision module of a report on standards. Stand alone assessments of the monetary and financial policy transparency code have been prepared by Fund staff as part of Article IV or Use of Fund Resources missions, technical assistance missions and, in some cases, separate missions dedicated to preparing reports on standards.²³

68.  Assessments of data dissemination practices have been developed as part of the monitoring of the SDDS, and have been adapted to deal with non-SDDS subscribers. Assessments of fiscal transparency have been prepared in conjunction with the preparation by national authorities of self-assessments against the fiscal code.

69.  Given the availability of these instruments, staff propose that the scope of future work on standards include the preparation of modules on data dissemination, fiscal transparency, monetary and financial policy transparency (addressing the transparency practices in areas covered by the FSSA, including payments systems, insurance and securities regulation, and possibly deposit insurance) as well as an assessment of banking supervision that addresses observance of the Basle Core Principles. Assessments reached in the first three of these areas should be seen as the ultimate responsibility of the Fund. Reports on standards should reflect the results of the FSSAs where available, addressing the substance of standards for payments systems, securities and insurance regulation. Modalities would need to be found to work cooperatively with the Bank in this regard, but given the joint role of the Fund and the Bank in the FSAP process, assessments in these later areas could be seen as the shared responsibility of the two institutions.

70.  In light of the initiatives underway within the World Bank, it would be desirable for the Bank to indicate a similar "domain of responsibility" as the Fund's core areas for which it is prepared to take responsibility for preparing modular assessments. Initially, these might include corporate governance, bankruptcy, and accounting and auditing. More generally, it could be desirable down the road to agree formally to expand the required elements of reports on standards into additional areas as other institutions develop the means of making assessments in these areas.

71.  Final estimates of resources required to prepare each of the second round case studies are not presently available as not all reports have yet been finalized. While preliminary estimates suggest that the resource costs for the second round case studies are likely to exceed those for the first round studies by a significant margin,²⁴ these estimates are likely to underestimate the ongoing resource demands associated with assessments of standards for several reasons:

• The participants in the first and second round case studies are likely to have been among the easier examples for case studies given the likely self-selection biases noted earlier. As a result, estimates of future resource implications based on the case studies may be biased downward.

• The nature of the experimental case studies has been changing in response to greater experience with the issues. Costs are likely to be substantially higher as the focus on quality and definition issues increases and the more coverage expands to include countries where staff knowledge of statistical systems, fiscal management systems and banking sectors is less thorough. The country experience in implementing standards is also likely to affect costs: some of the second round participants required substantially greater assistance in completing the staff's questionnaires than did countries in the first round. The 'one-off' nature of the case studies provides little guidance on the ongoing costs associated with monitoring the implementation of standards in the context of surveillance, and between updates of the reports on standards. While collaboration with other institutions should lead to aggregate cost savings in the sense that duplication of work would be avoided, there would be costs for the Fund (and other institutions) associated with enhanced collaboration. In the absence of a clearer sense of how such collaboration might evolve, it is difficult to estimate whether such costs would be significant.

• The expanded role for reports on standards that appears to be emerging in the international community, and the range of standards under development and for which assessments might eventually be sought, is likely to require a level of detail and effort not anticipated at the time the case studies were initiated.

• Finally, if reports on standards proceed, the Fund will need to develop a more formal internal review process, with additional demands on staff resources.

72.  A decision to incorporate assessments of core standards into Article IV surveillance for the membership as a whole would have significant implications for staff levels and other work priorities, notwithstanding the work already underway on FSSAs and other standards-related projects. Given the role of technical assistance and cooperation departments in preparing assessments, an expanded program of reports on the implementation of standards would likely reduce resources currently available for technical assistance. To the extent that members also require increased technical assistance to help implement standards, this will either require additional resources or require potentially significant offsetting reductions in other types of technical assistance. Even if reports were phased in across the membership over a period of 3-4 years for example, staff would still need to prepare around 50 reports per year. The heaviest workload would fall upon the three technical assistance departments as it is their specialized expertise that is required to prepare the bulk of the assessments (although area departments would shoulder an increasing burden of the work required as updating assessments ensued). If reports were confined only to the four previously identified core areas, STA and FAD would need to prepare around 50 modules annually, MAE would need to prepare around 100 modules, while area departments would need to provide substantial input into, and coordinate with the authorities on, around 200 modules annually. The resource implications of this work program for departments would be enormous and would require substantial increases in staffing levels. Even more resources would be required if staff assessments were to cover non-core standards as well.

73.  Staff will circulate a supplement to this paper closer to the time of the Board meeting that will provide quantitative estimates of the resource costs for the Fund associated with the individual case studies.²⁵ Where possible, it will also provide some insights into the resource demands on the participating national authorities. The supplement will also aim to provide estimates of the costs of producing assessments of standards on an ongoing basis.

74.  While the initial assessment would be more resource intensive than subsequent updates, the workload to monitor standards is unlikely to diminish significantly with time. As greater focus is placed on the implementation of standards, members are likely to move faster to improve their national practices, leading to demands for more technical assistance. Staff are also likely to receive requests for more frequent briefing and updates from members and the Board.

75.  As noted earlier, there are a number of threshold issues that are fundamental to consideration of preparing assessments of standards but which are generally independent of the experience of the case studies and the outreach program.

76.  Monitoring standards in core areas may be viewed as being consistent with the Fund's mandate and responsibilities under the Articles of Agreement. Article IV, Section I (ii) states that the general obligations of members include seeking "to promote stability by fostering orderly underlying economic and financial conditions and a monetary system that does not tend to produce erratic disruptions." To the extent that the widespread adoption of standards would help strengthen the financial architecture and reduce the likelihood of "erratic disruptions" in the form of financial market crises, policies designed to promote the development, dissemination and monitoring of recognized financial standards should properly fall within the Fund's broad macroeconomic surveillance responsibilities.

77.  Since developments in the non-core areas have had an increasing impact on the Fund's core areas, it would be difficult to gain a comprehensive picture of the issues relevant to the successful operation of a market economy without also considering the extent to which members have implemented standards that lie outside of the Fund's mandate and technical expertise. The Interim Committee's request to the Fund to develop its surveillance "process to encompass the standards and codes relevant to international financial stability" is recognition of the importance that all major standards be monitored in order to encourage orderly economic and financial conditions.²⁶ In this regard, if other institutions are to be involved in the preparation of assessments in areas beyond the Fund's mandate, the modalities for such an involvement would need to be worked out.

78.  While the evaluation of standards is not a new direction for surveillance, the magnitude of the extension currently being considered would have significant resource implications for the Fund (and other institutions) as well as implications for the way in which the Fund carries out its responsibilities. It is unrealistic to expect that full reports on standards can be prepared in the context of an Article IV consultation mission. While the preparation of the Tunisia report was seen as a useful input into, and benefited from, the concurrent Article IV consultation discussion, staff believe it is impractical and infeasible from a workload perspective to conduct both exercises simultaneously. Indeed, assessing the implementation of standards at the same time as an Article IV mission would diminish the ability of staff to address its other surveillance responsibilities. In addition, given the need for specialist expertise for both the initial stock take and the periodic updates, it would place a significant burden on national authorities to interact with a separate team contemporaneously with the Article IV consultation.

79.  However, the need to de-link in time the preparation of reports on standards from the Article IV consultation mission does not affect the linkage in substance between the preparation of the report and the Article IV process. Indeed, in the context of other work underway at the Fund involving standards, further consideration will need to be given to the modalities of collating and integrating the information provided in modular assessments into the surveillance process. In various ways, assessments of the implementation of standards could be produced off-cycle as modules, while the substance of those modules would influence subsequent Article IV consultation discussions and be reflected in documentation for the Executive Board.

80.  Even if reports on standards were to be prepared and updated only every few years, the resource demands will still remain substantial. One option would be to increase the prominence of standards in surveillance on a selective basis, as has been done in other aspects of the consultation process. Under this approach, reports on standards could be phased-in gradually, or considered for both a subset of the membership and for selected sectors, focussing on countries and sectors where standards and the existence of institutions to ensure their implementation are seen as essential to limiting the build-up of vulnerabilities and to promoting better informed investment decisions and better policies. While area departments would be best placed to identify the sectors in a country where standards are most in need of scrutiny, the priority countries would seem to be industrial and emerging market economies with access, or seeking access, to international capital markets.²⁷ For such countries, shortcomings in the implementation of standards would be seen as having potential macro-economic or regional and systemic implications. In particular, these would be countries with vulnerability to variable short-term capital flows and with potential as sources of contagion.

81.  If, however, the international community were to decide to use Fund assessments of the implementation of standards in the operational manner proposed by the Basle Committee and the G-7, for instance as an element of conditionality, this would likely create demand to prepare assessments across a much broader range of countries; in particular, extending reports on standards to a broader range of developing countries would have to occur sooner.²⁸

82.  As noted earlier, the primary objective of reports on standards is to help avoid a build-up of vulnerabilities and to strengthen monetary and financial systems. Is this best achieved by providing national authorities with a confidential assessment of their economy's adherence to standards along with advice on how this can be improved, or does it also involve providing information to the private sector? The G-22 argued that publication was required in order to maximize the incentives for the adoption of standards:

"Publication of transparency reports should not be voluntary; it would be paradoxical for a transparency report to be kept confidential."²⁹

83.  Staff believe that the publication of reports on standards would be in the best interests of members and of the international financial system. For members already observing standards, publication would be a helpful signal to markets. For countries making clear progress in implementing standards, preparation of assessments of standards would help provide a roadmap to priority areas while publication would also send a clear signal about the authorities' commitment to further improvement. In both cases, publication would be likely to provide the authorities with additional credibility.

84.  That being said, the issue of whether to publish assessments of standards is not clear cut, and some members may not yet be convinced of the virtues of publication. If reports on standards aim to provide members with advice on how to improve their implementation of standards, publishing details about observance of particular standards could be counterproductive if, in particular cases, it were to deter assessment of needs and action on recommendations in this area. On balance, the staff believes it would be desirable to establish a presumption in favor of publication, even though a member's consent would still be necessary. In this regard, the Fund's website would serve as one vehicle to disseminate these reports.

85.  Internationally recognized standards are being assigned an increasingly important role in strengthening the international financial architecture and the Fund is being asked to play a central role in this area. Indeed, there is significant momentum in the international community for the Fund to develop a mechanism to monitor the implementation of standards relevant for financial stability. The staff's experience with the first two rounds of case studies points to the need for further work to develop methodologies and techniques for conducting assessments of the implementation of standards and reporting thereon. To that end, and in order to avoid the sample selection bias discussed earlier, it would be desirable for the Board to encourage all members to agree to participate in the exercise if asked. It would also be desirable if the Board asked members to agree to the involvement of other agencies, where possible, in making assessments. This section presents a number of issues on which guidance from the Board is needed in order to move forward with further work in assessing standards.

86.  In order to avoid the constraints imposed by short deadlines and to permit better coordination within the Fund and with other institutions, it is proposed that staff report back to the Board on additional experience preparing reports on standards in a paper for consideration by mid-2000.

87.  As noted earlier, reports on standards provide assessments of countries' implementation of standards in addition to their disclosure practices. In this light, do Directors agree that a name along the lines of "Report on the Implementation of Standards and Codes" would be more appropriate than "Transparency Report"?

88.  While the Executive Board has indicated that a thorough understanding of country practices in the "core" areas is important for the Fund as part of its surveillance activities, incorporating assessments of standards into the surveillance process implies universality. However, the resource constraints, the need to coordinate work efforts within the Fund and with other institutions, and the different dimensions of each standard (including the treatment of the quality of information), imply that the simultaneous assessment of all standards for any given member, let alone for all members, would not be feasible. In this respect, do Directors agree that the most effective means of assessing standards would be in a gradual, rolling manner, or a "modular" approach, which would build on, and be integrated with, work underway elsewhere in the Fund? While this approach would still entail additional resources, it would permit focusing on countries and sectors where the observance of standards and the existence of institutions to ensure their implementation, are seen as important for limiting the build-up of vulnerabilities prejudicial to the international financial system.

89.  In view of the array of initiatives underway at the Fund which involve work on standards, further consideration would need to be given to the modalities for collating and integrating the information provided in modular assessments into the Article IV process. As work to produce further examples proceeds, staff propose to continue exploring options for presenting assessments of the implementation of standards to the Board.

90.  Do Directors agree that the issue of the quality of information should be pursued in a systematic and explicit manner, identifying concerns or questions about the underlying integrity of data as they arise without going as far as auditing the information released. In this respect, emphasis should be placed on the systems in place to provide assurances of quality, such as is required in the Code of Good Practices on Fiscal Transparency?

91.  Further consideration will need to be given to dovetailing the work on assessments of implementation of standards with other initiatives undertaken by the Fund and the international community more broadly. Should staff attempt to provide reports on the implementation of standards in the non-core areas, whether prepared by, or jointly with, outside institutions? In this regard, should other institutions, most notably the World Bank, be encouraged to take primary responsibility for assessments in these areas?

92.  Do Directors agree that a presumption in favor of publishing individual "Reports on the Implementation of Standards and Codes" should be established? In cases where members decided to publish reports, should the Fund website be considered as a venue for disseminating assessments on the implementation of different standards?

93.  Initial attempts have been made to estimate the costs of preparing the case studies. The overall costs and time required to prepare assessments may be higher as more difficult country cases are approached and as greater efforts are made to address the quality aspects. As noted earlier, a supplement to this paper which presents estimates of the resource costs to produce the first and second round case studies will be issued to the Board before consideration of this paper.