Some Points and Principles for Evaluating the Development Performance of
Different International OrganisationsA note prepared for the Japanese National Institute for Research Advancement
by Richard Jolly
In this note I provide my personal evaluations of the operations of UNICEF, UNDP, some other parts of the UN including some comparisons with the World Bank. Encouraged by the outline and request of Yasuo Sawai, I have drawn on three sources of information for what are my personal assessments:
The paper has three brief sections:
- My personal experience as Deputy Executive Director of UNICEF 1982-95, and Special Adviser to the Administrator of UNDP and Principal Co-ordinator of the Human Development Report 1996-2000
- Our findings on the UN's contributions to economic and social development 1945-2000, arising from the UN Intellectual History Project, for which I am one of three co-directors and a co-author of two volumes. (1)
- A recent paper on the UN's experience from 1960 to 2000 in using time dated quantitative goals to guide development programmes. This showed that over this period over 50 major development goals have been set by the UN, the first as part of the Development Decade 1961-70. Contrary to much casual opinion, there has been much more success in implementing these goals than many people realize. While few goals have been fully achieved (the eradication of smallpox and the reduction of infant mortality 1980 to 2000 being important exceptions) most goals have been partially or considerably achieved. Few have been totally neglected or are total failures. In many respects, the record of the UN goals has been more successful than that of the adjustment programmes of the Bretton Woods Institutions over the 1980s. (2)
1) The programme approach of UNICEF, UNFPA and UNDP - focusing on some key features which I believe have been responsible for success in country level implementation, especially in relation to goal achievement. 2) Comparative effectiveness of UN agencies- some methodological issues. 3) Four Conclusions for action Inevitably, my experience and my conclusions are selective and subjective but with respect to the operations of UNICEF and comparisons with those of the World Bank and other parts of the UN over the 1980s and 1990s, they are based on specific experiences as well as on the usual documentation. I believe the comparisons between the development efforts of the UN agencies and those of the Bretton Woods Institutions are enormously significant. Many donors are often much less aware of the actions and achievements of the UN agencies than they are of the World Bank - and the IMF and World Trade Organisation. This is unfortunate, since the historical record makes clear that the UN has often been in the forefront of international development thinking and practice. Frequently, the World Bank and the IMF have adopted ideas first pioneered by the UN, but often after an interval of five to fifteen years and even then, commonly pursuing approaches which involve higher costs and, being linked to conditionality, many times engender suspicion from developing countries.
The Programme Approach of UNICEF, UNFPA and UNDP
UNICEF's developed a system of "country programme support" in the early 1970s. This is of great importance for country level effectiveness. It became something of a model for other parts of the UN. in the 1980s and 1990s, notably UNFPA. There are several distinguishing features:
1. Each developing country receiving UNICEF funding obtains such support as part of a "country programme". Such a programme is based on a systematic analysis, prepared every five years or so, of the main needs of children in that country. This analysis is much deeper and broader than one simply focused on assessing the needs of that country for international support. Rather, it involves three elements: A]- an analysis of the situation of children in that country as a whole, their needs, their recent progress, their future, the adequacy of national policy and of international support. This document is usually produced by a group of people - some government, some professionals from universities and other institutions, some NGOS and UNICEF. Ideally, comparisons of the situation of children in the country concerned are made with the situation of children in other countries of a similar level of development - as well as how their situation appears when judged by the specifics of the Convention on the Rights of the Child. B]-As a second step, when the situation analysis is complete, the programming process considers the most important actions which need to be undertaken by the government and other groups in the country to deal with the problems identified. . C] Then, as a third step, the specifics of what UNICEF support can be provided are considered, mostly in the form of what catalytic support UNICEF can offer to assist national action to be undertaken or accelerated on a sustained basis. Sometimes UNICEF support is provided for just one or two components of national action - for example of vaccines or essential drugs or schoolbooks focused on issues of gender. Often UNICEF support has been provided to develop better statistics with which to monitor progress for children. 2. In the early 1980s, UNICEF adopted some global goals and priorities. Initially these focused on health actions to achieve a "child survival and development revolution". These included an acceleration of immunisation towards the target of 80% coverage by 1990 against the six main communicable diseases, responsible for some 5 million child deaths per year; promotion of ORT (oral rehydration therapy) to reduce deaths from diarrhoea, which at that time were killing 4 to 5 million children each year; the encouragement of breastfeeding and other actions for early child nutrition; and growth monitoring, to encourage steady growth for children in the first 2 or 3 years of life. Each of these became a focus for country-by-country action towards the more general goal of reducing infant and child mortality. By 1990, the target of 80% coverage of immunisation had been achieved on average in developing countries and within sixty to seventy countries individually. The estimated number of child deaths per year had fallen from 15 to some 12 million, a major reduction in spite of an increase in the number of child being born each year. This most positive experience in the use of the global goals, led UNICEF to propose the holding of World Summit for Children. This was held in the UN in September 1990 and was attended by 71 heads of state and representatives of some other 80 countries. The World Summit for Children built on earlier experience by agreeing a further set of ten major goals and 17 supporting goals for the 1990s. 3. Over the 1990s UNICEF continued its goal-oriented approach, albeit with some changes arising through the appointment of a new Executive Director in 1996. Notwithstanding, UNICEF over the 1990s achieved remarkable success in mobilising support for the goals agreed at the World Summit for Children. In 2001, a special session of the UN General Assembly was held in May to evaluate achievements. For this, UNICEF prepared and issued a remarkable document, "Progress since the world Summit for Children: A statistical Review". (3) This impressive document summarised progress, regionally and country-by-country, for all of the goals adopted at the World Summit for Children for which data was available. Moreover, UNICEF itself had worked with countries on a new form of low cost national survey to improve monitoring. These were the MICS - the Multiple Indicator Cluster Surveys - developed in 1997 to assess progress on all goals at the end of the decade. These MICS were conducted in 66 countries, by national government ministries with support from a variety of partners. The Demographic and Health Surveys also provided relevant data for 35 countries, thus helping fill many gaps in data on children. Through this system, statistical data on the situation of children in developing countries had improved incomparably, compared with the situation in the 1970s or before. As one example, data on child malnutrition by the year 2000 was available in over 100 countries, in some 70 of them with trend information. In 1975 such data was available for only about 5 countries. (4) 4. UNICEF experience played a major role in encouraging other parts of the UN to adopt human development goals in the 1990s and some, like UNFPA and UNDP, to use these goals as guidelines for their own programmes. In the case of UNFPA, it developed its own programming approach in the late 1980s, building up some 15 regional support offices. In my experience, this was a well judged and cost-efficient arrangement for an agency which had far fewer staff than either UNICEF or UNDP. 5. In the case of UNDP, its country programmes have increasingly emphasised issues of governance, environment, gender (including support from UNIFEM) with an increasing focus on human development. By 2002, some 125 countries had produced National Human Development Reports, some for several years in succession. These National Human Development reports not only make important contributions to national policy but often also provide guidelines for possible areas of UNDP support. 6. It may be useful to clarify here an important but less known contribution of Japan to the formulation of the Millennium Development Goals. In 1995 OECD was encouraged by the Japanese government to formulate some development goals that could be used by the donors and by developing countries as the basis for partnership in the 21st century. In the event, the draft produced by DAC in 1995 was inadequate, so informally the Human Development Report office of the UNDP was approached and a paper with more specific proposals for global goals was prepared in early 1996. These goals in turn were considered at the 34th High Level Meeting of the Development Assistance Committee of OECD in May 1996, where they were adopted. They formed the core of "Shaping the 21st Century: A Contribution of Development Co-operation". (5) Initially these goals were seen as those of OECD and DAC but in a skilful process involving the UN and the UN Secretary General himself, they became the core of the Millennium Development Goals, adopted at the Millennium Summit in September 2000. Methodology for Comparative Assessment of Different UN Agencies
1. UN mandates, duplication and value added. I would like here to comment on some of the methodological issues involved in making comparative assessments of the effectiveness of different UN agencies. One complication is that most of the UN agencies have different mandates and do different things, though with important areas of overlap. Often this has led some governments, particularly some donor governments, to argue that all overlap is a sign of inefficiency and of duplication. This is not true. UNICEF, for example, spends most of its programme resources working in areas where other UN agencies also have responsibilities - such as health, education, water, human rights. The result is usually not duplication but re-inforcement, with the result of collaborative efforts with WHO inhealth, UNESCO and the World Bank in education etc more effective than if either agency was working alone. In all these cases, in my experience, overlap is a point of strength not of weakness or duplication. A certain amount of inter-agency competition can also be positive. The test is not whther there is any overlap but whether each partner brings value-added. If a second or third agency working in a particular area adds value to the total effort, it should be encouraged not discouraged, providing they all work collaboratively not in isolation. 2. Humanitarian Affairs Humanitarian affairs and human rights are indeed particular responsibilities of UNHCR and UNCHR. But a major part of action in these areas is directly supported by the UN funding agencies, especially UNDP, UNICEF, UNFPA, WFP and UNIFEM. This is another area where overlap involves strengthening the total effort not duplication. Indeed, over the 1990s, rights-based programming was strongly encouraged by UNCHR and has now been adopted by several UN agencies - notably UNICEF, UNDP and UNFPA. Since these funding agencies have a high proportion of resources available for country support, their close collaboration with UNHCR and UNCHR should be welcomed, not treated as duplication. 3. Human Security The new concept of human security which in recent years has been strongly promoted by the governments of Japan, Canada and Norway, contains important implications for evaluating the contribution and effectiveness of different UN agencies. The broader the definition or human security, the greater the number of different UN agencies which have a role in promoting and supporting it. Thus an evaluation of UN effectiveness needs to be undertaken in relation to the different contributions to human security. As the Human Development Reports of 1994 and 1999 have argued, countries need to balance both their resources and the effort put into the different aspects of human security: human insecurities from health, food, unemployment, financial instability as well as from urban crime and terrorism. 4. Corruption Though all parts of the UN need to be on their guard against corruption, my experience is that it is not in general a major problem for the UN. Most UN agencies have internal audit systems and many are also audited a second time by external auditors, answerable directly to the agency Boards as well as to their governments. In the case of UNICEF, these audits took place for several months each year, covering field and regional offices as well as headquarters. More important than theft and financial corruption, in my experience, are the attempts by some governments to exercise inappropriate influence on key matters relating to personnel and voting. The attempts to win votes on the Security Council by allocations of national aid is one recent example; the introduction of political criteria in the approval of World Bank or IMF loans is a second; a third is the promising of agency resources or posts in exchange for support when a director general is running for re-election or when a new director is being appointed. There may be little that can be done for the first two categories of abuse, except that openness and transparency in reporting can help make clear what has happened. In the cases of appointments, many of the abuses can and should be avoided by i) always requiring an open process of advertising, assessment, interviewing and appointment for all senior UN positions and ii) sticking to open procedures in the allocation of resources from and to the different UN agencies. 5. Indicators of Effectiveness The effectiveness of individual UN agencies depends greatly on the strength and clarity of their corporate development objectives and the agency's structure, professionalism and programming approach. The proportion of an agency's staff located full-time in country or regional offices( as opposed to headquarters) is also critical. All these differ widely between UN agencies. UNDP and UNICEF, for example, both have over 80% of their staff based full-time in developing countries. Many other UN agencies have only a tiny proportion. All these could and should be measured in making assessments of comparative effectiveness. In Table 1 of the annexe, I provide six indicators of effectiveness of individual UN agencies, many of which could easily be quantified. It would be very helpful to collect such data for as many UN agencies as possible in order to assess effectiveness and later to guide allocations of resources and priorities for support. Four Conclusions for Action
I would like to end by summarizing what I believe are four important conclusions for international action. Each has grown from my experience and each would greatly help improve the effectiveness for development of the international system in general and of the UN in particular
1 A better balance needs to be struck between support for the UN and support for the Bretton Woods Institutions, with a rising share of finance and donor support being channelled to the UN. This balance needs to build more consciously on the comparative advantage of each of these two groups of institutions. The adoption of the Millennium Development Goals provides a new opportunity for this. 2 Within the UN additional resources are desperately required, both for core functions and for goal-oriented actions. Over the last ten to fifteen years, many parts of the UN, including the Secretariat itself, have been "starved into reform". As a number of commentators have made clear, the process of cutting back on UN resources has now become counter-productive. Worse still, it has often coincided with increased funding for the World Bank and other multi-lateral organisations, leading to imbalance and inefficiency. There are many examples of how UN agencies operate with greater cost effectiveness and lower costs per beneficiary than the World Bank. When managed by UNICEF, UNESCO and other UN agencies projects often cost less per beneficiary in education, water and health. 3 The MDGs provide a major opportunity for a new approach. There are also other goals in a broader range of areas which deserve support. Such support can be provided in ways directly linked to goal achievement and, if necessary, to particular countries. This is the system long adopted by UNICEF for what is termed "supplementary support" for "noted projects". This is preferable to other forms of trust fund support, since noted projects are part of total country programmes, thus linked to a coherent five-year programme approved both by the country and by the UNICEF Governing Board. 4 A new approach to Partnership between developing and developed countries is one of the most important needs. It forms the eighth MDG goal and one without which many countries will not be able to achieve most if any of the other goals. Partnership needs to establish a better relationship between the recipient and the donor countries. But it also needs to broaden the agenda of partnership beyond an overwhelming pre-occupation with aid. Access for a recipient country's exports, accelerated debt relief, more flexible provision of resources, simpler aid procedures, and more harmony of procedures with other donors - all these need to be brought into new partnerships. The Millennium development Goals offer great possibilities for improving the lives of many hundred millions of people in developing countries. Ultimately, partnership needs to be built on a shared commitment to achieve this vision.
Annexe: Indicators of "effectiveness for development" of individual UN agencies 1. Clarity of corporate development objectives of the agency
- How clearly specified are the agency goals and objectives?
2. Agency structure in relation to development objectives
- How clearly and seriously focused on MDGs?
- priorities in relation to field support
- monitoring process by Boards and top management
- % and trends of total staff in regional and field offices
3. Budget allocations
- extent to which field offices have decentralized authority within corporate policy guidelines and clear guidelines for responsibility and accountability.
- Adequacy of field support and field monitoring structure
- Management record of ensuring committed and effective field level management
-
- % and trends on country and field level operations
4. Programming structure
- % and trends on MDGs by goals
- Nature and quality of country programmes
5. Country offices
- Frequency of preparation and quality of guidelines and support
- Links with MDGs, direct and analytically indirect
- % and trends in concentration on least developed countries, other developing countries and high income countries.
6. Adequacy of coordination
- % and trends of time staff spent in field office, outside office and outside capital city and not in other countries.
- with recipient country
- with other UN and Bretton Woods
- with donors
Notes:
[ Back ]
National Institute for Research Advancement (NIRA) Home Page
Copyright (c) National Institute for Research Advancement (NIRA)