Cases and Evidence
The Case Studies Case Stories Map

Latin-American Curriculum Analysis: Learning from each other’s Learning Experiences

Organization(s)

UNESCO–OREALC, ICFES

Country (ies) and institutions

Countries: 15 Latin-American countries and the Mexican State of Nuevo Leon. Institutions: INEP Brazil (National Institute of Research on Education), MIDE UC in Chile (Measurement Centre of the School of Psychology at the Pontifical Catholic University of Chile), ICFES (Colombian Institute for Education Evaluation) and UNESCO-OREALC (see below, under ‘key players’, for the meaning of all these acronyms in their original language).

Diagram to describe how institutions were related to each other. (If available)

http://www.southsouthcases.info//ttssc/web/uploads/diagrams/

Overview

This case story relates the experiences around the curriculum analysis for the Third Regional Study of Learning Outcomes (TERCE) of the UNESCO-OREALC-coordinated Latin-American Laboratory of Assessment of Quality of Education (LLECE). This activity was coordinated by ICFES, the Colombian Institute for Educational Evaluation, with national teams in all 16 participating countries / territories, and UNESCO-OREALC. The activity provided learning opportunities for all people, institutions and countries involved, highlighting curriculum gaps, thus providing an opportunity to improve education quality.

Background and set-up

The Latin-American Laboratory for the Assessment of the Quality of Education (Spanish acronym: LLECE) is a UNESCO – OREALC coordinated network of 15 Latin-American countries, and the Mexican State of Nuevo Leon, which was founded in 1994, in Mexico City. Its main aim is to provide a space for technical and policy-discussion on the quality of education and factors associated with it. To this end, the LLECE conducts periodical studies of quality of education in the region, through large scale assessments of learning outcomes in mathematics, science and language (reading and writing), in primary education (actually, 3rd and 6th grade). While the first study, PERCE, took place in 1997, the second study, or SERCE, took place in 2006 and the Third Study (TERCE) is now in preparation. Results of the latter are expected in 2014. The focus of LLECE fits remarkably well within the global Education for All agenda, which includes the goal of improving quality of education (nr. 6). Learning outcomes may be perceived as proxy measures for this, and the recent OECD PISA 2009 study highlighted the urgency of this challenge for the region. >From the beginning, the focus of LLECE was not only on getting the studies done, but also on developing diagnostic capacity in the member states, by improving national educational evaluation systems. A ‘capacity development’ focus has thus been inherent in the Laboratory from the very start. Similarly, the persons involved in the LLECE are very aware that educational evaluation does not only have a descriptive function (providing a snapshot of ‘what is’) but also a normative or standard-setting one (focusing on ‘what should be’). That is to say, the studies of LLECE create a powerful incentive for regional administrations to not only develop educational evaluation systems, but also further improve educational delivery itself. A study like TERCE necessarily needs to go through a number of preparatory phases to ensure a final high-quality measurement. To ensure that the final study is largely representative of all participating education systems (and their aspirations), an analysis of the curricula of the relevant subject areas of the countries is conducted. This is a relatively complex task as it should not only review content (say, the proportion of algebra taught in mathematics), but also performance expectations as well as level of difficulty. Other variables may also be taken into account. In the case of TERCE, UNESCO-OREALC worked together with ICFES to coordinate the curriculum analysis. ICFES had already worked with OREALC for the curriculum analysis of the Second Study. ICFES will present the final results of the analysis at a forthcoming Seminar in Bogotá (25-27 May, see: http://portal.unesco.org/geography/en/ev.php-URL_ID=14082&URL_DO=DO_TOPIC&URL_SECTION=201.html). The curriculum analysis itself will inform the development of the items for the main test, which is being developed by MIDE UC, the measurement centre of the School of Psychology at the Pontifical Catholic University of Chile, and which is schedule to be piloted in 2012.

The partnership

To reinforce the regional dimension of the LLECE, one objective is to work with regional partners and it was agreed with Colombia that ICFES would undertake the curriculum analysis in lieu of Colombia's annual contribution. ICFES was initially the counterpart of UNESCO Santiago in the LLECE, on behalf of Colombia. It is well-known regionally for its expertise in the field of educational evaluation. MIDE UC, on the other hand, is an implementation partner. These partners are brought together through the coordination of UNESCO-OREALC (the technical coordination of LLECE, or CT LLECE), under the auspices of the General Assembly. While OREALC–UNESCO prepared in collaboration with MIDE UC the specifications for this work, which had to take place within the overarching framework and timelines of the Third Study, ICFES was in full charge of the activity itself (curriculum analysis). ICFES made a project leader available, who provides overarching leadership for this project. She worked with a coordinator, who communicated with designated focal points in the countries (one specialist for each subject area, or one person covering all areas, depending on the country) for the delivery of the information needed from the countries. This information was subsequently analysed by three subject/area teams. These teams were constituted by specialists from ICFES and from the countries, which implies that communications took place mainly by e-mail or telephone. UNESCO-OREALC provided mainly ‘troubleshooting’ support: some countries complained that the receipt of messages was not confirmed, or that information was not taken into account. These issues have been solved with negotiations. While MIDE and ICFES communicated as much as possible directly between each other, at times, UNESCO-OREALC had to act as a ‘mediator’. In general, UNESCO-OREALC provided generic quality control, but the draft document was also sent back to the country specialists for review. Given that this was quite a technical activity, political actors were not specifically involved in it. However, other NGOs and partners have shown interest, and PREAL (Programme for Education Reform in Latin America and the Caribbean), for example, will attend the dissemination event in Bogotá. In addition, UNICEF has agreed to co-sponsor this event.

Lessons learned

Although the activity has not been completely finalized, or systematically evaluated, there are already some lessons we can draw from this: (i) There has been great willingness in most countries to work on this task, and the national teams have stuck to the deadlines in almost all cases; (ii) Electronic communications did lead to obstacles in progressing with the work in a limited number of cases, where team members temporarily ‘dropped off the radar’; (iii) While Spanish has been the main language of communication for this activity, the Brazilian team had very little difficulties communicating with the Colombia-based coordinator; (iv) The activity seems to have helped the responsible units in government Ministries to review their own curriculum critically; (v) This project generates value added by providing a regional perspective. UNESCO’s expectation is that the final document will help subject evaluation teams, within Ministries, develop their understanding of where they stand with their curriculum with respect to other countries. This is critical for further advancing towards Education for All (Goal 6) in the LAC region, as earlier analyses indicated that there is room for improvement in curriculum development. Specific gains of working with a south-south modality crystallized through: (1) lower costs (the services of ICFES were provided in lieu of Colombia’s annual contribution to LLECE, which is US$ 14,000); (2) high engagement, created by working with and for local partners; (3) reduced ‘overhead’ (reduced travel and communication costs); and (4) very good ‘local knowledge’ of the partner. Until now, no English, French or Dutch speaking countries in the region are part of the programme, although discussions with (mainly) Suriname and, to a lesser extent, Haiti are ongoing. There is some interest from Guyana and Jamaica as well.

Complementarity with North-South cooperation

The activities taking place within the framework of the LLECE / TERCE are already being supported partially (and generously) by the Spanish Cooperation and, now, by UNICEF as well. The contribution of UNICEF comes within the context of a greater framework agreement between UNESCO-OREALC and UNICEF-TACRO.

How to share

UNESCO-OREALC is engaging with a wide range of (potential) partners to explore further possible synergies and possibilities for exchange of ideas. There are similarities, for example, between LLECE and the Southern and Eastern African Consortium for Monitoring Educational Quality (SACMEQ). Contacts have also been established with other organisations leading international large-scale assessments of learning (ILSAs), such as OECD (PISA) and IEA (TIMSS / PIRLS / ICCS), as well as with representatives of the academic community (inside and outside of the region) and other organizations such as the US-based Educational Testing Service. Such contacts are interesting in a number of ways, particularly in terms of quality assurance, inter-regional comparisons and the further expansion of south-south cooperation. ‘Traditional’ diffusion takes place via the LLECE website (http://www.llece.org); a designated restricted access website for national and regional counterparts; the UNESCO-OREALC website; as well as research publications. The final version of the regional curriculum analysis will be presented at a regional event (Seminar on Item Development) in Bogotá (link above). The keyword in LLECE’s knowledge exchange strategy is learning: while countries learn about how their children and young persons learn, the various partners in the network learn how it can further facilitate such learning. The LLECE is thus mutually enriching: from ‘south to south’, ‘north to south’ and ‘south to north’.

Duration

(YY-mm-dd) 2010-12-15-- (YY-mm-dd) Budget The countries contribute to this activity by making available a national team, but the curriculum analysis itself is conducted by ICFES. The countries paid nothing additional for this work (the annual contribution consists of US$ 14,000). MIDE UC is paid from countries´ contributions as well as through UNESCO´s extra-budgetary funds, mainly from Spanish-based development partners (AECID, DG Polde) although other contributions have also been important.

File

http://www.southsouthcases.info//ttssc/web/uploads/files/

Image

http://www.southsouthcases.info//ttssc/web/uploads/images/

Name of Primary Contact Person/s

For ICFES: Ms. Patricia PEDRAZA-DAZA; for UNESCO-OREALC: Mr. Moritz BILAGHER

Title of Primary Contact Person/s

Ms. Pedraza-Daza: Vice-Director of Instrument Design

City and country

Bogota, Colombia and Santiago Chile

Contact Email/s

ppedraza@icfes.gov.co; m.bilagher@unesco.org

Telephone/s

+57 - 1 - 338 7338, Ext. 1103

Key players

ICFES : Instituto Colombiano de Evaluación de la Educación, Ms. Patricia PEDRAZA (ppedraza@icfes.gov.co), Mr. William MANTILLA (analisisterce@gmail.com) INEP : Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira, Mr. Estevon NAGUMO (estevon.nagumo@inep.gov.br) MIDE UC : Centro de Medicion, Escuela de Psicologia, Pontificia Universidad Católica de Chile, Mr. Jorge MANZI (jmanzi@uc.cl), Ms. Paulina FLOTTS (mflotts@uc.cl), Ms. Catalina COVACEVICH (ccovacep@uc.cl) OREALC-UNESCO : UNESCO Regional Bureau of Education for Latin America and the Caribbean, Mr. Atílio PIZARRO (a.pizarro@unesco.org), Mr. Moritz BILAGHER (m.bilagher@unesco.org})