EDUCATIONAL EVALUATION IN EAST ASIA: EMERGING ISSUES AND CHALLENGES
No part of this digital document may be reproduced...
67 downloads
1750 Views
6MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
EDUCATIONAL EVALUATION IN EAST ASIA: EMERGING ISSUES AND CHALLENGES
No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form or by any means. The publisher has taken reasonable care in the preparation of this digital document, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained herein. This digital document is sold with the clear understanding that the publisher is not engaged in rendering legal, medical or any other professional services.
EDUCATIONAL EVALUATION IN EAST ASIA: EMERGING ISSUES AND CHALLENGES
SAMUEL S. PENG AND
JOHN CHI-KIN LEE EDITORS
Nova Science Publishers, Inc. New York
Copyright © 2009 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com NOTICE TO THE READER The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA Avaailable upon request ISBN 9781606928875
Published by Nova Science Publishers, Inc. New York
CONTENTS Introduction
1 Samuel S. Peng and John C. K. Lee
Part I. Practices and Challenges Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Education Evaluation and Assessment in Taiwan: Status, Innovations and Challenges Mu-lin Lu
13
Educational Evaluation in Singapore: Current Status and Future Challenges R. Subramaniam
23
Educational Evaluation in South Korea: Current Issues and Future Prospects Sun-Geun Baek
35
Development of Educational Evaluation and Challenges of Practicing Educational Evaluation in Thailand Sirichai Kanjanawasee
45
Educational Evaluation in Hong Kong: Status and Challenges John Chi-Kin Lee
61
Part II. Variations and Evoluation Chapter 6
Chapter 7
Chapter 8
11
73
Educational Evaluation and Quality Assurance: The Case of Hong Kong Nicholas Sun-keung Pang
75
School-Based Assessments (SBA): Implementation and Challenges in the Malaysian Context Suan Yoong
93
Current Issues Regarding the School Evaluation System in Japan Yasuhiko Washiyama
111
vi Chapter 9
Chapter 10
Contents Teacher Evaluation in Thailand: Current Issues and Future Prospects Pruet Siribanpitak
117
PISA as a Catalyst for Capacity Building for Educational Evaluation in Macao Kwok-cheung Cheung
129
Contributors
143
Index
147
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
INTRODUCTION Samuel S. Peng1 and John C. K. Lee2 1
Center for Research on Educational Evaluation and Development, National Taiwan Normal University, Taipei, Taiwan 2 Department of Curriculum and Instruction, The Chinese University of Hong Kong, Shatin, Hong Kong
BACKGROUND Educational evaluation plays an important role in the pursuit of educational excellence (Kennedy and Lee, 2008). At one level, it has been used in the evaluation of students' individual differences and performance. At another level, it has been used to contribute to an understanding of the overall effectiveness and efficiency of institutions and academic programs, as well as such matters as the specific qualities of school management, curriculum, teaching staff, pedagogical practices and educational policies. Educational evaluation is not a simple look in the mirror, a vain exercise; it is an effective way to provide evidence for all decision-making pertaining to resource allocation, program improvement, personnel hiring and promotion, selection of candidates, and educational policy-making. Evaluation is, in fact, an integral part of the multi-faceted process of enhancing quality and attaining excellence. Educational evaluation, as concept and as a set of practices, is not new in East Asia. However, in recent years it has received much greater attention from the general public as well as the educational community because of changes in the political and economic conditions in society. First, as the demand for educational resources increases, both the government and the schools as a whole need to know how well their resources have been effectively allocated and used; they also need to know the extent to which their objectives have been achieved. Evaluation is thus a necessary mechanism for monitoring the educational quality of institutions and programs. Secondly, as education in general and postsecondary education in particular have been expanded in response to global competition and the generally rapid rate of economic
2
Samuel S. Peng and John C. K. Lee
development in society, there are concerns about the equity and quality of education on the part of all stakeholders, including students, parents, teachers, school administrators and policy-makers. Subsequently, there have been strong demands for an effective evaluation system to ensure equity and quality, on the one hand, and proficiency and productivity on the other. Moreover, constant reforms of the educational system increase the demand for input, process and outcome evaluation. The great attention given to accountability and quality assurance makes the practice of evaluation more complex and innovative, with ever more diversified paradigms, methods and approaches. Thirdly, globalization increases awareness of the importance of a country’s international competitiveness and stimulates interest in international comparisons of student ability and competence. For example, the International Association for the Evaluation of Educational Achievement (IEA) and the Center for Educational Research and Innovation of the Organization for Economic and Development (CERI/OECD) have been carrying out international surveys of student achievement in science, mathematics, language and civic education. The results of their surveys have motivated the participating countries to consider how best to evaluate and improve their educational systems to ensure that their students are well prepared and competent. Consequently, the need for effective educational evaluation systems and methods is an important issue, one that is frequently studied and discussed in East Asian countries. However, the development of an effective evaluation system is not a simple task. Many factors need to be considered. Aside from measurement techniques and methodological perspectives, an effective system needs to be cognizant of the cultural, political, economic and social contexts in which an evaluation system is implemented. Consideration of such contexts is critical in East Asia for one simple reason – education is highly valued in our society due to its traditional Confucian culture. All high-stakes educational evaluations, such as senior high school entrance examinations, college entrance examinations and teacher qualification examinations, as well as school performance evaluations, often receive a great deal of public attention and scrutiny. Any official evaluation system, in terms of its contents, standards, and implementation procedures, may set the guidelines for teaching and student learning; schools tend to operate on the basis of, teachers tend to teach and students are likely to learn mainly what is to be assessed and evaluated. What is not tested or otherwise judged or assessed for “official” purposes usually gets less attention or is simply ignored. Thus, implementing a national evaluation system that is objective and fair without having any negative impact on schooling is a major challenge in most places in East Asia. Scholars in the field of educational evaluation are actively engaged in the search for an Asian model that will work well within all East Asian social and cultural contexts. East Asian countries serve as an ideal testing ground for judging the effectiveness of various evaluation models and programs because they share many things in common. Aside from the high value placed on positive attitudes toward education, educational governance at the systems level tends to be centralized and controlled by the central government. Given the similar cultural and social contexts, educators may be able to identify theories and practices
Introduction
3
that are most suitable for their students and educational systems by comparing the different evaluation systems or programs developed by other countries or special administrative regions. It was for this reason that the “Forum for Educational Evaluation in East Asia: Emerging Issues and Challenges” was initiated and held on October 2007 at National Taiwan Normal University. It brought together a group of distinguished scholars from Hong Kong SAR (China), Japan, Macao SAR (China), Malaysia, Singapore, South Korea, Taiwan and Thailand for scholarly discussion of the problem of developing, implementing and improving educational evaluation systems that are sensitive to the local cultures and social contexts. The breadth of issues discussed, free and open expression of ideas and opinions, and general sense of the success of this forum prompted the idea of publishing the presented papers in order to share the participants’ knowledge, ideas and experience with those who did not have an opportunity to participate directly in the forum. This book was therefore developed with the intention of providing a foundation for further discussion of educational evaluation models within the context of East Asian cultural and social conditions. It aims not only at identifying challenges of and strategies for educational evaluation but also at facilitating advanced research on educational evaluation in East Asia. Through an examination of the current principles, policies and practices of educational evaluation in this region, researchers will hopefully be able to further construct and refine evaluation theories and models rooted in the traditional cultures and educational values of East Asia. Of course, it is hoped that such theories and models will also be applicable to other regions of the world.
ORGANIZATION OF THE BOOK We have divided this book into two parts. Part I includes five case reports that center on the current status of educational evaluation programs and policies in Hong Kong SAR (China), Singapore, South Korea, Taiwan and Thailand. Specific policies, implementation procedures and the recent or ongoing development of educational evaluation programs are explored. In Part II, variations in educational evaluation policies and practices in the region are highlighted with chapters drawing on case studies of changes and strategies of educational evaluation in Hong Kong SAR (China), Japan, Macao SAR (China), Malaysia and Thailand. By providing case studies of the principles and practices of educational evaluation systems, the authors identify the current innovations, as well as challenges, associated with significant issues concerning the organization, management and implementation of educational evaluation in East Asia. The range of topics covered in the various chapters is summarized in Table 1.
4
Samuel S. Peng and John C. K. Lee Table 1. Topics covered by each author Topical area Evaluation policies
M.L. R.S. S.B. S.K. C.L. S.P. S.Y. W.Y. P.S. K.C. ©
Evaluation of educational progress Evaluation of basic education
©
Evaluation of the learning of reading, science and mathematics Evaluation of K-12 teachers
©
Evaluation of K-12 schools
©
Evaluation of higher educational institutions Evaluation of academic programs Evaluation of college candidates Evaluation methods
©
©
©
©
©
© ©
©
©
©
©
© ©
©
©
© ©
© © ©
© ©
Note: M.L. -- Mu-lin Lu; R.S. -- Ramanathan Subramaniam; S.B. -- Sun-Geun Baek; S.K. -- Sirichai Kanjanawasee; C.L. -- Chi-Kin John Lee; S.P. – Nicholas Sun-Keung Pang; S.Y. – Suan Yoong; W.Y. – Washiyama Yasuhiko; P.S. – Pruet Siribanpitak; K.C. – Kwok-cheung Cheung.
CHAPTER SYNOPSIS 1. Evaluation of Higher Education Mu-lin Lu, in the first of the case reports, focuses on the connection between higher education evaluation systems and the trend towards globalization. In Taiwan, under the revised University Act promulgated in 2005, the Minister of Education (MOE) took responsibility for organizing an evaluation committee and formulating evaluation policies and guidelines. Subsequently, in December 2005, the MOE and Taiwan’s universities and colleges jointly established the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT). Subsidized annually by the MOE, HEEACT is responsible for the general operation and administration of university evaluation, and is expected to safeguard the objectivity and accuracy of evaluations: the government’s financial support and the enrollment size of each institution are directly dependent on these evaluations. The MOE also expects the implementation of a systematic evaluation mechanism to lead universities toward greater autonomy and management efficiency. However, as a newly-founded public organization HEEACT confronts several technical problems, including the shortage of professional evaluators, the dependence on governmental funding, and the lack of consensus regarding evaluation indicators and methods and procedures for implementing evaluation. These challenges have made the goal of achieving fairness and objectivity in evaluation more difficult to accomplish. Dr. Lu argues that setting up a long-term evaluation mechanism to
Introduction
5
monitor the performance of universities is needed in order to assure teaching and learning quality, but also that universities still need time to accommodate and assimilate the “culture of evaluation.”
2. Evaluation of Mathematics and Science Learning Ramanathan Subramaniam focuses on educational evaluation in two areas: the learning of science and mathematics and the institutional evaluation of schools. He reports that Singapore places tremendous emphasis on the development of its human resources through education and its evaluation, which is necessary for improvement and innovation, because of the lack of indigenous natural resources. He also explains why the science and mathematics curricula in Singapore continue to set a very high standard. Dr. Subramaniam starts with the current status of and future challenges confronting educational evaluation in Singapore. He argues that reliance on centralized testing by an independent agency at the end of the last year of primary school, and the collaboration of this agency with an international examinations syndicate for the examinations coming at the end of junior and senior high school, have been major reasons why Singapore’s educational system is international in its practices and standards. However, he also points out that the present system of evaluation of science and mathematics learning, though now very efficient, may have to be fine-tuned in the future. For example, it may require such reforms as the use of 2-tier multiple choice questions, which overcome the problem of random guessing of correct answers by asking students to justify their answers on another level, i.e. from another choice of options (Tregust, 1988).
3. Evaluation for School Improvement Sun-Geun Baek describes the extent to which educational evaluation has been used as one of the main tools for the development and improvement of primary, secondary and higher education in South Korea. He refers to five educational evaluation programs in South Korea: • • • • •
the School Activities Records (SAR); the National Assessment of Educational Achievement (NAEA); the College Scholastic Ability Test (CSAT); the Teacher Competence Development Evaluation (TCDE); the Teacher Education Institute Accreditation System (TEIAS).
Baek points out that there have been ongoing conflicts among stakeholders on educational evaluation policies in Korea. For example, one of the teacher unions is strongly against the implementation of TCDE. Some teacher educators and institutes are against the implementation of TEIAS; furthermore, some teachers and professors disagree with changes in the educational evaluation system, and they also disagree with the implementation of the competition system. Nevertheless, Baek asserts that the future prospects of education in South Korea will be bright because Koreans not only acknowledge the importance of education but also are making substantial investments in education in order to build a powerful state.
6
Samuel S. Peng and John C. K. Lee
4. Evaluation for Quality Assurance Sirichai Kanjanawasee presents Thailand’s evaluation practices in relation to curriculum reform and quality assurance issues in that country. Based on the 1999 National Education Act, the establishment of educational standards, curricular revision and quality assurance are all part of the educational reform program. Kanjanawasee has introduced four levels of basic education in Thailand, and has indicated the need for effective in-service teacher training and for prototypes for integrated reading, thinking and writing assessment tools and procedure manuals. The national education policy should guarantee high-quality management by linking external and internal quality assurance mechanisms. In addition, there are challenges facing the development of the O-NET (Ordinary National Educational Test), which embodies content standards for different school curricula, as well as the A-NET (Advanced National Educational Tests), which are correlated with well-defined professional disciplines. Kanjanawasee also points out that much further research is needed in order to support system designs that are both meaningful and fair for all concerned.
5. Evaluation for Mixed Purposes and Educational Reform John Chi-Kin Lee emphasizes the problem of keeping the right balance between decentralization and accountability of educational evaluation, and invites readers to think of evaluation as being essentially an “accompaniment” rather than a “monitoring” device. Lee’s chapter provides a brief overview of current educational evaluation measures and issues in Hong Kong. With regard to such quality assurance mechanisms as school self-evaluation (SSE), external school review (ESR) and Territory-wide System Assessment (TSA), he leaves three questions open for discussion. First, how can we maximize the coherence, in both rhetoric and practice, between and among educational reform, educational supervision and evaluation policies? Second, how can we promote a systematic linkage between centralized and localized systems so as to drive effective school improvement and teacher development? Third, how can teachers’ workloads be reduced and teachers be relieved of their anxieties regarding educational evaluation, and how can we foster a harmonious educational-evaluation “community” of government agencies and teachers?
6. School Self-Evaluation Part II of the volume contains five studies of recent innovations in educational evaluation in East Asia. Nicholas Sun-Keung Pang introduces the details of a series of school-university partnership projects on school self-evaluation in Hong Kong SAR. He also reports on an ongoing investigation into the effectiveness of implementing self-evaluation in Hong Kong schools and the factors that have hindered and /or facilitated this self-evaluation movement. Pang suggests that there are two approaches to assuring the quality of education in Hong Kong: an external mechanism via inspection and an internal framework via self-evaluation. He concludes that, in order to facilitate change in schools, administrators should have the power and leadership capacity to unify a given school’s goals and specify the technology for achieving them, to promote consensus and the sharing of values among all members, and to
Introduction
7
focus attention by carefully selecting targets, controlling resources, and acting forcefully. Pang further suggests that effective leaders are those who can be flexible in adopting approaches to change that can cope with the challenges created by the ever-changing external environment, and in leading their organizations toward excellence.
7. Evaluation of Educational Progress Suan Yoong provides a critical review of the student assessment system currently used in Malaysia. Professor Yoong outlines the effects of the national assessment system, provides an overview of the strengths and weakness of an over-obsession with examinations, and explains the dominant forces shaping educational assessments in Malaysia, carefully taking into account the cultural contexts. In presenting the challenges faced by the implementation of school-based assessment (SBA) in Malaysia, Yoong points out that the examination-oriented tradition imposes a great pressure on schools as well as students to excel on examinations. Such pressure has led schools to attempt to control what students are to study, narrowing the school curriculum to include only those subjects and topics that are expected to be tested on exams and adopting policies that are geared solely toward helping students pass the highstakes examinations. He introduces the serious problems that have arisen from this narrowminded approach, including the failure to measure higher cognitive skills, the deprofessionalization of teachers and the taking of all the fun out of going to school. Yet Yoong also expresses concern that the Ministry of Education’s latest effort in examination system reform might find it difficult to overcome Malaysia’s deeply entrenched, examinationoriented culture, a culture that places the gaining of high test scores as the top priority of teaching and learning.
8. Evaluation of School Performance The chapter by Yasuhiko Washiyama deals with the notion of “autonomous school administration” in Japan and that country’s subsequent evaluation practices in response to the neoliberal ideas of deregulation and decentralization. Initially, Washiyama provides an historical overview of the school evaluation system in Japan. In the contexts of deregulation and decentralization, two evaluation-related policies have been developed: the Regulative Standards for School Establishment, revised in 2002, include new provisions on selfinspection and self-assessment; the Basic Policy of Economic Financial Administration and Structural Reform of 2005 mandates that all schools providing compulsory education be subjected to external evaluation. Professor Washiyama then introduces the Evaluation Guidelines for Schools of Compulsory Education. The purpose of these guidelines is to establish a Plan-Do-Check- Action management cycle for evaluation, to improve the quality of education, to provide for self-evaluation and external evaluation, and to create a more open school atmosphere. These guidelines cover the transition from the traditional quality control system to a system focusing on outcomes, various criteria for evaluation, more focused evaluations, and the establishment of a third-party evaluation organization. He also raises concerns about school evaluation, including the problem of a too-heavy evaluation workload for teachers and the fact that evaluation may become a form of routine paper work.
8
Samuel S. Peng and John C. K. Lee
Washiyama argues that these problems could be solved by the implementation of shadowing, the transition from school evaluation to school diagnosis, and the facilitation of collaboration among teachers. He further emphasizes that evaluation results must be communicated to teachers, students, parents and community members as favorable forces for improvement.
9. Evaluation of Teacher Performance Pruet Siribapitak first describes the number and qualifications of teachers in Thailand. He then discusses current issues in teacher evaluation, and explores the future prospects of teacher evaluation in that country’s public schools. Siribapitak discusses the various types of teacher evaluation for public school teachers, including induction program evaluation, annual performance evaluation, and academic status evaluation. He also points out that the Teacher Civil Service Commission of the Ministry of Education had set up a taskforce to review the teacher evaluation criteria in terms of three new dimensions: teacher discipline, morality and ethical conduct; the quality of teaching and student development; and the results of teaching and student development. Professor Siribapitak concludes that it might be more productive to design systems, not for their formative or summative orientations but in terms of the decisions that need to be made at different levels, e.g. by teacher, the principal, the school district, and the MOE.
10. Evaluation of Student Achievement In the final chapter, Kwok-Cheung Cheung offers an insightful analysis of educational evaluation and learning improvement in Macao. He first states that Macao’s educational system is characterized by a majority of privately-operated schools, and that the schools enjoy full administrative autonomy and complete freedom of instruction. Thus, there is not yet any form of external evaluation from kindergarten up to the terminal senior secondary level. The general public basically does not have a clear understanding of the educational achievement of schools, and educational evaluation is an area gravely in need of further scrutiny. PISA (the Program for International Student Assessment) was adopted as a flagship research program conducted by the Educational Testing and Assessment Research Centre (ETARC) of the University of Macau. Professor Cheung claims that PISA “serves as a catalyst for capacity building for educational evaluation in Macao.” There are five benchmarks: • • • •
Overall performance of the 15-year-olds in reading, mathematical and scientific literacy; Equity in the distribution of learning opportunities as measured by the impact of social, cultural and economic status of the home on literacy performance; Consistency of literacy performance standards across schools; Differential gender differences demonstrated in reading as well as in mathematical and scientific literacy;
Introduction •
9
Learning environments that can facilitate the life-long learning goals of 15-year-olds. Macao’s PISA then exemplifies a unique and effective international evaluation program
PERSISTING CHALLENGES AND CONDITIONS CONDUCIVE TO POSITIVE CHANGES In short, the authors of the essays in this book provide comprehensive information about their countries’ evaluation systems, programs and policies at various educational levels, and show how these are responding—or trying to respond—to the unique needs of their specific educational contexts. They also point out that there are still many problems to be solved. First of all, the evaluation of student achievement for the purpose of allowing entrance to higher levels of education, including senior high schools and colleges and universities, remains a challenge. Unlike the United States, most East Asian countries have a centralized system that requires students entering senior high schools and colleges and universities to take a national examination. Admissions are solely based on students' performance on a test which covers only selected subjects. “As a result, students tend to pay less attention to learning in classrooms and concentrate more on private tutorials in particular subjects tested in the entrance examination” (Kanjanawasee, p. 75). Such a system often hampers the normal developmental process of a pre-college education. Many individuals’ true potential cannot be fully expressed and developed because of the lack of opportunities and appropriate evaluation mechanisms. Thus, how to screen and identify studetns’ talents and abilities, and how to assure the high quality of education deserved by these students, remain serious questions. Thailand’s new program, which requires the use of high school grades and students’ performance outside of school, offers an alternative model for college admission practices. However, its success remains to be tested. The second challenge is how to evaluate teacher quality and performance in the context of fundamental educational reform. In particular, it remains hard to determine which characteristics of teachers are critical for effective teaching and other professional duties. From the perspective of East Asia’s Confucian heritage, a teacher should be able to fulfill three missions: cultivating a virtuous person, teaching life skills, and removing students’ doubts. This implies that teachers will not only need professional skills for effective teaching: they must also be able to serve as good role models for their students. In this age of specialization and professionalism, the traditional image of a teacher-saint has gradually faded away. Rather, there is a call for teacher-scholars who, as in China and Japan, are expected to do original research as well as teach effectively. Things have become more complicated in an era when more and more educators seem to view their students as mere “consumers.” Is it possible to talk about professional ethics when the pedagogical relationship has become a pedagogical contract that places more emphasis on responsibilities and rights than on the potential development of individuals? There may be two ways to approach the problem of teacher evaluation in our current society. One is normative, and seeks to define a qualified, professional teacher. The other is technical, and seeks to evaluate teachers to see if they are qualified and professional. Of course, we must have the definition or the standard before we can have evaluation. Thailand’s teacher evaluation program includes the evaluation
10
Samuel S. Peng and John C. K. Lee
of teachers’ ethics and moral behavior, and this approach is worthy of the consideration of other East Asian educators. Moreover, a national evaluation program needs to evaluate both for accountability and for innovation (summative and formative evaluation). Yet can we really rank schools and colleges only for the sake of accountability? And how can we lower the pressure placed on students competing for the best schools and colleges based solely on achievement test scores? And should qualitative evaluation be used, in order to improve both instruction and learning, rather than or together with the giving of merely quantitative ranks and grades? These are some of the several key issues that still need to be resolved. Finally, as shown in some of the essays in this book, successful implementation of educational evaluation measures necessitates supportive school leadership (Chapter 6), reduction of teacher anxieties and workloads (Chapter 5), enhanced communication among stakeholders (Chapter 8), and the fostering of shared decision-making across all levels, including government, school, parents and the community (Chapters 5 and 9). But to achieve these prerequisite goals will take the collective wisdom of scholars in the field of educational evaluation. In conclusion, evaluation is an integral part of the educational reform and innovation process. Assessment has the potential to act as a driving force for achieving educational excellence, but it must be carefully designed and implemented. Inappropriately designed educational evaluation programs or systems—those which cannot fit their contexts—may have a negative impact that could be very harmful to education. For example, in many East Asian countries where high-stakes examinations are prevalent, most students cram for exams and are good at taking tests but they have little time to pursue their own personal development, explore their talents, or reflect on the meaning of life. In other words, the pursuit of high scores on tests becomes in itself the central theme or purpose of their life. Yet as Lu in Chapter 1 points out, effective educational evaluation—with the ultimate goal of reforming or improving the educational system—needs the cooperation of the whole community: the parents (and larger public), students, teachers, school administrators and government officials. It will be above all this common or communal will that can take us to where we want to be.
REFERENCES De Grauwe, A. and Naidoo, J. P. (eds.). (2004). School evaluation for quality improvement: An ANTRIEP report. Paris: International Institute for Educational Planning, UNESCO. Kennedy, K. J. and Lee, J. C. K. (2008). The changing roles of schools in Asian societies: Schools for the knowledge society. London and New York: Routledge.
PART I. PRACTICES AND CHALLENGES
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 1
EDUCATION EVALUATION AND ASSESSMENT IN TAIWAN: STATUS, INNOVATIONS AND CHALLENGES Mu-lin Lu Ministry of Education, Taiwan, ROC
I. PREFACE Today, in the 21st century - the age of knowledge economy - a country’s competitiveness is measured by its ability to apply knowledge towards economic progress and the pursuit of excellence. In this light, higher education is regarded as the means to cultivating a nation’s high caliber workforce. As such, governments worldwide are performing evaluations of universities to ensure high quality within institutions of higher education, as well as to measure administrative performance and to determine how to allocate funding and resources. The number of Taiwan’s higher education institutions has expanded rapidly over the past decade, more than 70% of Taiwan’s high school and vocational school graduates advance to college or university. Therefore, the importance of controlling and safeguarding the quality of higher education continues to grow. Establishing and implementing an evaluation system is essential for improving the quality of university education. To this end, the Ministry of Education (MOE) authorized the independent Taiwan Assessment and Evaluation Association in 2004 to conduct evaluations of all universities in Taiwan. These evaluations focused primarily on university administration and secondly on academic disciplines, with the universities evaluated in groups. The evaluation results were announced as ratings. This round of evaluations motivated universities and colleges to make a greater effort toward selfimprovement, and it also triggered new thinking regarding the development of university evaluations in Taiwan. Based on the experiences of advanced countries, the MOE established the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT) in December 2005 to serve as an independent, dedicated agency handling all matters relating to the evaluation and assessment of higher education institutions. The goal of establishing of HEEACT aimed to make it easier to share evaluation experiences, accumulate data, develop
14
Mu-lin Lu
professional and objective evaluation criteria, train professional evaluators, and build a talent pool and database. In the following, we explain the historical development of Taiwan’s university evaluation system, its future development and current challenges.
II. STATUS A. Legal Basis – The University Act: The MOE’s Responsiblities in Evaluating Universities The revised University Act was promulgated in December 2005. Article 5, Paragraph 1, of the Act states that “Universities shall regularly carry out self-evaluations of teaching, research, services, academic affairs, administration, student participation and other proceedings; regulations for evaluations shall be formulated by the universities.” Article 5, Paragraph 2 stipulates that “In order to promote the development of universities, the Ministry of Education shall organize an Evaluation Committee, commission academic organizations or professional evaluators to carry out regular evaluation on the universities, and publish the results as a reference for educational subsidies from the government and the scale of adjustment and development of universities; the methods of evaluation shall be formulated by the Ministry of Education.” These two passages clearly stipulate that carrying out university evaluations is the joint responsibility of the MOE and the universities. We can only attain the goal of improving the quality of higher education if the two parties work together in a concerted effort to plan and promote university evaluations with a sense of direction.
B. Developing a Higher Education Evaluation Agency Suitable for Taiwan 1. The Establishment of Higher Education Evaluation Agencies a. Foreign Aapproaches. Evaluation agencies in other countries can be divided into three categories: “official” organizations such as in France; institutions that are independent from government agencies such as in Britain and Australia; and private-sector organizations such as in the United States. These three types of agencies share in common an emphasis on safeguarding objectivity and fairness, and thus their goal is to create and maintain impartial and unbiased institutions for performing professional evaluations. Their sources of funding can be roughly divided into government subsidies (Britain and France), membership fees (U.S.) and a combination of government subsidies and membership fees (Netherlands, Australia). In terms of function, these organizations carry out both institutional evaluations and academic discipline evaluations (also called specialized field evaluations). Depending on their focus, academic discipline evaluations can again be further divided into teaching evaluations and research evaluations. b. Taiwan’s Approach. Taiwan lacked an evaluation mechanism and a dedicated evaluation agency until only recently, when the Ministry of the Interior approved the establishment of the Taiwan Assessment and Evaluation Association (TWAEA) in August 2003. TWAEA is a private-sector agency that has carried various evaluation work on behalf of the MOE: These projects included the University Evaluation Planning and Implementation
Education Evaluation and Assessment in Taiwan
15
Plan of 2004, the University and College Faculty Development Center Evaluation Planning and Implementation Plan of 2005, the Evaluation of Universities of Science and Technology of 2005 and 2006, and the University Administration Follow-up Evaluation Plan of 2006. In addition, the MOE and Taiwan’s universities and colleges came together in December 2005 to jointly establish the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT), a public organization with the goal of serving in a capacity similar to the evaluation agencies of Britain and Australia, that are known for their “independence from official agencies.” However, as a sound evaluation institution cannot be created overnight, the MOE is currently supervising and assisting HEEACT to ensure that its development proceeds smoothly and efficiently. Presently, the MOE subsidizes funding that HEEACT requires for planning and evaluations, as it gradually works to establish a systematized, institutionalized dedicated agency for university evaluation. 2. Stage-by-Stage Establishment of a Higher Education Evaluation System a. Present Goals. Present goals for HEEACT include: to have HEEACT plan and implement all types of evaluation work so as to accumulate practical experience; to compile and study information on higher education evaluation in advanced countries, produce objective, professional evaluation criteria, and provide courses for the training of university evaluators; to develop the council into an evaluation planning institution; and to set up a diversified evaluation mechanism. b. Medium-to-Long Term Goals. Medium to long-term goals for HEEACT include: to gradually develop the Council into a dedicated, professional evaluation institution with the task of researching and planning the evaluation of Taiwan’s universities; to set up databases and talent pools; and to pass on these evaluations to other specialized evaluation institutions. 3. Present University Evaluations Conduct a. Putting into Practice University Self Evaluation. The University Act stipulates that universities shall regularly conduct self evaluations. Moreover for the sake of sustainable development, universities shall safeguard the quality of their administrative affairs and assume responsibility for administrative performance and improving their image and reputation. b. Assisting TEEAC in Implementing University Evaluation Work. Current Major Tasks: i. Planning and researching Taiwan’s university evaluation system. ii. Compiling and studying information on the evaluation of universities overseas. iii. Assisting universities in applying for international accreditation in all academic disciplines. iv. Developing and establishing qualification criteria for domestic and foreign professional evaluation institutions. v. Establishing talent pools and databases for the evaluation of universities in Taiwan. vi. Providing courses for the training of university evaluators. vii. Other evaluation-related matters. c. Implementing the University Department and Graduate School Evaluation Plan (20062010) based on a Five-Year Cycle i.
Goals of university department and graduate school evaluation
16
Mu-lin Lu a.
To gain an understanding of the current quality of university departments and graduate schools b. To determine and recommend status and deadlines for the quality accreditation of university departments and graduate schools. c. To encourage universities to establish quality improvement mechanisms for their departments and graduate schools. d. To assist university departments and graduate schools in developing unique characteristics and the pursuit of excellence. e. To use evaluation results as reference in formulating higher education policies for the government ii. Unique features of university department and graduate school evaluations a. b.
c. d. e. f.
Characteristic differences of university departments and graduate schools are respected; fixed quantitative indicators are not used University departments and graduate schools must explain on their own initiative to what extent they have progressed in fulfilling their founding mission and goals, in line with evaluation criteria. Evaluation content focuses on the assurance of teaching quality, but also pays attention to research quality. Evaluation process emphasizes that university departments and graduate schools should establish and implement self improvement mechanisms. Criteria-based self comparison is emphasized, rather than comparison with other universities or university departments and graduate schools. Evaluation results are reflected in an accreditation system, consisting of three categories: accreditation granted, accreditation on hold, and accreditation denied. Applicants that do not meet the criteria shall withdraw. Those whose accreditations that are on hold or have been denied shall accept a follow-up evaluation in the following year.
iii. Items covered in university department and graduate school evaluations Criteria 1: Goals, special characteristics and self improvement Criteria 2: Course design and teaching Criteria 3: Student learning and student affairs Criteria 4: Research and professional performance Criteria 5: Graduate performance iv. Dealing with evaluation results: Accreditations for university departments and graduate schools that are put “on hold” mainly for the following reasons: insufficient number of full-time teachers; insufficient learning space and resources; unclear goals; or course design does not reflect the goals of the university department/graduate school and lacks self improvement mechanisms. Accreditations are denied if the above-mentioned problems are severe enough that student learning and teaching quality are seriously affected. University departments and graduate schools whose accreditations have been granted shall submit a self improvement plan and
Education Evaluation and Assessment in Taiwan
17
implementation objectives based on the suggestions in the evaluation result report. They will not have to undergo another evaluation for the following five years. For those whose accreditations have been put on hold, HEEACT will conduct a follow-up evaluation regarding their shortcomings. Meanwhile, schools ‘on hold’ may not expand student recruitment or apply for the establishment of new graduate schools. For those whose accreditations have been denied, HEEACT will carry out a reevaluation of all items and the MOE will consider a reduction of student quotas. Should accreditation be denied after a follow-up evaluation or a reevaluation, student recruitment will be stopped. 4. Continuing with follow-up evaluations of university administrations 5. Encouraging universities to participate in international accreditation As Taiwan’s universities seek to develop unique characteristics and reach global standards, we cannot only use evaluations and assessments to strengthen the quality and development direction of academic disciplines. Universities and colleges are being encouraged to actively participate in international accreditation schemes as a way of obtaining international recognition. University departments and graduate schools that have already obtained international accreditation will be exempt from further evaluations during the accreditation period. Participation in international accreditation schemes will raise the quality of higher education and speed up the internationalization of universities in Taiwan, while also winning them the recognition in international academia. Universities and colleges in Taiwan currently participate in the following international accreditation schemes: a.
Medical school evaluation:
Since medicine is a specialized field, medicine department evaluations are planned and conducted by the Taiwan Medical Accreditation Council (TMAC). In 2002, TMAC’s evaluation system was approved by the National Committee on Foreign Medical Education and Accreditation in the United States. b. Engineering education accreditation: In 2005, engineering education accreditation system and schemes established by the Institute of Engineering Education Taiwan (IEET) were approved by the Washington Accord (WA) signatories. Engineering departments that have gained IEET accreditation can directly obtain WA recognition. Twelve engineering departments gained IEET accreditation in 2004, followed by 35 departments in 2005, and 41 departments in 2006, which shows that Taiwan’s engineering departments are actively participating in international accreditation. c.
Accreditation by the Association to Advance Collegiate Schools of Business (AACSB) in the United States:
The business management departments of Fu Jen Catholic University and National Sun Yat-sen University gained AACSB accreditation in April 2005, while National Chengchi University followed suit in December 2006. The business management departments of other universities including Asia University, Feng Chia University, National Cheng Kung University, National Chiao Tung University, National Chung Cheng University, National
18
Mu-lin Lu
Taiwan University, Tamkang University, and Yuan Ze University have applied for AACSB accreditation.
III. INNOVATIVE ACTIONS To truly achieve the goal of raising the quality of higher education, we must shed longheld thinking and approaches to university evaluations, and bring in new energy to create new possibilities for the future. In the following, we explain innovative actions being taken with regard to the university evaluation system, the role of universities and the application of evaluation results.
A. Creating a Well-Designed Evaluation System Evaluation work in Taiwan needs to develop toward systematization and institutionalization. We aim to create a well-rounded and sound evaluation system, building upon every single evaluation process. In order to do this, we must establish a team of evaluators and build a database, while properly integrating research, planning, implementation and auditing. Article 4 of the Regulations Governing University Evaluations stipulates that university administrations, departments and graduate schools, as well as course curriculums shall be evaluated every five to seven years. By putting into practice such evaluation work and building an evaluation system, we are bringing more effective management to our universities.
B. Leading Universities Towards Greater Autonomy Presently, Taiwan’s evaluation system is led by the government, with passive participation from universities. However, as the evaluation system evolves and universities mature with the evaluation process, we hope that all schools will understand the need for long-term self improvement. We also hope that evaluation results released by a professional, fair agency will serve as a reference for schools seeking to make improvements and develop unique characteristics. As soon as this practice takes root and a dedicated evaluation agency is able to function in its full capacity, we could consider establishing an evaluation system under which universities proactively apply for evaluation.
C. Improving the Quality of University Management, Efficiently Allocating Education Funding In Britain, higher education evaluation covers the two areas: teaching and research. The Quality Assurance Agency for Higher Education (QAA) is in charge of institutional and teaching evaluations, while the Research Assessment Exercise (RAE) is mainly in charge of evaluating research. Both agencies submit their evaluation results to the Higher Education
Education Evaluation and Assessment in Taiwan
19
Funding Council for England (HEFCE), which distributes grants and funding to higher education institutions based on these results. While the establishment of a systematized university evaluation system in Taiwan is still in the initial stages, we hope that the implementation of university evaluations will compel universities to attach greater importance to administrative performance. Moreover, the MOE will refer to evaluation results when deciding on matters relating to university development including funding, grants, and subsidies, the increases or changes to colleges, departments, graduate schools, and student quotas. The MOE’s overall aim in this regard is to achieve more efficient use of higher education resources, as well as their proper allocation.
IV. CHALLENGES In comparison with many advanced countries, the implementation of a university evaluation system in Taiwan is still in the initial stage. Numerous systemic or technical problems still need to be solved. Eight major challenges facing Taiwan’s university evaluation are listed below:
A. Training Professional Evaluators Evaluators presently selected by the dedicated evaluation agency can be divided into three categories based on the type of evaluation: 1. Veteran professors with research or practical experience in higher education administration; 2. Business or industry representatives familiar with specialized academic fields or university affairs; 3. Experts and scholars specializing in certain specialized fields. TEEAC’s evaluators hold an abundance of experience and knowledge in university administration or teaching in a specialized field. However, as university evaluation is a highly specialized undertaking, we need to address how the evaluation expertise of the evaluators can be raised through relevant courses, seminars and workshops.
B. Increasing Evaluation Funding In addition to TWAEA, which as a private-sector, nongovernmental organization, is independently funded and operated, the MOE and 153 universities and colleges have donated money to fund the Higher Education Evaluation and Accreditation Council of Taiwan (HEEACT). Since the council is a public organization, the MOE is fully subsidizing its operating costs during its initial stages. In the future, the MOE hopes that HEEACT will be able to engage in its own funding and create other revenue once operations are running smoothly and the entire system starts to take shape.
20
Mu-lin Lu
C. Designing Evaluation Indicators and Methods One example of the effort to design evaluation indicators and methods are the evaluations currently used for Taiwan’s university administrations, departments and graduate schools and colleges of medicine, where evaluation results come in the form of suggestions for improvement submitted by evaluators. This presents a new challenge to the schools and the public, who are accustomed to thinking in quantitative terms and are likely to note that evaluators evaluating different universities, departments and graduate schools are not the same. Our next step is to convince the schools and the public of the completeness of these indicators, the scientific soundness of the evaluation methods and the truth of the evaluation results.
D. Implementing Self Evaluation As the sanctuary of academia and high-level professional training, universities must assume responsibility for their own administrative efficiency, their research performance and social services. Moreover, universities should have a full understanding of their current strengths and weaknesses, to be gained continued and random self evaluations. Given this fact, universities can only achieve the goal of sustainable development if evaluations are incorporated into their organizational culture. However, Taiwan currently lacks a strong consensus regarding the essence and methods of university self evaluation. As a result, universities in Taiwan remain passive about accepting evaluation models based on outside performance requirements, which has not been helpful in their long-term development. Article 5, Paragraph 1 of the revised University Act stipulates that “universities shall regularly carry out self evaluation on teaching, research, services, academic affairs, administration, student participation and other proceedings; regulations for the evaluation shall be formulated by the universities.” In the spirit of this passage, the MOE will continue to encourage universities to create regulations for self-evaluation and to regularly carry out self-evaluations.
E. The Fairness of Evaluators In choosing the members of an evaluation committee for the evaluation of a university department or graduate school, HEEACT first submits a list of recommended evaluators to the department or the graduate school that is to be evaluated. The department or graduate school in question may reject committee members for any substantial reason. In principle, we already meet the general public’s expectations for an open and transparent evaluation procedure. Still, since Taiwan’s academic circles are rather small, it is difficult to completely draw a line between professors teaching at a department or graduate school that is being evaluated and the members of its evaluation committee. We still need to deliberate further on how we can truly ensure fairness and objectivity, and avoid conflicts of interest, when selecting evaluation committee members, as well as whether or not we need to bring in foreign scholars and experts.
Education Evaluation and Assessment in Taiwan
21
F. Procedure for Generating Evaluation Results A number of universities in Taiwan began department and graduate school evaluations in 2006. The review process for the evaluation results was mainly divided into the following three steps: committee members submit an evaluation report following an onsite visit, then the Academic Discipline Preliminary Review Committee reviews the report, and lastly a decision is made by the Accreditation Review Board (under the Council). If there are excessive discrepancies between the “Onsite Visit Draft Report” that is sent to all institutions under evaluation and the second-stage “Suggestions on Evaluation Results” submitted by the Academic Discipline Preliminary Review Committee or the third-stage “Report on Accreditation Results” by the Accreditation Review Board, the evaluated institutions will easily question the fairness of the evaluation process and reject the results. Therefore, we must consider how the procedure that generates the evaluation results can be designed with even more precision so that the institutions undergoing evaluation are not shocked when they obtain the results; that the evaluation results truly reflect the situation of the evaluated institution; and that there are no large discrepancies between the results and the institution’s situation during the onsite visit period.
G. Ways of Presenting the Evaluation Results Evaluation results can be presented as rankings, ratings, or the selected best. In the case of university administrations, and departments and graduate schools in Taiwan, the evaluation results are announced in the form of ratings, which complies with the spirit of the accreditation system. Furthermore, the criteria for deciding in favor or against accreditation include whether relevant measures taken by the institutions match their stated goals and whether they have implemented self improvement mechanisms. Feedback from the public shows that people ask themselves why the MOE does not announce university rankings to enable citizens to immediately clearly understand the strengths and weaknesses of each university and to help students make their choices. In actuality, the announcement of the evaluation results pertains to the goals and spirit of university evaluation. Therefore, there are certain differences in the way evaluation results are announced, depending on the type of evaluation and the evaluation goal. In the future, the MOE will hold more in-depth discussions with the evaluating agency regarding the form in which evaluation results will be announced.
H. Evaluation Follow-Up Mechanism The focus of evaluations is not based on grade or rank, but rather to gain an understanding of teaching quality and students’ learning performance in a systematic, scientific, and objective way. Thus, a long-term evaluation mechanism must be established so that the results of previous evaluations can serve as reference for monitoring a university’s performance. Only then will universities be able to keep improving their shortcomings and move toward excellence.
22
Mu-lin Lu
CONCLUSION In light of the trend towards globalization and internationalization, Taiwan’s higher education must be able to improve its quality, ensure performance, find its niche and develop unique strengths to gain a sure footing on the international stage and move towards excellence. University evaluation work plays a major role in this process. Our higher education evaluation work will succeed only if it is systematically and scientifically planned, implemented and reviewed. Moreover, we need to keep studying effective approaches used by advanced nations to serve as lessons for us. We hope that Taiwan’s higher education evaluation - with the assistance of an attentive public, active university participation, efficient government action and a well-rounded evaluation system-will be able to help our universities to achieve the vision of quality assurance and assume genuine responsibility for their performance.
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 2
EDUCATIONAL EVALUATION IN SINGAPORE: CURRENT STATUS AND FUTURE CHALLENGES R. Subramaniam National Institute of Education, Nanyang Technological University, Singapore
ABSTRACT As an island state, which is bereft of any natural resources, Singapore places tremendous emphasis on the development of its human resources through education. It has consistently ensured that its curricula are not only internationally competitive but its examination system is also benchmarked against international standards. Educational evaluation is a critical aspect of this quality control. This article focuses on educational evaluation in two areas: learning of science and mathematics at the primary, secondary and pre-university levels, and institutional evaluation of schools. It is shown that the use of external evaluation for assessing learning: centralized testing at the primary leaving level by an independent authority, a partnership model of this independent authority with an international examinations syndicate for testing at the secondary and pre-university leaving levels, and regular testing of science and mathematics achievement of students at the primary and secondary levels by an international evaluation association, have been factors which explain why the science and mathematics curricula in Singapore continue to be of a high standard. The use of academic attainment of students to band secondary schools and junior colleges in separate achievement tables, plus the inclusion of other academic and non-academic factors in these achievement tables is a key aspect of institutional evaluation, and explains why stakeholders in the country have been able to derive satisfaction from the government’s heavy investments in education. Some implications of these modes of educational evaluation are discussed.
Keywords: educational; institutional evaluation; Singapore; science and mathematics achievement; banding of schools.
24
R. Subramaniam
INTRODUCTION Education is probably the most important lever to empower a person in life as well as stimulate socio-economic development in a country. High levels of education of the citizenry have traditionally been correlated with enhanced developmental activities in a country. The challenges posed by globalization, new developments in science and technology, diffusion of information and communication technology in society, outsourcing, off-shoring and corporate downsizing have impacted tremendously on countries in the past ten years. Jobs have taken flight to countries which can provide best value for money. Since these developments affect the competitiveness of nations, they therefore impact not only on the employment of their workers but also on their education system since this prepares students for life in the workforce. Educational reform has been an important way in which many countries have been addressing these issues to varying extents, so that their students are better able to meet the challenge of the new economic landscape. Science and technology are potent agents for stimulating socio-economic development. The land-labor-capital model of economic development is a yesteryear paradigm that is increasingly giving way to a model based on science- and technology- driven economic development. The experiences of Western countries show that this development model has been very helpful in providing their people with a good standard of living. A number of countries in Asia have also successfully experimented with this model of economic development for nation building, for example, Japan and Singapore (Tan and Subramaniam, 1998). One of the key aspects of this model is that a country’s science and mathematics curricula need to be internationally competitive. Regular benchmarking of standards will help to ensure that educational reform is proceeding in the right direction and that the education system is well positioned for achieving the desired outcomes. Educational evaluation is a powerful tool to assess various aspects of the education process as it can produce performance data for the use of stakeholders. The origins of educational evaluation can be traced to the desire for quality control in the education process so that standards and accountability can be promoted. The large number of schools in a country, the diversity in their curricula and differences in assessment procedures at various levels pose issues of a systemic nature which, if not addressed properly, can affect standards. Educational evaluation affords directions in this regard as it can help to benchmark schools and subject offerings at the various levels, thus ensuring that they measure up to the desired norms. The epistemological foundations of educational evaluation have matured well over the past 40 years, and several evaluation instruments and methodologies are available to assess the efficacy of various aspects of the education process. Its importance has led to the formation of a number of professional bodies - for example, the International Association for the Evaluation of Educational Achievement (IEA), which has done yeoman service since its inception in 1959. The findings of this association have appeared in international journals and the media, and are highly regarded by the political establishment in several countries. The principal objective of this paper is to share some insights into the process of educational evaluation of science and mathematics learning at the primary, secondary and pre-university levels in Singapore. An additional objective is to describe an evaluation approach that has been used to appraise schools in the country. Whilst traditional modes of educational evaluation are deeply embedded in the education system in Singapore, the use of
Educational Evaluation in Singapore
25
educational evaluation of learning via leaving level examinations administered by an independent local authority or in conjunction with an international examinations syndicate provides the best assurance of quality control and academic rigor. Institutional evaluation must perforce be based predominantly on such results.
EDUCATION SYSTEM IN SINGAPORE To better appreciate the context of the ensuing discussion, a brief description of the education system in Singapore is given in this section (Lim and Tan, 1999). The medium of instruction in schools is English, except for mother tongue languages. Primary education is of six years duration. Students enter primary school at the age of seven. The core subjects are English, science, mathematics and mother tongue – mathematics is offered from the start but science is introduced only at the primary three level. At the end of primary four, there is a national streaming examination (Caleon and Subramaniam, 2005). Those who are channeled into EM1 and EM2 streams in Primary five are of above average abilities compared to those in the EM3 stream. Students in EM1 stream can take an extra subject (Higher Mother Tongue). However, from 2004, the EM3 stream was abolished. At the end of primary six, students sit for the Primary School Leaving Examinations. Based on the results of the Primary School Leaving Examinations, students are channeled into one of three streams at the secondary level: Special / Express, Normal or Technical. Such streaming allows students to learn at a pace that is best suited for their intellectual capabilities. Students in the special / express streams take the Singapore-Cambridge General Certificate of Education ‘Ordinary’ Level Examinations at the end of four years. Those in the latter two streams take this at the end of five years – prior to this, they would have taken the Singapore-Cambridge General Certificate of Education ‘Normal’ Level Examinations at the end of four years. The results of the Singapore-Cambridge General Certificate of Education ‘Ordinary’ Level Examinations determine how students progress further. Those with average grades can opt for polytechnic education to attain a diploma in a vocational discipline; those with weak grades can opt for technical education, which will equip them with a certificate in a technical specialty; and those who do well can opt for pre-university education at one of the junior colleges or a centralized institute – students enrolling in the former take the SingaporeCambridge General Certificate of Education ‘Advanced’ Level Examinations after two years while those who enroll in the latter take these after three years.. Those with good grades in the latter examinations can opt for university education to get a degree. The Ministry of Education is the agency responsible for overseeing education matters in the country. A particular source of strength is its implementation machinery, which ensures that policy matters are effectively translated into practice in the 350+ schools in the country.
EDUCATIONAL EVALUATION OF SCIENCE AND MATHEMATICS LEARNING AT THE PRIMARY LEVEL The curricula for science and mathematics at the primary level are prescribed by the Ministry of Education. At each level in the 6 years of education, individual schools
26
R. Subramaniam
administer their own examinations in all the subjects. Needless to say, there is bound to be some variability in the standards and difficulty levels for the various subject examinations. It is only at the Primary School Leaving Examinations (PSLE), which is held towards the end of primary six, that all students sit for the same examination in all the subjects and know where they stand at the national level. The PSLE is administered by the Singapore Examinations and Assessment Board (SEAB), the national independent agency for assessment and examinations. The PSLE is of a high standard for the following reasons: (a) The range of questions set in the subject examinations encompasses the different levels of Bloom’s taxonomy, with significant emphasis being placed on the higher levels. This is in view of the pronounced weightage given in the curricula on promoting thinking skills and problem-solving skills among students. (b) Even though the PSLE examination papers are not available in the public domain (as students are not allowed to take back the papers), anecdotal evidence based on the feedback provided by teachers and students as well as the attention occasionally given to these in the media indicate that the questions are intellectually challenging. (c) It is set by an independent authority, and there are rigorous quality checks on what appears in the test papers. Additional evidence that the science and mathematics standards at the primary level are internationally competitive can be drawn from the results of the Trends in Mathematics and Science Study (TIMSS), an examination set by the International Association for the Evaluation of Educational Achievement (IEA) every four years. These examinations are taken by students of many countries and afford a rigorous comparative benchmark of the science and mathematics curricula of a country. The findings from the TIMSS study reveal the following observations in relation to Singapore (Martin et al. 1997, Mullis et al. 1997; Toh et al. 2006): •
•
In TIMSS 1995, primary three students from Singapore were ranked 2nd in mathematics, while primary four students were ranked 1st in mathematics and 7th in science In TIMSS 2003, primary four students from Singapore were ranked 1st in mathematics and 1st in science.
Juxtaposing the observations made on the PSLE with the results of the TIMSS support the assertion that the curricula and assessment practices for gauging learning in science and mathematics at the primary level in Singapore are of a high standard.
EDUCATIONAL EVALUATION OF SCIENCE AND MATHEMATICS LEARNING AT THE POST PRIMARY LEVELS The post primary education system normally comprises 4 years of secondary education and 2 years of pre-university education. At all levels, examinations are conducted by
Educational Evaluation in Singapore
27
individual schools and junior colleges on a semester basis. There is bound to be some variability in the standards and difficulty levels of these examinations. However, the milestone examinations taken at the end of secondary and pre-university education are national examinations and, in effect, provide an evaluation of how well schools have prepared their students in the various subjects. What is significant is that these leaving level examinations are conducted jointly by two independent authorities: the Singapore Examinations and Assessment Board (SEAB) and the University of Cambridge Local Examinations Syndicate (UCLES), an international examinations syndicate. The external checks and validity provided in this manner constitute a reliable endorsement of the standards of the syllabus and curricula in the sciences and mathematics. The involvement of UCLES in educational evaluation of students’ learning, based on the syllabus prescribed by the Ministry of Education, offers a number of advantages: • •
• •
It is internationally recognized and has long years of experience in setting and marking examination papers. The examinations set and marked by it provide assurance that the standards are internationally benchmarked. (A few subject papers such as second language, are, however, marked locally). Involving an external agency to set and mark the examination papers ensures impartiality. Its co-branding of the leaving level examinations offers Singapore students recognizable entry level qualifications if they are to apply for placement in overseas schools or admission to foreign universities in the USA, UK, Australia and New Zealand, four of the popular destinations for higher education.
Additional support for the international competitiveness of the science and mathematics curricula at the secondary level is provided by data from the TIMSS studies done in 1995, 1997 and 2003. The findings are as follows for the Secondary two level: • • •
In TIMSS 1995, Singapore students were ranked 1st in science and 1st mathematics. In TIMSS 1999, Singapore students were ranked 2nd in science and 1st in mathematics. In TIMSS 2003, Singapore students were ranked 1st in science and 1st in mathematics.
Against the backdrop of the foregoing, it is clear that evaluation of science and mathematics learning at the secondary and pre-university levels are rigorous from an international standpoint.
INSTITUTIONAL EVALUATION OF SCHOOLS IN SINGAPORE Institutional evaluation is an integral aspect of the process of educational evaluation. Stakeholders and the public would like to know how effective schools have been in preparing students. A good evaluation system that cuts across all schools is indispensable from a
28
R. Subramaniam
national perspective as informed decisions can then be made on improvements to be made as well as lessons to be drawn from best practices. Since 1992, secondary schools and junior colleges in Singapore have been ranked on the basis of the academic standing of their students at the Singapore-Cambridge General Certificate of Education ‘Ordinary’ Level Examinations and Singapore-Cambridge General Certificate of Education ‘Advanced’ Level Examinations respectively. This has proved to be an effective tool, in effect producing a league table of schools and colleges to see where they stand nationally (http://www.moe.edu.sg). As the ranking is on the basis of leaving level examinations administered by SEAB and UCLES, they have the advantages of validity and reliability. Since ranking was introduced, it has been noticed that standards in schools and colleges have risen. The evaluation for schools works in this way. For the Normal stream, a composite index based on L1B4 is generated for each secondary school: average aggregate score of English (L1) plus the best four subjects taken by their students in the GCE ‘O” level examinations. Using the cut-off score of 21, the schools are distributed into 5 bands. For the Special /Express streams, a composite index (L1B5) weighted on the average aggregate scores of either English Language or Higher Mother Tongue (L1) plus the five best subjects taken by their students in the GCE ‘O” level examinations is assembled for each school. In this way, 9 bands below the cut off score of 18 points are generated. The lower the band score, the better is the performance of the school. For the year 2007, there were 58 schools in Special /Express stream as well as 51 schools in the Normal stream in the School Achievement Tables. The other important factors which are included in the Achievement Tables include: •
•
•
• •
Academic Value-added Performance: this is a measure of how well the schools are expected to perform in the GCE ‘O’ Level examinations based on the PSLE results of their students and how they actually performed. Physical and Aesthetic Achievement: the data for physical fitness is weighted on three scores – percentage of students who pass the National Physical Fitness Award test, percentage of overweight students, and fitness index. Best Practices: this focuses on effective practices and systems that lead to desirable outcomes in education – for example, organizational effectiveness, students’ allround development, staff well-being, teaching and learning. Character Development: this is to reiterate the point that values education and life skills are an integral aspect of students’ holistic education. National Education: this is to ensure that students are cognizant of Singapore’s history and of how the country has achieved much despite the absence of natural resources and the presence of other constraints.
A similar evaluation system holds at the pre-university level. For institutional evaluation, data on leaving level examinations are available with the Ministry of Education. Data on other aspects are obtained from schools via various means: for example, self-reporting by schools, interviews and external evaluation by teams from the Ministry of Education. Data on rankings (or banding) of schools provide the necessary information for schools to know where they stand with respect to others nationally and also provides the momentum for
Educational Evaluation in Singapore
29
them to improve their standings. The rankings done by the Ministry of Education have the advantages that the evaluation criteria are transparent and rigorous.
DISCUSSION Educational evaluation affords a systematic way of appraising the various processes in education. If carried out by a competent authority, the information it provides can offer an impartial perspective that would be of important value to stakeholders and the public. It would, in effect, contribute towards certification of the quality of education. In this chapter, we have focused on two aspects of educational evaluation: assessment of science and mathematics learning at the primary, secondary and pre-university levels, and evaluation of how schools meet their overall objectives. In a country with numerous schools and where there are bound to be variations in the standards and the difficulty levels of the examination papers set by these schools, a common leaving examination provides the best mode of evaluating how schools have prepared their students based on the syllabus prescribed by its Ministry of Education. This process of centralized testing evens out any differences in standards, difficulty levels and anomalies in content coverage that are inherently present in school-based examinations. This does not mean that school-based examinations have their drawbacks, it is just that the common denominator afforded by the same examination taken by all students provides more rigorous benchmarking of learning. More importantly, it is of interest to stakeholders to know how effective schools have been in their mission at a national level - a common mode of referencing provided by centralized testing offers validity and reliability in this regard. At the primary level, the examinations set by the SEAB, an independent agency, provides not only quality assurance but also third party evaluation of how schools have prepared their students for the examinations. Whist this mode of appraisal based only on the leaving level (primary six) to evaluate learning may be criticized for the absence of consideration of the results achieved by students at the other levels (primary one to five), it has to be borne in mind that the PSLE is not just an examination to assess what students have learnt in primary six but generally at the upper primary level (primary four to six) as well. That the examinations set at the primary leaving level are of a high standard can also be gauged from the fact that the national agency has been requested to provide testing services at the primary leaving level for other countries in the region such as Thailand, Philippines and Indonesia. On an international basis, the TIMSS results provided by the International Association for the Study of Educational Evaluation provide additional evidence that the science and mathematics curricula in Singapore are internationally competitive. At the secondary level, again a similar situation prevails – school-based examinations at all stages except towards the end of secondary education, where a nationwide examination for all subjects is taken. The examinations are administered jointly by SEAB and UCLES. The involvement of a renowned international examinations syndicate in national examinations provides strong support that international third party evaluation has its merits as it can ensure that high standards are maintained and that the examination results are internationally recognized.
30
R. Subramaniam
Additional support for the international competitiveness of the science and mathematics curricula at the primary and secondary levels is provided by the results of the TIMSS examinations administered by the International Association for the Study of Educational Evaluation. The situation at the pre-university level is also similar. Again, with the examinations jointly administered by the Singapore Examinations and Assessment Board and the University of Cambridge Local Examinations Syndicate, educational institutions know where exactly they stand at the national level in terms of the academic achievements of their students. A major reason why many countries participate in TIMSS is that it provides data for the evaluation of the effectiveness of their education system in science and mathematics, two subjects that count significantly in the science- and technology- driven world of today, with respect to those of other countries (Johnson, 1999). The testing is expensive and logisticsintensive but the data they provide is impartial and can catalyze much needed educational reform. Purves (1991, p34) has summarized this succinctly as follows: The world could be conceived as a huge educational laboratory where different national practices lend themselves to comparisons that could yield new insights into determinants of educational outcomes. The significance given to IEA surveys of science and mathematics achievement has led to a situation where rankings of countries have become a de-facto endorsement of educational indices of these countries (Johnson, 1999), not only in the international research literature but also in the popular media. Thus, the TIMSS results provide a solid endorsement of the educational standards in science and mathematics at the primary and secondary levels in Singapore. The system of streaming in schools, in Singapore has been criticized as being elitist and that it puts a stigma on those in the less desirable stream. It is a unique practice not found elsewhere. Critics of streaming tend to ignore the fact that streaming ensures that students are channeled into classes where they can study at a pace which suits their academic predisposition. Teachers are then in a position to provide better attention and support for their learning needs. One of the criticisms leveled against the ranking system is that it places too much emphasis on academic attainment. The process of education is more than just studying, it needs to develop students holistically through other means and these ought to be given weightage in the rankings – so say the critics! Also, as the premier schools get the better students (Tan and Gopinathan, 2000), the ranking is skewed in their favor. Moreover, with the excessive emphasis on examination results, schools may focus on producing examinationsmart students and not explore innovative approaches in their teaching. (Poh, 1999; Cheah, 1998). These concerns have led the Ministry of Education in 2004 to tweak the evaluation system by expanding the number of dimensions on which assessment is based. Instead of ranking lists, there are now School Achievement Tables, one for the Special / Express stream and another for the Normal stream. In the process, schools are now banded together – those with similar academic results are grouped together and are listed alphabetically, and it is not possible for the public to know the exact position of the schools in the tables. In the new evaluation system, predominant weightage is still given to the performance of the schools in the leaving level examinations as this is central to the mission of education but additionally other academic and non-academic factors are included in the achievement tables. This ensures that schools prepare students holistically, not just academically. In the new landscape,
Educational Evaluation in Singapore
31
significant emphasis is given to the promotion of creativity, innovation and enterprise among students (Tan and Subramaniam, 2002). The high stakes examination culture has put additional pressure on teachers to deliver and students to perform. Anecdotal evidence suggests that these may have implications on family life – for example, teachers putting in longer working hours and students clocking lengthier hours of study. There is, however, recognition that Singapore is unique – it is a small country with no natural resources and its survival is very much dependent on its linkages to the global economy and attracting of investments; education thus provides the best opportunity for its people to aspire for a better life and compete in the global economy. In such a scenario, it is inevitable that sacrifices have to be made. To a significant extent, the high salaries that teachers draw help to mitigate any such concerns they may have. Another aspect of the high stakes examination culture is the concern that late developers as well as those who are not academically inclined but who excel in pursuits such as the arts and sports are at a disadvantage. To address the former, multiple routes for entry to tertiary institutes are in place – for example, a student who has been in EM 3 stream in primary school and normal technical stream in secondary school can enter university after a stint in a technical institute and then at a polytechnic, provided that admission criteria to the university are met. To help meet the aspirations of the latter group of students, a School of the Arts as well as the Singapore Sports School has been established. In fact, the arts and sports scene in Singapore has undergone a renaissance in recent times, and job opportunities abound. The Singapore experience offers the following pointers to other East Asian countries which have less centralized system of education: •
• •
Ranking (or banding) of schools needs to be seriously considered so as to inject greater accountability in the system. Whilst there may be problems and constraints in introducing this in less centralized systems where there are differing examination systems, appropriate rubrics can be designed with suitable statistical normalizations to account for these differences. A national leaving level examination can be considered for the graduating years of schooling so that schools know how they have prepared their students. There needs to be pronounced emphasis on science and mathematics in the curricula as these subjects are crucial for the economy.
FUTURE SCENARIOS The 21st century presents new challenges to countries in various areas. No where is this more pronounced than in the area of education as it has the potential to influence developments in other sectors of the economy. Countries whose education systems are well positioned to address the new challenges can ensure that their students would be able to compete effectively in the new economy. Educational evaluation will thus continue to exert importance as it affords scope for enhancing institutional practice so that schools can position themselves for the future whilst evolving organically to the next level. In Singapore, the present system of educational evaluation of science and mathematics learning – use of national leaving examinations at the primary, secondary and pre-university
32
R. Subramaniam
levels, is not likely to change as it has served her well over the years. These are high stakes testing that have proved their utility value over the years. The examinations set by an independent authority at the primary leaving level affords third party validation of the quality and rigor of the educational standards prescribed by the Ministry of Education for schools. The links with UCLES at the post-primary levels are likely to be maintained as these have served Singapore well over the past few decades. In the author’s view, one aspect of testing that may be introduced in future examinations to gauge learning is the use of 2-tier multiple choice questions. Currently, multiple choice questions (MCQs) appear as part of most subject examinations at all leaving levels. The use of MCQs has the advantage that it can test a good number of concepts across the curricula in a limited time and also provides ease of administration and marking for teachers. One of the disadvantages of MCQs is the probabilistic dimension inherent in its format. In an MCQ with four options, the probability of getting the correct answer purely by guess work is 0.25. Also, by a process of elimination of options by intelligent guessing, it is possible to narrow down the answer to the correct option. A 2-tier MCQ addresses this significantly by asking students to justify their answers from another choice of options (Treagust, 1988). An example of a 2tier MCQ is given here:
Question Peter and Jane stand 300 meters apart in an open area where there are no buildings or trees. They shouted ‘Hello’ to each other at the same time. Given that Jane’s voice has a lower pitch compared to Peter’s and that there is no wind blowing, which one of the following statements is true:
Answer (a) (b) (c) (d)
Peter and Jane hear the word ‘Hello’ at the same time. Peter hears Jane’s voice first Jane hears Peter’s voice first None of the above
Reason (i) Since Peter’s voice has a higher pitch, it travels faster and Jane thus hears him first. (ii) A lower pitch sound carries less energy, and thus travels slower. (iii) The speed of sound is not affected by the pitch, hence Peter and Jane will hear at the same time. (iv) ____________________________________________________________ (Write your reason here if you do not agree with any of the above reasons.) Use of 2-tier MCQs would thus provide better indication of students’ understanding of a concept tested in a question. The use of 2-tier MCQs, however, is not without their
Educational Evaluation in Singapore
33
drawbacks, notwithstanding its acknowledged superiority over traditional MCQs. For example, the length of an examination paper will increase, considerable time and effort are needed to formulate 2-tier MCQs, and teachers need to be given training in formulating this kind of questions. To what extent this will impact on academic competencies of students is not clear but obviously they need to have an enhanced level of functional understanding of a topic. Lack or absence of testing instruments modeled on 2-tier MCQs in the various subject areas is another drawback. All these can come in the way of the wider acceptance of 2-tier MCQs in the examination system. With the pervasiveness of ICT in Singapore – high Internet penetration rates, high home PC ownership rates, broadband links in all schools and general cyber saviness of the younger generation, the author feels that there is some scope for reducing teachers’ workload by converting certain lessons into e-learning. This will allow students to take greater control of their learning needs and further immerse them in a media that they are comfortable with. Needless to say, this would have implications in the overall scheme of assessment. In more recent years, the term action research has appeared in the lexicon of teachers. Considerable encouragement is now given to teachers in schools in Singapore to engage in action research as a way of reflecting on their teaching as well as improving the effectiveness of various school programs. The reflective stance that action research provides as well as the critical dimension that it affords to re-look at school practices and programmes has implications on educational evaluation at the school level. In fact, Banfield and CayagoGicain (2006) argue that action research is a ‘powerful and empowering approach to educational evaluation” as it “has wide practical applicability from large scale evaluations that span the entire organization, to more modest evaluation of single programmes”. The infusion of action research in schools in Singapore is likely to lead to school-based educational evaluation contributing further to improved outcomes in the education process.
CONCLUSION Educational evaluation is a powerful approach to appraise various processes in the education system. It can provide important information to stakeholders. In this chapter, we have seen that the use of primary school leaving examinations conducted by an independent agency that is not connected to schools or the Ministry of Education provides the best form of educational evaluation of how schools prepare their students. In the case of leaving level examinations at the secondary and pre-univeristy levels, again the use of an international examinations syndicate in conjunction with an independent local agency provides valuable information on how effective schools are in preparing their students. Additionally, in the case of primary and secondary levels, the participation of students in the TIMSS examinations provides supporting evidence that the curricula in the science and mathematics in Singapore are internationally competitive. The high standing of Singapore students in science and mathematics in the TIMSS examinations in 1995, 1999 and 2003 shows that students are prepared to international standards. Moreover, the system of banding (previously ranking) of schools ensures that they are cognizant of the concepts of accountability and stakeholder value.
34
R. Subramaniam
Singapore has consistently sought international benchmarks for its education system as it provides the best assurance that the curricula and assessment are not only current but also follow international norms
REFERENCES Banfield, G. and Cayago-Gicain, S. (2006). Qualitative approaches to educational evaluation: A regional conference workshop, International Journal of Education, 7(4), 510-513. Caleon, I and Subramaniam, R. (2005). The impact of a cryogenics-based enrichment programme on attitude towards science and learning of science concepts, International Journal of Science Education, 27(6), 679-704. Cheah, Y. M. (1998). The examination culture and its impact on literary innovation: The case of Singapore, Language and Learning, 12, 192-209. Johnson, S. (1999). International Association for the evaluation of educational achievement: Science assessment in developing countries, Assessment in Education, 6(1), 57-73. Lim, E. P. Y. and Tan, A. (1999). Educational assessment in Singapore, Assessment in Education, 6(3), 391-404. Martin, M. O., Mullis, I. V. S., Beaton, A. E., Gonzales, E. J., Kelly, D. C. and Smith, T. A. (1999). Science achievement in the primary school years: IEA’s third international mathematics and science study (TIMSS). Chestnut Hill, MA: Boston College. Mullis, I. V. S., Martin, M. O., Beaton, A. E., Gonzales, E. J., Smith, T. A. and Kelly, D. C (1997). Mathematics Achievement in the Primary School Years: IEA’s third international mathematics and science study (TIMSS). Chestnut Hill, MA: Boston College. Poh, S. H. (1999). Assessment issues in Singapore, Educational Evaluation: Issues and Practice, 18, 31-32. Purves, A. C. (1991). Brief history of International Evaluation Association. In W. A. Hayes (Ed.), Activities Institutions and People, IEA Guidebook 1991, pp 34-48. The Hague: IEA. Tan, J. and Gopinathan, S. (2000). Education reform in Singapore: Towards greater creativity and innovation, NIRA Review, 7, 5-10. Tan, W. H. L. and Subramaniam, R. (1998). Developing countries need to popularise science, New Scientist, 2139, 52. Tan, W. H. L. and Subramaniam, R. (2002). Science and the student entrepreneur. Science, 298, 1556. Toh K. A., Riley J. P., Lourdusamy A., Subramaniam R. (2006). School and science achievement in Japan and Singapore: A tale of two cities. Educational Research for Policy and Practice, 5, 1–13. Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science, International Journal of Science Education, 10, 159-170.
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 3
EDUCATIONAL EVALUATION IN SOUTH KOREA: CURRENT ISSUES AND FUTURE PROSPECTS Sun-Geun Baek Department of Education, Seoul National University, Republic of Korea
ABSTRACT This paper describes five significant features of educational evaluation in South Korea. The first feature is ‘the School Activities Records (SAR)’, which is designed to evaluate primary, middle, and high school students based on academic achievements, as well as the development of social behaviors. The second feature is ‘the National Assessment of Educational Achievement (NAEA)’, which is to control the quality of the nationwide educational system by providing information on the overall educational achievement and where improvement might be needed. The third feature is ‘the College Scholastic Ability Test (CSAT)’, which is administered at the national level to screen the eligible candidates for higher education. The fourth feature is ‘the Teacher Competence Development Evaluation (TCDE)’, which is designed to evaluate primary, middle, and high school teachers based on their teaching skills and management abilities. The final feature is ‘the Teacher Education Institute Accreditation System (TEIAS)’, which is designed to evaluate and accredit each teacher education program and institute. In conclusion, current issues and future prospects of educational evaluation in South Korea will be discussed.
Keywords: SAR, NAEA, CSAT, TCDE, TEIAS.
I. INTRODUCTION Located in the eastern part of the Asian Continent, South Korea is a democratic state and a constitutional republic that holds a five-thousand year history. South Korea has 16 administrative units: The metropolis of Seoul, six metropolitan cities, and nine provinces. South Korea has a population of around 48 million.
36
Sun-Geun Baek Table 1. The Number of Schools, Teachers and Students in South Korea (2006) Levels
Schools
Teachers
Students
Kindergarten
8,290
32,096
545,812
Primary School
5,733
163,645
3,925,043
Middle School
2,999
106,919
2,075,311
High School
2,144
117,933
1,775,857
Junior College
152
11,857
817,994
College and University
201
54,967
2,368,169
Others
274
8,928
406,835
19,793
496,345
11,915,021
Grand Total
More than 10 million people live in Seoul, which is the country’s capital. For your reference, the number of schools, teachers and students in South Korea are indicated in Table 1. In addition, the enrollment rates for primary, middle, and high schools have reached about 100 percent since 1994. The ratio of high school students, who go on to a higher stage of education, reached 82.8 percent in 2007, which was the one of the highest ratios in the world (KEDI, 2007). In this paper, five significant features of educational evaluation in South Korea will be described. In addition, current issues and future prospects of educational evaluation in South Korea will be discussed at the end.
II. SCHOOL ACTIVITIES RECORDS (SAR) The School Activities Records (SAR) is designed to evaluate primary, middle, and high school students based on academic achievements, as well as the development of social behaviors. The Presidential Commission on Educational Reform announced on May 31, 1995 that school teachers should construct his/her students’ SAR. The purpose of this evaluation system was to get not only summative information, but also diagnostic and formative information of students’ academic achievements, as well as the development of social behaviors. Although there was a trivial revision in 2005, both the content and the format of the SAR are almost the same as the old version. Eleven main categories’ names and their contents in the SAR are as follows (MOE and HRD, 2007a): (1) Personal Information: Student's name, gender, identification number, address, parent names and their occupation, and special comments. (2) Educational Background: Student's schooling (entrance, transfer, and graduation) records, and special comments.
Educational Evaluation in South Korea
37
(3) School Attendance: Number of instructional days per year, number of absent days, the reasons for absences (sickness, accident, and others), number of late days, the reasons for lateness (sickness, accident, and others), number of absent classes, the reasons for absences (sickness, accident, and others), and special comments. (4) Awards: Prize name, level of prize, award date, and provider. (5) Certificates: Type of certificate, registration number of certificate, registered date, and provider. (6) Career Guidance: Student's special talents and skills, interests, student's prospective occupation, parent expectations, and special comments. (7) School Optional Creative Courses: Areas of specific activities for optional creative courses and comments. (8) Extracurricular Activities: Class activities, school activities, club activities, group activities, and general comments for each sub-category. (9) Volunteer Work and Special Experiences: Contents, frequencies, places, and periods of time for both volunteer work and special experiences. (10) Academic Achievement: − − −
For primary school students: Special comments for each subject per year without grade and rank, and special talents and interests. For middle school students: Raw score, five scale grade and rank for each subject per semester, and special talents and interests. For high school students: Raw score and stanine score for each subject per semester, and special talents and interests.
(11) Behavioral Characteristics and Comprehensive Comments: General comments on behavioral characteristics and overall comments per year. Each student's information on the SAR has been used to improve individual teachinglearning processes. In addition, high schools have used each student’s information to select their new students among middle school graduates. Colleges and universities have also used each student’s information in SAR to select their new students among high school graduates. Recently, the portion of each student’s information within total entrance examination scores for higher education is gradually increased.
III. NATIONAL ASSESSMENT OF EDUCATIONAL ACHIEVEMENT (NAEA) The National Assessment of Educational Achievement (NAEA) is to control the quality of the nationwide educational system by providing information on the overall educational achievement and where improvements might be needed. The NAEA is administered generally once in a year at the national level. The purposes of the NAEA are as follows (KICE, 2007): (1) To diagnose the educational achievements of elementary, middle, and high school students and to check systematically the trends of their achievements.
38
Sun-Geun Baek (2) To provide basic reference data for improving the national curriculum by analyzing the students' achievements of the goals of the national curriculum and checking the problems of the national curriculum implementation at the school and classroom levels. (3) To improve teaching and learning methods and to produce data to set up a learning encouragement policy by analyzing the test items and the relationship between the students’ achievements and background variables. (4) To improve student assessment methods in the schools by developing and utilizing new and appropriate assessment tools. (5) To explore new research designs and methods, in order to attain the purposes of the study of the NAEA.
Until 1988, several educational research centers, such as the Korean Educational Development Institute (KEDI) or the Central Educational Research Center had intermittent conduct with the NAEA. However, the National Board of Educational Evaluation had regular conduct with the NAEA from 1989 to 1997. Since 1998, the Korea Institute of Curriculum and Evaluation (KICE), which was established as a government funded educational research and testing center on January 1 in 1998, has had annual conduct with the NAEA. The general procedures for the NAEA are as follows (KICE, 2007): (1) (2) (3) (4) (5)
KICE sets up the basic policy for the NAEA. KICE works out the basic action plan. KICE develops the test items with primary, middle, and high school teachers. KICE develops survey questionnaire for students, teachers, and principals. Municipal and provincial educational authorities administer these tests to their students. (6) KICE analyzes about 3% ~ 5%-sampled data from total populations, and reports the test results.
Table 2 shows 2007 NAEA's research subjects, assessment areas, item types, and others, such as the listening comprehension test or questionnaires. Specifically, both Korean Language and English tests included listening comprehension through the Educational Broadcasting Systems (EBS), which was a nationwide broadcasting system in South Korea. In addition, KICE analyzed about 3% ~ 5%-sampled data from the total population and reported the results to the public. Additionally, almost all municipal and provincial educational authorities have administered the NAEA tests to their all students and analyzed their whole data according to their own purposes.
Educational Evaluation in South Korea
39
Table 2. Structure of 2007 NAEA Research Subjects
6th Grade
Assessment Areas Korean Social Studies Mathematics Sciences English
9th Grade
Korean Social Studies Mathematics Sciences English
10th Grade
Korean Social Studies Mathematics Sciences English
Item Types and Others
- Multiple-choice items and constructed response items for each s ubject were developed for each subject. - Constructed response items took 20-40% of the total score depen ding on the features of the subject. - Listening comprehension test was included in Korean and Englis h. - Questionnaires for students, teachers, and principals were develo ped to investigate the relationship between the background variabl es and the students’ academic achievements. - 2007 NAEA implemented on October 16-17, 2007.
IV. COLLEGE SCHOLASTIC ABILITY TEST (CSAT) The College Scholastic Ability Test (CSAT) is administered at the national level to screen eligible candidates for higher education. CSAT is administered once in a year at the national level. The purposes of the CSAT are as follows (KICE, 2007): (1) To screen the eligible candidates for higher education. (2) To contribute to the improvement of high school education. (3) To provide valid, reliable, and objective data for selecting new students for colleges and universities. In addition, the general procedures for CSAT are as follows (KICE, 2007): (1) The Ministry of Education and Human Resources Development (MOE and HRD) has set up the basic policy for CSAT. (2) KICE works out the basic action plan. (3) KICE develops the test items. (4) Municipal and provincial educational authorities administer these tests to their students. (5) KICE analyzes and reports the test results. About 580,000 applicants (12th grade students, high school graduates, and those having equivalent certificates) took the CSAT on November 15, 2007. The stanine score for each content area appeared exclusively on the student report card of CSAT in 2007. For your reference, the stanine score is a type of standardized score with nine levels (see Table 3).
40
Sun-Geun Baek
However, the stanine score, percentile score, and one standardized score within two types (one type’s mean equals 100 with standard deviation equals 20 or another type’s mean equals 50 with standard deviation equals 10) will be appeared on the student report card of CSAT since 2008. Table 3. Structure of Stanine Score Stanine Scores
1
2
3
4
5
6
7
8
9
%
4
7
12
17
20
17
12
7
4
100-97
96-90
89-78
77-61
60-41
40-24
23-12
11-5
4-1
Percentile
Table 4. Structure of 2007 CSAT Content Areas Korean Mathematics A or B English Social Studies (4 out of 11 areas) or Sciences (4 out of 8 areas) or Vocational Education (3 out of 17 areas) Foreign Languages or Chinese Characters and Classics (1 out of 8 areas)
Times (Minutes)
Maximum Scores
Items
80
100
50
100
100
30
70
100
50
30 for each area
50 for each area
20 for each area
40
50
30
Table 4 shows the 2007 CSAT's content areas, periods of time for tests, maximum raw scores, the number of items, and item types. The CSAT score is one of the most important data for getting admission into colleges or universities. For your reference, higher educational institutes in South Korea generally use 'the total entrance examination scores' to select new students among high school graduates or those who have equivalent certificates. The total entrance examination scores are consisted differently among higher educational institutes or even are consisted differently among departments, within the same institute. Some institutes (or departments) use only (i) CSAT scores, whereas some institutes use (i) CSAT scores and (ii) the information on SAR, and others use (i) CSAT scores, (ii) the information on SAR, and (iii) institutes' own entrance examination scores such as interview scores, essay writing scores, recommendation letters' scores, awards, and so on.
Educational Evaluation in South Korea
41
In order to present more substantial information about the college and university admission system in South Korea, Table 5 shows the case of the Seoul National University, which is the best university in South Korea. Please note there are a few differences in the contents of Table 5 depending on departments or colleges within Seoul National University. Table 5. Seoul National University’s Admission System in 2008 Admission Type
Regular
Occasional
Special
No. of Students (Total: 3,277)
1,401 (42.8%)
1,761 (53.7%) Type A: 831 (24.8%) Type B: 930 (27.7%)
115 (3.5%) Type 1: 95 (2.9%) Type 2: 20 (0.6%)
Qualification
-
Higher than level 2 (top 11%) in CSAT at least 2 out of 4 areas.
- The residents of rural or fishing villages, or disabled students. - Higher than level 2 (top 11%) in CSAT at least 2 out of 4 areas.
Materials
CSAT, SAR, essay writing, interview, etc.
SAR, CSAT, interview, documents (recommendation letter, awards, etc.)
The same as regular type.
Selection Step I (selecting 1.5 - 3 times of the quorum)
Selection Step II (final stage)
SAR: 100% CSAT: 100% Documents: 100%
SAR: 50% Essay Writing: 30% Interview: 20%
SAR: 80% Documents: 10% Interview: 10% 1) Documents: 50% Interview: 30% Essay Writing: 20% 2) Documents: 50% Interview: 50%
CSAT: 50% SAR: 30% Interview: 20% CSAT: 40% SAR: 20% Interview: 40%
42
Sun-Geun Baek
V. TEACHER COMPETENCE DEVELOPMENT EVALUATION (TCDE) The Teacher Competence Development Evaluation (TCDE), which was designed to evaluate primary, middle, and high school teachers based on their teaching skills and management abilities. The main purpose of the TCDE is for the development of primary and secondary teacher’s competence in order to increase the competitive power of public education. It will be fully implemented at all schools by 2008 (MOE and HRD, 2007b). According to the results of the first nationwide TCDE trial operation at 67 primary and secondary schools in 2006, 74 percent of the teachers thought favorably of the TCDE system. It worked as a means of self-reflection, helping them to analyze the strengths and weaknesses of each individual teacher, and to identify the specific needs of both students and parents. Students and parents also positively responded to the TCDE system. They thought that it helped increase trust for teachers and ensured that student opinions were adequately reflected in class and school operations. The government implemented the second nationwide TCED trial operation in 506 primary and secondary schools on March, 2007. Following the ministry’s master plan, at the end of the trial period, principals, vice principals, and teachers in 237 primary schools, 189 middle schools, and 80 high schools will have been evaluated by other teachers, vice principles, and principals based on their teaching skills and management abilities. Students and parents will also be surveyed to measure their satisfaction with teachers’ performance, as well as the principal’s competence.
VI. TEACHER EDUCATION INSTITUTE ACCREDITATION SYSTEM (TEIAS) The Teacher Education Institute Accreditation System (TEIAS), which was designed to evaluate and accredit each teacher education program and institute. TEIAS can be defined as an official verification system to see whether or not each teacher education program and institute met the prescribed standards for quality assurance and quality improvement. The main purpose of TEIAS is the improvement of teacher education programs and institutes, in order to bring up highly professionalized primary and secondary teachers. It will be fully implemented by 2009 (MOE and HRD, 2007c). There are two serious problems within the teacher education system. The first is the wide gap between the supply and the demand of initial teachers, and the other is the inadequacy of quality management for teacher education programs and institutes. Even though there are recently very low employment rates, there are still so many teacher education programs and institutes, as well as so many people who have teacher qualifications. For example, there are total 392 secondary teacher education programs and institutes, but less than the 20% nationwide employment rates of 2005 (Baek, 2007). Additionally, there is almost no quality management system for teacher education in South Korea. In order to solve these problems, the accreditation system of teacher education institutes should be implemented as soon as possible.
Educational Evaluation in South Korea
43
VII. CURRENT ISSUES AND FUTURE PROSPECTS Since 1998, the Korean society has been characterized with democratization and concerns for the public welfare, ICT (information communication technology)-oriented industrializations, openness and globalization. There have been many educational reforms to reduce private tutoring expenses, enhance the quality of human resources, build infrastructures for lifelong learning, and expand investment for the disadvantage. Educational reforms in South Korea will be an ongoing effort, striving to improve every sector of education, including educational evaluation. The current issues of educational evaluation in South Korea are almost the same as already described above, such as SAR, NAEA, CSAT, TCDE, and TEIAS. There are many ongoing conflicts among stakeholders with those educational evaluation policies. For example, teachers unions are strongly against the implementation of TCDE and many teacher education programs and institutes are against the implementation of TEIAS. Many teachers and professors still dislike any change or any reform within the educational evaluation system. Some people also dislike the implementation of a competition system, in order to improve the competence of teachers, as well as the quality of education. However, it is generally impossible to go against the current main stream of the 21st century, where one finds limitless competition age mixed with globalization. Educational evaluation has been used as inquiry and judgment methods, in order to optimize the evaluation object, in relation to its intended purposes or to help stakeholders determine whether the evaluation object is worthy of adoption, continuation, or expansion (Fitzpatrick, Sanders and Worthen, 2004). Therefore, educational evaluation will be used continually as main tools for the development and improvement of primary, secondary, and higher education in South Korea. South Koreans hope that every individual will have an equal and easy access to high quality education at any time and any place. There is an old proverb in South Korea: A father who wants to make plans for the next 10 years, plants a tree for his son, while a father who makes plans for the next 100 years, invests in the education of his son. We believe prospects for our education in the 21st century are very bright, because we not only acknowledge the importance of education, but are also making substantial investments to build a powerful state in education.
REFERENCES Baek, S. G. (2007). Political suggestions for the successful implementation of the accreditation system on teacher education institutes. Journal of Educational Evaluation, 20(2), 25-49. Fitzpatrick, J. L., Sanders, J. R., and Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). NY: Pearson Education, Inc. Korea Institute of Curriculum and Evaluation. (2007). The implementing plan for the college scholastic ability test in 2007. KICE homepage: http://www.kice.re.kr/. Korean Educational Development Institute. (2007). 2006 educational statistics in South Korea. KEDI homepage: http://www.kedi.re.kr/.
44
Sun-Geun Baek
Ministry of Education and Human Resources Development. (2007a). New teacher’s manual for school activities records. MOE and HRD homepage: http://www.moe.go.kr/. Ministry of Education and Human Resources Development. (2007b). New implementation plan for the teacher competence development evaluation. MOE and HRD homepage: http://www.moe.go.kr/. Ministry of Education and Human Resources Development. (2007c). New implementation plan for the teacher education institute accreditation system. MOE and HRD homepage: http://www.moe.go.kr/. Seoul National University. (2007). The selecting plan for new coming students. SNU homepage: http://www.snu.ac.kr/.
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 4
DEVELOPMENT OF EDUCATIONAL EVALUATION AND CHALLENGES OF PRACTICING EDUCATIONAL EVALUATION IN THAILAND Sirichai Kanjanawasee Center on Educational Testing and Evaluation for Educational and Professional Development, Faculty of Education, Chulalongkorn University, Thailand
ABSTRACT This paper presents a brief overview of current educational evaluation issues and challenges in Thailand. Thai basic education is divided into 4 levels: 1) Grades 1-3 2) Grades 4-6 3) Grades 7-9 ,and 4) Grades 10-12. At the final year of each level, students must pass 4 types of assessments correspondence to each institution’s standards: 1) 8 subject areas assessment, 2) reading, thinking and writing skills’ assessment, 3) desired learner attributes assessment, and 4) assessment of the participation in a developmental activities program. Therefore, we need to develop phototypes of effective integrated measurement tools and a procedure manual for a systematic assessment. The new university admission system was designed and implemented in 2006, using integrated multiple indicators to evaluate student achievement and performance. These included: 1) The overall cumulative grade point average (GPAX) for Grades 10-12, 2) the grade point average (GPA) in Grades 10-12 in 3 to 5 of the 8 subject groups of the core curriculum; and 3) results of the Ordinary National Educational Test (O-NET) and/or the Advanced National Educational Test (A-NET). These designs are facing new challenges regarding the practice of evaluation techniques that are meaningful and fair for all students.
Keywords: educational assessments, quality assurance, basic education, admission system.
46
Sirichai Kanjanawasee
INTRODUCTION This paper provides general information on Thailand’s education system relating to the basic education, professional standards for teachers and higher education admission system. It presents an overview on related aspects of academic achievement assessments in basic education and the evaluation of student achievement and performance for entering a new higher education admission system. It also addresses the challenges of the evaluation role to improve educational quality system and provides some policy recommendations. At present, the framework of education reform in Thailand is based on the 1999 National Education Act which provides principles and challenging guidelines for the provision and development of Thai education in order to prepare all Thai people to live in a learning society. In this regard, Thai people shall attain full development in terms of physical and spiritual health, intellect, morality and integrity, as well as a desirable way of life that focuses on living in harmony with other people. The educational system is classified into 3 types: formal education, non-formal education; and informal education. The formal education services are provided to these inside the school system, through both public and private bodies. It is divided into 2 levels: basic education and higher education. The basic education covers pre-primary, 6 years of primary education, 3 years of lower secondary education, and 3 years of upper secondary education. Higher education is provided at universities, institutions, colleges or other types of institutions. It is divided into two levels: associate degree and degree levels. The non-formal education services are provided by both public and private bodies to those outside the school system. Informal education enables learners to learn by themselves according to their interests, potential, readiness and the opportunities available that are organized by individuals, families, communities, authorities or other learning networks.
EDUCATIONAL STANDARDS AND QUALITY ASSURANCE As specified in the 1999 National Education Act, educational reform in Thailand must establish education standards which are the specification of desired educational quality for benchmarking, promoting, monitoring and quality assurance. There are currently three types of standards: national education standards, and standards for internal quality assurance and for external quality assessment.
National Education Standards The Office of the Education Council (OEC) is responsible for proposing national education standards. Consequently, sets of standards were formulated by the Office in cooperation with the offices responsible for basic, vocational, and higher education as well as the Office for National Education Standards and Quality Assessment. With approval from the Council of Ministers, on October 26, 2004, agencies providing education at all levels are expected to abide by the national education standards, which are comprised of three categories:
Development of Educational Evaluation and Challenges…
47
(I) Desirable characteristics of the Thai people, as both citizens of the country and members of the world community, consist of 5 indicators: 1) 2) 3) 4) 5)
sound physical and mental health; required knowledge and skills sufficient for leading a meaningful life and social development; skills in learning and self-adjustment; social skills; righteousness, public-mindedness, and consciousness of their citizenship of Thailand and the world.
(II) Guidelines for educational provision consist of 3 indicators: 1) 2) 3)
development of a diversified curricula and ambiance enabling learners to develop themselves in line with their natural inclinations and to the best of their potential; and systematic and effective development of administrators, teachers, faculty staff and education personnel; practice of school-based management.
(III) Guidelines for creating a learning society/knowledge society consist of 3 indicators: 1)
2) 3)
provision of academic services and establishment of cooperation between educational institutions and community, so as to transform educational institutions into a learning society/knowledge society; research and study, promotion of and support for learning sources and mechanisms; generation and management of knowledge for the benefit of all levels and components of the society.
The national education standards serve as the basis for setting assessment standards of internal and external quality assurance mechanisms.
Educational Standards for Internal Quality Assurance Currently, all agencies concerned have made progress in the development of relevant educational standards for internal quality assurance as a guideline in the provision of education by educational institutions under their supervision. For example, learning standards for basic education have been formulated to respond to the 2001 curriculum. In addition, the Committee for Development of a System to Evaluate Higher Education Quality has appointed a sub-Committee to develop and set national standards for higher education.
48
Sirichai Kanjanawasee
Educational Standards for External Quality Assessment The Office for National Education Standards and Quality Assessment (ONESQA) conducts external quality assessment at the levels of basic education and higher education (lower-than-degree and degree levels) in line with relevant educational standards that focus on assessment of educational institutions in the following respects: (1) educational achievement (output/outcome); (2) input/process; and (3) efficiency in administration and leadership.
1) Educational Standards for External Quality Assessment at Basic Education Level Educational standards for external quality assessment of basic education institutions were approved by the Council of Ministers, in January 2000. These standards, which are composed of 14 standards and 53 indicators, can be classified into 3 groups as follows: 1.1) Standards of Learners, consisting of 7 standards with 22 indicators, aim at physical, spiritual, intellectual and social development; 1.2) Standards of process, consisting of 3 standards with 21 indicators, focus on administrative and teaching-learning processes; and 1.3) Standards of Imput specify the characteristics or readiness of administrators, teachers and the curriculum. They are composed of 4 standards, with 10 indicators.
2) Educational Standards for External Quality Assessment at Higher Education Level Educational standards for external quality assessment of higher education institutions have been stipulated. These include: (1) Graduates Quality; (2) Teaching and Learning; (3) Academic Supports; (4) Research and Innovation; (5) Academic Services; (6) Preservation of Arts and Cultures; (7) Management and Administration; and (8) Internal Quality Assurance System.
DEVELOPMENT OF A QUALITY ASSURANCE SYSTEM A quality assurance system is comprised of quality control, quality audit and quality assurance. Quality assurance can be divided into 2 main factors: internal quality assurance; and external quality assessment. To enable linkage and consistency between internal quality assurance and external quality external quality assessment, the system criteria and methods for internal quality assurance as well as for external quality assessment have been formulated.
Internal Quality Assurance Internal Quality Assurance, primarily serves as a basis for external assessment. All educational institutions are required to implement an internal quality assurance system comprised of self-control, self-audit, and self-assessment toward their educational standards. The institutions must disseminate their Self-Study Report (SSR) each year.
Development of Educational Evaluation and Challenges…
49
The continuous process of internal quality assurance of all educational institutions consists of planning, assessment and improvement of their performance. Each educational institution is required to prepare its own educational development plan in line with the objectives and principles of the National Education Act, the National Education Standards as well as the aims, philosophy, and charter of the institution; clearly determine the time frame of implementation; continuously monitor and assess its own performance; and finally use the assessment results to improve and develop a high quality of education. Each educational institution is required to complete its internal quality assurance report before the beginning of the following academic year. The assessment results of educational quality as well as guidelines or programs for improvement and development of educational quality must be presented during the following year. As for the report, it must be submitted to the parent organization, the agencies concerned, the ONESQA and be made available to the public.
External Quality Assessment The Office for National Education Standards and Quality Assessment (ONESQA), established as a public organization in November 2000 to oversee external quality assessment. The ONESQA, utilizing qualified external assessors under contract to assess both basic and higher education institutions following standards relating to educational achievement (output/outcome); input/processes; and efficiency in administration and leadership. The external quality assessment of all institutions is conducted at least once every five years. The assessment outcomes will be submitted to the agencies concerned and made available to the general public. Besides the education standards, the National Education Act also stipulated that teachers’ Council of Thailand shall be regarded as the Teachers and Educational Personnel Council. It is incorporated with the principal objective to determine professional standards, issue and revoke licenses, supervise and monitor the compliance with the professional standards and code of ethics, including professional development, so that educational professional practitioners, namely, those who are licensed to practice the teaching profession, educational institution administrators, educational administrators and other educational personnel (educational supervisors), shall have knowledge and understanding in the practice of educational profession which is the licensed profession. The Secretariat Office of the Teachers’ Council of Thailand therefore established these Educational Professional Standards. The Educational Professional Standards, which consist of standards of professional knowledge and experience, standards of performance and standards of conduct, have been announced as the Teachers Council of Thailand Regulation on Professional Standards and Ethics in 2005. In the educational professional practice, those who enter such profession shall meet the specified standards of professional knowledge and experience, so as to be eligible to obtain the license to practice the profession.
50
Sirichai Kanjanawasee
PROFESSIONAL STANDARDS FOR TEACHERS Standards of Professional Knowledge and Experience •
Standards of Teachers’ Knowledge
Have minimum qualifications with Bachelor’s degree in education or the equivalent or other degrees as accredited by the Teachers’ Council of Thailand, with knowledge in the following areas: 1) 2) 3) 4) 5) 6) 7) 8) 9)
Language and technology for teachers. Curriculum development. Learning management. Psychology for teachers. Educational measurement and evaluation. Classroom management. Educational research. Educational innovation and information technology. Teachership.
•
Standards of Experience for Teachers
Have completed their teaching operations at educational institutions under the educational degree curriculum for a minimum of one year and passed the criteria for evaluation of the teaching operations in accordance with the rules, procedures and conditions as set out by the Teachers’ Council of Thailand Board as follows: 1) Training in professional practice during study. 2) Teaching operations in educational institutions on specific subjects.
Standards of Performance 1) Regularly practice academic activities relating to the development of the teaching profession. 2) Make decisions to practice various activities taking into account consequences for the learners. 3) Be committed to developing learners to reach their full potential. 4) Develop teaching plans for effective implementation. 5) Regularly develop instructional media to be effective. 6) Organize instructional activities focusing on permanent results for learners. 7) Systematically report on results of learners’ quality development. 8) Conduct themselves as good role models for learners. 9) Constructively cooperate with others at the educational institution. 10) Constructively cooperate with others in the community.
Development of Educational Evaluation and Challenges…
51
11) Seek and use information for development. 12) Create opportunities for learners to learn under all circumstances.
Standards of Conduct •
Personal ethics 1)
•
Professional ethics 2)
•
4)
5) 6)
7)
Education professional practitioners shall care for and be merciful to, pay attention to, assist and encourage their students and clients on an equal treatment basis, in accordance with their roles and duties. Education professional practitioners shall encourage their students and clients to achieve learning, skills, good and proper conduct, in accordance with their roles and duties, to their full capability and generosity. Education professional practitioners shall conduct themselves as a good role model in terms of physical conduct, speech and mental conduct. Education professional practitioners shall not act in any manner, which is against the physical, intellectual, mental, emotional and social development of their students and clients. Educational professional practitioners shall provide service honestly and equally without asking for, accepting or acquiring any interests, which would be considered abuses of their authority.
Collegial ethics 8)
•
Education professional practitioners shall have love, faith, integrity and responsibility for the profession and act as good members of professional organizations.
Client centered ethics 3)
•
Education professional practitioners shall have self-discipline and improve their professional practice, personality and vision to keep up with the academic, economic social and political development.
Education professional practitioners shall provide assistance and support to one another constructively by adhering to the morality and for crative synergy among themselves.
Societal ethics 9)
Education professional practitioners shall conduct themselves as leaders in economic, social, religious, art and cultural, intellectual and environmental
52
Sirichai Kanjanawasee conservation and development as well as preserve the public interests and adhere to the democratic regime of government with the King as Head of the State.
According to the teachers and Educational Personnel Act of B.E.2547 (2004), there is a six-scale teacher classification framework based on academic status for the new salary structure. The classification ranges from assistant teachers, teachers, experienced teachers, higher experienced teachers, expert teachers, to specialized teachers. The Teacher Civil Service Commission sets up the criteria for performance-based teacher evaluation system including three areas: (1) The discipline, morality and ethical conduct, (2) Performance quality/core and specific competencies, and (3) The performance results. The evaluation system tries to promote teachers to higher academic status. Teachers who get promoted to a higher level will get extra monthly allowance. In addition, all public school teachers will be evaluated their performance every six months for a salary increase.
BASIC EDUCATION In accordance with the focus on educational reform, many steps - from national policy to institutional level – are being taken to develop the basic education curriculum and the teaching-learning process, to improve the assessment and evaluation of achievement, and to expand access to formal education.
Present Status of Basic Education In 2002, in accordance with the National Education Act, 12 years of free basic education was made available to students throughout the country, for the first time. The government later increased that number to 14 years to include two years of pre-primary education. Basic education covers pre-primary instruction, 6 years of primary, 3 years of lower secondary, and 3 years of upper secondary education, with the 9 years of primary and lower secondary levels considered compulsory.
Basic Education Curriculum A 2001 update of the Basic Education Curriculum called for “core requirements” as prescribed by the Basic Education Commission, and “specific content”, is directly related to local needs and contexts, to be developed by the educational institutions themselves.
(1) Basic Education Curriculum at the National Level The 2001 Curriculum for Basic Education covers 12 years of basic education (Grades 112), and is divided into 4 three-year stages, consisting of 1,000 – 2,000 hours per year. In this curriculum, the knowledge and skills specified in Section 23 of the National Education Act have been grouped into 8 subject areas: Thai Language, Mathematics, Science, Social Studies, Religion and Culture, Health and Physical Education, Art, Career- and
Development of Educational Evaluation and Challenges…
53
Technology- Related Education, and Foreign Language. Additionally, activities that focus on responding to the learner’s specific interests are also included. In 2005, following nationwide training programmes for administrators, supervisors, teachers, and personnel in related departments, the 2001 Curriculum for Basic Education became fully implemented for all grades.
(2) Basic Education Curriculum at the Institutional Level As stipulated in the National Education Act, educational institutions are required to develop curriculum content relating to the needs of the community and society, and include local wisdom and desirable attributes for members of the family, community, society and the nation. The proportion of core curriculum to local content developed by the institution should be approximately 70:30 flexibly applied in compliance with the nature of each subject.
ACADEMIC ACHIEVEMENT ASSESSMENTS IN BASIC EDUCATION Basic Education in Thailand is divided into 4 levels: Level 1. Primary 1 – Primary 3. (Grade 1 – Grade 3) Level 2. Primary 4 – Primary 6. (Grade 4 – Grade 6) Level 3. Secondary 1 – Secondary 3. (Grade 7 – Grade 9) Level 4. Secondary 4 – Secondary 6. (Grade 10 – Grade 12)
Educational Assessments at the Institutional Level At each level, Grades 3, 6, 9, and 12, students must pass four types of assessments correspondence to each institution’s standards, they are: 1) Eight subjects areas’ assessment, 2) Reading, critical thinking and writing skills’ assessment, 3) Desired learner attributes’ assessment, and 4) Assessment of participation in developmental activities program.
Educational Tests at the National Level Since 2001, students studying in Grades 3, 6, and 9, are obliged to take national educational tests administered by the Office of the Basic Education Commission (OBEC) in 2 to 5 subjects. Beginning in 2005, Grade 12 students wishing to pursue higher education must take the tests administered by the National Institute of Educational Testing Service (NIETS) as shown in Table 1. The national tests for Grade 12 students are of 2 types, the Ordinary National Educational Tests (O-NET) and the Advanced National Educational Test (A-NET). Both O-NET and ANET require tests in 5 subjects; however, A-NET focuses more on critical thinking and analytical skills. Results from O-NET and A-NET are used as one of the factors in the revised higher education admission system.
54
Sirichai Kanjanawasee Table 1. Subjects Required in National Educational Tests Levels of Education
Mathematics
Grade 3 Grade 6 Grade 9 Grade 12
x x x x
Subjects Required Thai Science Language x x x x x x x
English x x x
Social Sciences
x x
Organizers OBEC OBEC OBEC NIETS
Source: Bureau of Educational Testing, OBEC and The National Institute of Educational Testing Service.
Current Issues and Policy Recommendations Educational assessments of basic education are facing multi-level problems as follows.
At the National Level 1. Quality Assurance System Schools are carrying out their internal quality assurance not for self-development to improve student learning but to avoid being compared with the external standards and criteria set by ONESQA. The National education policy should be to apply quality assurance as quality management for linking external quality assurance with internal quality assurance to ensure that schools are fulfilling their own purposes and meeting their own standards, as well as meeting predetermined minimum criteria or national quality standards. 2. Teacher Evaluation Based on performance-based teacher evaluation systems, applying teachers are evaluated by authorities from schools and local educational service areas. Many teachers are passed and get promoted to a higher academic status, whereas students’ achievement remains the same, and in some schools students’ achievement becomes worse. The Teacher Civil Service Commission must make clear the goals and roles of the teacher evaluation. The goals must be to elevate the quality of teaching and learning, while the roles must involve both formative and summative purposes. Additionally, the evaluation criteria needs to be revised to focus on classroom practices, student development, and learning outcomes. At the Institutional level Three problems with assessments of students in schools are obvious. 1. Teachers assess students’ learning in eight subject areas, by focusing on memorizing knowledge. 2. There are no well-constructed tools for assessing students’ ability in reading, thinking and writing.
Development of Educational Evaluation and Challenges…
55
3. The assessments of desired learners’ attributes are based on the subjective judgment of teachers. The policy recommendations are as follows: 1. The school principal should impose effective teacher training regarding the assessment of the eight subject areas that focus on higher ability measurement. 2. One key role for school principal is to ask academic staff and assessment experts to develop prototypes for integrated reading, critical thinking and writing assessment tools for teachers. 3. The school principal needs to ask academic staff and assessment experts to create a procedure manual for systematic assessment of desired learners’ attributes.
ACADEMIC ACHIEVEMENT ASSESSMENT FOR HIGHER EDUCATION Higher education is essential to human resource development, especially in this era of globalization, where a country’s international competitiveness depends a great deal on the capability of its citizens to thrive in a knowledge-based economy and society. In response to government policy, the Office of the Higher Education Commission proposed a roadmap to serve as guiding principles for promoting and developing the quality of Thai higher education. Covering the years between 2005 and 2008, the roadmap consists of measures to enhance the quality of graduates, lecturers, research projects and the management of higher education institutions. To increase access to and improvement in the quality of higher education, several endeavors have been undertaken: the provision of the Income Contingent Loan; establishment of new universities and the transformation of existing public institutions into autonomous universities; reform of the central university admission system; and the promotion of research and innovation of Thai higher education institutions.
Access to Higher Education The previous national admission system, in effect since 1967, it was well-accepted as one of the most fair educational procedures in Thailand. However it depended solely upon an entrance examination that evaluated a student’s academic performance only through that particular test. As a result, students tended to place less importance on learning for knowledge and skills in their regular classrooms and concentrated their efforts on tutorials for particular subjects, in order just to be successful on the entrance examination. Consequently, responsible agencies, such as the Office of the Higher Education Commission, the Council of University Presidents of Thailand, and the educational institutions themselves called for a change from the previous approach, with a new Central University Admission System (CUAS).
56
Sirichai Kanjanawasee
Reform of the Higher Education Admission System Beginning in 2006, students completing Grade 12 are obliged to take national educational tests of 2 types: the Ordinary National Educational Test (O-NET) and the Advanced National Educational Test (A-NET). Both examinations are administered by the National Institute of Educational Testing Service to assess student proficiency in Mathematics, Thai language, Science, English language and Social Studies, Religion and Culture. The A-NET tests focus more on critical thinking and analytical skills, as well as aptitude in relation to professional disciplines. The newly-modified university admission system can be summed up as follows:
(1) The Direct University Admission System Under this system, each higher education institution determines and administers its own admission criteria and procedures within certain fields of study consistent with its mission and philosophy. For admission to some fields of study, the Direct Admission System may also include O-NET and/or A-Net results. Aptitude tests not included in the national tests, but required for certain fields of study, will be administered by the particular institution. (2) The Central University Admission System (CUAS) The admission system was redesigned and implemented in 2006, using integrated multiple indicators to evaluate student achievement and performance. These include: (1) the overall cumulative grade point average (GPAX) for Grades 10-12; (2) the grade point average (GPA) for Grades 10-12 in 3 to 5 of the 8 subject groups of the core curriculum; and (3) results of the O-NET and/or A-NET Tests. The agencies involved are striving to further improve the CUAS so that students will be able to study in the university programs they select, while at the same time focusing on attaining a well-rounded education at the secondary level. (3) Special Programmes and the Quota System Apart from the normal admission system, a number of students are also admitted through special programs and a quota arrangement set by the institution. For example, in cooperation with the Ministry of Public Health, the faculties of medicine at Chulalongkorn, Chiang Mai, Khon Kaen, Mahidol and Prince of Songkla universities admit about 15 percent of their students through the ‘Promotion of Medical Science Education for Rural Areas’ Project. Other programs admitting students through special university quotas include the ‘Development and Promotion of Science and Technology Talents’ Project, the Sports Promotion Programme and the Arts Promotion Programme.
New Designs, New Challenges and Policy Recommendations (1)New Designs Higher education admission system is always a high stake and crucial issue in Thailand. It means the future life for students and their families, and there are many stakeholders involved.
Development of Educational Evaluation and Challenges…
57
Table 2. Components of the New Central University Admission Systems Components 1. GPAX 2. GPA (3-5 subjects areas from 8subjects areas) 3. O-NET* 4. A-NET*
Academic Year 2006 10%
2007 10%
2008 10%
20%
30%
40%
35-70% 0-35%
60%
50%
*
Remark Both O-NET and A-NET examinations are administered by the National Institute of Educational Testing Service (NIETS).
The Office of Higher Education Commission and the Council of University Presidents of Thailand has concluded the integrated multiple indicators to evaluate student achievement and performance for entering higher education institutions as shown in Table 2.
(2) New Challenges The O-NET and A-NET, national standard tests are constructed by the newly established public organization, NIETS. How did the NIETS develop the O-NET tests, which represent subject area content for different school curricula? How will the NIETS develop the A-NET tests, which correspond to subject areas for different disciplines? How will the NIETS manage the large scale testing effectively? The students’ GPAX and GPA are grading from each school which is different in typical curriculum, different tools, and probably different quality standards. How will we figure out school’s GPA inflation rates and how can we make adjustments for GPAX and GPA? (3) Policy Recommendations The O-NET and A-NET National Standard Tests: •
•
•
The Basic Education Commission must call for national content standards of core curriculum in the eight subject areas. The NIETS should develop the O-NET Tests which cover the national content standards for each subject area. The NIETS should develop the A-NET, Tests which focus more on critical thinking and analytical skills as well as aptitude measures in relation to well-defined groups of professional disciplines, such as medicine, engineering, education, architecture, foreign languages, fine arts and music. The NIETS should establish testing standards for test construction, test administration, test scoring and reporting. The NIETS should prepare for computerized adaptive testing.
The Students’ GPAX and GPA:
58
Sirichai Kanjanawasee •
•
•
The Basic Education Commission should promote investigative research regarding basic education achievement, GPA inflation, development of GPA inflation rates, and linking of GPA and GPAX. The Higher Education Commission should encourage university scholars to develop and compare GPA equating and linking methods among the linear method, equipercentile method, regression method, and item response theory method to find out the most appropriate model. The Council of University Presidents of Thailand should set up a taskforce to conduct research on weighting the integrated multiple indicators and develop composite index calibration methods that are meaningful and fair for all students.
CONCLUSION As specified, in the 1999 National Education Act, educational reform in Thailand must establish educational standards, revised curriculum, and quality assurance. Currently, there are three types of standards: national standards, and standards for internal quality assurance and for external quality assessment. In accordance with the focus on educational reform, the Basic Education Commission called for curriculum change at the national level (core – curriculum) and at the institution level (local – curriculum). The proportion of core curriculum to local content developed by the institution should be approximately 70:30 flexibly applied in compliance with the nature of each subject. The Basic Education is divided into 4 levels: 1) Grades 1-3, 2) Grades 4-6, 3) Grades 7-9, and 4) Grades 10-12. At each level, upon leaving grades 3, 6, 9, and 12 students must pass 4 types of assessments correspondence to each institution standards: 8 subject areas’ assessment; reading, thinking and writing skills’ assessment; desired learner attributes assessment; and assessment of the participation in developmental activities program. The Thai Basic Education System is facing many assessment barriers. The national quality assurance policy should link external quality assurance with internal quality assurance to serve local wisdom and to meet minimum national standards. The new teacher evaluation approach should focus on classroom practices, student development and learning outcomes. The system needs effective in-service teacher training on the assessment of learners’ higher ability. It also needs to develop phototypes of integrated reading, thinking and writing assessment tools and procedure manual for systematic assessment of desired learners’ attributes. The previous national admission systems for Higher Education depended solely upon an entrance examination that evaluated a student’s academic performance only through that particular test. As a result, students tended to pay less attention to learning in classrooms and concentrated more on private tutorials, in particular on subjects that counted for the entrance examination. Consequently, responsible agencies called for a change in the previous approach to a new Central University Admission System (CUAS). The system was designed and implemented in 2006, using integrated multiple indicators to evaluate student achievement and performance. These included: 1) The overall cumulative grade point average (GPAX) for Grades 10-12, 2) the grade point average (GPA) in Grades 10-12 in 3 to 5 of the 8 subject groups of the core curriculum; and 3) results of the Ordinary National Educational Tests (O-
Development of Educational Evaluation and Challenges…
59
NET) and/or the Advanced National Educational Tests (A-NET). The admission system is facing the challenges of developing the O-NET tests that represent content standards for different school curricula; the A-NET tests that corresponds to well-defined groups of professional disciplines. The system also needs to develop effective GPA – equating methods and composite index calibration methods, probably based on IRT, that will be meaningful and fair for all students.
REFERENCES Kanjanawasee, S. et al. (2007). The research and development of monitoring and evaluation system on Thailand education. Bangkok: Office of the Education Council. Kanjanawasee, S. et al. (2006). The achievement evaluation of Ministry of Education’s performance project. Bangkok: Faculty of Education, Chulalongkorn University. Kanjanawasee, S. (2000). Learning assessment : Executive recommendations. Journal of Research Methodology, 13(1), 75 – 91. Kanjanawasee, S. (2007). Modern test theories. (3rd ed.). Bangkok: Chulalongkorn University Book Center. Kolen, M. J. and Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer. Office of the National Education Commission. (1999). The National Education Act of B.E. 2542(1999). Bangkok: Seven Printing Group. Office of the Education Council. (2006). Education in Thailand. Bangkok: I.C.C. International Public Co., Ltd. Office of the Education Council. (2005). National education standards. Bangkok: VTC communication Ltd. Partnership. Secretariat office of the teachers’ Council of Thailand. (2005). Education professional standards. Bangkok: Professional Standards Bureau.
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 5
EDUCATIONAL EVALUATION IN HONG KONG: STATUS AND CHALLENGES John Chi-Kin Lee Department of Curriculum and Instruction, The Chinese University of Hong Kong, China
ABSTRACT This paper provides a brief overview of current educational evaluation measures and related issues in Hong Kong. The foci of measures for discussion are quality assurance mechanisms especially school self-evaluation (SSE) and external school review (ESR), and Territory-wide System Assessment (TSA). Three questions are posed for discussion: (a) How can we promote coherence among educational reform (highlighting students’ whole-person development) and educational supervision and evaluation policies (stressing academic performances) in both rhetoric and practice? (b) How can we promote a systematic linkage between centralized systems (ESR, SSE, BCA/TSA) and localized systems (teacher evaluation) so as to drive school improvement/effectiveness and teacher development/effectiveness? (c) How can we help relieve teachers’ workload and anxieties linked with educational evaluation and foster a shared community of educational evaluation among government agencies, SSB/SMC/IMC and teachers?
Keywords: Educational evaluation; quality assurance; Hong Kong.
I. INTRODUCTION The term “educational evaluation” has diverse meanings and its scope is wide encompassing many sub-fields and topics ranging from student evaluation (e.g., external (public) examinations), personnel evaluation (e.g., teacher and principal evaluation) to programme/project evaluation, school evaluation and system evaluation (e.g., international studies of educational achievement, state and school district evaluation) (Kellaghan and
62
John Chi-Kin Lee
Stufflebeam, 2003a, 2003b). In this Forum for Educational Evaluation in East Asia – Emerging Issues and Challenges, topics of presentations and sessions reveal the diversity ranging from school evaluation systems, school-based assessments, teacher evaluation, language proficiency assessment, international assessment (such as PISA), information technology and psychological applications, to mobile devices and learning assessment, and institutional evaluation (capacity building of evaluation personnel). While I shall give a very brief overview of some of these topics, the paper will concentrate on the issues related to school evaluation and teacher evaluation. According to a seminar organized by the Asian Network of Training and Research Institutions in Educational Planning (ANTRIEP) (De Grauwe and Naidoo, 2004, p.22), school evaluation is “part of the decision-making process in education; it involves judgments about the performance of schools through systematically collecting and analyzing information and relating this to explicit objectives, criteria and values. Ideally, school evaluation involves an (internal and external) assessment that covers all aspects of a school and their impact upon student learning…From this perspective, school evaluation is not an end in itself, but the first step in the process of school improvement and quality enhancement.” In that seminar, three main evaluation tools, (external review or inspection, examinations and school self-evaluation or internal evaluation) were discussed. With regard to teacher evaluation, this covers a broad range of issues, such as licensure testing for beginning teachers, innovations in teacher assessment and teacher tests and connections between teacher evaluation, accountability and pay (Pearlman and Tannenbaum, 2003).
II. CONTEXT AND OVERVIEW OF EDUCATIONAL EVALUATION IN HONG KONG Since 1997, Hong Kong has experienced not only political changes but also the gradual introduction of a comprehensive range of educational reforms, aimed at enhancing quality in education (Lee, 2005, 2006). Of the various reform measures, one is school-based management (SBM). Schools are expected to transform themselves into dynamic and accountable professional learning communities, to develop a proper performance appraisal system for the evaluation and development of staff and to engage in internal self-evaluation with external assessment (Advisory Committee on School-based Management, 2000). Assessment for learning and school-based assessments in the area of student evaluation have become increasingly important in the educational agenda in Hong Kong. This is partly because of the advocacy of curriculum reform and partly because of the intended introduction of school-based assessment (SBA) as a component of many future senior secondary (grades 10-12) subjects’ public examinations. Concomitant with educational reforms, schools have encountered the challenges of closure resulting from falling school enrollment, a consequence of a sharp decline in the birth rate. Against this background, the Government and the school sector have given more attention to educational evaluation issues, which are to some extent linked with the notions of accountability, competition and the quest for excellence. Before going into details about system evaluation, programme evaluation as well as external school review and school self-
Educational Evaluation in Hong Kong: Status and Challenges
63
evaluation, it is useful to understand briefly the main bodies for coordinating and arranging different evaluation policies and measures in Hong Kong. As shown in Figure 1, most public examinations and assessment measures (e.g., Basic Competency Assessment and Territorywide System Assessement) are under the purview of the Hong Kong Examinations and Assessment Authority (HKEAA) while the Education Bureau (EDB) (formerly known as the Education and Manpower Bureau (EMB)) has a Curriculum and Quality Assurance Branch to implement quality assurance measures such as external school reviews and quality assurance inspections. On the other hand, the EDB may commission external bodies (e.g., universities) to conduct evaluation studies or large-scale assessments such as the PISA (Programme for International Assessment). With regard to system evaluation, Hong Kong has taken part in a number of large-scale international studies of educational achievement such as PISA, TIMSS (Trends in International Mathematics and Science Study) (e.g., Leung, 2005) and PIRLS (Progress in International Reading Literacy Study) (e.g., Tse and Loh, 2007). Taking PISA, organized by the OECD (Organization for Economic Co-operation and Development) as an example, the analysis of findings from PISA revealed that “Hong Kong’s education system has achieved both excellence and equality of opportunity when compared with other participating countries…Hong Kong has the highest percentage of immigrant students among the participating countries/regions, and the achievement gap between the non-native students and local students is substantial” (Ho, 2005a, p.2). The findings from PISA inform us about other issues, such as the influence of students’ self-regulated learning on their achievements (Ho, 2004) as well as student performance in various subjects in different medium-of-instruction schools (e.g., Ho and Man, 2007).
Figure 1. Main educational bodies and selected educational evaluation measures affecting Hong Kong schools.
64
John Chi-Kin Lee
On the other hand, there are public examinations and territory-wide assessments organized by the Hong Kong Examinations and Assessment Authority (HKEAA). The former are exemplified by the Hong Kong Certification of Education Examination (HKCEE) at the end of secondary five (grade 11) and the Hong Kong Advanced Level Examination (HKALE) at the end of the sixth form (secondary seven, grade 13). It is worth noting that “standards of performance on the HKCEE and HKALE have for many years been benchmarked against standards in comparable subjects on British GCE and A/AS-Level examinations”. (http://eant01.hkeaa.edu.hk/hkea/switch.asp?p_left=ir_left.aspandp_clickurl=http://www.hkea a.edu.hk/zh/ir_intro.htm access: 1-11-2007). The latter refers to the Territory-wide System Assessment (TSA), which will be discussed in the following section. Moreover, the Education Bureau implemented a policy of language proficiency assessment for teachers teaching English and Putonghua in which the HKEAA is responsible for arranging the Language Proficiency Assessment for Teachers (LPAT). As regards programme evaluation, schools have started to become acquainted with monitoring or review of projects, especially those submitted to the Quality Education Fund (QEF), established in 1998 to promote school-based, educational innovations. The QEF stipulates that leaders of QEF projects need to submit progress reports, involving selfevaluation of project effectiveness (with regard to attainment of the stated objectives, success of the activities in the light of the impact on the participants and resources (e.g., people, time and equipment, etc.) used for the activities). For the final report, the project impact in terms of a project’s effects on learning effectiveness, professional development and/or school development should be addressed and evaluated using an evidence-based approach. In addition, evaluation of a project needs to include the quality and dissemination value of the project deliverables (http://qef.org.hk/eng/main.htm?plan/plan01.htm, access: 26-7-2007). On the other hand, teachers in some schools may have the chance to engage in small-scale action research (e.g., Lee and Leung, 2006; Lee, 2007; Lee, Yin and Zhou, 2008) or learning studies (Lo, 2006) through participation in university-school partnership projects (Lee and Williams, 2006).
III. QUALITY ASSURANCE, SELF-EVALUATION AND SCHOOL IMPROVEMENT FOR HONG KONG SCHOOLS In 1997, the Quality Assurance Inspectorate (QAI) was established and since 2003 there has been “School Development and Accountability” (SDA) framework, set up to promote school improvement through a combination of School Self-evaluation (SSE) and External School Review (ESR). Such a framework reflects a gradual shift from an emphasis on external inspection to a school’s own review of its performance as a starting point, to be validated by ESR and the school as the center for change and improvement (Tang, 2008). In the handbook Self-evaluation: Background, principles and key learning (MacBeath, 2005), published by the National College for School Leadership, there is a section on “Selfevaluation and school review in Hong Kong” (pp.39-40). This praised Hong Kong’s model as “exemplary of the sequential model, one that lies closest to the New Relationship in Schools in England and one that offers some pointers to both the strengths and potential pitfalls of
Educational Evaluation in Hong Kong: Status and Challenges
65
shorter, sharper inspection centred on the school’s own internal evaluation” (p.39). It also highlights the following issues (p.39): • •
There may be disagreements over the final grade of different aspects of school quality or a lack of democratic process in arriving at the final grade; and Downgrading of grades by the external school review (ESR) team may sometimes lead to disappointment or resentment but in most cases the school accepts the professional judgment and opinions of the ESR team
In Hong Kong, some school improvement projects were launched that helped schools to familiarize themselves with, and implement, self-evaluation. Pang (2004, p.5), for example, initiated projects adopting a bottom-up approach that aimed to help schools: develop their own models of school-based management in the spirit of the recommendations of the Education Commission Report No.7; institutionalize a self-evaluation framework in daily practices for continuous improvement; and develop their own sets of school-based performance indicators for use in school self-evaluation. On the other hand, the Education and Manpower Bureau of the Hong Kong SAR Government launched a partnership project known as the School Development through School Self-evaluation Project with twenty-one schools (McGlynn, 2004; QAC, EMB, 2003a, 2003b). The project proposed three focuses or circles of evaluation: learning at the center or first level; culture, which refers to the enabling conditions and ethos conducive to student learning, as the second level; and leadership as the third level (Kennedy and Lee, in press). The SSE activities chosen by the project schools ranged from reading and/or writing, peer observation/collaborative teaching, students’ evaluation of teachers’ performance, using a staff appraisal process to promote school self-evaluation, teachers’ view of the ideal and actual school, appraising the school’s performance through surveys of stakeholders to students’ perceptions of the quality of school life and comparison with teachers’ views to evaluation of the programme on personal and social education, evaluation of the staff development programme in a new school, evaluation of mathematics teaching and students’ views on their first week in primary school (QAC, EMB, 2003a, p.12). In addition to the efforts by various universities and the Government, some professional organizations played a part in promoting school self-evaluation. Under the auspices of the District Teachers’ Network, the Hong Kong Primary Educational Research Association and the Education and Manpower Bureau co-organised with the School of Educational Management of Beijing Normal University a “Sharing and Action Research on School Selfevaluation in Primary School” project involving nine schools (Hong Kong Primary Educational Research Association, 2003). The project aimed to promote a culture of school self-evaluation through workshops on using schools’ performance indicators, teachers’ and students’ self-evaluation, peer observation and action learning on teaching practices (Kennedy and Lee, in press; Wu and Lee, 2005). MacBeath and Clark, commissioned by the Quality Assurance Division (QAD), Education and Manpower Bureau (EMB) (now called the Education Bureau), led and conducted two phases (Phase I and II) of “The Impact Study on the Effectiveness of External School Review in Enhancing School Improvement through School Self-evaluation in Hong
66
John Chi-Kin Lee
Kong” (QAC, EMB, 2005, 2006). It is noteworthy that “the role and task of the SIT/SSE team holds the key to embedding self-evaluation in the school (QAC, EMB, 2005, p.11) and “school leadership is ultimately the single most important ingredient in making SSE work.” (QAC, EMB, 2006, p.14). According to a Legislative Council paper submitted by the Education and Manpower Bureau in June 2007 (LC Paper No. CB(2)2071/06-07(01)), the major achievements and observations of the SDA framework are (EMB, 2007, pp.4-5): (a) (b) (c) (d)
giving impetus to nurturing the culture of school self-evaluation (SSE); promoting the use of data and evidence as a basis for SSE; creating a greater sense of openness and transparency within schools; and inducing a positive impact on learning and teaching.
In the second cycle, scheduled to start in 2008/09, the following guidelines will be used (based on p.7 and p.8): (a) the process will continue to be improvement oriented; (b) the process will become more focused and school-specific, making reference to the findings of the first ESR report for individual schools; (c) the process will embed SSE in ongoing school practice, strengthening SSE and ESR as integral parts of the school improvement process; and (d) the process will streamline the SDA process (such as refining performance indicator (PI) areas and Key Performance Measures (KPMs)), further reducing teachers’ workload.
IV. BASIC COMPETENCY ASSESSMENT AS “STATE” EVALUATION: A FORM OF SYSTEM EVALUATION The Basic Competency Assessment (BCA) has two components: student assessment and system assessment (now called the Territory-wide System Assessment (TSA)). The former refers to a resource bank on assessment for learning provided through the internet and the latter is administered at the territory level at the three levels of Primary three, Primary six and Secondary three. The TSA is developed as a low-stake assessment tool intended to provide feedback to schools about their standards in the three subjects of Chinese language, English language and mathematics so that schools and teachers could make plans to improve effectiveness in learning and teaching (http://cd1.edb.hkedcity.net/cd/eap_web/ bca/index3.htm access 25-72007). Despite the good intention of having TSA as a low-stake assessment, some schools worried that the data, if unfavourable to the school, would affect the parents’ and EDB image of the school and be used as a reason for school closure by the government. Some schools were found cheating by encouraging weak students to abstain from the examinations while some others arranged intensive drilling for students. There were also concerns that the TSA scores of new immigrants and special needs students would be calculated on a separate basis as this was perceived to be unfair to schools.
Educational Evaluation in Hong Kong: Status and Challenges
67
V. TEACHER (STAFF) APPRAISAL AS A FORM OF SCHOOL-BASED TEACHER EVALUATION In Hong Kong, it is basically up to the individual School Sponsoring Body (SSB) and/or School Management Committee (SMC)/Incorporated Management Committee (IMC) to decide upon their own teacher appraisal system (Lee, Lam and Li, 2003). These observations are pertinent: •
• •
•
While staff appraisal systems and the peer observation in schools are more popular, they are still in various stages of development depending on individual SSB, SMC or IMC. As staff appraisal systems are not usually and directly linked with incentives, the assessment does not tend to be very rigorous (meeting the threshold and having a central tendency in the case of having different raters). Most teachers do not feel threatened by in-house evaluation. Teacher evaluation places more emphasis on overall professional duties than on actual classroom teaching practices and/or student learning effectiveness/quality. There is also an increasing tendency to involve more than one evaluator in the process of school-based teacher evaluation and having principal evaluation conducted by teachers. There is little linkage or loose coupling between teacher evaluation and school improvement.
VI. INITIAL OBSERVATIONS ON PRINCIPALS’ AND TEACHERS’ PERCEPTIONS OF QUALITY ASSURANCE MEASURES (E.G., ESR AND SSE, TSA AND TEACHER APPRAISAL) According to a survey conducted by the Hong Kong Primary Educational Research Association and Education Convergence (2006), primary school teachers perceived that the following policy measures had pressured them (in rank order): Territory-wide Assessments (TSA), Language Proficiency Assessment (Benchmarking) for language teachers; implementation of inclusive education; self-evaluation mechanism under school management; and external school review. For secondary school counterparts, the policy measures perceived to have pressured them (in rank order): external school review, senior secondary curriculum reform, change to a 3-34 system (six years’ secondary school system and four years’ university system); Language Proficiency Assessment (Benchmarking) for language teachers; and self-evaluation mechanism under school management. These evaluation and assessment measures that have generated teachers’ anxiety should not be taken lightly by policymakers. The writer has recently conducted studies of principals’ and teachers’ perceptions of quality assurance measures (e.g., ESR and SSE, TSA and teacher appraisal) in five primary schools in Hong Kong and the following are initial observations:
68
John Chi-Kin Lee •
•
•
•
While different schools revealed varying stages of development in school selfevaluation, almost all principals saw the positive benefits brought about by school self-evaluation with external school review. Some teachers felt that SSE and ESR brought an increased workload in preparing documents, together with threats of uncertainty, while some others perceived that ESR could provide a recognition of a school’s accomplishments. Some teacher interviewees were still worried that less than outstanding and good TSA scores would be used as evidence of unsatisfactory performance in external school review. This could be used as an implicit reason for school closure and follow-up monitoring by the EDB. While most schools used TSA as feedback for improving learning and teaching, they placed different emphasis on and had varying modes (e.g., extra lessons, TSA format assessment, soliciting parental support in speech training) of preparing students for TSA.
VII. THE WAY FORWARD Educational evaluation is a powerful approach to appraise the effectiveness and quality of inputs, processes and outputs in an educational system. It is also a useful approach to provide opportunities for self-evaluation, reflection and continuous improvement as well as to provide timely and important information to various stakeholders. Nonetheless, in an education system like Hong Kong, accountability and performance are highlighted, which may induce stress to teachers and school resistance to the adoption of evaluation measures. Also, the cultural heritage emphasizing academic excellence in Hong Kong may exacerbate competitiveness among students, teachers and schools, defeating the good intentions of evaluation (such as in the case of TSA and school self-evaluation) in providing feedback for improvement. It is therefore imperative for policymakers, university professors and frontline practitioners to explore solutions for resolving challenges that result from evaluation policies and these solutions entail: (a) promoting coherence among educational reform (highlighting students’ wholeperson development) and educational supervision and evaluation policies (stressing academic performances) in both rhetoric and practice; (b) promoting a systematic linkage between centralized systems (ESR, SSE, BCA/TSA) and localized systems (teacher evaluation) so as to drive school improvement/effectiveness and teacher development/effectiveness; and (c) helping reduce teachers’ workloads, relieving any anxieties linked with educational evaluation and fostering a shared community of educational evaluation among government agencies, SSB/SMC/IMC and teachers.
Educational Evaluation in Hong Kong: Status and Challenges
69
ACKNOWLEDGMENT The author would like to thank EDB’s colleagues, Mr Tony Tang, PAS(QA) for his considerable support in providing useful information and Ms Po-ling Wu, PEO(QA) for her help in interviews conducted for this study. Sincere thanks are extended to Professor Michael Williams for his advice.
REFERENCES Advisory Committee on School-based Management (2000). Transforming schools into dynamic and accountable professional learning communities: School-based management consultation document. Hong Kong: Printing Department. De Grauwe, A., and Naidoo, J. P. (2004). Seminar report: School evaluation for quality improvement: issues and challenges. In A. De Grauwe and J. P. Naidoo. (Eds.), School evaluation for quality improvement: An ANTRIEP report (pp.15-40). UNESCO: International Institute for Educational Planning. Education and Manpower Bureau (EMB) (2007). Report on the implementation of the school development and accountability and the planning of the second cycle. LC Paper No. CB(2)2071/06-07(01). Kennedy, K., and Lee, J. C. K. (in press). The changing role of schools in Asian societies: Schools for the knowledge society. London: Routledge. Ho. E. S. C. (2004). The nature and impact of self-regulated learning on student’s achievement: What we have learned from the first cycle of PISA. Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research, The Chinese University of Hong Kong (in Chinese). Ho, E. S. C. (2005). Can basic education system in Hong Kong be equal and excellent: Results from PISA2000+. Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research. Ho, E. S. C. and Man, E. Y. F. (2007). Student performance in Chinese medium-ofinstruction (CMI) and English medium-of-instruction (EMI) schools: What we learned from the PISA study. Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research. Hong Kong Primary Educational Research Association (Ed.). (2003). Action research and sharing of primary schools Self-evaluation. Hong Kong: Hong Kong Primary Educational Research Association. (in Chinese). Hong Kong Primary Educational Research Association (HKPERA) and Education Convergence (EC) (2006). A research study on Hong Kong teachers’ stress: Preliminary analysis. Educational Research Report Series No. 2. Hong Kong: HKERA and EC. Kellaghan, T. and Stufflebeam, D. L. (Eds.). (2003a). International handbook of educational evaluation (Part one: Perspectives). Dordrecht: Kluwer Academic Publishers. Kellaghan, T. and Stufflebeam, D. L. (Eds.). (2003b). International handbook of educational evaluation (Part two: Practice). Dordrecht: Kluwer Academic Publishers. Lee, J. C. K. (2005). School effectiveness and improvement in Hong Kong: Review and prospect. Educational Journal, 33(1-2), 1-23. (in Chinese).
70
John Chi-Kin Lee
Lee, J. C. K. (2006). Commentary on papers presented at the MORE symposium at the ICSEI Annual Conference, Barcelona, Spain, 2005. Educational Research and Evaluation, 12(6), 595-600. Lee, J. C. K. (Li Zijian) (2007). Foreword: University and school partnership research: From 4P to 4R. Shanghai Research on Education, 8, 1. Lee, J. C. K., Lam, W.P. and Li, Y. Y. (2003). Teacher evaluation and effectiveness in Hong Kong: Issues and challenges. Journal of Personnel Evaluation in Education, 17(1), pp.41-65. Lee, J. C. K. and Leung, Y. M. J. (2006). Exploring the core elements of improving classroom learning and teaching. Hong Kong: Hong Kong Institute of Educational Research, The Chinese University of Hong Kong. Lee, J. C. K., and Williams, M. (Eds.) (2006). School improvement: International perspectives. New York: Nova Science Publishers, Inc. Lee, J.C.K., Yin, H. and Zhou, Z. (2008). Facilitating teacher professional development through ‘4-P’ model: Experiences from Hong Kong’s Partnership for Improvement of Learning and Teaching (PILT) project. Journal of Educational Research and Development, 4(2), 17-47. (in Chinese) Leung, F. K. S. (2005). Some characteristics of East Asian mathematics classrooms based on data from the TIMSS 1999 video study. Educational Studies in Mathematics, 60(2), 199215. Lo, M. L. (2006). Learning study – the Hong Kong version of lesson study: Development, impact and challenges. In M. Matoba, Crawford, K. A., Mohammad, R. S. Arani (Eds.), Lesson study: International perspective on policy and practice (pp. 133-157). Beijing: Educational Science Publishing House. McGlynn, A. (2004). Self-evaluation or self-delusion: The choice is yours. in N.S.K. Pang, J. MacBeath and A. McGlynn (Eds), Self-evaluation and school development (pp.17-22). Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research, The Chinese University of Hong Kong, Hong Kong. MacBeath, J. (2005). Self-Evaluation: Background, Principles and Key Learning. Nottingham: National College of School Leadership. Pang, N. S. K. (2004). Developing schools through self-evaluation, in N. S. K. Pang, J MacBeath and A. McGlynn (Eds), Self-evaluation and school development (pp.1-9). Hong Kong: Faculty of Education and Hong Kong Institute of Educational Research The Chinese University of Hong Kong. Pearlman, M., and Tannenbaum, R. (2003). Teacher evaluation practices in the accountability era. In T. Kellaghan and D. L. Stufflebeam (Eds.), International handbook of educational evaluation (Part two: Practice) (pp.609-642). Dordrecht: Kluwer Academic Publishers. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2003a). Is our school a Hong Kong school of today and tomorrow: A self-evaluating school is the way to success. Retrieved July 26, 2007, from http://www.edb.gov.hk/FileManager/ Tc/Content_705/SSE_Pub1.pdf. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2003b). School self-evaluation: Making it happen in twenty one schools. Retrieved July 26, 2007, from http://www.edb.gov.hk/FileManager/Tc/Content_705/SSE_Pub2.doc. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2005). The impact study on the effectiveness of external school review in enhancing school
Educational Evaluation in Hong Kong: Status and Challenges
71
improvement through school self-evaluation in Hong Kong. Phase I report executive summary. Retrieved July 24, 2007, from http://www.edb.gov.hk/filemanager/en/ content_176/phase_i_impact_study_on_esr_executive_summary_eng_0505.pdf. Quality Assurance Division, Education and Manpower Bureau (QAC, EMB) (2006). The impact study on the effectiveness of external school review in enhancing school improvement through school self-evaluation in Hong Kong. Phase II report executive summary. Retrieved July 24, 2007, from http://www.edb.gov.hk/filemanager/en/ content_176/phase_ii_impact_study_on_esr_executive_summary_eng_0806.pdf. Tang, T.F.Y. (2008). School development and accountability in Hong Kong: A quality assurance framework. Journal of Quality School Education, 5, 39-55. Tse, S. K., and Loh, E. K. Y. (2007). The impact of PIRLS in the Hong Kong, SAR. In K. Schwippert (Ed.), Progress in reading literacy: The impact of PIRLS 2001 in 13 countries (pp. 109-126). Münster: Waxmann Verlag GmbH. Wu, S. W. and Lee, K. K. (2005). Improve school self-evaluation by practice. Hong Kong Teachers' Centre Journal, 4, 109-14. (in Chinese).
PART II. VARIATIONS AND EVOLUATION
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 6
EDUCATIONAL EVALUATION AND QUALITY ASSURANCE: THE CASE OF HONG KONG Nicholas Sun-keung Pang Department of Educational Administration and Policy, The Chinese University of Hong Kong, China
ABSTRACT In recent years, the Education and Manpower Bureau (now Education Bureau) has implemented a two-pronged approach to assuring the quality of education in Hong Kong: an external mechanism via inspection and an internal framework via self-evaluation. However, both of these are mainly top-down and less effective than expected. This essay argues that quality assurance in collaboration with an external independent agency through a bottom-up approach is a more effective way to sustain development in schools. Schools can only improve continuously when they have institutionalized a bottom-up approach to school self-evaluation. After introducing the details of a series of schooluniversity partnership projects concerned with school self-evaluation in Hong Kong, this paper investigates the potential effectiveness of implementing self-evaluation in Hong Kong schools and the factors that have both hindered and facilitated such a policy.
Keywords: quality assurance, school self-evaluation, school inspection, sustainable development, school self-renewal, bottom-up approach.
THE QUALITY ASSURANCE MOVEMENT IN HONG KONG There have been rapid changes in both the educational system and the schools themselves in Hong Kong, due to the recommendations of the Education Commission Report No. 7 (ECR7) issued in 1997. The ECR7 recommended a two-pronged approach to ensure the quality of education in Hong Kong: an external assurance mechanism and an internal quality assurance framework. While the external quality assurance mechanism was achieved through
76
Nicholas Sun-keung Pang
the establishment, in 1997, of a Quality Assurance Inspectorate (QAI) to which schools were accountable, the internal quality assurance framework relied on schools’ own ability to evaluate themselves as part of the process of school improvement. The external quality assurance mechanism worked by adopting a whole-school approach to inspection by the QAI, which assessed schools' effectiveness, identified their strengths and weaknesses, made suggestions on ways of improvement and development in the schools, and released inspection reports for public reference. In order to continuously improve the quality of school education, all schools were also expected to engage in cyclical processes of planning, implementation and evaluation (Pang, MacBeath, and McGlynn, 2004). Every school was expected to work towards meeting the educational needs of its students as effectively as it could, and self-evaluation would provide information on which to base plans for improvement. As for this self-evaluation, all schools were to produce documents which outlined the long-term goals, prioritized development areas, set out specific targets for implementation, evaluated progress made in work during the previous school year, and set improvement or development targets for the coming year (Ministry of Education, 1984; Scottish Office, 2002). However, in the initial stage, the two strategies in this two-pronged approach to assuring educational quality in Hong Kong might not have been as effective as expected. For one thing, as for the external assurance mechanism, whole-school inspections could only take place for two- or three-week periods (each time) and the QAI would not re-visit the same school for at least five to six years, given the QAI teams’ manpower. Thus while wholeschool inspections were implemented in Hong Kong in the 1997/98 school year, between that year and 2001/02 the number of schools inspected was only about 11.4% of the total. Secondly, in conducting whole-school inspections in over 200 schools from 1998 to 2002, the QAI found that most of these schools had not established a self-evaluation framework, and in fact at that time there were still no appropriate school-based indicators for school selfevaluation. Schools can only improve continuously when they have institutionalized a self-evaluation framework in daily practice and when there is a set of valid, reliable and school-based performance indicators available for use in self-evaluation. Practicing self-evaluation enables schools (i) to develop formal procedures for setting school goals; (ii) to have the participation of teachers, parents and alumni in school management, development, planning, evaluation and decision-making; (iii) to assess their progress towards goals as well as their own performance over time; and (iv) to take appropriate steps toward self-improvement. When school-based indicators are derived from the schools’ own stated aims, they are useful tools for measuring and monitoring school performance in areas of interest. Self-evaluation with appropriate school-based indicators provides information to schools, teachers, parents, students and the community, using for reference the general profiles of schools and the comparison among schools of similar background or quality. School self-evaluation and school-based performance indicators are the crucial elements needed for continuous improvement in and of schools (Cuttance, 1994; Scottish Office, 2002). A summary of the pros and cons of this two-pronged approach to quality assurance in education is given in Table 1.
Educational Evaluation and Quality Assurance
77
Table 1. The pros and cons of the two-pronged quality assurance strategy Quality Assurance 1. Approach 2. Changing agent 3. Changes 4. Drives 5. Pace of Change
External Mechanism An innovative approach Quality Assurance Unit A “crash-through” approach with radical “surgery” Top-down and external Fast, on schedule and intermittent
Internal Mechanism An incremental approach School Self-Evaluation An evolutionary development Bottom-up and internal Slow, continuous, and long lasting
QUALITY ASSURANCE THROUGH SELF-EVALUATION Evidence-based organizational change is a very recent trend in the school reform and improvement movement. It is important that schools’ organizational change should be based on objective and reliable evidence of school performance. Once they have implemented selfevaluation, schools should have a self-renewal mechanism for managing change, one which can be built upon (1) clear and appropriate diagnosis of the school as an organization, and (2) the role of the administration. Experience in both research and practice has shown that if school reforms are to succeed, organizational changes need the active support of the principal. The principal needs to be an advocate of self-evaluation and be prepared to articulate a vision of self-renewal for the school. School development cannot be copied or otherwise imposed from outside. The principle needs to understand the current situation, including strengths and weaknesses, opportunities and threats (SWOT) to the organization; he or she needs to determine the goals to be attained within the next 1-3 years, and develop the strategies to be pursued in order to achieve them. Institutionalization of self-evaluation in the organizational framework and daily managerial practices allow the principal to lead the school towards effective educational change (MacBeath, Jakobsen, Meuret and Schratz, 2000; Gamage and Pang, 2003). To successfully institutionalize a self-renewal framework in daily managerial practices as well as to lead and manage change effectively, the principle or leader needs to: (1) acquire appropriate knowledge and understanding of the theoretical framework of school selfevaluation; (2) develop and acquire the necessary skills and attitudes for self-evaluation and manipulation of performance indicators; (3) think through his/her leadership role as a guide to action; and (4) clarify for him/herself the strategic elements essential to an effective implementation of the school development plan. Then the principal should examine the types of knowledge, skills and attitudes which need to be developed for successful implementation of organizational change.
What Is School Self-Evaluation? School self-evaluation (SSE) is a mechanism through which a school can help itself review the quality of the education it provides, improve continuously and develop itself into a more effective school. The three major questions usually asked in the process of school selfevaluation are: (1) How can we know about our school’s present performance? (2) What is
78
Nicholas Sun-keung Pang
our school's present performance? (3) What will we do after knowing this? These seem to be simple questions, but if we want to have a full picture or thorough understanding of our own school, we need to go through a systematic and objective self-evaluation.
The Basic Steps in School Self-Evaluation School self-evaluation contains several identifiable stages. These are: problem recognition, identifying priorities, defining important questions, data collection, data analysis, reporting and communicating, school development planning, school culture building, and feedback and evaluation. All these stages need to be fully implemented in a sequential order. Thus, the following schema will be useful to a leader initiating change in the organization. It needs to be re-emphasized that the leader should have a good understanding of the concept and process of school self-evaluation and consequent self-renewal. Here is a fuller description of these stages: Problem identification: An organization becomes aware of the existence of a problem that needs to be fixed. The leader either recognizes and confronts it, or overlooks and ignores it. Identifying priorities: There may be problems in a school’s various domains, for example, organization and management, teaching and learning, ethos and support, academic and affective performance. However, a school cannot solve all these problems within a single year. Administrators should prioritize the problems based on the teachers' will and students' needs. Defining important questions: For any identified problem, the school should know the key questions involved. These will be answered through a systematic process of data collection and analysis. Data collection: Data can be collected through questionnaires, observations and/or interviews, in order to ascertain whether the problem has been fixed or still exists. Consideration should be given to the sources of data which may make significant contributions to assessments of the school's performance. Data analysis: On the basis of data collected through questionnaires, observations or interviews, attempts should be made to identify, clarify, and re-define the problems. Reporting and communicating: Staff should be briefed on diagnostic data and involved in developing strategies to solve the problems, providing opportunities for staff training on group dynamics, communication techniques, and goal setting. School development planning: An attempt should be made to bridge the gap between the current situation and what should have happened. A consultant or expert can help in determining: What steps should be taken? By whom? When? And how? Implementation of steps should be monitored to resolve any difficulties which may arise. School culture building: Efforts should be made to build trust and confidence, improve communication, build teams, develop skills in problem solving, and strengthen cooperation among different subsystems of the school. Feedback and evaluation: Feedback should be provided to staff as the final step in the school self-evaluation cycle. A new cycle then needs to be initiated in order to
Educational Evaluation and Quality Assurance
79
institutionalize school self-renewal (self-development) as a continuous process of innovation and change (Rudd and Davies, 2000). Schools should gather information based on the perceptions of administrators, teachers, parents, students and the community, for comparison with schools of similar background or within the same “quality circle” in order to promote continuous improvement and development.
A Bottom-Up Approach to School Self-Evaluation in Hong Kong After conducting the first few cycles of whole-school inspection starting in 1998, Hong Kong’s QAI (in 2003 renamed the Quality Assurance Division, QAD) found that a selfevaluation framework had not been generally established in Hong Kong schools and that no appropriate school-based indicators had been developed for use in school self-evaluation. In response to these weaknesses the author, with the support of the Hong Kong Centre for the Development of Educational Leadership (HKCDEL) and the Quality Education Fund (QEF), launched a series of school improvement projects to help Hong Kong schools institutionalize a self-evaluation framework in daily practice, acquire the skills and techniques needed for school self-evaluation, and develop a set of valid, reliable and school-based indicators for use in self-evaluation. These projects aimed at helping schools to (i) develop their own models of school-based management in the spirit of the recommendations of Education Commission Report No. 7; (ii) acquire the skills and techniques of school self-evaluation (SSE) and institutionalize a self-evaluation framework in daily practice for continuous improvement; (iii) develop their own sets of school-based performance indicators for use in school self-evaluation; (iv) enhance the capability of coping with challenges arising from educational reforms that would have a great impact on schools in the near future; and (v) keep developing continuously and fostering a culture of organizational learning among all participating schools. In total, 107 kindergartens, 154 primary schools and 197 secondary schools in Hong Kong have taken part in these projects in the period 2000-2010 (see Table 2). Table 2. The school improvement projects organized by the School Development and Evaluation Team (SDET) in Hong Kong Period
Project Project Title KinderDuration gartens 2000-2002 2-year School Self-evaluation and Schoolbased Performance Indicators (Quality Education Fund) 2001-2002 1-year School Self-evaluation and Renewal 2002-2003 1-year
SSE Training Programs for Heads and 22 Teachers of the Catholic Church
2002-2004 2-year
Self-Evaluation and School Development
Primary Schools 10
Secondary Total Schools 10 20
2
2
4 22
10
10
20
80
Nicholas Sun-keung Pang Table 2. (Continued)
Period
Project Project Title KinderDuration gartens 2003-2004 1-year Initiating Organizational Change via School Self-evaluation 2004-2005 1-year School Self-evaluation: Reinforcing Organizational Change 2004-2005 1-year Self-Evaluation, External Review and School Development 2004-2006 2-year Transforming Schools into Learning Organizations (Quality Education Fund) 2005-2006 1-year School Self-evaluation, External Review and Sustainability 2005-2006 1-year Self-evaluation, External Review and Extended School Development (Teaching and Learning) 2005-2006 1-year School-based Self-evaluation and Development 2006-2007 1-year School Self-evaluation: Surviving Educational Reform 2006-2007 1-year Self-evaluation of Teaching and Learning 2006-2008 2-year Learning to Learn: The Application of Metacognition in Learning and Teaching 2006-2008 2-year Metacognition in Learning and Teaching: Supporting Students’ Learning Needs (Quality Education Fund) 2007-2008 1-year School Self-evaluation: Surviving the Educational Reform 2007-2009 2-year Metacognition in Learning and Teaching 2007-2009 2-year School Self-evaluation: Optimization 42 of School Management 2008-2010 2-year Use of Metacognitive Strategies in Promoting Self-regulated Learning 2008-2010 2-year
2008-2010 2-year
Enhancing Effectiveness of Early Childhood Education through the Development of a Self-evaluation Culture From Assessment for Learning to Promoting Self-regulated Learning in Early-childhood Education (Kindergarten and Lower Primary Levels) (Quality Education Fund) Total No. of Schools
Primary Schools 23
Secondary Total Schools 27 50
10
14
24
25
36
61
10
10
20
20
36
56
2
12
14
2
3
5
4
5
9
2
6
8
3
0
3
9
9
18
1
2
3
4
4
8 42
7
11
33
18 33
10
10
107
154
20
197
458
Educational Evaluation and Quality Assurance
81
SETTING-UP A SELF-EVALUATION FRAMEWORK In order to establish a climate of quality assurance and to cultivate a quality culture in the participating schools, a normative re-education strategy was used to initiate changes in these schools: this was a user-centered, bottom-up approach to school reform. In the improvement projects all teachers in a school were allowed to participate in shaping the reform by identifying their own needs, assessing the school’s present conditions and performance, and formulating school development plans. Hence the activities organized in each school included (i) setting-up of a School Development and Evaluation Committee (SDEC) in the management structure; (ii) providing staff development programs for members of the SDEC; and (iii) conducting workshops for all teachers in the school, in order to facilitate the practice of school self-evaluation and the formulation of a school development plan. Each of the participating schools established a School Development and Evaluation Committee (SDEC) to facilitate efficient and effective school self-evaluation in daily practice. In each school, the principal and a group of three senior teachers were SDEC members. The major duties of the SDEC members were: (i) identifying priorities in the areas related to school development; (ii) defining important questions to be investigated and answered in self-evaluation; (iii) collecting appropriate data for consideration and reference; (iv) analyzing data for interpretation and sharing; (iv) reporting and communicating results of self-evaluation to concerned stakeholders; and (v) formulating a school development plan and an action plan. A staff development program was provided to all SDEC members in the participating schools. This staff development program, which consisted of ten modules covering a wide range of topics, helped the schools to practice school self-evaluation, to develop school-based performance indicators for use in self-evaluation, and to grow, develop, and improve continuously and effectively (Pang, 2007). The modules of the program were as follows: Module 1: School-based Management and School Improvement Module 2: School Development Planning Module 3: School Self-evaluation and School-based Performance Indicators Module 4: Theory and Practice in Teacher Supervision: A New Perspective Module 5: Home-School-Community Co-operation Module 6: Staff Growth and Development Module 7: Curriculum Design, Implementation and Evaluation Module 8: Strategic Planning and Financial Management in Schools Module 9: Problem-based Management and Conflict Resolution Module 10: Management Data Analysis and Basic Statistics Two concurrent programs were run for SDEC members, one for the primary schools and the other for the secondary schools. Modules 1-5 were conducted in the first year, and Modules 6-10 in the second year. The programs aimed at enabling the principals and teachers to grow and develop and to initiate changes in their own schools. Activities in the form of workshops were designed to facilitate principals’ and teachers’ reflection on the concepts introduced in the courses. There were training packages for each module; that is, in total ten different packages for the program. The training packages were designed and produced for
82
Nicholas Sun-keung Pang
self-learning purposes, with contents that were adaptable with minimal preparation or background training. A participatory approach was adopted for conducting the school self-evaluation exercises. It should be stressed that practicing self-evaluation is the responsibility not only of the SDEC members but also of the stakeholders in the schools, including school managers, administrators, teachers, parents and students. Through the processes of self-evaluation a selfrenewal strategy has been institutionalized in the schools’ management structure, and through participatory school self-evaluation a quality culture has been fostered in these schools. The projects lasted for either one or two years, depending on each school’s own preference. Four workshops within two years were offered for all teaching staff from each of the participating schools. The first whole-school workshop in Year 1 aimed to (i) facilitate setting up a self-evaluation framework in each of the participating schools; (ii) help teachers incorporate self-evaluation exercises in their daily work; and (iii) enable teachers to acquire the skills of self-evaluation and knowledge of how to use performance indicators. The second whole-school workshop in Year 1 aimed to facilitate a participatory approach, in order to (i) interpret the results of self-evaluation; and (ii) formulate a school development plan for the next year. These schools were also provided with consultancy and in-service training in interpreting self-evaluation reports and formulating school development plans in the workshops. The first whole-school workshop in Year 2 aimed to (i) reinforce and consolidate the school’s self-evaluation culture; (ii) facilitate the school’s development of its own ‘quality culture’; and (iii) foster a culture of organizational learning in the school. The second wholeschool workshop in Year 2 aimed to (i) present the results of school self-evaluation for the past two years to all staff members of the school; (ii) facilitate a participatory approach to interpreting the longitudinal self-evaluation data for the purpose of comparison; and (iii) facilitate a participatory approach to the formulating of a school development plan for the next few years.
Relationships among Performance Indicators, School Improvement and Educational Research School self-evaluation (SSE) and performance indicators (PIs) are both essential to schools’ continuous improvement. Schools should have information about the perceptions of administrators, teachers, parents, students and the community for reference and also for comparison among schools of similar background. It is difficult to “see” or demonstrate continuous school improvement, if appropriate performance indicators have not been developed for use in school self-evaluation. A theoretical framework for the relationships among performance indicators, school improvement and educational research is proposed in Figure 1. When we evaluate the performance of a number of schools against a standardized set of performance indicators, we are obviously doing one kind of educational research. The group of schools we are looking at in our research will form a quality circle for comparison. Findings from such research will inform us about the best schooling practices and tell us what we can learn from the experience of failure or poor performance.
Educational Evaluation and Quality Assurance
83
PI 10 PI 9
School Improvement
PI 8 PI 7 PI 6 PI 5 PI 4
Educational Research
PI 3 PI 2 PI 1 Sch 1 Sch 2 Sch 3 Sch 4 Sch 5 Sch 6 Sch 7 Sch 8 Sch 9 Sch 10 Figure 1. The Relations among Performance Indicators, School Improvement, and Educational Research.
On the other hand, when a school’s performance is evaluated against different sets of indicators, the results of this evaluation will show the school’s performance in different areas of concern, and thus it will become clear which areas are weakest and most in need of improvement for a particular school.
Theoretical Framework for Educational Evaluation In light of the above considerations, it will be clear that the development of sets of performance indicators (PIs) for use in self-evaluation is also an important task within the larger project of school self-improvement. Two kinds of PIs were employed in these school improvement projects: (1) a standardized set of PIs for comparing the performance of the participating schools, and (2) sets of school-based PIs (SBPIs) developed by teachers for their own use in selfevaluation in different schools. A theoretical framework for educational evaluation is proposed in Figure 2. This framework allows the School Development and Evaluation Committee (SDEC) members to consider the kinds of indicators that will suit their needs, the areas of evaluation that they are interested in, and the sources of data that they will need in order to design a set of school-based performance indicators for their own use in self-evaluation. The probable areas to be evaluated within the schools include administration and management, teaching and learning, quality of school life for teachers and students, staff development and organizational learning, school ethos and culture, home-school-community cooperation, and so forth.
84
Nicholas Sun-keung Pang
Figure 2. Theeoretical Framework for Educational Evaluation.
A Standardized Set of Performance Indicators for Comparison For the sake of comparison and forming a quality circle, all schools participating in the projects have used these sets of PIs for evaluating school performance. These PI sets were mainly developed by the School Development and Evaluation Team (SDET) of the Chinese University of Hong Kong. These sets are theoretically grounded and have good validity and reliability. Furthermore, the areas for evaluation are different in the two-year school improvement projects. The areas to be evaluated in the participating schools for Year 1 are summarized in Table 3, for Year 2 in Table 4. Table 3. Areas for evaluation in Year 1 Quality of Students’ School Life Students’ Perception of School Climate Students’ Socio-Economic Status Students’ Perception of Parental Styles
Teachers’ Feelings about School Life School Organizational Values School Organizational Learning Parents’ Perception of the School
Table 4. Areas for evaluation in Year 2 Students’ Meta-cognition Students’ Emotional Intelligence Teachers’ Meta-cognition Teachers’ Emotional Intelligence Teachers’ Job Satisfaction
Instructional Leadership Instructional Climate Teaching Effectiveness Effectiveness of the Appraisal System Parents’ Perception of the Child
Educational Evaluation and Quality Assurance
85
The combinations of different evaluation areas allow principals and teachers to understand a given school’s strengths and weaknesses from different perspectives and thus allow the triangulation of findings.
Sets of School-Based Performance Indicators The sets of school-based performance indicators (SBPIs) were independently developed by the members of School Development and Evaluation Committees (SDECs). The SDEC members had to identify their own problems in the schools and define their priorities with regard to school improvement. They had to develop a set of SBPIs for use in self-evaluation to suit their individual needs. The SDEC members acquired the skills and techniques needed for developing SBPIs when they attended the training program offered in these projects. It must be stressed that the PI sets developed should be school-based, i.e., grounded in the context of a particular school and suited to the needs of that school. Thus in the choice, development and testing of these PI sets the SDEC members should already have consulted most teachers concerning their school’s own preferences and interests. In this way, consensus and compromise could be reached prior to the full use of SBPIs in school self-evaluation.
CONCRETE BENEFITS TO SCHOOLS Since change and transformation are slow, progressive, incremental processes, it would be unrealistic to expect or seek sudden, disruptive changes in schools. Nevertheless, concrete benefits of the project were as follows: (i) schools started to foster a culture of self-evaluation, that is, a culture of organizational learning for continuous improvement (Pang, 2006a); (ii) a few training packages on school self-evaluation and the use of school-based performance indicators were developed, which were useful for other schools engaged in a similar endeavor; and (iii) publications resulted, and these disseminated good practices in school selfevaluation. The participating Hong Kong schools have been benefited by these projects in four main ways. First, a self-renewal strategy has been institutionalized in the schools through establishing a self-evaluation framework and using school-based indicators in self-evaluation; this self-renewal strategy allows schools to enhance their capacity for change and develop continuously. Secondly, school administrators’ and teachers’ skills and techniques in selfevaluation have been enhanced through a series of training courses designed specifically for them; their professional competence, confidence and performance is promoted by their participation in these programs and courses. Thirdly, the learning of students will be improved when schools’ effectiveness has been enhanced and when a quality culture has been cultivated in these schools. Fourthly, parents and the wider community will be benefited because the school self-evaluation process has led to annual reports containing fair, reliable and objective information about the schools. These reports ensure that schools are accountable to parents and to the society.
86
Nicholas Sun-keung Pang
THE EFFECTIVENESS OF THE IMPLEMENTATION OF SCHOOL SELF-EVALUATION IN HONG KONG After conducting the school-university partnership projects over the course of a few years, the author conducted follow-up qualitative research to solicit the principals' and teachers’ views on the effectiveness and usefulness of these projects and explore the factors that seemed to help and hinder the implementation of school self-evaluation (SSE) in Hong Kong schools. A sample of 20 schools was randomly selected for this qualitative research project, and 18 of their principals were interviewed. Teachers' views on the same subject matter were also solicited during whole-school workshops held in the schools. In total, about 900 teachers from the 20 schools participated in group discussions through the activities organized in the workshops. The principals’ and teachers' views and opinions, thus collected, were summarized and transcribed, and subsequently analyzed and categorized into themes. These are summarized below.
Factors That Hindered the Implementation of SSE Generally, most principals and teachers opined that school self-evaluation had not been the normal practice in schools and it was, to them, a new and innovative concept. They thought that since the implementation of SSE involved a paradigm shift in school management and change in normal school life for all teachers, external support including financial resources, staff development programs and in-house and consultancy services should be provided. In addition, most principals and teachers would like SSE to be implemented phase by phase, thus giving them more time and “space” to acquire the relevant new knowledge and skills. They reported that in the present turbulent school environment, where there were already many school reforms and innovations, the further introduction of new concepts such as school self-evaluation would inevitably lead to resistance. There were also many specific factors that hindered the implementation of SSE in Hong Kong schools, and these are summarized and classified below at the system level and the school organizational level.
Hindrances at the System Level 1. A loosely-coupled system. The Hong Kong education system is still a loosely-coupled system, with subsidized schools forming the major sector. About 80% of all schools in Hong Kong are subsidized schools, 5% are government schools, 5%, are direct subsidized schools (DSS), and 10% are private schools. While subsidized schools receive financial support from the government, they have their own school sponsoring bodies and management committees. Compared to government schools, subsidized schools have greater autonomy and more freedom of choice in responding to requests for change and the implementation of new educational policies by the Education Bureau. That is, resistance to change in the Hong Kong educational system is much greater than that in the educational systems of other countries, where the state or government schools form the largest sector.
Educational Evaluation and Quality Assurance
87
2. A too-ambitious plan. There are approximately 1,000 schools, including primary and secondary, in the Hong Kong education system. Conducting whole-school territorywide inspections or external school reviews within a few short years was an unrealistic plan. Expecting most schools to be self-reliant in conducting selfevaluations, and to become more open to change within a year or so, was again impractical and unattainable. 3. Too many existing reforms. There have already been many new reform proposals for the educational system in Hong Kong in the 21st century, in addition to those left over from the last decade. Furthermore, the implementation of these reforms and policies has mainly been attempted without good planning and co-ordination. Schools have been suffering from the great confusion and other burdens arising from these reforms. Any introduction of further new reforms and programs in schools would cause, at best, indifference and perhaps resistance because of the tremendous pressure and workloads already existing in schools. 4. School self-evaluation is a complex process. The implementation of school selfevaluation involves a change of school culture and a change of general practices in school life. Such changes cannot be achieved only through directives issued by the education authorities, but need a well-planned, bottom-up strategy of initiation and introduction which means extra resources and support from external sources. 5. Lack of resources. Effective implementation of new reforms or initiatives needs extra resources and support. At a time of economic recession in the early years of the new millennium in Hong Kong, the shortage of financial and human resources has created more difficulties for the implementation of school self-evaluation throughout the territory.
Hindrances at the School Organizational Level Implementation of SSE at the school level is not an easy task, given the present turbulent environment and conservative culture found in most Hong Kong schools. Based on the above-mentioned research, the author observed several major factors that hindered the effective implementation of school self-evaluation. Since these factors are commonly found in most Hong Kong schools, they are worthy of the special attention of school leaders and administrators. 1. The plurality of categories of stakeholders and the diversity of views and opinions in schools may have led to many excellent reforms being opposed. 2. Past failures in the implementation of educational policy caused schools to take a passive and conservative approach to educational reforms. 3. Schools are inevitably political arenas and power struggles are common. These created resistance to educational policy change in the schools. 4. School leaders and teachers in some schools might have been embroiled in conflicts which caused tension, fear, and low morale among teachers, not conducive to selfreflection and self-evaluation. 5. The communication breakdown between teachers and administrators found in some schools meant a very weak basis for professional collaboration and commitment. 6. Most teachers and principals had little knowledge of or skill in school selfevaluation, and misconceptions about SSE were common.
88
Nicholas Sun-keung Pang 7. There were no guidelines and other criteria for measuring success that were commonly agreed upon for use in evaluation. 8. There were no formal, systematic, in-depth, well-designed professional training programs to train people in the implementation of SSE. 9. Most schools were either passive or actively opposed to change, and their organizational learning culture was not well-developed. 10. Most schools lacked a long-term vision of, or ability to plan for, school development and improvement.
Factors That Facilitated the Implementation of SSE Though there were a predominance of factors that seem to have hindered the effective implementation of self-evaluation in Hong Kong schools, there were also a few factors, at both the system level and the school level, that facilitated self-evaluation in the school context.
Facilitators at the System Level 1. A leaner, flatter governance structure of the central education authority. Based on the recommendations of the report on the Education Department (Education and Manpower Bureau, 1998), the Hong Kong Government successfully merged the Education and Manpower Bureau (EMB) with the Education Department (ED) in 2002. Therefore the governance and ruling structure of the school educational system has changed from a three-tier structure (EMB-ED-Schools) to a two-tier one (EMBSchools). The interdependence between policy making and policy implementation has consequently been strengthened, and the school educational system has become more tightly organized. The central educational authority is now more interactive, and more responsive in addressing the problems and difficulties which arise in the implementation of quality assurance mechanisms. 2. An evolving model of quality assurance. Though the two-pronged strategy (an external assurance mechanism and internal quality assurance framework) for guaranteeing the quality of education in Hong Kong has remained unchanged since 1997, the internal framework for quality assurance has been evolving in order to meet the new needs of schools in the ever-changing external environment. A new framework, which enhances school development and accountability through school self-evaluation (SSE) and external school review (ESR), was introduced in 2003 (Education and Manpower Bureau, 2003). This framework stresses the internal mechanism more than the external one, and more resources have been put into the promotion of school self-evaluation. 3. Availability of tools for self-evaluation. Following a few years of development and continuous testing, the former Education and Manpower Bureau was successful in developing some sets of tools for use in schools’ self-evaluation. These tools included performance indicators and key performance measures in the domains of management and organization, teaching and learning, school support and ethos, and academic and non-academic outcomes (Quality Assurance Division, 2002). They provided balanced coverage and a common platform for assessment of different
Educational Evaluation and Quality Assurance
89
aspects of schoolwork and student performance for SSE and ESR, and also established territory-wide norms against which school performance could be measured and assessed. Stakeholders’ questionnaires for teachers, pupils and parents, and other tools in the social and affective domains for students, were developed for use in SSE, and schools were allowed to choose the relevant indicators for their own needs and uses.
Facilitators at the School Organizational Level A few schools participating in the projects successfully created a culture of selfevaluation and organizational change. Such a culture already existed, at least in potential form, in some of these schools before they took part in the projects; others had to create this sort of culture after joining the projects. The school improvement projects provided guidance and consultancy on school development, which facilitated the implementation of selfevaluation in these schools and enhanced their transformation into learning organizations (Pang, 2006a). Factors that facilitated the implementation of school self-evaluation in these schools are summarized below. 1. Enhanced leadership. The management of organizational change calls for “strong” leadership (Gamage and Pang, 2003), and there was a tendency at this time toward stronger leadership in the schools that was better able to succeed in implementing school self-evaluation and initiating organizational change. While some organizational components have a limiting influence on others because of the presence of multiple and often conflicting goals, the success in achieving beneficial organizational change in these schools was due to a strong leadership that eliminated these tensions by deciding upon unified goals and clarifying technology. 2. Shared values. There were, to a considerable extent, shared values among the staff members in the schools which promoted school self-evaluation in the management and organizational structure. Sharing values is the one fundamental activity that holds staff together and unified when they are faced with changes in long-term and short-term goals. If organizations are determinate means-ends structures for attaining preferred outcomes, then agreement about preferences is the only source of order that is left. 3. Focused attention. There was special attention given to human relations in the management system in the schools that had successfully implemented school selfevaluation. Small-step strategies within a confused, turbulent and ever-changing environment may produce more effective, efficient, interesting, varied, and thoughtful organizational changes. Leaders in these schools have compensated for multiple and conflicting goals by carefully selecting targets, controlling resources, and acting forcefully. 4. Good team spirit, high staff morale and a strong sense of professionalism. The schools that were very successful in the implementation of SSE possessed a very strong teaching force that had good team spirit and high staff morale, as well as a strong sense of professionalism. The formation of the strong, professional teaching force was not an accident but the result of deliberate and careful selection during the recruitment of personnel. High teacher morale and a strong team spirit were also the result of the enhanced leadership and effective management systems in the schools.
90
Nicholas Sun-keung Pang
The above are then the findings of the author’s qualitative research into the effectiveness of implementing self-evaluation in schools and the factors that both hindered and facilitated organizational change. These findings shed light on how school administrators can lead and manage the kind of organizational change that will really promote school development and improvement.
CONCLUSION Evidence-based organizational change has become a very recent trend in the school reform and improvement movement, a movement in which school self-evaluation plays an important role. School self-evaluation provides a framework which allows school leaders to institutionalize a self-renewal strategy in daily managerial practices as well as to lead and manage change (Pang, 2006c). Due to various hindrances at both the educational system level and the school organizational level, many Hong Kong schools have not been effectively establishing a self-evaluation framework, and many still have a relatively weak culture of self-renewal. Nevertheless, a normative re-education strategy may be effective in helping schools to surmount the resistance and the hindrances found at both system and organizational levels. The school improvement projects launched in this study are a case in point, showing that organizational change can be initiated through the putting into practice of school self-evaluation. The practical experience gained from even a few successful schools in the project can shed light on the problem of how to help other schools transform into learning organizations through this implementation of SSE. In order to facilitate change in schools, administrators need to demonstrate strong leadership that can unify their school’s goals and make clear what kind of technology is needed for achieving them. Such leadership is also necessary in order to promote the sharing of values among all members and their agreement on preferences. It is necessary for the focusing of leaders’ attention by carefully selecting targets, for controlling resources, and of course for acting forcefully (Gamage and Pang, 2006). A good team spirit, high staff morale and strong sense of professionalism do not only form the crucial basis for change, they also help reduce resistance to change (Pang, 2006b). Effective leaders are those who can adopt these approaches in order to be flexible in coping with the challenges created by an everchanging external environment, and in continuing to lead their organizations towards excellence.
REFERENCES Cuttance, P. F. (1994). Monitoring educational quality through performance indicators for school practice. School Effectiveness and School Improvement, 5(2), 101-126. Education and Manpower Bureau. (1998). Review of the Education Department. Hong Kong: The Government Printer. Education and Manpower Bureau. (2003). Enhancing school development and accountability through school self-evaluation and external school review. EMB Circular No. 23/2003. June 12, 2003.
Educational Evaluation and Quality Assurance
91
Gamage, D. T. and Pang, N. S. K. (2003). Leadership and management in education: developing essential skills and competencies. Hong Kong: The Chinese University Press. Gamage, D. T. and Pang, N. S. K. (2006). Facing the challenges of the 21st century: Preparation of school leaders in Australia and Hong Kong. Educational Research Journal, 21(1), 21-46. MacBeath, J., Jakobsen, L., Meuret, D., Schratz, M. (2000). Self-evaluation in European Schools: a Story of Change. London: Routledge/Falmer. Ministry of Education. (1984). School self-evaluation. Victoria: Ministry of Education. Pang, N. S. K., MacBeath, J., and McGlynn, A. (2004). Self-evaluation and School Development. School Education Reform Series No.19. Hong Kong: The Faculty of Education of the Chinese University of Hong Kong and Hong Kong Institute of Educational Research. Pang, N. S. K. (2006a). Schools as Learning Organizations. In J. C. K. Lee, and M. Williams, (Eds.), School Improvement: International Perspectives (pp. 65-86). New York: Nova Science Publishers Inc. Pang, N. S. K. (Ed.). (2006b). Globalization: Educational research, change and reforms. Hong Kong: The Chinese University Press, the Hong Kong Educational Research Association and the Hong Kong Institute of Educational Research. Pang, N. S. K. (2006c). Managing school change through self-evaluation in the era of globalization. In N. S. K. Pang (Ed.), Globalization: Educational research, change and reforms (pp. 293-313). Hong Kong: The Chinese University Press, the Hong Kong Educational Research Association and the Hong Kong Institute of Educational Research. Pang, N. S. K. (2007). The continuing professional development of principals in Hong Kong. Frontiers of Education in China, 2(4), 605-619. Quality Assurance Division. (2002). Performance Indicators for Hong Kong Schools. Hong Kong: Education Department. Rudd, P. and Davies, D. (2000). Evaluating school self-evaluation. Paper presented at the British Educational Research Association Conference, Cardiff University, September 710, 2000. Scottish Office. (2002). How good is our school? Self-evaluation using performance indicators. Edinburgh: the Scottish Office of Education and Industry Department.
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 7
SCHOOL-BASED ASSESSMENTS (SBA): IMPLEMENTATION AND CHALLENGES IN THE MALAYSIAN CONTEXT Suan Yoong Educational Testing Center, Universiti Pendidikan Sultan Idris, Malaysia
ABSTRACT National examinations are important annual events in Malaysia. This paper briefly provides a critical review of the assessment system currently used in Malaysia, outlines the effects of over-obsession with examinations, provides an overview of the Ministry of Education’s efforts to dilute the obsession with examinations, reviews the proposed implementation of school-based assessment, and explains the dominant forces shaping educational assessments in Malaysia, above all taking into account the cultural contexts.
Keywords: School-based assessment, high-stakes assessment.
1. INTRODUCTION National examinations are important annual events in Asia, especially when the stakes are high in determining entry into premier universities. Come examination time, it has become a common sight in many East Asian cities to see large crowds of parents or relatives waiting in the vicinity of the examination halls, giving moral and material support to their loved ones who are taking the examination. In fact, around examination time it is not uncommon to see parents readying their children for the last dash through the “dreaded and stressful” hurdles of examinations. They may do this by preparing “dietary tonics and fortifying medicines” (e.g., jinseng) and other food essences to supplement the examinee’s diet, or by extending this “fortifying culture” in the form of supplementary tutoring by ferrying students to and from private tuition centers. Indeed, the glaring presence and
94
Suan Yoong
vigorous growth of these cram schools, designed purely for examination preparation in Japan, South Korea, Mainland China, Taiwan, Hong Kong, Macao, Singapore and Malaysia, is further testimony to the presence of East Asia’s examination-oriented culture (Bray, 2003; Cheng, 1990; 1995; Chew and Leong, 1995; Foondun, 1992; 1998; Harnisch, 1994; Hussein, 1987; Kim, 2000; Kwok, 2004; Rohlen, 1980; Sorensen, 1994; Zeng, 1999). The internationally renowned “boulevard" of tuition centers on Nanjing Street, near the Taipei Train Station in Taiwan, is perhaps the most vivid example of this culture (Chen, 1996). Come D-day and the announcement of examination results, the local media rush to flash news reports that may highlight the most successful high schools, glorify outstanding individual performances, and present ranking tables that compare school performances. Of course, learning cultures in other parts of the world are also to some degree examination-oriented. However, in East Asia this phenomenon and its side-effects are more pronounced, and thus here especially they are felt to warrant a closer examination. The extraordinary emphasis placed by East Asian students and parents on examinations is also one of the pre-eminent characteristics of the educational system in Malaysia. While the examination orientation has been a force driving many students to excel in their studies, this obsession has also created both educational and social problems. Many Malaysian educators, practitioners, curriculum developers and policy makers have long recognized the negative impacts and detrimental effects of over-obsession with high-stake assessments, and have criticized such practices on the grounds that they distort the basic objectives of education. Like a record playing over and over again, these concerns have been publicly voiced year in and year out, especially after public examination results are splashed in the print media. This paper briefly provides a critical review of the assessment system currently used in Malaysia, outlines the effects of the national assessment system, provides an overview of the advantages and disadvantages of an over-obsession with examinations, reviews the implementation of the school-based assessment (SBA) system in Malaysia, and explains the dominant forces shaping educational assessments in Malaysia, with a particular emphasis on the cultural contexts.
2. THE MALAYSIAN EDUCATIONAL AND EXAMINATION SYSTEM Malaysia is a multi-ethnic country, being 66% Bumiputra (mainly Malays plus a small proportion of other indigenous groups such as Ibans, Kadazans and Bidayuh), 25% Chinese, 8% Indian (Hindu), and 1% other ethnicities (Malaysia, 2006). This Southeast Asian country has inherited a pluralistic system of education from its colonial master, England, one that mirrors to a large extent the legacy of the system in Great Britain. The difference is that, as in most Asian countries, the educational system of Malaysia is highly centralized under the full control of the Ministry of Education (MOE). The educational system in Malaysia is essentially three-tiered—primary, secondary and tertiary education—with the tiers linked to each other through a series of formal, nationallyconducted public exams (see Figure 1). At the age of 6 plus, Malaysian children begin a formal primary education that will last for 6 years. Malay, Chinese and Tamil government primary schools exist side by side, each using their respective mother-tongues as the main medium of instruction. Government secondary schools, which use Malay as the main medium
School-Based Assessments (SBA)
95
of instruction, offer 3 years of lower secondary and 2 years of upper secondary education. Students are streamed into academic or technical and vocational tracks at upper secondary levels. Pre-university education (sixth form or matriculation classes) is offered in selected secondary schools for 2 more years for students wishing to enter government-sponsored public universities.
Figure 1. Malaysian Educational System and Examinations.
2.1. Assessments and Public Obsessions Students completing primary education must sit for the UPSR (Primary School Assessment) Examination. After completing their lower secondary education, they must take the PMR (Lower Secondary Assessment) Examination. Upon completing their upper secondary education, students in the academic and technical tracks will sit for the SPM (Malaysian Certificate of Education) Examination (equivalent to the British O-level examination, now referred to as the GCSE), while students in the vocational track will sit for the SPMV (Malaysian Certificate of Education -Vocational) Examination. Students at preuniversity levels must sit for either the STPM (Higher School Certificate) Examination (equivalent to the British A-level examination) or for matriculation examinations accredited by the local universities. All public examinations at the end of the various levels, namely primary, lower secondary, upper secondary, and pre-university/matriculation, are controlled and administered by the Malaysian Examination Syndicate (MES) set up by the Ministry of Education. Although these public examinations were initially designed for selection purposes, student
96
Suan Yoong
promotions through primary to upper secondary grades were made automatic when Malaysia instituted 11-year free universal education. However, SPM results will determine whether or not students can further their education in government pre-university or matriculation classes, while the STPM matriculation exams remain the major selection examinations for entry into local government universities. Although promotion of students from grade to grade is automatic until the secondary level, the obsession with public examinations remains very great in Malaysia. Education as a social activity carries with it, after all, the social values specific to particular societies and cultures. Yet while in many societies having a prestigious career requires attending an elite university, in Malaysia (and countries like it) attending an elite university means passing examinations at the end of every tier, and ultimately passing the university entrance examination with a high enough grade. Furthermore, in order to pass the latter with a high enough grade to enter one of the best schools, students must begin preparing at an early age (Feiler, 1991). The final prize of an elite higher education being so attractive, the sociocultural value of assessment results over-ride their academic value. For example, though the UPSR and PMR have since been designed as diagnostic examinations, their results are still used primarily as a basis for channeling students into premier secondary schools or streaming them into or onto academic or technical-vocational tracks at the upper secondary level (Ministry of Education, 2004). Thus the SPM and STPM are high-stakes exams that determine entry into national universities and/or local private universities and colleges with overseas university collaboration programs. Through a policy of rewards and other forms of recognition dating back to the colonial era, education has, in other words, become the most significant instrument in determining one's future job (career) and consequent social status. Students therefore need to overcome a series of examination hurdles in their pursuit of the final prize, and this “scramble” to gain entry into premier elitist schools starts from an early age. It is only natural that examinations gained real legitimacy in Malaysia as a reliable means of facilitating and regulating upward social mobility.
2.2. Chinese High Schools Apart from the public secondary schools, there also exist 60 privately-managed Chinese independent high schools where most subjects are taught in Chinese. About 20% of the students who completed primary education in public Chinese primary schools continue their education in Chinese independent high schools. Chinese independent high schools offer 3 years of Junior Middle and 3 years of Senior Middle education. Upon completing each tier of high school education, students must sit for the UEC (Unified Examination Certificate for Chinese Schools) Examination conducted by a private Examination Board that is itself set up by a united Chinese school community organization, the Dong Jiao Zong. The Senior Middle UEC is recognized by over 300 universities and colleges around the world. With the UEC results, independent high school students can further their studies in overseas universities, especially in Taiwan, China, Japan, the United States, Canada, the United Kingdom, Australia and New Zealand, or enter local private colleges which have twinning programs with overseas universities. Some Chinese independent high schools offer dual programs, teaching in both Malay and Chinese, so that the students can sit for the PMR, SPM, STPM and/or UEC.
School-Based Assessments (SBA)
97
The UEC is an equally high-stakes exam as far as the students and parents of Chinese independent high schools are concerned, since it offers the opportunity for students to gain entry into overseas universities in the United States, Canada, the United Kingdom, Australia and New Zealand. Moreover, the Chinese independent high schools are also more stringent in their automatic promotion policy, retaining students longer at each grade level if they fail to perform adequately at that level. A significant number of premier Chinese independent high schools also control their student intake by conducting entrance exams and only admitting the top students from Chinese public primary schools. Thus an extraordinary emphasis is placed on education in Malaysia and especially by Chinese students and parents, necessitating a high-stakes examination system.
3. THE NEGATIVE INFLUENCE OF THE EXAMINATION ORIENTATION Public exams in Malaysia are essentially assessments of achievement in a number of core and elective subjects, focusing mainly on cognitive aspects of learning based on the national curriculum. Because the educational system is intensely competitive and hierarchical, only a small percentage of students are selected for the top stream and an even smaller percentage making it into a local university. For the Malaysian Chinese there is an added element: the Confucian heritage influences their perception of public examinations. For over a thousand years, formally conducted public examinations in China have given millions of families hope for a better future for their children and grandchildren, even though extremely few have fulfilled this hope (Ho, 1986; Lee, 1996). Upward social mobility has long been achieved in China primarily through education. The reliance on examinations, closely tied to the Confucian heritage, has thus become deeply embedded in Chinese culture. This traditional examination-orientation has also been extended to other East Asian countries, notably Japan and Korea, whose cultures were themselves long ago influenced by Chinese Confucian culture: hence university entrance exams are important annual events in Japan, Korea, Taiwan, Singapore, Malaysia and almost all overseas Chinese communities in the region (Rohlen, 1980; Sorensen, 1994). It is perhaps no exaggeration to say that an exam-orientation is the soul of Chinese educational culture, whether inside or outside China (Cheng, 1995). Furthermore, this ethos has been extended from the Chinese community to the other ethnic groups in Malaysia.
3.1. Pressure to Excel The consequences of this traditional examination-orientation has had a great impact on schools in various countries. The pressure to excel has resulted in schools’ attempting to control their student intake by selecting the most able students ahead of those with learning difficulties in order to raise or maintain test performance levels. The exam-orientation causes schools to adopt policies that are only useful for passing high-stakes exams rather than focusing more generally on skills useful in helping prepare students for later life. Indeed, “teaching to the test” by focusing intensively on test-preparation activities, using test-
98
Suan Yoong
preparatory texts and past test papers is a distinct instructional style that can be seen at all levels in most Malaysian schools. The obsession with teaching-to-the-test has also led to the phenomenon of curriculum "narrowing" on the part of teachers (William, 2001; quoted in Gregory and Clarke, 2003), that is, de-emphasizing or neglecting subject areas not tested or emphasized on exams, and precluding any supplementary curricula that encourage students’ all-round development. Classroom instructional and assessment strategies that mirror the content and format of the examination tend to be emphasized, thus leaving significant portions of the “non-examinable” content untaught or quickly skimmed through. Concerned educators worry that subject areas not tested (for instance, fine arts, physical education, music, and other “low-stakes” subjects) are accorded less class time or set aside altogether. Malaysian schools in particular are also devoting non-critical subject times to examination subjects, even in the early grades.
3.2. Exam-Orientation in Instructional Strategy and Private Tuition However, public examinations are obviously overly simplistic, overly objective, reductionist and thus very limited; these paper-and-pencil assessments tend to measure only a “myopically defined, narrow, test-defined set of skills” (Madaus and Clarke, 2001; quoted in Horn, 2003). They often fail to measure higher cognitive skills, a vital part of a person’s complex mind and personality. Public exams have been criticized for promoting rote memorizing of facts on the part of students, rather than problem-solving skills or (of course) creativity. Critics argue that this “spoon-fed” education legitimized by the examination culture produces students who can only regurgitate uncritically what they have acquired through rote learning. Such practices not only hamper real intellectual growth but mean the assessment is less able to select students with active, independent, creative minds, since those who succeed on exams may have passed by means of memorizing standard answers to stereotyped examination questions. This exam-orientation in instructional strategy has also led teachers to teach in ways that contradict the notions of sound educational practice, resulting in the “de-professionalization” of the teachers themselves. It is a common observation that practicing teachers who have been trained to teach problem-solving strategies are compelled by examination pressure to teach the subject matter in science and mathematics as knowledge to be memorized rather than as concepts to be applied to real problem-solving or inquiry. Similarly, science laboratory experiment sessions have also been compromised by the drive to prepare students for public examinations. In effect, high-stakes assessment brings with it a "default" philosophy of education that “holds in high regard a narrow bundle of knowledge and skills” (Gunzenhauser, 2003) to be memorized. Because of the power of this default philosophy, teachers in the exam-orientation climate may find themselves doing things that fall short of their visions of themselves as true educators, such as spending extra time drilling students on practice tests, de-emphasizing or eliminating untested subject matter, or teaching to the test (Jones, Jones, Hardin, Chapman, Yarborough, and Davis, 1999). Critics have also charged that over-obsession with exams is draining the fun out of schooling, and placing undue pressure on young children by increasing their levels of anxiety, stress, and fatigue. The anxiety has driven the majority of them to attend test preparation activities such as personal tutoring sessions, coaching, and extended test-oriented classes,
School-Based Assessments (SBA)
99
which in turn increase the children’s stress and fatigue. The demand for tutoring is spawned by the exam-oriented learning culture, that is, by the increasingly high levels of competition for social mobility (Bray, 2003), but also by the multi-faceted relationships between and among supply and demand patterns in the economy and other factors leading to the desirability of tutoring jobs (Kwok, 2004). Small wonder then that private tutoring that is parasitic on daytime schooling has become popular and is developing into a thriving enterprise in many countries, including more recently Malaysia. Moreover, students who view the passing of exams as an insurmountable barrier may ultimately drop out of school early. High-visibility examples of security breaches, rampant cheating in examinations and test-tampering have also been reported; leakage of examination questions has been reported as occurring sometimes even at the primary school level in Malaysia (Kong, 2007).
4. DILUTING THE INFLUENCE OF EXAM-OBSESSION: THE MINISTRY’S EFFORTS There have been large-scale school and educational reforms initiated at primary and secondary levels, with the goal of enlarging educational opportunities by reducing the hurdles at each level of education, minimizing the backwash of public examinations, and fostering continuous assessment or SBA. Since 1970, the Ministry of Education has initiated curriculum reforms, notably in 1976, 1983, 1995, and 1999, to enhance the quality of education by enriching and diversifying the curriculum, focusing on student-centered pedagogical changes, and stressing outcomes-based learning. However, none of these has involved any serious attempt to reform the public examination system in Malaysia until recently. Malaysia began to democratize its educational system in the early 1990s by introducing a policy of 11 years of free universal education. This new policy included a change in the way of grading public examinations at the end of primary (UPSR) and lower secondary (PMR) school: now the goal of grading would be diagnostic in nature rather than, as before, strictly serving a selective function—that is, promotion to the educational tier or level. However, these exams still retained their high-stakes character since their results were still being used to select students into premier secondary schools and stream them into science, arts, technical or vocational tracks at the upper secondary levels.
4.1. School-Based Projects and Formative Assessments Then in 1996 the Ministry of Education introduced the Secondary School Progressive Assessment or PKSM. This required students who sat for the PMR (lower secondary) and SPM (upper secondary) examinations to submit coursework output in the form of portfolios and projects for History, Geography, and Integrated Living Skills and Design. These were school-based assignments or projects graded by the school teachers following standardized marking schemes supplied by the Malaysian Examination Syndicate (MES), and the grades were submitted to the MES to be included in the final stage of PMR or SPM grading. Similar practices had also been instituted by the Chinese Independent Schools Examination Board for
100
Suan Yoong
the UEC examination. For the PMR examination, oral assessments are also planned for Malay and English languages, while practical assessment is planned for science. Unfortunately, grade inflation and cheating in the form of non-authentic or plagiarized work are so widespread that the school-based assignments or oral practice sessions have been criticized heavily by the public. Teachers are unhappy because these have also increased their workload considerably. Moreover, the effect of both practices on reducing examination pressure was minimal. However, both of them—school-based assignments and oral practice sessions—are still in use today. Most of the Ministry’s subsequent efforts seemed to be aimed at circumventing the system by enhancing the teaching and learning process as a means to reduce the pressure of final public examinations on students, and to reinforce the importance of consistent learning throughout the school years. This initiative was promoted through PKBS (School-Based Progressive Assessments). PKBS was a program of formal assessments in the form of weekly, monthly or term tests, projects and regular class assignments that were set by the respective subject teachers and conducted throughout the year, in order to evaluate pupils’ progress at all levels in the cognitive, affective and psychomotor domains. These assessments were done through classroom observations, orals, and written works. The information from these non- standardized assessments was used to reinforce the instructional process. However, teachers were unhappy with this program because their workload was increased considerably. Moreover, the results of these assessments were neither included in the public examination grades nor needed for the selection and promotion of students to the next grade. As a result, the teachers kept reverting to their old ways of teaching to the test.
4.2. The Planned New Role of ICT in Assessment and Smart Schools Then in 1996, Malaysia began to plan its own Multimedia Super Corridor (MSC) as a dynamic hub for the global ICT industry, in order to leap forward into the information and knowledge age. The Multimedia Super Corridor is a 15km by 40km area, encompassing the Kuala Lumpur City Centre and Kuala Lumpur International Airport, which will be developed as a high growth centre. MSC Malaysia now hosts more than 1000 multinationals, including foreign-owned and home-grown Malaysian companies, focused on multimedia and communications products, solutions, services and research and development (MSC Malaysia, 2008). The Malaysian government has equipped core sectors of MSC Malaysia with highcapacity global telecommunications and logistics networks. Six innovative flagship applications have been developed in MSC Malaysia to accelerate its growth, among them the Smart School flagship application (Malaysia, 1997). The Malaysian Smart School Flagship was premised on the strong belief that information and communications technology is a key enabler, that it can stimulate a desire to learn in everyone. Deemed one of the most forwardlooking ICT-mediated learning initiatives in the world, the Smart School Flagship was established in order to reinvent the core teaching-learning processes by integrating ICT with the curricula, pedagogy, and assessment practices of Malaysia’s schools. The goal would be to help students learn more effectively and efficiently while simultaneously gaining a competitive position in an increasingly globalized, ICT-driven market. Computerized management software was developed to help the “smart schools” manage more efficiently and effectively the resources and processes required to support teaching-learning functions,
School-Based Assessments (SBA)
101
thus enabling students to practice self-accessed and self-directed learning, and making the teaching-learning process easier, more fun and more effective. The Smart School system was piloted in 87 schools beginning in 1999, and will have been expanded to include 10,000 schools by 2010. A key element of the Smart School system is a smart assessment system that involves a significant departure from the “current assessment system which is characterized by a single delivery (one session only) at a fixed time of the year to students of a given age group” (Malaysia, 1997). Within this system, and integrated with ICT, criterion-referenced on-going assessment would be emphasized to provide a more holistic and accurate picture of a student’s performance. Teachers, students and parents will be able to access on-line assessment items as well as get a more accurate picture of a student’s achievement, readiness, progress, aptitude, learning styles, and abilities. Smart School assessment will be flexible and learner-friendly, while the quality of the assessment information will be assured by using multiple instruments and approaches. This will lead to a flexible assessment and certification system that involves using a combination of classroom-based, school-based, and centralized assessments, using authentic, alternative and performance-based assessment instruments and approaches. The Smart School assessment system not only attests to a student’s cumulative accomplishments but will also promote continued improvement on a lifetime basis. According to the plan, assessment for achievement tests will be implemented first, and assessment for readiness, progress and aptitude tests will be developed as teachers grow more capable in the use of the skills and tools that will be made available to them. The Smart School management system would have the capability to store assessment records that document students’ accomplishments during their entire educational career, to be identified by a single ID (Malaysia, 1997). However, evaluation of the Smart Schools program revealed that although ICT holds great promise, the reality in the Malaysia Smart schools can hardly match or reflect the stateof-the-art in technological advances, and the application of ICT to instruction, assessment, and testing in the Smart Schools is still in its infancy (MDC, 2004; Frost and Sullivan, 2004). Therefore, despite these proposed innovative efforts, the smart assessment system has yet to surface in the Malaysian school system, and even in the pilot Smart schools there seems to have been no noticeable change in the Malaysian examination culture.
4.3. Blueprint for Educational Development In 2001 the Ministry of Education unveiled a 10-year educational development plan (Ministry of Education, 2001), reiterating the need to strengthen SBA as a way to reduce emphasis on centralized examinations while integrating the elements of criterion-referenced assessments into existing assessments and strengthening formative evaluation. The practical assessment of science in the UPSR examination was proposed; SPM would be made an open certificate examination, allowing students to takes at least 6 core subjects and open up more electives, with no limit on the number of subjects to be taken (Ministry of Education, 2001: 99). While this system allows the students to choose examination subjects based on their own interests and potential, the effort was counter-productive as many students chose to take many more examination subjects than were required - some as many as 17 per sitting.
102
Suan Yoong
More recently, the Ministry of Education unveiled the Blueprint for Educational Development 2006-2010 in an effort to reform the educational system and "improve the longterm international competitive edge of Malaysia” (Ministry of Education, 2006). The Blueprint is specifically for the Ninth Malaysia Plan: 2006-2010, which stresses that the implementation of the SBA system will be accelerated to enable continuous evaluation of, and support for, creativity as well as analytical problem-solving skills (Malaysia, 2006). This Blueprint specifically notes that “presently emphasis has been placed on summative evaluation that focuses on student achievements and not on identifying strengths and weaknesses in teaching-learning” (Ministry of Education, 2006). According to the Blueprint, the assessment system will be modified so as to make schooling less oriented toward examinations; this will be done by reducing the number of public examination subjects, while other subjects will be assessed at the school-based level. The Blueprint also charts the way toward achieving a new examination reform, a new examination policy that will strengthen SBA because it will be holistic, task-based and criterion-referenced, and will also promote character-building. Specific recommendations include changing the emphasis of public examinations: rather than content-based they will now be skill/ability-based; they will assess ability rather than achievement. Also, there will be more room for alternative assessments and for the quality assurance of assessments, and national benchmark standards will be established (Ministry of Education, 2006).
5. SCHOOL-BASED ASSESSMENT: THE ROAD MAP “We will empower teachers to conduct SBA and give them the tools and training to carry it out effectively…School assessment will be given due recognition and acknowledgement.” These are the words of Dr. Adi, the Malaysia Examinations Syndicate Director of Examinations, who delivered the keynote address at a forum on educational assessment in Kuala Lumpur, Malaysia (Adi, 2007; Simrit Kaur, 2007). According to Dr. Adi, the ultimate aim is to reduce over-reliance on data about students obtained through centralized public examinations. “There has been too much attention given to public examinations. School is now looked upon as a process of preparing students for public examinations. School assessment has lost its significance and the outcome is not taken into account in students’ final grade,” said Dr. Adi. “The focus should not only be on academic aspects. We should also assess students’ personal and character development and involvement in extra-curricular activities.” (Bernama, 2007). Accordingly, a new assessment system for schools will be implemented by 2010, in order to make way for a major change in the Malaysian educational system: centralized examinations will give way to a combination of centralized examinations and SBA. For the first time, Dr. Adi has gone a step further to announce that, following the proposed introduction of SBA, the Malaysian Examinations Syndicate proposes the abolishment of two major public examinations – the UPSR (at the end of the primary level) by 2012 and the PMR (at the end of lower secondary level) by 2015. Five forms of assessment for students in the national education system have been proposed:
School-Based Assessments (SBA) • • • • •
103
SCHOOL assessment – planned, developed, conducted, examined and reported by teachers in schools. CENTRAL assessment – involving standardized instruments, data analyses and guidelines provided by the MES. CENTRAL examinations – fully conducted by the MES. PSYCHOMETRIC tests – to measure students’ innate abilities. PHYSICAL activity assessment –to evaluate and measure students’ performance and involvement in extra-curricular activities.
Dr. Adi nonetheless noted that this will change the whole culture of knowledge acquisition in Malaysia, for students may begin to feel that learning can actually be fun. However, he also said that it would take time, because the transition would require several stages—from getting input from academicians, teachers, parents and students to choosing the right methods of assessment in the Malaysian context (Koh, 2007a).
5.1. Public Reactions Public reactions to the announcement ranged from expressions of full support to those of having reservations. The following news report is typical: A parent, V. Vasantha, said she had reservations about teachers being objective when it came to school-based assessment. Psychometric testing, for example, is something new and will require training for teachers, or they could come out with "10 conclusions about one student." And could teachers cope with assessing so many students? There may be adjustment problems but "We do not want any bias in the grading or assessments, so the standards must be comparable for schools to want to compete with each other" (Koh, 2007a). [I’m assuming these news reports were translated by author from Chinese or Malay?] No! it was actually from the Online news report - Yoong
Many had urged the Education Ministry to be cautious in its implementation of the system, saying it would really need to depend on teachers’ performance and the guidelines for all schools when it came to assessment (Chok, 2007; Koh, 2007b, Kong, 2007). "It is good that it will be more school-based but there should be a guarantee that there is a standard for all. Some others find hope in the Education Ministry’s plan to reform the assessment system for students, which will shift the emphasis from examinations to continuous forms of SBA such as aptitude, personality and diagnostic tests” (Koh, 2007a). The following excerpt from a local news report records a typical reaction to the recent proposal to scrap the UPSR and PMR examinations: “I remember a time when students had to do well enough in Primary school, passing all the exams before they were allowed to go to Secondary school. And then you had to pass your SRP before you could move to Form 4. And then our Government decided to make things easier for students, dumbing down the content of the education, and now we are even thinking of removing Public Examinations! Do you really trust individual schools to set their own test and examinations? I don’t. Not that I don’t trust teachers, but it’s more that I don’t trust the schools. There will be leaks in the papers, and with no proper guidelines for marking and
104
Suan Yoong grades, what will happen is that on paper the Government will suddenly show how “Smart” our students are, when in reality our educational system is going down the drain. I think we should be making our exams harder, getting some quality students out there, not people who can get an A in English yet have really terrible grammar and not be able to hold a conversation. Gone are the days when our education meant something, and when students took exams seriously. Back in the day, you studied for the SRP1 because if you failed, you would be held back a year!” (Simrit Kaur, 2007)
5.2. A Culture with an Entrenched Examination-Orientation In Asia, public examinations have always meant an extremely high status for education. An understanding of the cultural dimensions of East Asian societies, including Malaysia, and of the role in them of education will prove helpful in understanding the deeply-entrenched examination-oriented culture in Malaysia, especially when it comes to educational evaluation or assessment—for this is seen by the community as being very important, as having high stakes. In Malaysia, different sectors of the community—whether teachers, parents, students, administrators, businessmen, political leaders, or even professionals—all seem to share a very similar cultural attitude toward what education should be and how children should be educated. The fear of losing a competitive edge in the international arena also foster concerns about the quality of education, the latter being most often cast in terms of education attainments, competitiveness, world class standards, and academic excellence. Thus, the Ministry of Education has always focused on enhancing the knowledge, skills and expertise of future generations: these will be needed to support a knowledge-based economy and realize Malaysia’s vision of becoming a developed nation by the year 2020. The centrality of education in East Asian societies with identifiable common goals has fostered a virtual coalition between parents and teachers in the quest for excellence in education. As a result, students are required to adapt themselves to the system, and indeed adaptability is an essential crucial quality in the Asian education process. The dialectics of work and play, happiness and pain, give and take, tolerance and gain, ease and difficulty, are deep-rooted in these societies and could explain the rationale for adaptability behind many educational practices. Cheng (1995) noted that Chinese parents and students prefer to adapt to the system instead of intervening in the educational process, and thus that they do (like their children) expect learning to continue beyond school hours. In such a socio-culturaleducational context, the students have little room for any activities apart from formal academic learning, and the latter can take place in or out of schools, can extend into private homes (including those of private tutors) and after-school hours. A clear percentage of students adapted well and achieved an excellent academic performance. Unfortunately, a large proportion of those who could not make it were “legitimately” ignored for failing to do so. Given the fact of this Malaysian culture with its deeply-entrenched examinationorientation, it is not surprising that the public generally supports public exams as the best way to maintain objectivity, higher standards, and stricter accountability in the educational system. The Malaysian public feels strongly that examinations, while not perfect, are probably the 1
Note: SRP examination was replaced by PMR in the 80s.
School-Based Assessments (SBA)
105
best way to hold all students accountable for learning what they are expected to learn during their school careers. Given this entrenched attitude, it is clear that the Ministry of Education will have a hard time reducing (let alone eradicating) the public’s obsession with examinations as the fairest means of student evaluation.
CONCLUDING REMARKS One must be clear about the policy goals of examination reform: know the strengths and weaknesses of all testing instruments; recognize the political, social, and educational tradeoffs involved in using one form of assessment over another; and, most importantly, make no demand of any testing instrument for which it was not designed. One-shot assessment, no matter what the format, cannot address multiple needs (Goertz and Duffy, 2003). Public examinations are designed to meet certain purposes, notably to provide reliable and comparable scores for individuals as well as groups. They are cost-efficient in terms of development, scoring, and administration, and sample a broad set of curriculum objectives within a limited time frame. In general, the competencies tapped are those that lend themselves to being assessed in these ways, while aspects of learning that cannot be assessed under such constrained conditions are not addressed. To design new kinds of assessment methods as alternative approaches to large-scale assessment for capturing the complexity of cognition and learning will require breaking out of the current paradigm. It is now a common practice to characterize student achievement in terms of multiple aspects of proficiency rather than relying on a single score. It is also possible to chart students' progress over time instead of simply measuring performance at a particular point in time. Schools should use multiple paths in evaluating and look for alternative forms of valued performance. Many of these methods are not yet widely used because they are not easily understood or packaged in accessible ways. Furthermore, having a broad array of models does not mean the measurement problem has been solved. There is a need to focus, in psychometric model-building, on the critical features of achievement that we want to assess. Information and communication technologies are helpful in removing some of the constraints that have limited the practice of evaluation in the past, and new ICTs are expanding the types of constructs that can be tapped through assessment. By enriching assessment situations through the use of multimedia, interactivity, and control over the stimulus display, it is possible to evaluate a much wider array of constructs than was previously possible. The Smart School assessment system offers opportunities in this direction. New capabilities afforded by technology include directly assessing problem-solving skills, making visible the sequences of actions taken by learners in solving problems, and modeling complex reasoning tasks (National Research Council, 2002). Technology also makes possible the simplification of teachers’ tasks. (Chudowsky and Pellegrino, 2003) Introducing SBA into the public examination system improves the validity of assessments. Some important outcomes which cannot be evaluated within the context of a written examination can readily be assessed through SBA (e.g., practical work, theatres and workshops, research projects, portfolios of creative work, active artistic or design projects). SBA must be designed so that evaluations become an integral part of teaching and learning within school subjects and not an ‘add-on’. There should be no significant increase in the
106
Suan Yoong
workload of teachers. In order to carry out SBA in a rigorous and consistent manner, teachers will need to receive appropriate training and support. They need to develop a wider range of assessment skills which will enable them to better facilitate the implementation of SBA. In the implementation of any educational reform, the capacity of the system to support change in practice must be addressed. Research on SBA programs shows that clear goals and incentives are necessary but not sufficient to motivate teachers to reach their school's student achievement goals. Teacher motivation is also influenced by the presence of various capacitybuilding conditions, most obviously the opportunity for meaningful professional development. In addition, teacher knowledge and skills related to improved instruction are important (Sloane and Kelly, 2003). Moreover, teacher education must be funded along with the various complementary testing programs. In other words, schools need knowledge, human and financial resources to successfully implement any reforms. The Ministry of Education’s latest effort in examination-system reform will involve a huge effort to overcome a deeply-entrenched exam-taking culture. Cultural reform is very difficult to achieve; deeply-entrenched beliefs and practices are hard to eradicate, let alone transform. The public needs to be more broadly educated about the possible range of testing and test-interpretation formats, if people are ever going to shake off their traditional concept of assessment and embrace a new assessment culture that is forward-looking, psychometrically sound, and humanistic. Such a reform must be accompanied, then, by a structural change in the society and culture as well as in the educational system. The highly centralized Malaysian system of education must be further decentralized and democratized, and schools must be appropriately empowered. This depends, once again, on a clearer understanding of the deeply-entrenched examination-orientation of Chinese and Malaysian culture, the tendency to see exam-taking as a kind of one-shot, high-stakes “gamble.” An example showing that progress in this direction is possible is the PTS (Level One Assessment) aptitude test, introduced to select grade 3 students so that they might skip grade 4 and proceed directly to grade 5. (This was in line with the new education policy which stipulated that primary education can be completed in five to seven years.) Pupils were tested on their abilities and potential in verbal, quantitative and thinking skills. Of course, this PTS inevitably became a high-stakes test in the eyes of teachers and parents as well as students; fortunately, due to the pressures it generated, it was discontinued two years later. On the other hand, allowing almost everyone to enter university does not necessarily reduce examination pressure, as students will still be competing to get into the best universities. Thus, irrespective of what changes are made in the assessment mode, the newly-introduced measure always becomes high-stakes unless there is a strong will to dilute and diversify the incentives and reward systems that bind upward social mobility so tightly with education. We must be able to deal with the fact that if and when School-Based Assessments are given a more significant weight in university entrance selection, they, too, will become relatively high-stakes—but hopefully relatively less high-stakes than winner-take-all exams. We are still a long way, in Malaysia, from the goal of overcoming a deeply-entrenched examination-orientation culture. There are no simple solutions. The key point to keep in mind is that the final goal of standards and assessment systems is not the race for ever-higher scores, but the race for students’ solid preparation for the workplace or postsecondary education. Furthermore, even if the SBA program is increasingly implemented, accountability remains an important problem to be reckoned with, most notably in the context of the
School-Based Assessments (SBA)
107
documented gaps in achievement between students of different economic, ethnic, and racial backgrounds.
REFERENCES Adi B. T. (2007). A proposal towards a more holistic education assessment system in Malaysia. Keynote address at International Conference on Assessment (KLICA) of the National Educational Assessment System (NEAS): Looking Ahead, organized by the Examinations Syndicate, Ministry of Education, Malaysia. 7-9 May 2007 at Sunway Resort Hotel and Spa, Pelaling Jaya, Malaysia. Bernama. (2007). New Assessment System for Schools To Be Implemented In 2010. Bernama News Agency, Malaysia. May 17, 2006. Bray, M. (1999). The shadow education system: Private tutoring and its implications for planners. Paris: UNESCO / IIEP. Bray, M. (2003). Adverse effects of private supplementary tutoring. Paris: UNESCO / IIEP. Chen, J. (1996). Cram Culture - A Fresh Look at the Buxiban. Retrieved July 15, 2007, from http://home.sina.com/sinorama/1196/article2/index.html. Cheng, K. M. (1995). Excellence in education: is it culture-free? Keynote address presented at the 9th annual meeting of the Educational Research Association, November, 22–24, Singapore. Cheng, K. M. (1990). The culture of schooling in East Asia. In N. Entwistle (Ed.), Handbook of education ideas and practice (pp. 163-173). London: Routledge. Chew, S. B., and Leong, Y. C. (Eds.). (1995). Private tuition in Malaysia and Sri Lanka: A comparative study. Kuala Lumpur: University of Malaya, Department of Social Foundations in Education. Chok S. L. (2007). It takes courage to cut the exam knot. New Straits Times, June 3, 2007. Chudowsky, N. and Pellegrino, J. W. (2003) Large-scale assessments that support learning: what will it take? Theory Into Practice, Winter, 2003. Feiler, B. S. (1991) Learning to bow - Inside the heart of Japan. New York: Ticknor and Fields. Foondun, A. R. (1992). Private tuition in Mauritius: The mad race for a place in a ‘five-star’ secondary school. IIEP Research and Studies Programme, Monograph Vol. 8. Paris: IIEP. Foondun, A. R. (1998). Private tuition: A comparison of tutoring practices at the primary level in Mauritius and some South East Asian countries. Bangkok: UNICEF East Asia and Pacific Regional Office. Frost and Sullivan. (2004). Benchmarking of the Smart School Integrated Solution. Retrieved July 15, 2007, from http://www.msc.com.my/smartschool/downloads/benchmarking.pdf. Goertz, M. and Duffy, M. (2003). Mapping the landscape of high-stakes testing and accountability programs. Theory into Practice, Winter, 2003. Gregory, K. and Clarke, M. (2003) High-stakes assessment in England and Singapore. Theory into Practice, Winter, 2003. Gunzenhauser, M. G. (2003). High-stakes testing and the default philosophy of education Theory into Practice, Winter, 2003. Harnisch, D. L. (1994). Supplemental education in Japan: Juku schooling and its implications. Journal of Curriculum Studies, 26(3), 323-334.
108
Suan Yoong
Ho, D. Y. F. (1986). Chinese patterns of socialization: A critical review. In M. H. Bond (Ed.), The Psychology of the Chinese People. Hong Kong: Oxford University Press. Horn, C. (2003). High-stakes testing and students: stopping or perpetuating a cycle of failure? Theory into Practice, Winter, 2003. Hussein, M. G. A. (1987). Private tutoring: A hidden educational problem. Educational Studies in Mathematics, 18(1), 91-96. Jones, G. M., Jones, B. D., Hardin, B., Chapman, L., Yarborough, T., and Davis, M. (1999). The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan, 81(3), 199-203. Kim, I. (2000). Private tutoring in Korea: A cultural analysis. Paper presented at the Annual Conference of Comparative and International Education Society (CIES), March 2000, San Antonia, Texas, USA. Koh L. C. (2007a) School-based assessment: Thumbs up mostly, but some foresee teething problems, NST Online. Retrieved March 6, 2007, from http://www.nst.com.my/ NST/index_html Koh, L. C. (2007b). Taking formal education beyond exams. Retrieved March 22, 2007, from http://www.nst.com.my/Current_News/NST/Sunday/SundayInterview/20070304082920/ Article/index_html. Kong S. H. (2007, September 14). Parents want probe into exam ‘leaks’. The Sun. p.8. Kwok. P. (2004). Examination-oriented knowledge and value transformation in East Asian cram schools. Asia Pacific Education Review 2004, 5(1), 64-75. Lee, W. O. (1996). The cultural context for Chinese learners: Conceptions of learning in the Confucian tradition. In D.A. Watkins and J.B. Biggs, J.B. (Eds), The Chinese Learner: Cultural, psychological and contextual influences, (pp. 25-41). Hong Kong: Comparative Education Research Centre and Victoria, Australia: The Australian Council for the Educational Research. Madaus, G., and Clarke, M. (2001). The impact of high-stakes testing on minority students. In M. Kornhaber and G. Orfield (Eds.), Raising Standards or Raising Barriers: Inequality and High Stakes Testing in Public Education (pp. 85-106). New York: Century Foundation. Malaysia, Government of (1997) The Malaysian smart school: An MSC flagship application, a conceptual blueprint. Kuala Lumpur: Government of Malaysia, Smart School Project Team. Malaysia, Government of (2006). The Ninth Malaysia Plan (2006-2010). The Economic Planning Unit, Prime Minister's Department, Putrajaya. 2006. p. 255, Ch.11 on Enhancing Human Capital. Kuala Lumpur, Malaysia: Government Printing Press. Ministry of Education. (2001). Pembangunan Pendidikan 2001-2010: Perancangan Bersepadu Penjana Kecemerlangan Pendidikan. (Educational Development 2001-2010: Integrated Planning towards Excellence in Education). Kuala Lumpur, Malaysia: Ministry of Education. Ministry of Education. (2004). The Development of Education: National Report of Malaysia. Kuala Lumpur, Malaysia: Ministry of Education. Ministry of Education. (2006). Pelan Induk Pembangunan Pendidikan 2006-2010 (Blueprint for Education Development 2006-2010). Kuala Lumpur, Malaysia: Ministry of Education.
School-Based Assessments (SBA)
109
MDC, Multimedia Development Corporation. (2004). Multimedia Super Corridor: Impact Survey 2004. Multimedia Development Corporation Bhd, MSC Headquarters, Malaysia: www.msc.com.my. MSC Malaysia. (2008). MSC Malaysia Status / MSC Malaysia Status Check. Retrieved July 15, 2007, from http://www.mscmalaysia.my/topic/MSC+Malaysia+Status+Companies. National Research Council. (2002). Technology and assessment: Thinking aheadProceedings of a workshop. Washington, DC: National Academy Press. Rohlen, T. P. (1980). The juku phenomenon: an exploratory essay. Journal of Japanese Studies, 6(2), 207-242. Simrit Kaur. (2007, May 9). Assessment to replace UPSR and PMR. The Star. Sloane, Finbarr C. and Kelly, Anthony E. (2003). Issues in high-stakes testing programs. Theory into Practice, Winter, 2003. Sorensen, C. W. (1994). Success and education in South Korea. Comparative Education Review, 38(1), 10-35. Zeng, K. M. (1999). Dragon gate: Competitive examinations and their consequences. London: Cassell.
In: Educational Evaluation in East Asia Editor: Samuel S. Peng and John Ch-Kin Lee
ISBN 978-1-60692-887-5 © 2009 Nova Science Publishers, Inc.
Chapter 8
CURRENT ISSUES REGARDING THE SCHOOL EVALUATION SYSTEM IN JAPAN Yasuhiko Washiyama Tokyo Gakugei University, Japan
1. HISTORICAL DEVELOPMENT OF THE SCHOOL EVALUATION SYSTEM IN JAPAN Evaluation of schools in Japan was conducted according to the “Local Inspector System” until the end of World War II, and under the supervision of local “Boards of Education” thereafter. Rather than actual evaluation, however, these systems were more in the nature of compliance-oriented supervision and guidance provided to each school on an individual basis, and with no effects outside the school in question. School “evaluation” in the true sense of the term has only taken off in Japan in the last 10 years or so. The pursuit of deregulation and decentralization has occasioned a review of the regulations on school administration, which prescribe the authority held over schools by Boards of Education. This review has resulted in an expansion of schools’ discretionary powers, making the concept of “autonomous school administration” a reality. The introduction of the school councilor system in 2000 brought school administration and development beyond the limited control of the principal and teaching staff and enabled active participation by students’ guardians—here defined as their parents or primary caretakers—as well as the local community. In the context of these trends, the need emerged for a system to evaluate each school’s autonomous activities and to effect improvements in educational quality. As a first step towards promoting evaluation within schools, the “Regulative Standard for School Establishment” applied by the government to new kindergartens, elementary, junior and senior high schools was revised in 2002 to include new provisions on “self-inspection and self-assessment.” Next, a clause to the effect that all schools operating within the established framework of compulsory education, hereafter called “compulsory education schools,” should be subjected to external evaluation was included in the 2005 “Basic Policies for Economic and Fiscal Management and Structural Reform.” In that same year, the Central
112
Yasuhiko Washiyama
Council for Education released its report on “Redesigning Compulsory Education for a New Era,” which proposed the formulation of guidelines in order to enhance evaluation systems for schools. The “Evaluation Guidelines for Compulsory Education Schools” were released in 2006, marking a new beginning for school evaluation in Japan.
2. EVALUATION GUIDELINES FOR COMPULSORY EDUCATION SCHOOLS I would like to introduce three concepts which provide the framework for the Evaluation Guidelines for Compulsory Education Schools: Goals, Methods and Indicators. The following four points are listed under “Goals”: (1) To establish a Plan-Do-Check-Act management cycle, realizing systematic and continuous improvement in each school through setting goals, monitoring progress and reviewing initiatives; (2) To make the results of both self-evaluation and external evaluation public, creating an open school atmosphere founded on the understanding and involvement of guardians and the wider community; (3) To clarify responsibilities for both education and management, enabling schools and their teachers to discharge their duties to the full and the school’s founding body to provide support and establish the conditions for operation of schools in accordance with evaluation results, thereby assuring and enhancing educational quality; (4) To apply evaluation of lessons and of teaching staff to the development of scholastic achievement guarantees, clear academic goals and guidelines, and school administration systems that are directly connected to overall school evaluation processes. “Methods” includes the following three points: (1) Self-evaluation (goal-setting, collecting and consolidating data, implementing evaluations, producing evaluation reports, identifying points for improvement); (2) External evaluation (establishing an external evaluation committee, implementing evaluations, producing evaluation reports); (3) General information for guardians and community members, and general measures for improving the school, provided by the school’s founding body. Ten points are listed as “Indicators”: (1) Curriculum and academic guidelines; (2) student behavioral guidance; (3) student career and further education guidance; (4) safety management; (5) health management; (6) special needs education; (7) organizational operations; (8) teacher training; (9) collaboration with guardians and members of the local community; (10) facilities.
Current Issues Regarding the School Evaluation System in Japan
113
3. SCHOOL EVALUATION AS A MEANS OF STRUCTURAL REFORM OF THE EXISTING COMPULSORY EDUCATION SYSTEM Until recently, evaluation of compulsory education providers was practiced from a “compliance” perspective. Inspections were carried out to ensure that the curriculum followed the Education-Ministry-approved course of study, that the required number of class hours was being offered, and that the school was being operated correctly in accordance with legal stipulations. Educational quality was maintained and enhanced through the application of uniform national standards to school “inputs and outputs.” The recent round of reforms has been characterized by the shift to a quality assurance approach, focusing on evaluation of educational outcomes. The initial “input stage” involves laying the foundations for school operations: addressing legal requirements, setting goals, ensuring fiscal stability, and so on. In the “process stage” each school must develop autonomous initiatives, assuming greater authority over school activities, setting specific objectives and finding ways to achieve them in collaboration with the community. The “outcomes stage” entails continuous and progressive inspection and evaluation by reference to educational outcomes, addressing matters such as learning effectiveness and the development of teachers’ professional capabilities. This process requires the formulation of a new evaluation system within each individual school. This is to be a three-tiered system comprising the school’s “self-evaluation” of its own activities and operations by reference to its own educational objectives, “external evaluation” by students’ guardians, local residents and other stakeholders, and “third party evaluation” conducted objectively and scientifically by outside groups unfettered by any association with the school. The system, then, is founded first and foremost on the “self-evaluation” process. The combination of quantitative evaluation, based on numerical targets and performance indicators, with qualitative evaluation that focuses on activity indicators reflects the importance of properly evaluating educational processes, each child’s degree of development, stakeholder satisfaction, and the nature of linkages with the wider community. To date, self-evaluation has been implemented in over 95% of all schools in Japan’s compulsory education system, while around 80% have undergone external evaluation. It is believed that an organization will be established to undertake third-party evaluation following analysis of the outcomes of trial programs conducted in the 2006 and 2007 school years. The challenge is how to establish school evaluation as part of the Plan-Do-Check-Act management cycle, ensuring that the system of setting goals, monitoring progress and reviewing initiatives will generate systematic and ongoing improvement. It is vital that the items for evaluation, the evaluators themselves, and the results of evaluation all be structured appropriately. This structuring would clarify what needs to be achieved, by whom, and when, facilitating strategic improvement in and of schools through across-the-board collaboration.
4. CONCERNS ABOUT THE SCHOOL EVALUATION SYSTEM While aiming for improvement of educational activity across the board, the central concern of school evaluation is obviously that of guaranteeing sound scholastic ability on the
114
Yasuhiko Washiyama
part of students. Scholastic ability is thus employed as a principal indicator in the evaluation process, which highlights the issue of degrees of scholastic achievement. In April of 2007, a national test of scholastic ability was conducted in Japan for the first time in 43 years. Students in the sixth year of elementary school and the third year of junior high school were tested in Japanese and arithmetic/mathematics, with almost all national and other public school students participating along with 60% of those in private schools. This test is an important tool for comprehending the true current state of scholastic achievement in Japan and applying this knowledge in guidance and instruction activities. However, there is also the potential for this test to be used in ways which are highly problematic. One such case has already emerged in Adachi Ward, part of the Metropolis of Tokyo, which has been conducting its own scholastic ability test since before the national test was instituted. It was revealed that teachers proctoring the test at schools in Adachi Ward had been communicating answers to students taking the test by means of signals or cues. The worst-performing area in Tokyo, Adachi Ward authorities had been under pressure to improve test results. They sought to achieve this goal by making all the schools in their jurisdiction compete in the test, publishing a ranking of schools based on students’ test results, reflecting these results in budget allocations to schools, and encouraging families to refer to the results when choosing schools to attend. The improper conduct during the test on the part of both teachers and students was a direct outcome of this competition-based approach. As this example shows, scholastic ability tests run the risk of being used to exacerbate competition, generate hierarchies and create disparities between and among schools – outcomes not envisaged in the original concept of school evaluation. This is an important problem and one which calls for careful consideration; school administrators must be able to approach the publication and use of test results in a manner which accords with the spirit of honest and objective school evaluation. If asked to identify three more general problems which may arise from the new school evaluation system, many would name the following: (1) greater disparities between and among different schools and teachers and more cases of loss of motivation and mental illness in the teaching profession; (2) encroachment by the evaluation workload on the time teachers usually spend with their students and on studying and preparing teaching materials; (3) the treatment of evaluation as a routine practice, carried out as a matter of form, with content being buried in the demands of paperwork. Measures must be instituted to forestall these problems. Evaluation is often thought of as the responsibility of principals, head teachers and other school administrators, and not the concern of either mid-career or recently-qualified teachers. However, it is important that evaluation be pursued as a school-wide initiative. Teachers must look beyond the confines of their own classroom, adopting a broad and multi-layered perspective which incorporates the overall organizational activities of the school and shares ideals regarding the school’s direction. Teachers must work together with guardians and members of the wider community in activities aimed at enhancing not just students’ scholastic ability but also their study habits and lifestyles. The crucial point here is to use the evaluation process as a means to develop mutual trust and confid