Getting Schools Working: Research and Systemic School Reform in South Africa

Getting schools working RESEARCH AND SYSTEMIC SCHOOL REFORM IN SOUTH AFRICA

G NICK TAYLOR, JOHAN MULLER & PENNY VINJEVOLD

Pearson Education South Africa Forest Drive, Pinelands, Cape Town www.pearsoned.co.za Copyright © Maskew Miller Longman (Pty) Ltd 2003 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the copyright holder. First published in 2003 ISBN 1 86891 242 6 Published by Delina Venter Edited by Gwen Hewett Index by Ethné Clarke Cover design by Visual Partners Typesetting by Stacey Gibson

Contents Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v About the authors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi List of abbreviations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Part I Systemic school reform and research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 1 Accountability and support in systemic school reform . . . . . . . . . . . . . . . . . . . . . . . 3 Chapter 2 Researching school reform in South Africa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Chapter 3 Lessons from learner assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Part II Explaining the outcomes of schooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Chapter 4 Factors which influence learner performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Chapter 5 A social theory of schooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Part III Implications of the model for classrooms, schools and districts . . . . . . . . . . . 85 Chapter 6 Curriculum delivery in the classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Chapter 7 Organising schools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Chapter 8 Why districts matter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

iii

Preface This book has its origins in the research commissioned by the President’s Education Initiative, published as Getting Learning Right (Taylor and Vinjevold, 1999). Taken collectively, that research enabled a broader scrutiny of teaching and learning in South African schools not hitherto available to the education community. However, valuable as that starting point was, the investigation itself displayed a number of conceptual and methodological gaps and shortcomings. One feature immediately apparent was that the studies remained independent and self-standing, and were not amenable to aggregation or generalisation. Getting Schools Working provides a sustained reflection and meta-analysis of generalisable knowledge of schooling in South Africa, through a survey of the stream of research between 1998 and 2002, and the construction of a theoretical model of the factors which influence learning. The aim of the book is to contribute to the establishment of a consolidated knowledge base that might be of use for researchers, policy makers and practitioners. The work occurred under the auspices of the Research on Schooling initiative, managed by JET Education Services. While the authors take responsibility for the descriptions and views expressed in the text, the finalisation of this volume would not have been possible without the support of the following: • The Board of JET Education Services, who financed the Research on Schooling initiative. • JET staff members who made substantive contributions to the research and background information gathering. In particular, Jennifer Roberts wrote sections of Chapters 7 and 8, Thabiso Nyabanyaba contributed to Chapter 6, Kholofelo Sedibe assisted with Chapter 7, Maureen Mosselson chased down innumerable references, and Ruth Mogoro helped in the collection and management of information and other resources. • Participants in the JET Research Training Course provided formative critical comments: Tim Dunne, Paula Ensor, Jaamiah Galant, Anthony Gewer, Joanne Hardman, Thomas Kgame, Philip Kutame, Lesley le Grange, Johann Mouton, Lebs Mphahlele, Thabiso Nyabanyaba, Peter Present, Cheryl Reeves, Jennifer Roberts, Yael Shalem and Lauren Wildschutt. • Luis Crouch for untiring support, technical advice and critical comment. • Helen Perry, Lebs Mphahlele, Brahm Fleisch and Charles Simkins undertook research specifically commissioned for the project, as acknowledged in the text.

v

About the authors Nick Taylor Nick Taylor taught maths and science for 10 years, and served as subject advisor in Soweto from 1988 to 1984. He was a policy researcher in the field of schooling at Wits University Education Policy Unit, from which he ran the National Education Policy Initiative in behalf of the NECC. He is the co-editor of Getting Learning Right, published by JET in 1999. He is currently CEO of JET Education Services. Johan Muller Johan Muller was the first director of the Education Policy Unit at the University of the Witwatersrand. He has taught curriculum theory and policy at the universities of Witwatersrand and Cape Town and served on the Review Committee for Curriculum 2005. His book Reclaiming Knowledge was published by RoutledgeFalmer in 2000. He currently holds the Chair of Curriculum at the University of Cape Town. Penny Vinjevold Penny Vinjevold has an M.Ed from the University of the Witwatersrand. She worked as a teacher and teacher educator for sixteen years in Johannesburg, Mafeking and Soweto. Thereafter, she was appointed General Manager of Evaluations at the Joint Education Trust. She managed the research component of the President’s Education Initiative and is co-editor of Getting Learning Right. Ms Vinjevold has experience in both quantitative and qualitative research methodology, especially the development, administration and analysis of learner performance assessment instruments. She was appointed a member of the Review Committee for Curriculum 2005 and of the Ministerial Project Committee that oversaw the development of the National Curriculum Statement for Grades R–9. She is currently Chief Director of Planning in the Western Cape Education Department.

vi

Abbreviations AC AIDS

assessment criteria acquired immuno-deficiency disease syndrome C2005 Curriculum 2005 CDE Centre for Development and Enterprise CEPD Centre for Education Policy Development CIE Catholic Institute of Education CSDI Cadbury School Development Initiative DAS development appraisal system DDSP District Development and Support Project DET Department of Education and Training DFID Department for International Development DoE Department of Education E English EAZ Education Action Zone EC Eastern Cape ECD early childhood development ELP expected levels of performance EMIS Education and Management Information System EQUIP Education Quality Improvement Project ESA Eric Schollar Associates FRD Foundation for Research and Development FS Free State Province GDE Gauteng Department of Education GET General Education and Training GETINSET General Education and Training in-service education and training GNP gross national product GP Gauteng Province GTZ Deutsche Gesellschaft für Technische Zusammenarbeit HG Higher Grade HIV Human immuno-virus HLM hierarchical linear modelling

HoA HOD HoD HoR HR HSRC IEA INSET JET KZN LOI LP LSM M MEC MLA MoE MP NC NCS NCTM NGO NLNS NP NW OBE OD OFSTED PCK PEI QLP READ S SADTU SAILI SAQA

House of Assembly Head of Department House of Delegates House of Representatives human resource Human Sciences Research Council International Association for Educational Achievement in-service education and training JET Education Services KwaZulu-Natal language of instruction Limpopo Province (formerly Northern Province) Learning support materials mathematics Member of Executive Council Monitoring Learner Assessment Ministry of Education Mpumalanga Province Northern Cape Province National Curriculum Statements National Council of Teachers of Mathematics non-governmental organisation National Literacy and Numeracy Strategy Northern Province (now called Limpopo Province) North-West Province outcomes-based education organisational development Office for Standards in Education pedagogical content knowledge President’s Education Initiative Quality Learning Project The READ Trust science South African Democratic Teachers Union South African Institute for Leadership in Industry South African Qualifications Authority

vii

Abbreviations SDU SEI SES SG SGB SMT SO SRN STAR TIMSS TIMSS-R

viii

Schools Development Unit at the University of Cape Town School Effectiveness Initiative socio-economic status Standard Grade school governing body school management team specific outcomes School Register of Needs Supplementary Test of Achievement in Reading Third International Mathematics and Science Study Third International Mathematics and Science Study Repeat

UCT UNESCO

UNICEF UPE US AID WC WCED wrt WSE

University of Cape Town United Nations Educational, Scientific and Cultural Organisation United Nations Children’s Fund University of Port Elizabeth United States Agency for International Development Western Cape Western Cape Education Department with regard to Whole School Evaluation

part

I Systemic school reform and research This book is about systemic school reform and the research base that informs it. It aims to provide a commentary on systemic reform efforts since 1995, and then goes on to discuss in greater detail the kinds of research, both empirical and theoretical, that has and that could take us beyond common sense, suggestive trend data, and rich depictions of the schooling terrain that light up particular features but leave systemic implications uncertain. This is a rather trickier endeavour than the bald statement of intent above conveys, and much of the book deals with the travails of less than perfect designs, less than stable data and less than comprehensive conceptual frameworks. This is the day-to-day reality of the craft of educational research for policy, which this book attempts to capture and take forward. Part I introduces the status of both systemic reform and research since 1995. The main purpose is to take stock of • the lessons that have emerged concerning the state of public schooling and attempts to reform it; and • the state of research in schooling and the lessons that can be drawn for future research programmes. Chapter 1 describes current efforts at systemic reform in South Africa. The chapter begins by describing the considerable financial and other resources directed both

by government and the private sector at improving the schooling system. These efforts are broadly categorised into accountability and support measures. The chapter describes in some detail current government accountability and support measures. Although it is clear that school reform programmes are moving away from only support measures and now generally incorporate both accountability and support measures, accountability ‘bite’ remains weak, and research impact on policy remains tangential. This does not mean that school-based research has not been conducted in the period 1995–2002. On the contrary, there has been a burgeoning of such research. The resulting increased information base has come from four main sources: government, international comparative studies, university-directed studies and donorcommissioned evaluations. Chapter 2 describes a sample of this research. Because this volume is concerned with systemic reform, the chapter restricts itself to studies that included both accountability measures (learner assessment) and support measures (input or process indicators). The chapter describes the purpose and design of these studies, particularly the sampling processes. It also describes the successes and challenges facing researchers in gathering and analysing contextual or home background data; input factors (human and

1

part I Systemic school reform and research

physical resources) and educational process factors, that is, the transactions in schools and classrooms that lead to learning. Chapter 3 describes the learner assessment studies conducted in South Africa since 1997. The assessment of learner achievement has been a controversial development in South African research. While much work has to be done to improve test construction and administration to strengthen the validity and reliability of the tests, the chapter sets out to show that testing has become an important measure for holding teachers, schools and districts

2

to account and for providing relevant and appropriate support for systemic reform, a trend that is on the increase in this country as it is in schooling systems worldwide. All the results so far at our disposal point to disturbingly low levels of reading and numeracy among South African learners. It is likely that these levels of attainment will increasingly become a matter of concern for communities, governments and educators alike, and that they will become the litmus test against which evolving models of school reform will be judged.

chapter

Accountability and support in systemic school reform Introduction By any developing country’s standards, South Africa provides access to schooling for a very high proportion of its young people. Participation rates at the primary level are close to 100%, and at the secondary level they are high and increasing (Simkins, 2002b). Since 1994 the government has made impressive gains in terms of what Fiske and Ladd (2002) call distributional equity – that is, equalising spending across the system. This has involved significantly increasing spending in the poor sectors of the system, which serve the black majority, and reducing the allocation of public resources to the more privileged sectors (Taylor, 2001; Van der Berg, 2001b). In addition, through the application of the Norms and Standards for Funding Schools (DoE, 1998b), which differentially allocates the non-salary component of school funding towards poorer schools, government is making a serious effort to redress inequities in the system arising from past discrimination.1 However, increased financial allocations are clearly not sufficient to overcome the inequitable provisioning of the apartheid past. Indeed, increased spending on its own is likely to have very little effect on the poorest parts of the system, given some provinces’ inability to spend current allocations, a point we discuss in more detail below. Much more must be done about improving the quality of the learning experiences in schools and classrooms if we are to offer real educational opportunity to young citizens. To this end dozens of school development programmes have been established over the last decade, and more are commissioned every year. Up to the end of the 1990s these initiatives came largely from the non-government sector, although government had some level of involvement in almost all of them. It is estimated that something in the order

1

of 10% of the nation’s nearly 30 000 schools are involved in donor- and NGO-initiated development projects of one or other kind, with a total off-budget expenditure of up to R1 billion annually.2 This includes fiveyear commitments of R120 million by US AID, R300 million by the Business Trust, a new five-year allocation of R350 million by the British Department for International Development (DFID), following the completion of the R90 million Imbewu programme; a pending award of some R300 million by DFID to Limpopo Province; smaller but still very significant contributions by JET Education Services (JET), the Tshikululu programme, the National Business Initiative, the Royal Netherlands Embassy and the Danish International Development Agency; and dozens of smaller projects supported by a host of local and offshore donors. Since 2000 the government sector, both national and provincial, has begun to initiate its own programmes of targeted reform, such as the Dinaledi project, the National Strategy for Maths, Science and Technology, and the Gauteng Department of Education’s Strategy for Mathematics, Science and Technology Education. These government-initiated projects are over and above the new state programmes aimed at improving quality, such as the Whole School Evaluation and the Systemic Evaluation initiatives (see Mseleku, 2002). In addition to the above programme activity there has been considerable school- and classroom-based research activity. While the information base on schooling in South Africa was insubstantial in the early 1990s, the research tempo has begun to pick up since the late 1990s, and much data has been accumulated from government, the universities and NGOs. In order to collate, extend and make sense of this work, JET Education

3

part I Systemic school reform and research

Services established the Research on School Development programme in 2000. In particular the research programme aims to describe the scale of school development activity in the country, to understand how the different initiatives are structured and to try to establish what the success factors might be through carefully designed research projects. The overall objective of the research is to promote a more informed debate among the actors – government, development agencies, donors and researchers – on how best to proceed with school development. The present volume is part of this initiative.

Approaches to school reform Inside–out and outside–in The JET Research on School Development programme began with an analysis of the literature on school reform (Muller and Roberts, 2000). Muller and Roberts’ analysis describes a decades-long standoff between defenders of the ‘outside–in’ (standards-based, school effectiveness) approach and proponents of the ‘inside–out’ (school-focused or school improvement) approach. Outside–in initiatives generally start with a set of standards, since their point of departure is that this is the locomotive that pulls the learning train. In contrast to earlier efforts which focused on overly general standards that proved too vague to provide firm guidelines to teachers, the outside–in reformers have come to realise that standards should be clear, parsimonious and rigorous. Further, these should be accompanied by targets for achieving the standards, and incentives for reaching the targets. Assessment of student performance provides the hard data which enables outside–in initiatives to ‘steer by results’, and gives all actors in the system the summative outcomes of their combined efforts. The problem with this approach on its own, as several critics have pointed out (see, for example, Elmore and Burney, 1999), is that school managers and teachers are often expected to perform at levels for which they are not equipped. In other words, a different level of systematic support is required to help

4

educators meet the new accountability requirements. By contrast, inside–out reformers have tended to focus primarily on issues of the organisational culture of schools: shared values, vision and teamwork. Later inside–out programmes have shifted to an explicit focus on improving classroom instruction as a prerequisite for improving learner achievement, and to a growing awareness that school improvement should be tailored to the specific state of development of the school. Thus, severely dysfunctional schools (Type I schools, in the terms of Hopkins and MacGilchrist, 1998) require organisational stabilisation, the establishment of basic management systems, and governance and management training, in order to set the conditions conducive to effective teaching and learning. Only once a threshold level of institutional functionality has been achieved (Type II schools) can interventions at the classroom level be effective. The tension between outside–in and inside–out approaches to school reform is parallel to the distinction made by Bernstein, whose work we discuss in some detail in Chapter 5, between performance and competence models of pedagogic practice. Bernstein (1996) lists the following as the chief characteristics of competence pedagogy: • It assumes a universal democracy of acquisition: all learners are assumed to be inherently competent and there are no deficits, only differences. • The learner is assumed to be naturally active and creative in the construction of a valid world of meanings and practice. • The learner is assumed to be self-regulating and her development is not advanced by formal instruction. • Any hierarchical conception of teaching is treated with suspicion: teaching should not go beyond facilitation, accommodation and context management. Performance models, on the other hand, emphasise particular outputs that the acquirer is expected to produce, and specialised skills

chapter 1 Accountability and support in systemic school reform

necessary for the production of these outputs. The teaching resources required by competence models are less likely to be explicated through standard textbooks or teaching routines, and more likely to be constructed by the teacher in response to a specific context, while performance pedagogies are more likely to depend on standard texts. And whereas the recognition and realisation rules for achievement are implicit in competence models, they are more explicit in performance approaches. Competence models, according to Bernstein, require a relatively high degree of autonomy on the part of teachers and institutions in order to respond to the needs of particular pupils. Performance models, in contrast, demand that acquirers’ performance is subordinate to external regulation, and hence that teachers and schools are relatively less autonomous. In the latter case, accountability is facilitated by the ‘objectivity’ of the performance – in the sense that the rules of recognition and realisation are clearly explicated and are thus more amenable to measurement and optimisation. Muller (2000, 103) notes that competence models, on the contrary, are “seriously opposed to graded assessment” on the grounds that such assessment may neither give a fair reflection of pupils’ competence, nor predict future performance accurately. On the issue of costs, Bernstein (1996) notes that the transmission costs are likely to be higher in the case of competence models, both in terms of the direct costs of the intensive training of teachers required by the sophistication of such approaches, and in terms of the hidden costs in teachers’ time spent in providing individual attention to pupils, socialising parents, and constructing context-specific resources. The position we advance in this book is that a particular kind of rapprochement between competence and performance, or between the ‘inside–outs’ and the ‘outside–ins’, is both feasible and desirable. While Bernstein draws a clear analytical distinction between competence and perform-

ance models, he emphasises the importance of understanding that, in practice, the two may not necessarily be mutually exclusive, but present the option of combining specific elements of each in what he refers to as a pedagogic pallet (ibid., 70). Empirical research has begun to demonstrate which features of the pedagogic pallet are most effective with particular kinds of learners, as the discussion in this volume hopes to show. Agreement on student achievement as the ultimate measure of the health of both individual institutions and the school system as a whole is now, if not quite a shared article of faith, at least a point of agreement between the ‘inside–outs’ and the ‘outside–ins’. There is also now much wider appreciation of the fact that a significant component of learner performance is a reflection of the home environment, and that it is the value which a school adds to student entry level performance (the ‘residual variation’) which measures the worth of the school. Finally, there is a growing realisation that a systematically constructed combination of accountability and support measures is required to break the very poor record, internationally and in South Africa, of success in improving poorly functioning sections of the school system.

Systemic reform: accountability and support measures Systemic reform programmes may be seen as a combination of outside–in and inside–out approaches. Whereas outside–in programmes employ mainly accountability measures, and the inside–out initiatives focus mainly on support activities, systemic reform is premised on the need to align and mediate accountability and support. The rapprochement that is occurring between the two broad models of school reform is leading to a convergence around the importance of linking classroom instruction to an external accountability system. There is general agreement that, without an explicit focus on schools and classrooms, improved learning is very difficult if not impossible to achieve. And without attention to building capacity in higher levels

5

part I Systemic school reform and research

of the system, change cannot be directed and monitored effectively, nor is it likely to be sustained beyond the life of the project, or to be replicated in non-project schools. Large-scale systemic reform programmes have been gaining ground in the last decade in the USA (Elmore et al., 1996), and in 1997, in what may prove to be the largest and most successful such initiative, the National Literacy and Numeracy Strategy (NLNS) was launched in England (Fullan, 2001). In announcing the programme, the Secretary of State for Education and Employment set targets for the improvement of the national average for literacy scores for 11-year-olds from 57% to 80% by 2002, and an increase in numeracy scores from 54% to 75%. He promised to resign if these goals were not met. Although the Secretary was promoted to Home Secretary in 2001, it would seem that his job would have been safe had he stayed on in education. By 2000 literacy scores had risen to 75% and numeracy scores to 72%. Michael Fullan (op. cit.), the evaluator of the programme, describes these results as “astounding”, given that 20 000 schools and 7 million children are involved. He has no doubt that the 2002 targets will be met.3 Fullan ascribes this success to a number of features of the programme, including: • a national plan, setting out targets, actions, responsibilities and deadlines • a substantial investment, sustained over at least six years and skewed towards those schools most in need • an expectation that every class will have a daily maths lesson and a daily literacy hour • both initial teacher training and ongoing professional development of administrators, principals and teachers, designed to enable every primary school teacher to understand and be able to use best practice in teaching literacy and numeracy • a major investment in books (over 23 million new books since 1997) • regular monitoring and extensive evaluation by OFSTED

6

Clearly the programme has been designed to line up and integrate accountability and support measures so that they operate in tandem, respectively pulling and pushing the schooling system to higher levels of performance. Fullan’s diagnosis is that almost all the gains can be attributed to an increase in teacher motivation. He is also convinced that the improvements in learning performance are valid, in other words, that the results are not just a trick of measurement. He is sure that children actually are reading, writing and doing maths significantly better than they were before, although he has some reservations as to whether the programme may be confining its target to too narrow a band of knowledge and skills. However, he does raise a question as to how lasting the gains might be. In emphasising the key role of government in large-scale reform, Fullan identifies three elements. Government should: • demand accountability of schools and teachers and set targets, • provide incentives to perform better, and • build capacity to manage and teach more effectively. While the National Literacy and Numeracy Strategy has been very successful at implementing the first two elements, Fullan contends that deep-rooted capacity is not being built. This may seem to be quibbling in the face of a massive achievement, especially in view of the fact that the programme has focused on the professional development of teachers. But what Fullan means by his criticism is that schools are not undergoing the fundamental transformation required to turn them into learning organisations: only when this happens will the achievements of the programme be truly sustainable.

School reform in South Africa Inside–out Up to 1995, INSET for teachers or principals was the predominant form of activity aimed at school quality improvement in South Africa. In that year the Teacher Education

chapter 1 Accountability and support in systemic school reform

Audit estimated that there were over 100 programmes of this type in operation in the NGO sector alone (CEPD, 1995). These INSET programmes were mounted exclusively by NGOs and universities which had stood outside the apartheid fold. These were classic inside–out approaches, as were the Whole School Development (school-by-school) programmes which began to emerge in the midto late 90s. The main assumptions of these approaches had much in common with Bernstein’s competence model outlined above, and generally incorporated most of the following: 1. The child is the centre of all learning. As much individual attention as possible should be focused on each child in order to direct learning towards her specific needs, interests and capacities. 2. The unit of focus of the programmes is the individual teacher (rather than, for example, the school – Whole School Development approaches excepted here). Thus, while the teacher becomes more of a facilitator of learning than the all-knowing fount of knowledge which the child uncritically ‘consumes’, change at the school and classroom levels depends essentially on the motivation and initiative of the individual teacher. 3. Consequently, such programmes should place a heavy, often exclusive, emphasis on teaching method (rather than on subject knowledge), and in particular, on child-centred methods. 4. Customised worksheets, which assist teachers to promote pupil activities in the classroom, should constitute a central component of the programmes. 5. School knowledge must start with and be directed towards the ‘real world’: ‘relevance’ should be a key criterion in determining the school curriculum. (Chapter 5 elaborates some of these points.) The exclusive focus on persons rather than systems by inside–out approaches is simultaneously the greatest strength and fundamental flaw of this perspective. On the positive side,

there is no doubt that the development of inquisitive, active young citizens, able to interact critically and constructively with the world around them, is key to the growth of a vibrant society able to hold its own in a competitive world, and that highly motivated and skilled teachers, able and willing to exercise initiative and responsibility, are prerequisite to the evolution of our schools into institutions which nurture such citizens. On the negative side, the inability of inside–out approaches to see beyond the individual to the institution remains a very significant inhibition to school reform. Institutions are far more than the sum of the individuals that inhabit them. Well-functioning schools also require efficient and effective operating procedures and systems which regulate and integrate the efforts of teachers, managers, parents, pupils, and the many suppliers of goods and services, situated both in higher levels of the bureaucracy and in the private sector. Without these mechanisms of institutional functionality, principals and teachers remain unaccountable to the noble goals of inside–out approaches, and the positive energy generated by visioncrafting exercises and participative mission statements is not necessarily translated into better knowledge and skills for pupils.

Outside–in Outside–in reform initiatives have been rarer in South Africa, but one significant example has been the Education Action Zone (EAZ) programme adopted by the Gauteng Department of Education (GDE) in 2000. This was designed as a comprehensive systemic initiative which would include monitoring schools – through monitoring attendance and punctuality of teachers and pupils, providing ‘pace setters’ to encourage coverage of the curriculum, and setting common examinations – and providing support and training to principals, teachers and pupils. Although it has been argued that the EAZ combined accountability and support measures (RivettCarnac, 2002), we agree with Fleisch that the programme focused largely on the former (Fleisch, 2001; 2002) and thus did not fully

7

part I Systemic school reform and research

meet its systemic intentions. Furthermore, a project approach was adopted in administering the programme, rather than strengthening the systems and capacity for school monitoring and support in the standard line functions of the GDE. Thus, the EAZ was managed from the provincial head office, with special units responsible for earmarked schools and reporting directly to the provincial Minister of Education (MEC) and the Head of Department. We will argue below that this was an unsustainable strategy which, whatever its initial gains, would reach a ceiling fairly rapidly. Nevertheless, the EAZ did inject a renewed respect for the legitimate authority of government and began the process of building a culture of accountability towards pupils, parents and the taxpayer, on the part of schools, principals and teachers. The EAZ was accompanied by an impressive rise in matric results in targeted schools, both in the absolute sense and relative to non-EAZ schools. It would seem likely that this improved performance is a direct result of the programme. The following are among the most noteworthy features of the EAZ project: • Of Gauteng’s high schools which offer Grade 12, 67 schools, or 14% were involved. • The worst performing schools in the province were targeted: in the 1999 matric exams all 67 schools achieved pass rates of 30% or below, with 64 of the schools at 20% or below. • In 2000 only 29 schools remained at 30% or less, with only 13 at 20% or below. • The targeted 5% improved pass rate was achieved by 90% of EAZ schools. • The aggregate pass rate for EAZ schools improved by an average of 14.5%, which exceeds the improvements shown by both other former DET schools in the province (up 10.1%), and all public schools in Gauteng (5.3%). • The number of matric passes in EAZ schools increased from 1677 in 1999 to 2313 in 2000 (up 38%). • The number of distinctions achieved by EAZ schools increased by 422%, from 37 in 1999 to 193 in 2000.

8

• The number of university exemptions increased by 47%, from 107 to 157. • These developments were accompanied by a marked decrease in the number of candidates enrolled for the exam at EAZ schools. While there was a small overall decrease of 1.7% for the province as whole, and a drop of only 3.4% for former DET schools, EAZ schools showed an aggregate decrease of 25.4% (Fleisch, 2001). The fall in enrolments at EAZ schools may be due to parents and pupils voting with their feet and moving to schools with better prospects, or to EAZ schools applying stricter criteria for examination registration, or even to students unable to cope with what must have been an increased scholastic load. It was probably a combination of these factors. Fleisch (op. cit.) speculates that, whatever the origin, the smaller numbers of candidates may have contributed significantly to improved achievement, by providing greater access on the part of students to resources, and changing the climate of Grade 12 classes. These conditions were reinforced by the increase in study time provided by the monitoring of attendance and punctuality. One reservation to this conclusion is provided by a number of EAZ schools (8, or 12%) which improved their results while increasing their roll, or at least holding steady. Overall, the EAZ results are noteworthy because they do at least provide evidence for dramatic improvement over the short term in a field of evaluation studies that all too often fails to show any measurable improvement.

Systemic reform in South Africa Although it is probably safe to say that most school development programmes currently in operation in South Africa lean towards the inside–out approach, a new generation of initiatives has begun to emerge within the last decade, which attempt to incorporate both accountability and support measures. For example, the Quality Learning Project (QLP), the District Development and Support Project (DDSP), the Cadbury Schools Development

chapter 1 Accountability and support in systemic school reform

Initiative (CSDI), the 1000 Schools READ Project, the Mahlahle Project and the SAILI Primary School Project among others have introduced some elements of accountability into their operation. All these projects, for example, include learning assessment at various stages of their implementation as a means of assessing the performance of learners against South African curriculum guidelines or standards, and for assessing the impact of the interventions. However, with the exception of the QLP and READ, there are no clear targets set for achieving these standards nor are there incentives for reaching them (Roberts, 2001). The absence of accountability measures in South African school reform programmes can be explained in two ways. First, because measures for systemic alignment, with the exception of the matric exam, do not yet bite down to the school level, schools are almost entirely unaccountable for the outcomes of their learners. Consequently, training programmes and other support measures, because they have no outcome indicators of change, tend to focus on issues such as institutional vision and culture, and not on the technicalities of, for example, procuring and managing textbooks and stationery, or quality assuring the delivery of the curriculum. The participants are free to implement the lessons of this training in their districts, schools or classrooms, or they may decide not to. No one would know the difference because of the absence of monitoring and other accountability sub-systems. It is a premise of our systemic model that the impact of programmes of this nature would be immeasurably increased if they were linked to defined targets of improved learner performance. Managers and teachers would know what is expected of them, and be better motivated to utilise the services of the training to assist in meeting these performance targets. A second reason inhibiting these programmes from moving from inside–out to systemic mode is that the training providers have often developed a theoretical or ideological commitment to the former. For all their evident virtues, it must be asked

whether the short and episodic bursts of workshop-based training offered by NGOs and consortia in these school development projects can build the deep knowledge structures and professional comportment among teachers and managers required to improve the quality of schooling. A related question is whether the training programmes for individuals offered by training institutions can have an impact on the system if they are not linked to institutional development.

Current trends in accountability and support in the South African system Two kinds of measures are available for improving the equity, efficiency and quality of public schooling. Accountability measures give direction, set performance standards and targets, and monitor outcomes; they are used to manage staff and resources; they offer incentives, and administer rewards and sanctions as a consequence of performance. Support measures empower individuals to meet the expectations set by these demand drivers. They build capacity, develop an ethos of institutional coherence and commitment, provide training, establish systems and structures, and distribute resources. Mechanisms designed to hold institutions and individuals accountable include curriculum frameworks, assessment and certification systems, school inspection, performance management reviews and financial auditing. They are administered mainly by the government at different levels – national, provincial, district and school. Researchers and the media also play a crucial role in informing and propagating public debate. Support mechanisms include training programmes and the provision of buildings, utilities, books, stationery and other equipment. Appraisal schemes, such as the proposed development appraisal system (DAS), have the potential to play an important role in identifying individual training and support needs. The principal agents of support measures are government officials at provincial, district and school levels. Training institutions,

9

part I Systemic school reform and research

NGOs, teacher unions and other professional associations are important in designing and delivering training programmes, and enhancing the professional status of teaching.

Accountability measures The new government inherited a system of education in which the authority of the state had been steadily eroded over a period of two decades. Instilling the idea of legitimate authority, as well as setting up accountability systems for the exercise of this authority, has proved to be one of the most daunting endeavours of the last eight years. This is true of every sphere of the public sector. Since the end of the 1990s the 10 Departments of Education have begun systematic efforts to improve accountability, in terms of directing, monitoring and steering the system (MoE, 2001). The most encouraging sign of this is the recent report on the financing of education (DoE, 2003).

Directing The new National Curriculum Statements (NCS) provide a clearly defined framework of what teachers and pupils should be doing and achieving in classrooms at the General Education and Training level (Grades 1 to 9). This makes possible the co-ordination of the efforts of departmental officials, principals, teachers, textbook writers, and assessors in directing, delivering and monitoring teaching

and learning. The NCS constitute the intended curriculum, which sets the goals for learning. We discuss the NCS in greater detail in Chapter 5. A new curriculum for the Further Education and Training level (Grades 10 to 12) will be implemented in 2006.

Monitoring Virtually the only performance monitoring system in place at present is the matriculation exam (the main exceptions will be discussed in Chapter 3). A push on the part of government since 2000 to improve exam scores would appear to be bearing some fruit, with the 2000 results for Gauteng quoted above replicating themselves in most provinces. Taken together these results were impressive: in 2000 there were more matriculants (nearly 33 500, or 13.4%), the quality of these matriculants improved (nearly 5000, or 7.7%, more exemptions) and the number of candidates was reduced by over 4%, hence achieving significant cost savings in the process (see Table 1.1). That said, the drop in enrolments, even more marked in 2001, is a phenomenon that requires investigation (Schindler, 2002). Nevertheless, quantity, quality and efficiency were all improved simultaneously in 2000. There was a slight drop in these indicators in 2001, against the backdrop of a fall of 8% in enrolments, and another significant rise in 2002. It would seem that the inevitable disruptions caused by the thoroughgoing

Year

Candidates

Total passes

%

University exemption

%

Total failures

%

1994

495 408

287 343

58

88 497

18

208 065

42

1995

531 453

283 742

53

78 821

15

247 711

47

1996

518 032

278 958

54

79 768

15

239 074

46

1997

555 267

261 400

47

69 007

12

293 867

53

1998

552 384

272 488

49

69 856

13

279 954

51

1999

511 159

249 831

49

63 725

12

261 328

51

2000

489 941

283 294

58

68 626

14

206 004

42

2001

449 371

277 206

62

67 707

15

172 126

38

2002

471 309

324 752

69

75 048

16

146 557

31

Source: Schindler (2002) except 2002 figures which are unconfirmed at the time of writing.

10

Table 1.1

SENIOR CERTIFICATE EXAMINATION RESULTS, 1994–2002

chapter 1 Accountability and support in systemic school reform

restructuring instituted by the new government in 1994 have begun to stabilise, and that the system is beginning to produce steady gains in performance. Comparing the provincial results for the last two years is instructive. Table 1.2 shows significant variations across the provinces. The obvious problem of publishing the matric results in this sort of table is that the figures give no indication of the value added by each province. The dramatic improvements in the Northern Cape and Limpopo Province require an explanation. It is improbable that such significant gains could happen without some factor demonstrably responsible for them. The more questions we ask the more we realise that we need more data, and to analyse it at ever more detailed levels. Within provinces, for example, it would be instructive to compare categories of schools, by socio-economic status and by performance. Within and between individual schools, it would be instructive to look at the differential effects of school management, teacher characteristics and instructional practices on pupil performance. We pursue this issue in Chapter 4. Can a systemic theory of school reform explain the differential improvements across provinces in the 2000 matric exam, bearing

in mind that, because they are not adjusted for added value, we are not comparing apples with apples? A plausible hypothesis, supported by Michael Fullan’s speculation about the role of teacher motivation being behind the success of Britain’s NLNS, would be that the accountability measures or heightened expectations of schools gave principals and teachers something to aim for, something concrete, measurable and achievable. This hypothesis is certainly given strong support in the literature. Nevertheless, it would be interesting to put it to the test in the South African context. In particular such an explanation would have to demonstrate, for example, that expectations of improved matric results were conveyed with greater urgency in Limpopo Province than they were in Mpumulanga, leading to an increase in performance in the former and a decrease in the latter in 2001. This line of argument is supported by the Limpopo Province’s claim to be the first province to institute common exams in all secondary school grades (The Star, 27 December 2001), thus increasing expectations (as well as improving exam techniques) throughout the high schools. Circumstantial arguments which further support the hypothesis are that other accountability measures in the pipeline have not yet begun to be imple-

PASS RATES (%) 2000

Change on 1999

2001

Change on 2000

2002

Change on 2001

Northern Cape

71.2

+6.9

84.2

+13.0

89.9

+4.7

Western Cape

80.6

+1.8

82.7

+2.1

86.5

+3.8

Gauteng

67.5

+9.5

73.6

+6.1

78.1

+4.5

Limpopo Province

51.4

+13.9

59.5

+8.1

69.5

+10.0

KwaZulu-Natal

57.2

+6.5

62.8

+5.6

70.6

+7.8

Free State

52.7

+10.6

59.0

+6.3

70.7

+11.7

North-West Province

58.3

+6.2

62.5

+4.2

67.8

+5.3

Mpumalanga

53.2

+4.9

46.9

–6.3

55.8

+8.9

Eastern Cape

49.8

+9.6

45.6

–4.2

51.8

+6.2

TOTAL

58.0

+9.2

61.7

+3.7

68.9

+7.2

Province

Table 1.2

SENIOR CERTIFICATE PASS RATES FOR 2000–2002 BY PROVINCE

Source: Schindler (2002) except 2002 figures which are unconfirmed at the time of writing.

11

part I Systemic school reform and research

mented, as we elaborate below, while support measures such as improved textbook distribution, or training programmes – increasing the cognitive resources of the system – would require a longer period of development before they began to have an effect. It might also be surmised that the very poor performance of Mpumalanga and the Eastern Cape in 2001, in sharp contrast to some sparkling results in the Northern Cape, Limpopo Province, KwaZulu-Natal and the Free State, is related to the ongoing leadership crises in those two provinces. While the gross matriculation pass and exemption rates seem to be steadily improving in the majority of provinces, the mathematics results, a key indicator of system quality, are more mixed. Table 1.3 shows that, although the proportion of candidates writing and passing maths has increased since 1996, neither the absolute number nor the proportion passing on the Higher Grade (HG) has improved. The sharp light thrown by the annual matric exam on what learners have learnt by the end of their schooling careers is in strong contrast to the murk which shrouds this question throughout all other grades. The good news in this regard is that the long-awaited Systemic Evaluation system, which aims to sample learner performance across the country, was piloted at Grade 3 level in 2001. Once this is fully in place we will have indicative data for a representative sample of schools

across the country. This data will be invaluable in designing intervention programmes and will immeasurably improve the monitoring of performance by province. What little we do know at present about learning at Grades 3, 6 and 9 is very disturbing, and indicates that the reason there are so few matric passes, when compared with total enrolments in the primary school grades, is because children are not learning what we expect them to learn in each of their grades, and that this effect rapidly accumulates as they fall further and further behind the levels specified by the national curriculum. We detail some of these results in Chapter 3. Very few school development programmes currently operating in South Africa attempt to ascertain in any detail the knowledge needs of the pupils, teachers and managers in participating schools. While the intended curriculum gives a map and shows the destination of schooling, if we do not know where our children are on the map then we do not know what direction to proceed in. Consequently, intervention programmes cannot be designed around the specific needs of their beneficiaries, nor can they assess progress of the programme against learner performance. Part of the problem is that the architects of such programmes often assume that probing knowledge needs does violence to the selfimage of pupils and teachers, and implies a deficit model of school reform. Consequently

1996

1999

2000

2001

2002

No. of candidates

518 225

511 225

489 941

449 371

444 259

No. writing maths

218 090 (42%)

281 303 (55%)

284 017 (58%)

263 945 (59%)

260 916 (59%)

Total maths pass

108 910 (21%)

122 225 (24%)

128 142 (26%)

123 149 (27%)

143 195 (32%)

Pass on SG

86 494 (17%)

102 371 (20%)

103 265 (21%)

103 645 (23%)

120 763 (27%)

Pass on HG

22 416 (4%)

19 854 (4%)

24 877 (5%)

19 504 (4%)

22 432 (5%)

Source: Schindler (2002) except 2002 figures which are unconfirmed at the time of writing.

12

Table 1.3

MATHS SENIOR CERTIFICATE RESULTS, 1996–2002

chapter 1 Accountability and support in systemic school reform

they are self-conscious about including accountability measures in their programmes. Certainly the process and results of research into the problems which give rise to the poor learning situation in so many South African schools need to be handled with sensitivity and according to the highest ethical standards. However, any systemic theory of school reform predicts that omitting the use of achievement data in designing and monitoring these interventions robs them of significant power, and ultimately does far more damage to the lives of the pupils, through lost opportunity, than any amount of testing could ever do. We return to this issue in Chapter 5. Applying monitoring mechanisms requires not only that district officials and school principals support these processes, but indeed that they put their full authority behind driving them. This is the ideal of systemic reform: institutional managers should be instrumental in identifying their own needs, formulating appropriate support measures and monitoring progress. Under these conditions, the support and training agencies assist in achieving the policy and practice priorities of the public sector.

Financial management and absorptive capacity Many of the provinces experienced substantial overruns on educational expenditure in 1997. Since then the Minister of Finance has not only exercised strict control over spending, but has also effected measures to shift the spending ratio of personnel to non-personnel (that is, learning resource) costs so as to free

up more money for capital items. While there is now much better financial control, many provinces are still having difficulties in spending their budgets. For example, the provinces only managed to spend 71% of the amounts budgeted and allocated for school infrastructural improvements in 2001 (MoE, 2002). It is a great irony in a country of such great poverty where hundreds of thousands of children are schooled under very difficult conditions, often without roofs over their heads or in very inadequate buildings, that hundreds of millions of Rand remain unspent every year because of management inefficiencies in the civil service apparatus. It is obvious that until management inefficiencies can be ironed out of the system, allocating further resources to education will have little or no effect in a number of provinces. The point is emphasised by the poor record of spending on conditional grants, funds which are allocated for special purposes. Table 1.4 shows spending against three conditional grant line items: quality improvement and financial management, HIV/AIDS, and early childhood educare programmes.

Organisational development and management The management of schools appears to be slowly improving, as indicated by attendance figures for the first day of the 2002 school year (95% for educators and 90% for learners), and by the proportion of schools which commenced teaching that day (70%) (MoE, 2002, iii). The Minister explains this success as follows: “This can be attributed, in part, to the continued surprise visits by the Ministry,

Expenditure as % of allocation Line item Quality improvement & financial management

2000/1

2001/2

45.00

60.00

Notes

HIV/AIDS educational programmes

39.45

EC: 6.39% LP: 16.00%

Early childhood development (ECD)

15.00

EC: 0.25% FS: 0.90%

Table 1.4

EXPENDITURE ON CONDITIONAL GRANTS, ALL PROVINCES, 2001

Source: MoE (2002).

13

part I Systemic school reform and research

officials from national and provincial departments and community leaders.” Although the Minister would seem to have little else at his disposal, like the monitoring strategies used in the EAZ approach described above, these are very blunt instruments of management. While they may be efficacious in the short term, if they are not replaced by increased capacity within provincial departments, they are soon likely to reach a ceiling and be short-lived in duration. It has been shown in a number of successful system reform initiatives in other countries that combining accountability and support measures is not an additive process, but one requiring the alignment of the two, and one which requires a careful balancing act. On the one hand, as Elmore (2002, 7) points out, accountability measures on their own have a habit of mutating into caricatures of themselves, all too often being reduced to highstakes standardised testing with wide margins of error, and measuring a relatively narrow range of skills. On the other hand, support without accountability, as is the case in South Africa at present, leads to a situation in which the support measures lack direction, focus and coherence, and thus are inefficient in achieving quality gains. As we show in Chapter 5, without a coherent accountability framework, principals and teachers often do not know what is expected of them. Elmore’s contention that staff within a school should share a coherent, explicit set of norms and expectations about what a good school should look like before they can use signals from the outside to improve student learning (ibid., 4) makes a lot of sense, and indeed this is the ideal of systemic reform. The conditions for generating such a set of norms in dysfunctional schools are absent by definition. For Hopkins and MacGilchrist (1998), a prerequisite for making any progress in such situations is to replace the principal before commencing any kind of reform programme, and to initiate the process through outside intervention. Seen from this perspective, the accountability procedures instituted in the EAZ schools might be seen as holding

14

corrupt principals and teachers to the fundamental terms of their employment contracts (such as showing up for school every day), rather than, as Shalem (2002) puts it, bullying a model of reform onto the school. Our problem with the EAZ model is of a different type. While a heavy-handed intervention may have been necessary in the short term, if the initiative is to be sustainable then capacity must be built into the system. First, standard operating procedures must be put in place in order to regulate the flow of goods and services within the system. For example, the building of schools with money that is available will certainly be facilitated if all the officials, at various levels of the system, have a shared set of ideals. However, if there is no clear set of procedures to be followed for planning and building the schools, for integrating the work of a number of teams from different directorates, for ensuring that every cog in this complex process works as it should, and for holding individuals accountable should they be responsible for holding up the process, then no degree of shared ideals will get the schools built. While it is true that the work of senior civil servants is beginning to be regulated through performance contracts, these officials have few tools at their disposal to ensure that their subordinates, in turn, play their respective roles in meeting performance targets, and in identifying and overcoming bottlenecks, be these due to system malfunction or lack of capacity. Without efficient management systems, the only means at the disposal of senior managers are the blunt instruments of threat, exhortation, cajoling, and management by ‘walking around and shouting’. It is even difficult, in the present climate, to apply the measures required in cases of criminality or gross dereliction. As a result, dealing with disciplinary cases can take many months and even years (MoE, 2001). However, any management system designed as a means for demanding accountability only could all too easily slip back into the authoritarian practices common under apartheid. If it is to serve the purpose of align-

chapter 1 Accountability and support in systemic school reform

ing accountability and support measures, then two further purposes of the management system must be to facilitate the generation of a shared institutional culture within work teams, and to identify and remedy support needs, be they in the form of a procedural bottleneck, or the capacity needs of individuals. In particular, such a system will explicitly utilise information from accountability measures such as Systemic Evaluation (DoE, 2001) and Whole School Evaluation (DoE, 2000a; 2000b; 2000c), both as starting points for planning and delivering more effective learning experiences for children, and for measuring progress towards achieving these goals. Unfortunately, these two key monitoring mechanisms have not yet moved beyond the pilot phase.

Support measures Development appraisal The development appraisal system, proposed by the unions and agreed by government at the end of the 1990s, is being implemented in one or two provinces, but in general the initiative seems to be in limbo, and therefore represents a missed opportunity for identifying the development needs of teachers and managers, and for tracking the results of support measures.

Provision of textbooks, stationery and other cognitive resources We argue in Chapter 5 that progression in school learning is essentially about learning the values held to be important by the society, and learning to read and write at successively higher levels of cognitive complexity. Within this understanding, the different

school subjects represent distinct areas of specialised knowledge and language. It follows that the quality of learning at each level crucially depends on the presence and productive use of good textbooks and other reading and writing materials. Following the expenditure overruns in many provinces in 1997, and the strict regime demanded by the national treasury in subsequent years, spending on books and stationery plummeted from a total of around R900 million in 1995/6 to a low of about R80 million in 1997/8 (Taylor and Vinjevold, 1999). Table 1.5 shows that in this area, too, government is steadily improving the budget allocation for books and stationery. Expenditure also seems to have improved, although a number of provinces regularly fail to spend their book budgets. In terms of the delivery of books to schools in time for the start of the 2001 calendar, the provinces exhibited mixed fortunes. Success in the provinces was generally around 80–90%, except for the Eastern Cape, Mpumulanga and the Northern Province, where delivery was estimated at 24%, 60% and 70%, respectively. There was no information available for KwaZulu-Natal (MoE, 2001). This is a continuing problem, and in 2002 the Minister reported that only four provinces (Free State, Northern Cape, North-West Province and Western Cape) had managed to deliver 75% of books ordered by the start of the school year (MoE, 2002).

Educator development: provision The state of learner performance alluded to above and described in Chapter 3 supports the conclusions of classroom-based research

Year

Budget allocation

Percent increase

Expenditure

Expenditure as a % of budget

1998/99

392.6

1999/00

794.7

102.0

769.4

96.8

2000/01

920.2

15.8

651.5

70.8

2001/02

1162.8

26.4

Table 1.5

BUDGET AND EXPENDITURE ON BOOKS AND STATIONERY, 1998–2000 (R MILLIONS)

Source: Compiled from MoE (2001; 2002).

15

part I Systemic school reform and research

(Taylor and Vinjevold, 1999) concerning the low levels of subject and pedagogical content knowledge on the part of teachers concerning the subjects they teach. In this regard, structured reading and numeracy INSET programmes stand out as urgent priorities for teachers at the Foundation and Intermediate phases, as do programmes which systematically take Senior phase teachers through the content of their specialised subject areas. Much of the INSET associated with school development programmes at present is undertaken by NGOs, through short workshop-based courses. Such courses can be effective in providing information and orientation to new policies, inspiring and planning individual and institutional change, and developing management systems. However, this form of INSET is a very weak intervention in building the deep knowledge structures and professional ethos required for the long-term qualitative improvement of teaching and learning. The universities have largely not been involved in this kind of work, but opportunities now abound for them here, and already there are some very promising developments, with training institutions beginning to participate in some large school development programmes. There would seem to be a historical opportunity for providers to offer accredited two- or three-year courses for school managers and teachers, directed by a focus on improving the delivery of the curriculum, by strengthening school-level management and classroom instruction. The single-minded focus in the past on pedagogy through INSET courses for teachers, a tendency amplified by the processoriented Curriculum 2005, must be augmented by an approach which places centre stage the quality of the knowledge transactions which occur between teacher and learner. This would include, first, the subject knowledge of teachers and second, their pedagogical content knowledge. Coverage of the curriculum to the standard appropriate to the grade being taught as well as the effective use of reading and writing activities should be integral to such programmes.

16

Conclusion The South African schooling system is characterised by the absence of accountability measures with ‘bite’. This results in a significant diminution of the opportunity to learn, particularly in poorer schools. Inefficiency thus exacerbates inequality. Under these conditions, it is likely that small moves in the direction of improved accountability will result in significant gains in performance. This may be the explanation for improvements in the matric results since 2000. This form of accountability is a blunt instrument when directed from Pretoria and the provincial capitals, and on its own, its impact is likely to reach a ceiling rather soon. Since the late 1990s an enormous amount of data has begun to be collected about South African schooling. However, most of this information, as we see in Chapter 2, remains in the private domain. This applies as much to work commissioned or undertaken by government as it does to donors operating from the corporate sector or sphere of international development aid. There are two main problems with this situation. First, a wealth of information which should be informing the practices of government, donors and NGOs lies fallow, and opportunities for learning from the successes and failures of others are missed, resulting in duplication of effort and endless repetition of mistakes. Second, the design and methodologies of the research studies themselves remain opaque to interrogation and improvement, resulting in a parallel set of missed opportunities and repetition of poor research. Given this state of affairs, the main purpose of this volume is to collate and analyse representative examples of research that has been done on schools in South Africa in recent years in order to take stock of: • the lessons that have emerged concerning the state of public schooling, and attempts to reform it, and • the state of research on schooling and the lessons that can be drawn for future research programmes.

chapter 1 Accountability and support in systemic school reform

Our starting assumption is that schooling is a systemic enterprise which can be improved on any real scale only by means of efforts which target key accountability and support measures at strategic leverage points. It follows, therefore, that a significant share of research on schooling should be targeted at a systemic perspective, and be directed towards identifying these key measures and leverage points (Muller, 2002). Aligning the many components of the schooling system so as to impact maximally on learner performance requires the orchestration of pressure and support at three key levels: district, school and classroom. The model is illustrated in Figure 1.1.

The chapters that follow explore these key nodes in greater detail. Chapter 2 has a dual purpose. First, it lists the studies from which the data analysed in later chapters is drawn, and characterises the schools which constitute the foci of these research studies. Second, the chapter provides an overview of the main design and methodological features of these studies, and draws conclusions concerning the sorts of lessons and insights that are likely to be derived from this work, given design constraints. Chapter 3 presents and analyses the results of a number of studies which have assessed the performance of pupils on written tests, principally in numeracy and literacy. Much of

Effectively functioning district office • Effective OD, planning and management • Effective HR management • Effective financial management

Effectively functioning school • Effective school management • Effective HR performance monitoring • Effective school administration (tracking of learners)

Effective school support

Figure 1.1

KEY NODES OF SYSTEMIC REFORM

Effective school monitoring

Effective curriculum management • Monitoring delivery of curriculum • Support of teachers • Instructional leadership

Effective school development planning

Effective educator • More effective management of learning programmes • Improved assessment practices • More effective use of LSM

Effective curriculum management • Improved learner participation in class • Improved learner performance

17

part I Systemic school reform and research

this information has never been made public before, and begins to provide the much needed picture of the state of learning in South Africa below the level of Grade 12. In an attempt to understand the kinds of results detailed in Chapter 3, Chapter 4 surveys the available studies which set out to identify the factors which influence pupil learning. The chapter ends with a delineation of research priorities for investigating schooling, and in particular with the need to design programmes which bridge the gap between statistical studies – which are proving to be successful at identifying macro-scale influences in the social environment, but which have been singularly unsuccessful to date in illuminating school-level factors – and microscale case studies – which are effective in illuminating interactions between individuals, but are unable either to generalise these lessons, nor to apply their methods at the institutional level. The cause of understanding the systemic features of schooling are poorly served by either approach on its own. Chapter 5 begins with the problem of making sense of the long list of factors which have been identified by educational research since the mid-1970s as being associated with improved learning, and with a number of problems which arise in attempting to

18

use these results to improve schooling. Based on the assumption that the best way around these problems is to develop a principled understanding of how schooling is intended to function, the chapter lays out a theory of schooling, drawing on the work of Basil Bernstein, among others. This must be seen as an early, crudely developed framework, which provides coherence for our understanding of schooling by postulating causal links between the component elements, but which at the same time requires much elaboration through testing against empirical reality. In Chapters 6, 7 and 8 the theory is used to analyse a sample of the data outlined in Chapter 2 at classroom, school and district levels respectively. This is an ex post facto analysis, and suffers all the usual shortcomings of work of this kind. However, the results of the analysis are compatible with the predictions of the model, and thus encourage the development of research programmes designed within the assumptions of the theory. The Conclusion draws together the findings of the preceding chapters and makes detailed recommendations for research that holds the promise of identifying the systemic leverage points for improving the quality of schooling in South Africa.

chapter

Researching school reform in South Africa Introduction It is common cause that until recently there has been little data available on the schooling system in South Africa. There is simply too little reliable information that allows us to know with any degree of assurance both what is happening in the system, and why. The grounds for this state of affairs were preliminarily explored in Getting Learning Right (Taylor and Vinjevold, 1999). The foundations of a data-poor environment were laid by the Department of Education under apartheid. Seekings (2001a, 23) comments: “Unsurprisingly, the apartheid state collected little data on the causes of educational performance among black students. More surprisingly, it collected little data on white students. The post-apartheid state inherited an educational system that collected little data on any education indicators.” Crouch (1999a) goes on to say that the tradition is hard to break, and that the use of educational indicators in South Africa is still sadly undeveloped. Systematic research data is equally thin. Researchers were blocked by the political climate from doing systematic work in schools even if they were willing. Jubber (1988; 1994a; 1994b) is one of the few exceptions. It must be admitted though that in common with their peers in sociology and politics, progressive educational researchers were, by and large, not keen to do systematic quantitative work anyway, and some are still markedly reluctant to do so. During the 1980s, as Seekings (2001a, 7) points out, Marxist and neo-Marxist approaches predominated in the English academy and, we may add, hermeneutic ones in the Afrikaans academy. Although these produced a rich qualitative body of work, both of them shared an “emphatic antipathy to quantitative work” (ibid., 8). For the former, quantitative work smacked of complicity with ‘bourgeois social

2

science’, if not with apartheid itself. This antipathy to quantitative work is not unique to South African progressive scholars, as Seekings quotes Erik Olin Wright in pointing out. Nevertheless, it has contributed to an ignorance of methodological and statistical procedures, which the general lack of “training in research methodology and statistical analysis” (Gouws and Gibson, 2001, 111) has compounded and reproduced over time. If we add to this the fact that substantial quantitative work is “very costly” (ibid.), and that, up until very recently, significant funding for educational research has simply been unavailable, then we should not be surprised at the lack of an indigenous research tradition of generalisable empirical investigation in South African education. The tide has begun to turn since 1990, especially since the mid-1990s. We have come some way since then towards a more effective empirical mapping of the schooling terrain. Impetus towards an increased information base has come from four important directions. The first is the National Department of Education (see Mseleku, 2002), which has introduced, or is set to introduce: • Systemic evaluation (Grades 3, 6 and 9). Grade 3 testing has been completed and the final report is eagerly awaited. • Whole School Evaluation. This has, as of the time of writing, not yet begun, largely due to opposition from the unions (see Taylor, 2002). • A national Education Management Information System (EMIS). This systematic national data base, together with other national sources of data such as the October Household Survey and national census data, is bound to be a rich source of information, but has so far been little used by education researchers.

19

part I Systemic school reform and research

The second source of information on schooling in South Africa consists of international comparative studies, most importantly TIMSS-R and MLA, which have set out to provide systematically collected, properly sampled independent sets of data for the entire population of schools in the country on pupil achievement, as well as on various contextual and school-based variables. These data sets are rich, and, like the national data sets referred to above, have by no means been mined for their potential contribution to our understanding of schools (Seekings, 2001b), although Howie (2002) has made a good start. For the first time we now have systematic achievement data about the country as a whole, where before we had data from the Senior Certificate only, which is, as it stands, a highly selective and non-random measure, since only 30% of pupils entering the system continue to Grade 12. The third source of data comprises studies conducted at higher education institutions. A large part of this work has been undertaken by education schools and faculties, but as the recent 1990–1998 survey reported by Gilmour (2001) shows, very little of this work – no more than 8% of the total published – deals with teachers and teaching, and this has been mainly small-scale and qualitative. More recently, economists and statisticians have begun to do important research in education (see for example the work of Fedderke et al., 1998; Bhorat and Hodge, 1999; Seekings, 2001b; Van der Berg, 2001a and Simpkins, 2002b). These two traditions of research have many sympathies in common, but have yet to communicate or collaborate productively with one another. We look more closely at some of this work in Chapter 4. The fourth impetus for increased knowledge of the education system comes from the donor community. It has been estimated that there are funded interventions in about 10% of all schools in the country (see Chapter 1, and Taylor, 2002) and grant makers have for some time now been requiring concrete demonstrations of efficacy before renewing funding. This has sparked some

20

important evaluations of interventions or projects. These evaluations have had mixed effects, as we will see below, but they have had at least one very important spin-off. They have sparked a small industry in research instrument design and construction in order to assess ‘before’ (and later ‘after’) baseline pupil competency via an independent measure. The upshot here has been that we have emerging data-sets of pupil achievement for various grades that can be used to supplement the Senior Certificate, MLA and TIMSS-R. These are described in Chapter 3. Donor-driven research and evaluation reports have almost without exception remained in the private domain, and in restricted circulation. The majority of these studies are mimeos, and some are quoted verbally so often that they have attained a quasimythical status. Names like Imbewu, DDSP, Mahlahle, GTZ and so on are now, practically speaking, part of public educational discourse, but the reports themselves are rarely seen, and the writers usually too busy to write them up for peer review and publication in respected public journals, with some important exceptions (like Schollar, 2001a). The result is that researchers and evaluators in the developing pool rarely refer to one another, and the educational knowledge base in the country proliferates but hardly accumulates or grows. One intention of this volume is to put some of this restricted-circulation work up for systematic scrutiny and assessment. The main purpose of the present chapter is to explore research conducted in South Africa that might inform systemic reform. We saw in the previous chapter that the model of systemic reform adopted in this volume includes both accountability and support components. In order to explore research that might shed light on that model of systemic reform, we sought studies conducted in the five-year period 1998–2002 that include learner assessment (accountability components) and input or process indicators (support components). While many other school- and classroombased studies have been completed in that

chapter 2 Researching school reform in South Africa

period, sixteen studies were chosen because they include both learner assessment data and related data on all or some of the following: teachers, learners, classrooms, schools and districts. Table 2.1 on page 22 provides an overview of the scope and design of the sixteen studies selected. Although these studies form the focus of this chapter, other studies conducted in the period are referred to in respect of their contribution to understanding the typology we develop here. The sixteen studies included for analysis are drawn from each of the four sources described above – government, international comparative studies, university-generated research and donor-commissioned evaluations. Two are studies commissioned by government: the Grade 3 Systemic Evaluation 2001, and the Western Cape Education Department (WCED) Grade 3 Learner Assessment study of 2001. We know of no other government-commissioned studies that include learner assessment. The study of the Education Action Zone (EAZ) project of the Gauteng Department of Education was commissioned through the JET research project described in Chapter 1. Two of the sixteen studies are the only international studies in which South Africa participated in this five-year period: the Third International Mathematics and Science Study (TIMSS-R) (Howie, 2001) and the Monitoring Learner Assessment (MLA) study (Chinapah et al. 2000). Another two studies analysed in this chapter are studies conducted by faculty at the University of Cape Town (UCT), namely ‘Teachers, textbooks and pedagogy: Studying maths for all in primary classrooms’ (Ensor et al., 2002) and ‘Baseline study of Western Cape schools: Final research report’ (Baxen and Gilmour, 2000). The remaining nine studies are evaluations of donor-sponsored interventions conducted in the period. Two of the interventions are national interventions: the Quality Learning Project (QLP), a systemic intervention in 525

schools in all provinces, and the READ Business Trust intervention, based in 1000 schools across the country. The District Development and Support Project (DDSP) operates in four provinces, namely Limpopo, KwaZulu-Natal, Northern Cape and Eastern Cape. The other evaluation studies included in our sample are interventions in single provinces: the Phalaborwa and Mahlahle Projects in Limpopo, the Imbewu Project in the Eastern Cape, Siyathuthuka in KwaZuluNatal and the SAILI and Cadbury School Development Initiative (CSDI) in the Western Cape. Analysis of these studies showed that there are three main domains for which indicators have been constructed in research and evaluation. These are: • indicators of context and input, meaning those factors in the contextual environment that create systematic patterns before schooling itself begins. For example, home background, resources such as buildings, water, electricity, textbooks, learner/ teacher ratio, learner/classroom ratio, teacher experience and teacher qualifications. • indicators of the process of schooling, like planning and management, monitoring and support, and pedagogy, which, as we have shown in the preceding chapter, can be examined at district, school and classroom level. • indicators of the success of schooling, like pupil attainment and achievement scores, dropout rates and completion rates. In the rest of this chapter we explore how the sixteen studies shown in Table 2.1 gather and use data on these indicators, and how the different designs allow conclusions to be drawn about relationships between these indicators. In order to do this, we examine first the purpose and design of the studies and the sampling procedures used.

21

22

Quasiexperimental

Repeat survey

Repeat survey

Repeat survey

Repeat survey

Repeat survey

Repeat survey

Repeat survey

DDSP Grade Learner Attainment Study

Siyathuthuka

Imbewu

SAILI

CSDI

Phalaborwa

Mahlahle

Survey

GTZ (Baxen and Gilmour, 2000)

READ Business Trust

Survey

MLA

Quasi-experimental

Repeat survey

TIMSS-R

Quasiexperimental

Survey

EAZ

Ensor et al. (2002)

Repeat survey

WC Grade 3 Learner Attainment

QLP

Survey

Research design

National Grade 3 Systemic Assessment

Name of study

83 – all schools and four control schools

All schools in intervention

All schools in intervention

All schools in intervention

All schools in intervention

All schools in intervention

All schools in intervention

All schools in intervention

525 – all schools in intervention

14 – all schools in study

All WCED schools with Grade 3 and 6 classes

All schools with Grade 4 classes

7234 – all SA schools with Grade 8 classes

1138 – all WCED schools with Grade 3 classes of >15 learners All schools in intervention

Schools with Grade 3 classes of >30 learners

Population

Research universe

36

10

8

11

70

26

453

89

102

14

39

400

200

67

100

5%

Sample

3, 6

3, 6

3, 6

3, 6

5, 7

3, 6

3

3

9, 11

7

3, 6

4

8

12

3

3

Grade

1 370 1 361

373 378

305 282

396 439

1 144 1 368

970 936

14 700

3 824

3 363 2 985

538

1 027 1 052

10 483 10 759 10 438

8 147

8 029

3 059

108 000

No. of learners

Indicators of success: number of learners and level-of-attainment scores

DESIGN FEATURES OF 16 SCHOOL-BASED STUDIES

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

Y

N

Y

Y

Contextual

Y

Y

Y

Y

Y

Y

N

Y

Y

Y

Y

Y

Y

N

N

Y

Input

Contextual, input and process indicators

Y

Y

Y

Y

Y

Y

N

Y

Y

Y

Y

N

Y

N

N

Y

Process

Table 2.1

chapter 2 Researching school reform in South Africa

Purpose and design The design of a study determines and constrains the kinds of inferences that can be drawn from its results. Aspects of design adequacy can be independently assessed, but the most crucial determinant of appropriate design is the question being asked: “certain designs are better suited to address particular kinds of questions under particular conditions than others.” (Shavelson and Towne, 2002, 98). The lack of appropriate theory and reliable descriptive data is probably the greatest constraint on robust experimental designs capable of generating strong and generalisable inferences, as we shall see (ibid., 108). Shavelson and Towne distinguish between three classes of research question: • descriptive questions, asking, “what is happening?” • causal questions, asking, “is there a systematic effect?” • process questions, asking, “why or how is it happening?” Many studies are primarily descriptive, or causal, or process in design, while others are mixed. Mixed designs are particularly common in programme evaluation studies. TIMSS-R is an example of a large-scale study that is essentially descriptive, which allows for the establishment of simple (correlational) relationships between various contextual and school indicators, and performance on the TIMSS tests. In other words, TIMSS-R was asking a classical, large-scale “what is happening?” question. Because the TIMSS-R study used a national random sample, the significance of the relationships can plausibly be generalised to schools across the country. Studies of this sort are most useful when we know relatively little about a given domain. When they are robust, they allow us to discern patterns, which may allow us to proceed to surmising what exactly it is that is causing the pattern, and then how. Descriptive studies themselves are not designed to answer such questions. Studies of simple relationships are non-directional. They cannot tell us

which comes first, which is the cause and which the effect. In order to proceed to this point, we need a theory which coherently and plausibly specifies a model of what causes what. In addition, the theory must be able to explain why the possible rival explanations are ruled out. The best kind of design to test such a hypothesis would be one that sets up two groups for comparison, which differ only with respect to the hypothesised causal variable. Randomisation4 (that is, random allocation to treatment or non-treatment groups) is the best way to ensure that the comparison groups differ only with respect to the treatment, because it allows for the independent effect of the hypothesised causal variable to be satisfactorily isolated, and allows all systematic biases to be excluded. Randomised field trials are common in medical and especially pharmaceutical research, but rare in schooling research. Perhaps the most famous and influential randomised schooling experiment was the Tennessee Project STAR classsize study, where students were randomly assigned to classes of different sizes, and then monitored over time (Finn and Achilles, 1999). This study found that small-class students outperformed their peers in the control group. This was especially marked in the case of minority students. Although the experimental group learners returned to normal class sizes after four years in their experimental group, they continued to outperform their peers right up to the university entrance exam. Randomisation is frequently not feasible in schooling, for both practical and ethical reasons. The next best design to test for causality is the so-called quasi-experimental or observational design, where naturally occurring groups are selected for comparison. This is a design increasingly used in programme evaluation studies, where the treatment group (the schools or students undergoing the programme) is matched with an equivalent group that is as alike as possible to the treatment group except with respect to the treatment. The largest threat to

23

part I Systemic school reform and research

establishing strong causal conclusions here is the possibility that some unmeasured, prior-existing difference, rather than the treatment, is causing the effect. The strength of the conclusion that can be drawn depends upon the ability to eliminate plausible competing hypotheses, which is less easy to do authoritatively than with a true experiment. Nevertheless, if strong effects are found, the ‘real life’ groups of the quasiexperiment can be more persuasive to policy makers than the artificial groups of randomisation, which might be unfeasible to generalise to a larger group. Schollar (2001a) and Ensor et al. (2002) are good examples of programme evaluations that have used quasiexperimental design. Finally, to discover that x causes an effect is not yet to understand why or how it does so. Conclusions here are particularly important once the presence of a causal effect has already been established. Shavelson and Towne (2002) distinguish between process studies where theory is fairly well established, and those where it is not. In the case of the former, they show that the mechanism at work can be fairly rigorously established. A common example is the stable effect of superior attainment scores produced by learners in private or Catholic schools as compared to those produced by learners in ordinary public schools in the USA. What is it about the private or Catholic school that actually produces the difference? It is easy, but almost certainly misleading, to automatically assume that it has something to do with religion. Bryk et al. (1993) developed and tested a series of models to explain how the effect was produced. They conclude that it was through a combination of school factors that together enhanced a sense of community. More commonly, though, mechanism studies have no clear prior idea of what the mechanism might be, and thus often have a strong descriptive component. In-depth case studies are common here. Such studies are, again, most useful when the causal effect has already been established. There are numerous small-scale process studies that assume an effect (learner-

24

centred teaching, for example) and proceed to unpack how it works. These studies run the risk of begging the question, since such studies are trying to determine the mechanism of an effect whose efficacy has yet to be established. In some of the first generation of descriptive studies, like the Western Cape Grade 3 Learner Assessment study, the purpose is principally to map the distribution of attainment scores in a particular area, usually a district or province, sometimes a whole country. The aim here is to see how the public education system is performing, so such descriptive studies are sometimes called systemic studies. The most comprehensive of such studies is TIMSS-R. The planned systemic assessment of Grades 3, 6 and 9 by the DoE will be another such mapping or descriptive enterprise. TIMSS-R highlights the fact that such descriptive studies often do collect collateral information regarding the environment, and that this data can provide highly suggestive information about the patterns in the attainmentscore data. As we commented before, such studies are rarely properly exploited by their original sponsors, and require secondary analyses. These can be done quite independently of the original baseline study, and the usefulness of secondary analyses is being shown locally by analysts who are correlating baseline data (like matric scores) with census or October Household survey data (see Anderson et al., 2001) to generate suggestive correlations. The majority of studies on our list (see Table 2.1) are thus descriptive studies, and we have no reason to think that this is not the case for South African schooling research as a whole. What distinguishes descriptive studies is that there is no experimental treatment condition – called an ‘intervention’ in local NGO and donor work – which is assessed for efficacy either in a before/after or experimental/control design format. There is, in other words, no explicit hypothesis being examined. One or more repeat studies may be, and in fact usually are, done in order to gain a descriptive map over time to check trends.

chapter 2 Researching school reform in South Africa

But the principal aim in descriptive studies is to map the performance of a domain with respect to various indicators, rather than to assign explanatory priority to this or that factor. In Mahlahle (JET, 1999; 2000), for example, the design presumes that textbook availability and use is important for learner performance, but the study can only report on changes in availability and use without attributing causality to them. As we saw above, explanatory or quasiexperimental studies are characterised primarily by a treatment, which is then more or less systematically tested for impact on one or more outcome. By this definition, there have been very few education studies that are properly quasi-experimental. Most explanatory studies that have been done so far have been evaluations of interventions (Schollar, 1999, and Ensor, 2002, are representative exemplars). In many other cases the treatment has not been uniformly applied to all the schools concerned, nor is the treatment a single and thus isolatable factor. The ‘dosage’ is thus variable across the schools or classes. This is clearly a consequence of the different purposes of support interventions, which are often planned and implemented well in advance of planning the programme evaluation. Experimental treatments in conventional educational experiments, which aim for generalisable data from the outset, are usually planned as an integral part of the experiment, so that treatments can be uniformly assigned to systematically sampled schools or classes in comparable dosages.

Sampling The relatively large-scale studies that have been conducted in South Africa since 1997 have occasioned an increased interest in, and use of, sampling. This is a development that can only be welcomed. But the very intention of these studies, which is to make inferences about a relatively large number of schools, has at times been undermined by less than careful sampling procedures. In a number of cases, the sampling procedures followed do not allow us to make the inferences that the

studies were designed to make (Crouch, 2000). This is true of both descriptive and explanatory studies, of relatively large-scale countrywide studies as well as of smaller-scale evaluation studies. Crouch argues that if we want to make inferences about the whole population of South African schools, then a purely random sample of South African schools would have to be drawn. The TIMSS-R and MLA studies wanted to make just such inferences about South African schools. However, they also wanted to make comparative inferences about the provincial sub-populations. This dramatically increased the number of schools required in the sample, since the total number of schools in the sample must grow in linear proportion to the number of sub-populations. Consequently, the South African samples had to be relatively large compared to those of other countries. In fact, in the TIMSS-R study (see Table 2.2 on page 26), provincial reporting meant that South Africa had the third largest number of schools participating in the study (225) after Canada (410) and the USA (250). Similarly, over 400 primary schools participated in the MLA study in order to allow for provincial inferences. The first step in the sampling process for both TIMSS-R and MLA was to select schools from the entire South African population. In the case of TIMSS-R, all South African schools with Grade 8 classes constituted the population; in the case of MLA, all South African schools with Grade 4 classes were included. The number of schools for each province was determined by probability-weighting, to take into account the different number of learners in each province, and further stratified within each province by size of school, language of instruction and source of funding, in the case of TIMMS-R, and by school type, size and location (urban, semi-urban, rural) in the case of MLA. In the Western Cape Education Department (WCED) 2001 Learner Assessment study (JET, 2002), a random sample of 150 schools was drawn from the 1138 schools in the province that had Grade 3 classes of more

25

part I Systemic school reform and research

Province

Total no. of schools sampled

Ineligible

Participating

Nonparticipating

Eastern Cape

25

0

25

0

Free State

25

1

21

3

Gauteng

25

2

18

5

KwaZulu-Natal

25

0

25

0

Mpumalanga

25

1

21

3

North-West

25

0

16

9

Northern Cape

25

1

22

2

Northern Province

25

0

22

3

Western Cape Total

25

1

24

0

225

6

194

25

Table 2.2

SAMPLING STATUS FOR SOUTH AFRICA IN TIMSS-R

Source: Howie (2001).

than 15 learners. This sample was then used to draw a 100-school sample which was weighted to represent the relative size of the seven districts of the province, as well as to represent the relative proportion of schools from former racially-defined education departments (Cape Education Department, House of Representatives and Department of Education). For example, if 20% of the schools with Grade 3 learners in the Western Cape are ex-HoR schools and are situated in the Overberg district, then 20% of the schools in the sample must be ex-HoR schools from the Overberg district. This weighting is simply to get the provincial proportion correct. It does not mean that the study can make inferences about districts in the province or about how schools from former departments are faring. In order to do so, the sample would have had to be further proportionally increased for each sub-population. The TIMSS-R, MLA and WCED Learner Assessment study are the only studies that we know of that explicitly set out to make inferences about all South African schools and/or all schools in a province. All the other studies have as their total population only the schools involved in a particular treatment or intervention. Because the purpose of the majority of interventions in South Africa is to improve provision in disadvantaged schools, this is generally the target population of such interventions, although ‘disadvantage’ is

26

rarely systematically assessed and these schools are not systematically chosen. Therefore these studies are not able to make inferences about all disadvantaged schools or even about disadvantaged schools in a particular province or district. The recent emphasis on district development in South African systemic reform has meant that the most recent intervention projects have included all schools in certain districts. All secondary schools in 18 districts across South Africa are involved in the QLP intervention, for example, and all primary schools in 18 districts in four provinces are included in the DDSP intervention. In the DDSP, all the primary schools with Grade 3 classes participated in the study, which means that the entire population took part (see Table 2.3 on page 27). In this case, sampling only took place at the school level in that the test administrators were instructed to test a minimum of 40 Grade 3 pupils from each school, unless the number of Grade 3 pupils in the school was less than this number. If there was only one class, all Grade 3s were tested up to a maximum of 50 pupils. In schools where there was more than one class, the same number of pupils were selected randomly from each of the classes to make up a total of 40 pupils. This study then allows for strong inferences about the primary schools in these 18 districts. But equally strong inferences would have been possible if an appropriate

chapter 2 Researching school reform in South Africa

Province

District

KZN

Chwezi Ekhombe Godide Sibudheni Sigananda

EC

Herschel Queenstown West Queenstown East Cofimvaba Cala Lady Frere

Number of primary schools with Grade 3 in district

NC NP

Vuwani Mkhulu Palala Hlanganani Apel Polokwane

Grand Total

Number of schools visited in the DDSP Baseline Study

Number of Number Grade 3 of learners in learners DDSP schools tested visited

20 32 33 23 28 136

20 32 33 23 28 136

1 091 1 152 1 218 1 282 1 794 6 537

660 643 816 726 1 004 3 849

18 13 27 12 7 10 87

17 13 26 12 6 9 83

1 108 864 1 903 532 493 433 5 333

570 423 810 390 225 234 2 652

53

51

2 327

1 256

39 39 34 28 26 27 193

39 39 34 28 26 27 193

2 309 4 047 1 855 2 648 1 428 2 008 14 295

499 325 091 118 898 1 013 6 944

459

453

28 492

14 700

Table 2.3

NUMBER OF GRADE 3 LEARNERS TESTED AS PART OF THE DDSP GRADE 3 BASELINE STUDY

1 1 1 1

Source: Vinjevold and Crouch (2001).

school sample from the population had been drawn. From the point of view of inferential adequacy, the study could thus have been a less expensive exercise. A number of other projects, such as the EAZ, Cadbury School Development Initiative (CSDI) and SAILI have included all schools in their evaluations (Fleisch, 2001; JET, 2001a; 2001b; 2001c). In the EAZ, all the schools in Gauteng with less than a 30% pass rate in the 2000 Senior Certificate exam were included. In SAILI, the project population was formed from a selection of schools which had volunteered for the project, and which came from no particular area or grouping. (See Table 2.4 on page 28.) All these studies allow inferences about the intervention population only, not about any particular type of school. Their generalisability is thus circumscribed.

In other studies of interventions, a sample of the project schools has been chosen for the evaluation. In the QLP study, for example, a stratified sample of 102 of the 525 schools in the project was randomly selected (Kanjee et al., 2001). The sample was stratified by size of school, former education department and pass rate in the 2000 Senior Certificate, in order to represent these categories in the sample in proportion to the larger population. In the Imbewu, Siyathuthuka, and Phalaborwa evaluations, the schools were a selected sample of the schools in the projects (Schollar, 2001b; JET 2001d; 2001e; 2001f; 2001g). While attempts were made to ensure that schools from a range of districts or clusters within districts were selected, sampling in these studies was neither random nor purposive. In most cases, ease of access or

27

part I Systemic school reform and research

Project

Province

No. of districts in intervention

CIE

NP

1 district

30

10

Siyathuthuka

KZN

2 districts

30

10

No. of schools in study sample

525

70

Imbewu

EC

9 districts

Cadbury

WC

1 district

8

8

Mahlahle

NP

2 districts

50

36

Phalaborwa

NP

1 district

26

10

SAILI

WC

many

11

11

SDU

WC

1 district

38

38

communication played a role in determining which schools were selected. How does one know what number of schools makes an appropriate sample? Crouch warns against percentage rules for sampling, like the 20% rule, for example, and points out that “the ideal sample size for estimating a population parameter is determined really by only four factors”: the confidence level desired, the margin of error accepted, the variability of the phenomenon being measured and loss of precision because of intra-cluster correlation of observations, unequal probabilities of selection, and/or differential rates of non-response (Crouch, 2000, 2–3).

Contextual factors Since the Coleman study (Coleman et al., 1966), scholars and policy makers alike have accepted that home background factors play a major role in determining schooling outcomes. This still accounts for up to 80% of the variance in American studies, though the measured variance in South Africa is considerably lower (Crouch and Mabogoane, 2001), as we see in Chapter 4. This is also the case in many Third World countries. The implication of this finding is that schooling may, in contexts like South Africa, have a larger impact on life chances, but before we can assert this with any confidence, we must be sure that contextual factors are properly measured and controlled for. Indeed, it is because the Senior

28

No. of schools in intervention

Table 2.4

SAMPLE SIZE

Certificate does not or cannot control for background that it is sometimes said that the Senior Certificate is not a good indicator of school performance. As we saw in the previous section, many of the treatment studies we discuss here have located their treatment/intervention in ‘disadvantaged’ districts and schools, chosen often on the basis of accessibility, provincial department decisions or even donor behest, as in the case of the GTZ study. The assumption is that these schools are homogeneously similar in terms of disadvantage or social class. This may well be the case in relatively isolated rural areas – say, of the Eastern Cape – but this is rarely if ever put to the test or assessed, even after the fact. Consequently, in the DDSP, SAILI, Mahlahle, Siyathuthuka, CSDI, Imbewu, and Phalaborwa studies, the only background data collected concerns learners’ gender, age, and years in school. An interesting exception is Perry’s (2002) analysis of the links between DDSP test results on the one hand, and school conditions and socio-economic status (SES) on the other. A SES index was constructed from SES status of the Census enumerator-area within which the schools fell, which was in turn based on four factors: proportion of functionally illiterate persons in the area, economic dependency ratio of the area, proportion of households without electricity and proportion of households without water. Perry found that literacy and access to water are statisti-

chapter 2 Researching school reform in South Africa

cally significantly associated with learner achievement, but that the difference is not significant. Nevertheless, it does show the possibilities of using collected contextual data in post hoc analysis. Economists who correlate large household data-sets with attainment data conventionally use this form of analysis. Anderson et al. (2001), for example, used 1995 October Household Survey data to examine the relationship between a mother’s educational level and her child’s progress at school. MLA and TIMSS-R are two trans-national studies that contain a wealth of student home-background data. MLA, for instance, looks at number of people in the home, whether parents are married or single, whether they are employed, home language used, newspapers and books in the home, number of meals, attitude to school and teachers, and parental education. Conventionally, large studies of this kind aggregate clusters of items into composite indices on the basis of statistical weighting. So, for instance, MLA constructed a home learning-support index, a home readingmaterials index, parent opinion about education index, school access index, attitudes to school and teacher index, access to information index, and a parent-education index. The QLP has adapted the TIMSS-R instrument to obtain learner background data to correlate with learner attainment. The following items were extracted by Simkins and Patterson (2002) from the learner questionnaire to construct a household effect index: language spoken at home, frequency of use of language of instruction at home, household wealth, study aids, meals, home reading, parental support and time use. All the studies which have collected more detailed background information than the strictly numerical data on gender, age and years in school have encountered common challenges related to the reliability of selfreport data. In the QLP study, for example, the all-important variable of parental educational level was found, on comparing reported levels with mean levels as reflected in the Census data, to be grossly inflated, in fact

double that in the Census. This data has had to be discarded. Learners’ responses to background questions in the TIMSS-R and QLP indicate poor understanding of the questions, possibly related to low literacy levels. In the MLA study the data collected from learners and parents was severely compromised by the low literacy levels of these respondents. We return to this issue in Chapter 4. We can conclude that, despite background factors being a well-established cause of schooling attainment, South African studies have by and large not measured these in satisfactory ways. This is especially true of what Crouch has called the cognitive dimensions of socio-economic status – parental educational level, books in the home, and factors to do with language, especially language of instruction.

Input factors A wide variety of input data is collected in the studies that inform this chapter. This input data can be divided into human resource (educator) data and physical resource (classroom and school) data.

Human resources In general, two types of human resource input data are collected in school-based studies in South Africa, namely personnel background data (age, gender, qualifications, experience etc) and attitudinal data (what the personnel of schools think about schooling, learners and themselves). The majority of studies collect background data on teachers. Considerably fewer studies gather similar data on principals and school management teams, and only a handful report background data on school governing bodies. There is also less data available on attitudes than on background, although the TIMSS-R, MLA and GTZ studies have made advances in this area. The background data on teachers generally relates to gender, age, teaching experience and teaching qualifications. Data on the first three of these categories is easy to collect, and represents reliable (if inconsequential) data for correlating the human-resource input in schools with learner achievement.

29

part I Systemic school reform and research

The qualifications data has not proved as reliable for making correlations. This is because the data is often collected in non-comparable formats. For example, some studies ask for years in post-secondary education and do not distinguish between academic and professional qualifications. Others only ask for the post level of the qualification, which does not distinguish between qualifications with very different levels of quality. The very variable nature of teacher education courses offered, especially at the former DET colleges in South Africa, suggests that the nature of the qualification (degree or diploma/certificate) and the name of the credentialing institution might provide more useful information on teacher input. Nevertheless, Crouch and Mabogoane (2001, 65) found that an increase in roughly one year of qualification could be associated with a substantial increase in pass rates. Background data on principals and school management teams has been collected by the QLP, Mahlahle, MLA, GTZ and Imbewu studies. As with teachers, this data is mainly concerned with age, gender, experience and qualifications. In the case of principals, both the experience and qualifications data has not been collected in a way that allows comparison across studies or sufficient variation. Questions on the experience of principals often do not distinguish between teaching experience, management (HOD, deputy principal) experience and experience as a principal. The data collected on principal qualifications suffers from the same problems outlined above for teachers, but presents another problem: principal qualifications generally refer to teaching qualifications and not to qualifications in management and administration. The GTZ study is an exception in that the study collects data on management and leadership courses attended by principals and in-school support received from departments of education and other providers. The GTZ study also adds another dimension to the human resource input data, namely the occupations of SGB members of schools in the study. Finally, many studies collect attitudinal information from teachers, and sometimes, as

30

with MLA, from parents too. While this might seem somewhat counter-intuitive, one of the stable international findings is that positive teacher attitudes to learners, or rather, high expectations, correlate well with learner attainments. This is sometimes called ‘school-based social capital’ (Dika and Singh, 2002), a factor discussed further in Chapter 4. In the main, however, not much is done with this data locally.

Physical resources South African school-based studies have collected large amounts of data on classroom and school resources. The majority of studies use individual indicators to index this data. The most common of these indicators collected for the classroom are numbers of tables, desks, chairs, cupboards, chalkboards and wall charts. Less commonly collected data, but correlationally more important, is that on the availability of textbooks and exercise books. The school-level input data collected in the studies focuses on the availability of water, electricity and sewage, the number and state of repair of classrooms and other buildings, and the state of repair/cleanliness/ security of the school grounds. In the majority of studies, this physical resource data is used simply to report on the physical conditions of schools and not to understand learner performance or other success indicators. The GTZ, READ, TIMSS-R and MLA have however examined the impact of learner–teacher ratios on learner performance. The influence of the number of classrooms on class size suggests that learner–classroom ratios are a more important consideration in disadvantaged schools. This data, plus much else besides, is reliably and comparably provided in the national School Register of Needs (SRN), the 2000 edition of which shows the learner–classroom ratio to be a factor of declining importance (Seekings, 2001a). Crouch and Mabogoane (2001) use this database to investigate the influence of school resources on learner performance. This SRN data collected periodically (so far only in 1996 and 2000) provides a rich and largely

chapter 2 Researching school reform in South Africa

under-utilised set of data. Use of this data saves costs, since it need not be re-collected, and allows comparisons across studies, since the collection categories are the same, avoiding a problem that bedevils cross-study comparisons. Perry (2002) has used the SRN data to construct a ‘school condition’ index for the DDSP. This index consists of four items: availability of electricity, availability of water, condition of school building and learner–classroom ratio. This index is then correlated against the Grade 3 learner assessment scores. As we mentioned in the contextual data section above, data from this source can be post hoc correlated with any attainment study data as long as the school’s EMIS number is known, and need not be built in from the outset. Having said that, physical resources seem to be less consequential than human ones in poor South African schools. Teacher qualifications, textbooks, management and other ‘cognitive’ resources have far greater impact on learners’ outcomes than “the ‘bricks and mortar’ and easily quantifiable resources” (Crouch and Mabogoane, 2001, 64). That successive studies continue to spend a great deal of time and money painstakingly compiling lists of physical resources is either because these are apparently easier to collect (though not necessarily to compare, as we said above), or because researchers are unaware of the findings in the literature. Either way, the collective knowledge base is not significantly advanced.

Educational process factors Studies using educational process indicators are interested in the actual delivery of educational goods, what TIMSS-R calls the enacted or implemented curriculum. Research efforts in this category are concerned to construct indicators that will tap into the relevant parts of the educational process that help maximise the opportunity for learners’ exposure to the intended curriculum. Efforts have so far been directed mainly at the classroom and sometimes the school level. The QLP innovatively attempts to assess district level delivery as well. If the vast sums spent on INSET, pedagogi-

cal and textbook interventions are anything to go by (see Chapter 1 for estimates), the implicit wager in much South African school reform has been that the key process factors lie with classrooms and teachers. Indeed, so fixed on the teacher (at the expense of the learner) is the research gaze that most studies so far completed have confined themselves to assessing whether the intervention has changed teacher practice, not examining its impact on learner attainment. A recent study has queried whether attainment is in fact a fair test of such interventions (Adler and Reed, 2002). This restricted focus is a consequence of the adoption by these studies of what we referred to in Chapter 1 as inside–out, or competence models of school reform. More recently, management and leadership factors have also begun to come under the research spotlight. However, none of these factors has so far been particularly well captured by the research, and which process factors actually make the difference to learner attainment in South Africa remains something of a mystery. Small wonder then that the ‘whole school’ branch of school improvement, which banks on not differentiating between process factors, continues to find favour with funders. There are two major modes for data collection of process information: dependent (mainly reported) and independent (mainly documentary and observed). Each can vary as to its openness or closedness (Ensor, 2002). As a rule, the large national studies like TIMSS-R and MLA are more or less compelled to use a closed reporting mode because of cost, though many small studies also use reported data. This means that teachers and principals are the main data sources for these studies. There is a question mark hanging over the reliability of reported data, as we saw above with Simkins and the QLP. Similarly, the various studies of Getting Learning Right (Taylor and Vinjevold, 1999) regularly reported disparities between what teachers reported they were doing and what they actually did. Nevertheless, it would be wrong to discard this mode altogether. There are ways to

31

part I Systemic school reform and research

crosscheck for accuracy (using repeated items and other forms of triangulation) and where possible, this method should be combined with less dependent measures. READ, for example, combined checklist and interview data with up to 20 visits to schools to check book use (Schollar, 2001a). Other countries have found this a respectable source of data. Perhaps the lesson from Getting Learning Right is that in matters of pedagogical innovation, where exemplars of the new are not known at first hand – for example, where teachers are told to use ‘group work’, but have in fact never seen ‘good practice’ exemplars of group work in real life – teacher judgement of their own practice should be approached with circumspection. In any case, with TIMSS-R and the MLA, there is now a large bank of reported process data that is barely mined as to its impact on attainment. Many researchers will prefer to use observed data, collected by a trained observer by means of an open or closed protocol. The larger the study, the greater the inclination will be to use closed instruments in order to maximise reliability. However, many scholars and pedagogues will argue that there are certain kinds of data that are not easily captured by a closed instrument. These kinds have to do with teaching and its tacit nature. In particular, it is not easy to capture ‘teaching for conceptual understanding’ on a checklist (Ensor, 2002). In other words, these scholars are reminding us that some educational realities cannot be fixed into closed observational measures without sacrificing some degree of validity, or ‘richness of meaning’ (Babbie and Mouton, 2001, 125). There is no question that master teachers know and can judge the craft and specialised practice of teachers more reliably and validly than anyone else. But we do not have enough master teachers in our schools, let alone in the numbers that would be required to generate research information systematically (Crouch, 2002). Smaller-scale studies can thus afford to use master teachers working with relatively open observational schedules, but it is doubtful whether this will be feasible in larger-scale

32

studies. Standardised research instruments are one method the research community has so far devised to do this job. Doubtless better methods will be devised in time. Instruments differ as to their stability in reliably capturing the same phenomena in the same categories across time and across different raters. The problem of inter-rater reliability is commonly dealt with through thorough training of the observers so as to generate a common interpretive frame of reference. This is particularly necessary when the observation instrument is long and complex. None of the projects reported here (MLA and TIMSS-R perhaps excepted) report indices of inter-rater reliability. Much more seriously, the training of coders reported in many of the larger studies seems to be inadequate, and may well compromise the reliability of the data. An idiosyncratic, but effective, method of controlling for inter-rater reliability is that of Eric Schollar who does all his observations himself (Schollar, 2001a; 2001b). This is not likely to become a generalised solution. A second problem of rater reliability lies with the level of expertise of the observers, especially when it comes to coding specialised practices like maths and science. Skilled teachers can steer the course of learning in ways that may well be invisible to the non-adept. One way around these difficulties is to have expert observers describe classroom practices, and then to code these descriptions post hoc. We take this route with the data we report on in Chapters 6, 7 and 8. There is a final problem with many South African observation schedules, that may perhaps best be called a problem of construct validity: which educational process is this proxy indicator representing? Is it representing what we think it is representing? Conversely, what do the various proxy indicators have in common, and how do they differ? When we observe something as rich in meaning as the delivery of educational goods, what are we actually capturing? We tend to talk about this problem as one of theory, and we generally say that the observational categories are under-theorised (that is, they

chapter 2 Researching school reform in South Africa

float free from any theoretical construct). This is the besetting problem of all schooleffectiveness research. The categories are aggregated statistically, but the factors in the statistical clusters do not always seem to have anything in common – the classical problem of empiricism. We have the converse problem too: of researchers deploying a normative theory which predicts specific empirical relations, but where the coding categories are derived from the theory in a very loose sense only. This problem cannot be pursued here, but its solution surely lies in drawing the internal and external languages of description into closer alignment, so that

theory speaks more systematically to the observational categories and vice versa (Moore and Muller, 2002). Herein lies the solution also to the problem of empirical indicators aggregated by statistical means into clustered indices that we mentioned above. Indices can always be inductively created, but unless such an inductive index can be reconciled with a theoretical expectation, we literally do not know what the index means. Here South Africa joins the international world of research into school effectiveness in mutual perplexity. In Chapter 5 we attempt to derive a principled set of relationships between theory and evidence.

33

chapter

Lessons from learner assessment Introduction The administration of learner tests is a recent and controversial development in schoolbased research in South Africa. While some of the controversy around testing relates to a specifically South African concern that the results of the tests may be used to substantiate racial prejudices regarding ‘intelligence’, the South African testing debate also reflects contemporary international debates. The first area of contention centres on the purpose of learner testing. The opponents of testing argue that because test results are inevitably used to judge learners, teachers, schools or interventions, the pressures created are counter-productive to learning. Shephard (2001), for example, reviewing the effects of high-stakes testing in the USA, found that testing contributed to teaching to the test; curriculum distortion through emphasis on particular areas of the curriculum; and loss of intrinsic motivation to learn. In contrast, those who argue for responsible use of high-stakes assessment argue that assessment can foster high expectations and support high levels of learning. Proponents of thoughtful and meaningful assessment, such as the National Council of Teachers of Mathematics (NCTM, 2000), claim that learners’ scores can provide important information which, when combined with information from other sources, can lead to decisions that promote student learning and equality of opportunity. Schoenfeld (2002) contends that when assessment, curricula, and professional development are appropriately aligned then clear and dramatic evidence of improvement in learner scores and understanding is produced. The second major debate revolves around what tests can tell us about what learners know. So, for example, Greaney (2001, 106) dismisses Warwick Elley’s Grade 4 STAR reading tests on the following grounds: “A problem with this (and most group-

34

3

administered reading tests) is that it is very difficult for the teacher to assess the reading processes. Raw scores only suggest that the answer was correct or incorrect not why the incorrect choices were made.” A third debate revolves around what should be tested and how this ‘content’ should be tested. So, for example, Greaney claims that Elley’s tests purport to test decoding skills but in reality do not do so. According to Greaney (2001, 107), there is an apparent reluctance on the part of test designers to accept research findings implicating “efficient phonological processing deficits as being a major cause of poor reading progress”. This is evident for Greaney in the make up of the STAR assessment. Similarly, Garden (2002, 5) argues that differences in beliefs about mathematics teaching and learning lead to differences of opinion on what should be assessed and how. “Inductive versus deductive approaches, emphasis on utilitarian aspects of mathematics rather than mathematics as a discipline, didactic teaching approaches as opposed to child-centred discovery all carry implications for how achievement may be measured.” This ongoing debate led the TIMSS test constructors to include both multiple choice and free response items in the Repeat Study. Extensive measures have been taken in recent years to defuse the debate by improving the validity and reliability of tests, but even when there are high levels of validity and reliability it is important to be aware of where the strengths and limitations of test instruments lie. The future legitimacy of assessment will depend on strengthening these two dimensions and so ensuring that the important information generated by assessment can, as the NCTM argues, lead to decisions that “promote student learning and equality of opportunity” (cited in Schoenfeld, 2002, 21).

chapter 3 Lessons from learner assessment

Although there were many formal evaluations of teacher education programmes in the 1990s, few, if any, of these studies included tests of learner achievement. In the mid-1990s Eric Schollar (2001a), among others, began to test learners as part of evaluations of NGO classroom- and school-based interventions. In the PEI research (Taylor and Vinjevold, 1999) eleven of the studies included learner testing. These studies served notice of the complexities of test construction, validation and standardisation. Since the PEI research, there has been a significant increase in the number of schoolbased studies in which tests are administered. This development has both informed and been driven by the Department of Education quality assurance framework. (DoE, 1998a). Systemic assessment, which is central to this quality framework, sees learner assessment as an essential output indicator which must be linked and interpreted in relation to other systems and processes that make up the schooling system. The previous chapter and Chapter 4 of this book describe how schoolbased studies are attempting to understand the link between learner assessment on the one hand, and context and education processes on the other. The present chapter describes a sample of studies conducted in South Africa between 1998 and 2002 which have a learner assessment component. In particular, the chapter examines: • The scope of the testing – the grades in which tests were administered, the number and location of schools in which tests were conducted and the numbers of learners tested. • What has been tested – the subject content knowledge and skills tested. In the majority of studies reviewed the tests have assessed literacy/language skills and numeracy/ mathematics skills. The chapter examines issues that influenced the construction and design of tests. • Issues of reliability and validity – the chapter will discuss the steps taken to address questions of validity and reliability. Issues of language are integral to these concerns.

• The results of the tests – and what these tests tell us about learners’ knowledge and skills.

The scope of assessment in South African studies In the period 1998–2002, school-based research projects in South Africa administered a variety of tests to learners in all grades of the General Education and Training band except Grades 1 and 2. In most cases numeracy/ mathematics tests and reading and writing tests were administered. Exceptions were the TIMSS-R study, which focused on mathematics and science, and the MLA, which tested numeracy, literacy and Life Skills. Table 3.1 on page 36 provides an overview of the testing conducted in the period. The table shows that only the tests administered in the QLP, MLA, Systemic Evaluation and TIMSS-R were administered in all provinces. The HSRC administered reading and writing and mathematics tests to learners in 102 schools involved in the countrywide 500-school QLP intervention. In this study the tests were administered to 3363 Grade 9 learners and 3567 Grade 11 learners. In the MLA study, tests were administered to 10 500 Grade 4 learners from 400 schools across the country, while 8147 Grade 8 learners from 194 schools were tested in the TIMSS-R. Close to 108 000 learners from across South Africa were tested in the Foundation Phase Systemic Evaluation undertaken by the Department of Education in 2001, although the number of schools is not specified in the draft report (DoE, 2002b). All three of these studies employed random sampling, which meant that learners from a wide range of backgrounds and schools, from among the poorest in the country to the wealthiest and most advantaged, were tested. In all the other studies listed in the table, research agencies administered tests to all schools or samples of schools involved in an intervention or project. For example, the JET Grade 3 tests were administered in eight different studies to Grades 3, 4 and 5 learners in over 800 schools in five provinces. In the

35

part I Systemic school reform and research

Test

Domains

Language

TIMSS-R

Maths and science

English Afrikaans

8

9

194

8 147

MLA

Numeracy, literacy and Life Skills

English Afrikaans

4

9

400

10 483 10 759 10 438

Systemic Evaluation Grade 3 2001

Numeracy, literacy and Life Skills

All SA languages

3

9

not known

GTZ 3

Numeracy and literacy

English Afrikaans

3

1

39

1 027

GTZ Gr 6

Numeracy and literacy

English Afrikaans

6

1

39

1 052

JET Gr 3

Numeracy and reading

8 SA languages

3, 4, 5

5

735

18 185

JET Gr 6

Numeracy and reading

English Afrikaans

6 7

5 1

70

Eric Schollar

Maths, reading and writing

English Afrikaans

5

9

not known

5 000

Eric Schollar

Maths, reading and writing

English Afrikaans

7

9

not known

5 000

HSRC (QLP)

Maths, reading and writing

English Afrikaans

9

9

102

3 363

HSRC (QLP)

Maths, reading and writing

English Afrikaans

11

9

102

2 985

majority of cases the learners were from disadvantaged schools, that is, former DET schools, or, to a much lesser extent, former HoR schools in the Western Cape and Northern Cape. This does mean that the schools or learners come from a circumscribed sub-set of all schools and learners, as we shall see later. An exception to this pattern was the Western Cape Grade 3 Learner Assessment 2001 Study, which tested learners from a random sample of schools. Eric Schollar and Associates have administered mathematics and reading and writing tests in the many evaluation studies they have conducted in South Africa in the period 1998 to 2002. These evaluation studies include the Imbewu evaluation, the Business Trust 1000 School study, and various evalua-

36

Grade

Provinces

Schools

Learners

Table 3.1

OVERVIEW OF SELECTED STUDIES WHICH INCLUDED LEARNER ASSESSMENT, 1998–2002

108 000

1 144

tions of the READ project, the Mpumalanga Primary Science Initiative and many others. One set of tests has been administered to over 5000 Grade 3, 4 and 5 learners in nine provinces and another set to 5000 Grade 6 and 7 learners in nine provinces. The table shows that only the JET Grade 3 numeracy and reading tests and the Grade 3 Systemic Evaluation were administered in the mother tongue of the learners. In all other cases the tests were administered in either English or Afrikaans.

Issues in test construction Most of the tests used in the school-based studies conducted in the period 1998–2002 focused on reading and mathematics. Exceptions to this were the TIMSS-R, which

chapter 3 Lessons from learner assessment

tested mathematics and science, and the MLA, which tested Life Skills as well as literacy and numeracy. In all cases problems were encountered in the construction, validation and standardisation of the tests. Construction problems arose from five major sources: under-specification of curriculum, multiple specifications, lack of performance specification, learner performance levels and context.

Under-specification of curriculum content To allow a claim of validity, any learning assessment instrument must assess a specific content set at a level of cognitive demand appropriate to the learners. The ‘appropriate’ content for system-wide assessment or assessment aimed at understanding the health of the school system is best derived from the intended curriculum of the country. In South Africa the under-specification of the knowledge content in Curriculum 2005 documents (see Chapter 5 for further discussion) has bedeviled test construction in the GET band. The Curriculum 2005 Specific Outcomes (SOs) and Expected Levels of Performance (ELPs) do not provide the level of specificity required for the development of systemic assessment instruments. In response to this under-specification, test designers developed alternative curriculum frameworks within which to locate their assessment instruments. The most ambitious of these constructions was the JET Grade 3 and Grade 6 tests in which international benchmarks were superimposed on the South African curriculum outcomes. In the case of the numeracy instruments, Dutch numeracy outcomes were used, and in the case of the reading test, the International Association for Educational Achievement (IEA) framework for nine-year-olds was used. National and provincial Department of Education officials and teachers familiar with the context of a wide variety of schools commented on the alignment of these frameworks with the South African curriculum and therefore their suitability for the South African situation.

The process of establishing the correct ‘fit’ was time-consuming and laborious. The Revised National Curriculum Statements discussed in Chapters 1 and 5 have greatly expedited the construction of tests, as in the case of the WCED system-wide testing at Grade 3 level, conducted in October/ November 2002.

Multiple content specifications International comparative studies that include learner assessment face a problem different from the under-specificity of the curriculum. They have to construct an assessment instrument from the intended curricula of a range of countries. The test constructors for the TIMSS-R and the MLA studies developed overarching curriculum frameworks from which to develop test items. The TIMSSR used the ‘old’ South African curriculum, Report 550, because Curriculum 2005 had not yet been implemented in Grade 8 at the time of the TIMSS-R. The challenge for both studies was to ensure that the instruments used for assessment did not unduly advantage learners from some countries at the expense of the others. Analysis of the TIMSS-R items shows that 82% of mathematics items are covered in the South African Grade 8 curriculum, while only 42% of TIMSS-R science items are covered in the South African science curriculum (Howie and Hughes, 1998). This latter figure is a worrying index of the degree to which the South African curriculum falls short of international benchmarks.

Lack of competency specification Competency specification refers to the kinds of competencies that learners are expected to demonstrate whilst engaged with the content. Neither the old curriculum, Report 550, nor Curriculum 2005 specifies these to any degree. TIMSS-R recognises six main categories of competency expectations in mathematics: knowing, using routine procedures, investigating and problem solving, mathematical reasoning, proportionality, and communicating.5

37

part I Systemic school reform and research

Learner performance levels The fact that South African learners perform at levels way below the implicit requirements of South African curriculum frameworks has had a profound effect on test construction, validation and standardisation. In the QLP study Focus on Four (Reeves and Long, 1998) and TIMSS-R, South African learners barely scored on tests based on South African curriculum frameworks. Consequently, these assessment instruments have not been able to tell us very much about what learners know, other than that they are not competent at the levels of the curriculum intended for their age or grade. JET met the same problem in the development of the Grade 3 literacy and numeracy tests. The results of the pilot testing of the numeracy instrument indicated that the tests were much too difficult for the learners in the schools tested, and that the potential for diagnostic analysis was thus limited. Additional Grade 1 and 2 items were added to the test and some of the more difficult items were dropped. This allowed a range of items to be included, not only to ensure that learners at the bottom end were registered, but also to ensure that comparisons could be made with learners in other parts of South Africa and the world. But the consequence is that the JET Grade 3 numeracy test reflects the South African Grade 2 curriculum rather than the Grade 3 curriculum. The pilot JET Grade 3 reading test consisted of a sentence completion task and six reading comprehension passages. The results showed that the sentence completion items discriminated well for the learners but that the reading passages were too difficult. In the post-pilot adjustment of the test the four easiest passages were chosen. Furthermore, because a significant number of learners scored zero for the reading test, an additional task (word recognition) was added to the test for the main study. These items were added to ensure that the test could be used to report on the reading ability of all learners. The results of the Grade 3 tests showed that learners were performing below the

38

curriculum requirements for their grade (JET, 1999). This suggested that it would be inappropriate to administer Grade 6 level tests to learners in the same schools. This supposition was supported by the pilot study conducted in schools in the Northern Province. As a result, the Dutch numeracy learning outcomes for Grades 4 and 5 were used to develop the JET ‘Grade 6’ numeracy test. In the case of the reading test, JET decided to use the Grade 3 reading test to assess the reading levels of Grade 6 learners in the research schools. Because the language of teaching and learning in the schools from Grade 4 is English, JET administered the English version of the Grade 3 reading test to Grade 6 learners. The HSRC tests, designed for the QLP evaluation, used Report 550 to develop their Grade 9 and 11 tests. The test items were developed by a team of practising teachers from a variety of contexts and overseen by the appropriate subject associations. Despite this informed input, the learners fared poorly on the test, to the extent that it is not possible to know at what levels these learners are operating. This section has shown that all the tests reported here had to be adjusted downwards in order to register performance differentials between all the learners, and thus obtain a distribution of scores that allows discrimination between their performances.

Context All assessment tasks face the challenge of taking into account how different contexts may affect (aid or disadvantage) different learners. This challenge is particularly difficult in language (reading and writing and maths wordproblems). In South Africa the huge inequities in society and the different languages and socio-economic settings of learners exacerbate this problem. The development of each of the tests described above included an attempt to take account of these challenges.

chapter 3 Lessons from learner assessment

Issues of validity and reliability We reported in the previous section that test constructors in South Africa have attempted to address issues of content and construct validity in four ways: • Superimposing international curriculum frameworks on the Curriculum 2005 outcomes. • Using national and provincial DoE officials and teachers familiar with the context of various school types to formulate and moderate test items. These were criterionreferenced exercises aimed at establishing what learners can do at the end of certain grades. • Piloting the instruments in disadvantaged schools and adjusting them by including significantly greater numbers of items from lower grades, in order to ensure that a spread of results across the items would be obtained within the poorer performances. • Permitting learners significantly longer periods to complete the test; for example the JET Grade 3 reading test when compared with the IEA instrument. Despite these attempts to deal with the special problems in South Africa, there remain specific threats to reliability and validity. These include translation and administration threats.

Translation of the tests In South Africa the language of the test is a serious challenge to the validity of tests. Most tests have been developed in English but not translated into the first language of learners because of various problems. For example, the language used in translation may be technically correct but at a different level of complexity (easier or more difficult) than the language from which it is translated. Translations can also be ‘direct’ and not take account of the meaning and purpose of each question. Finally, words and phrases in a reading test can be selected for their relevance, use and phonetic and grammatical structure for, say, Grade 3 English speakers, but the translated words may not fulfil this same function. The JET Grade 3 test was

translated into eight languages, and attempted to address these problems. There were criticisms that incorrect words or spellings were used in these translations. However, investigation found that within a language, vocabulary and spelling of words varies from province to province and even from village to village. Thus a loop of forward-translation of an instrument from English into another language, followed by independent re-translation back into English, need not result in a true equivalent of the original English version. And, even if it did, there would be no guarantee that the other language version would be equally comprehensible across its own settings. These and other complications of translation have had the consequence that, at levels above the Foundation Phase (Grades 1–3), tests are generally administered in the languages of teaching and learning in South Africa, namely English and Afrikaans.

Familiarity with the format of test items Internationally, test administrators attempt to reduce the influence of item format by visiting schools ahead of the testing to introduce teachers and learners to the different types of questions which will be posed in the test. In South Africa this strategy was not possible for practical and financial reasons. In order to overcome the problem, the test administrators included a number of trial items at the beginning of the test to familiarise learners with the test format. However, learners’ familiarity with the test items remained uneven and may have affected reliability.

Length of the tests Because the JET tests aimed to maximise understanding of what learners can and cannot do, the amount of time allocated to each task was generous. This meant that Grade 3 learners often spent three and a half hours writing the tests, albeit with breaks in between. As a result the learners were exhausted by the time they completed the test. This might have affected their performance in the latter parts of the test.

39

part I Systemic school reform and research

Even when tests were of much shorter duration, administrators found that learners’ capacity to concentrate was often limited. The relatively short periods of time spent on learning activities in many South African classrooms, combined with the low cognitive/intellectual expectations of learners reported in many schools, may explain low concentration levels in tests. This may have consequences for inferences of reliability.

Late starts and interruptions to the testing The rural and remote nature of South African schools, combined with poor maps and signposting, meant that in some instances test administrators arrived at schools after the tests were scheduled to begin, even though they had often left their lodgings at very early hours. In other cases, schools had not been informed of the testing and valuable time was lost explaining to the principal and teachers what the testing was about and why it was being done. In other cases, administrators were often the only persons at the school at the official starting time of the school. Learners in some schools arrived well after the official starting time and this affected the administration of the tests. Learners in many rural schools go home at lunchtime, leaving school for over an hour. This practice also interrupted the testing procedure.

Learners’ low levels of literacy In the JET Grade 3 tests considerable time was taken by many learners completing personal details such as their first name and surname and the name of the school. In some cases the names were illegible and the test administrators were forced to re-write names. A common observation across grades was that learners had little idea how to spell the name of their school and had probably never seen it in print. Learners in primary schools also appeared to have little experience of test writing, as they did not turn the pages of the tests and had to be continually reminded to do this by the test administrators. This may have affected the reliability of the test results.

40

Poor conditions of schools In some of the schools where tests were conducted classrooms had no windows, doors or ceilings, while some classrooms had walls that did not go up to the ceiling. In these cases the noise from other classes and passing teachers and learners affected the testing. This was particularly a problem during breaks, as the school breaks did not coincide with the breaks in the test. There were some instances of mud or wooden shelters, and in a few cases the schools where tests were conducted had no buildings at all. Testing proved very difficult in these schools. Shortage of furniture in some schools meant that learners had to write the test kneeling. In the end, the contextual problems discussed above related mainly to rural and remote schools and not to the majority of schools. In addition, the problems were not serious enough to threaten the validity of the general diagnostic picture painted in the section on learner results.

Tests of reliability Statistical tests of the validity and reliability of test instruments have been conducted in few South African studies. One exception is the DDSP study, which used the JET Grade 3 tests. Vinjevold and Crouch (2001, 32) report that, in terms of statistical validity, the results of the tests are strong: “Most mean ratios correct are statistically reliable to approximately 1/2 of 1 percentage point (that is, the standard error is around 0.005) at the ‘big group’ level (province or language) and about 2 percentage points at the smaller group level (district)”. Thus, the authors conclude, the learner assessment instrument represents a statistically valid and rigorous baseline for DDSP itself. They also assessed the reliability of the Grade 3 JET instrument by estimating one reliability measure, Cronbach’s Alpha, which varies between 0 and 1. A value of 0 implies that there is essentially no correlation in how the learners interpret and answer the questions and suggests that the instrument

chapter 3 Lessons from learner assessment

has no coherence. In contrast, a value of 1 means that all items are “essentially equivalent to each other; there is perfect inter-correlation amongst all items, so any child that fails in one item will fail in every other item” (ibid., 24), so that the instrument has many redundant items. In short, the instrument has coherence but no substance. They conclude that “generally, one should be happy for a test of this nature to have a reliability coefficient of 0.85” (ibid., 24). This measure was calculated for the DDSP sample as a whole (14 700 learners in 453 schools) as well as by language group for the overall score, and separately for numeracy and literacy scores. Table 3.2 reports these measures. The lowest coefficient across the languages was 0.93, and the values are mainly from 0.95 to 0.97. Reliability was only a little lower in literacy than numeracy, and did not drop below acceptable levels when separate languages were considered. The scores indicate that the instruments are highly reliable.

Results of the tests Studies conducted in South Africa from 1998 to 2002 suggest that learners’ scores are far below what is expected at all levels of the schooling system, both in relation to other countries (including other developing countries) and in relation to the expectations of the South African curriculum.

Learner achievement in relation to other countries South Africa participated in two international comparative studies in this period. The Grade 8 mathematics and science study, TIMSS-R, was administered in 38 countries across the world, while the MLA was a Grade 4 study of literacy, numeracy and Life Skills in 12 African countries. In both studies South African learners performed well below the levels of their counterparts.

Third International Maths and Science Study Repeat (TIMSS-R) In the TIMSS-R the South African mean scores of 275 for maths and 243 for science are well below the international means (487 and 488 for 38 countries) and below the mean scores of all participating countries including two African countries, Morocco and Tunisia. Table 3.3 on page 42 reports the mean scores for mathematics and science for South Africa and ten other countries. Although the countries in the table are not in all respects similar to South Africa, they are selected here because they are comparable with South Africa on five economic/social indicators that are considered to influence education levels: per capita GNP, expenditure on education as a percentage of GNP, unemployment rates, life expectancy and adultliteracy rates (Martin et al., 2000; Mullis et al., 2000).

Language

Overall score

Numeracy

Literacy

Afrikaans

.99

.99

.97

N Sotho

.96

.96

.93

Setswana

.97

.96

.94

Xhosa

.97

.96

.96

Xitsonga

.97

.96

.95

Zulu

.97

.96

.96

All

.97

.97

.95

Table 3.2

ESTIMATING THE RELIABILITY OF THE JET GRADE 3 INSTRUMENTS: CRONBACH’S ALPHA

Source: Vinjevold and Crouch (2001).

41

part I Systemic school reform and research

Table 3.3

TIMSS-R MEAN SCORES (MATHEMATICS AND SCIENCE) Maths Mean score out of 800 points

Chile

Science

Std error

Mean score out of 800 points

Std error

392

4.4

420

3.7

Czech Republic

520

4.2

539

4.2

Indonesia

403

4.9

435

4.5

Korea

587

2.0

549

2.6

Malaysia

519

4.4

492

4.4

Morocco

337

2.6

323

4.3

Philippines

345

6.0

345

7.5

South Africa

275

6.8

243

7.8

Thailand

467

5.1

482

4.0

Tunisia

448

2.4

430

3.4

Turkey

429

4.3

433

4.3

International Average

487

0.7

488

0.7

Country

Source: Howie (2001).

The results of the mathematics test show that: • none of the South African pupils achieved the International Top 10% benchmark6 (616 out of 800 points) • 1% of South African pupils reached the International Upper Quarter benchmark (555 points)7 • 5% of South African pupils reached the International Median (= 50%) benchmark (479 out of 800) • 14% of South African pupils reached the International Lower Quarter benchmark 396 points (Mullis et al., 2000a) In the science test: • none of the South African pupils achieved the International Top 10% benchmark (616 out of 800 points) • 2% of South African pupils reached the International Upper Quarter benchmark (558 points) • 6% of pupils reached the International Median benchmark (488 out of 800) • 13% of South African pupils reached the

42

International Lower Quarter benchmark (410 points); (Martin et al., 2000)

Monitoring Learner Assessment (MLA) This study was commissioned by the National Department of Education, which participated for the first time in the Joint International Unesco–Unicef Monitoring Learning Achievement Project (Chinapah et al., 2000). More than 10 000 Grade 4 learners participated in the South African study, and they scored an average of 30% for numeracy. This mean was the lowest of the 12 countries. A large proportion of learners scored below 25%, while only about 2% obtained scores in the 75–100% range. In Life Skills, the South African learners’ mean was the second lowest, and the literacy mean the fourth lowest of the twelve countries. Though the Western Cape performed the best of all South African provinces, its performance was only marginally better than two of the participating African countries, and worse than the other nine. (See Figure 3.1 on page 43.)

chapter 3 Lessons from learner assessment

70% 60%

Figure 3.1

SOUTH AFRICAN PERFORMANCE ON THE GRADE 4 MLA NUMERACY TEST, 2000

50% 40% 30% 20%

Tunisia

Mauritius

Morocco

Botswana

Uganda

Madagascar

Mali

Malawi

Senegal

Niger

Zambia

South Africa

W Cape

0%

Mpumalanga

10%

Source: Chinapah et al. (2000).

Learner achievement in relation to South African grade level benchmarks We have seen above that learners in South Africa perform below their grade counterparts from other countries. Local studies show that South African learners also perform below the expectations of the South African curriculum. This is demonstrated by examining the results of some of the tests administered between 1998 and 2002.

QLP Grade 9 and 11 language and mathematics tests The QLP study conducted by the HSRC in 102 schools found that in the Grade 9 reading comprehension test, learners experienced problems in the areas of word recognition, language conventions and organisation skills. In particular, learners could not explain the meaning of words in passages or texts; they experienced difficulty finding clues in the text; they lacked skills for organising and accessing information in a contents table and for arranging words in alphabetical order; and they seemed uncertain about the use of inverted commas. In the Grade 11 reading test, “it was clear that learners did not understand the written

text due to poorly developed grammatical skills and vocabulary and poor knowledge of common idioms and idiomatic usage” (Kanjee et al., 2001, 48). For example, learners demonstrated limited vocabulary, poor knowledge of grammatical structures and lack of knowledge regarding the specific subject being addressed. The writing component of the Grades 9 and 11 instruments showed that across all districts the performance of learners on the tasks of ‘Providing Information’ was statistically significantly better than those in the ‘Creative Writing’ tasks. Learners were consistently better at providing information, for example filling in a form, than writing a creative text. The overall results of the Grades 9 and 11 mathematics tests portrayed “a very bleak picture regarding understanding and knowledge of Mathematics” (Kanjee et al., 2001, 60). In every district and across all schools sampled, the mean performance of schools was “unacceptably low” (ibid.). District mean scores ranged from 18 to 27%, a few learners scored above 50%, while none scored above 80%. Generally, learners demonstrated poor understanding of concepts in all topics of the mathematics syllabus, especially in geometry.

43

part I Systemic school reform and research

Both Grades 9 and 11 learners had “difficulty in answering questions where they were required to reproduce knowledge, which forms the basis of Mathematical questions. They had limited knowledge of mathematical rules and concepts. As a result of this, learners were not able to answer questions of higher order cognitive thinking, such as problems on the application of mathematical concepts” (ibid., 61). Further analysis of the Grades 9 and 11 tests indicates that learners had difficulties in simplifying numbers in exponential form; the majority of learners were unable to simplify algebraic fractions with a monomial denominator; most learners could interpret information from a graph but were unable to represent given data on a graph; learners either did not know the basic theorems and subject matter or were unable to apply the information to solve basic geometry questions.

JET Grade 3 and 6 numeracy and reading tests The poor performance in Grade 9 and Grade 11 suggests that these learners had not been exposed to basic reading, writing and numeracy skills. This is confirmed by the results of the tests conducted at the Grade 3 and Grade 6 levels. We saw above that the JET numeracy test was benchmarked at the Grade 2 level. This test, which consisted of four tasks (counting

and ordering, addition, subtraction and multiplication), was administered to over 18 000 pupils in 5 provinces. The results of these studies are presented in Table 3.4. In the numeracy instrument, it is striking that learners in all studies did considerably better on the addition and subtraction tasks than on the counting and ordering task. This unexpected finding may be explained by learners having an inadequate exposure to the structure of the number system, and a reliance on rote methods. Although the above data provides useful information on the learners’ relative strengths and weaknesses in numeracy strands, it does not indicate what numeracy content the learners had mastered. This information can be better provided by item analysis. The content and results of a selected sample of items serves to indicate the level at which the Grade 3 learners in the various studies are operating. Table 3.5 on page 45 shows the mean percentage correct for learners by district on a sample of 12 of the 30 items in Task 2, the addition task. The top row of the table provides the content of the item. The letter ‘C’ indicates word or application problems. The table shows, for example, that 89% of learners in the DDSP study were able to answer Item 1 (7 + 5 = ___ ) correctly while 44% solved Item 6 (70 + 24 = ____ ) correctly. In

Study

44

Province

Task 1 (counting and ordering)

Task 2 Task 3 Task 4 (addition) (subtraction) (multiplication)

DDSP

NC, LP, KZN, EC

16

34

29

WCED

WC

42

54

44

24 39

SAILI

WC

43

54

44

41

SDU

WC

34

46

36

31

Cadbury

WC

41

55

44

43

Mahlahle

LP

15

38

35

26

CIE

LP

26

51

39

39

Imbewu

EC

36

60

53

45

Phalaborwa

LP

30

50

42

34

Siyathuthuka

KZN

31

51

42

39

Table 3.4

AVERAGE PERCENTAGE CORRECT FOR THE NUMERACY TASKS, BY STUDY

chapter 3 Lessons from learner assessment

Study

7+ 5

18 + 7

42 + 34 + 16 + 6 8 50

70 + 30 + 50 + 165 + 240 + 400 + 37 + 24 9 C 60 400 60 205 37 C

DDSP

89

84

74

57

51

44

19

19

16

14

16

9

WCED

90

86

79

78

66

62

48

47

48

40

44

25

SAILI

92

88

82

82

65

64

41

46

41

36

37

20

SDU

88

79

68

70

56

50

42

41

39

32

35

18

Cadbury

91

80

76

71

65

58

48

54

48

40

41

30

CIE

92

93

87

70

64

62

33

42

38

31

37

21

Imbewu

92

91

88

81

73

67

45

54

48

53

49

29

Phalaborwa

95

92

85

81

71

60

43

43

38

32

37

12

Siyathuthuka

91

90

78

74

70

58

50

44

32

46

41

22

summary, the table shows that learners’ capacity to add one- and two-digit numbers decreased rapidly as the numbers involved in the problem increased in magnitude. Learners struggled particularly with addition of numbers requiring ‘carrying’ and with word or application problems. Large numbers of learners were not able to correctly solve items requiring ‘carrying’ or crossing over the number 100. Furthermore, a majority of learners used ‘concrete’ methods for both addition and subtraction tasks: for example, drawing 7 marks and a further 5 marks, to find the solution to 7 + 5. These ‘baby’ methods are acceptable in the early stages of numeracy but become an impediment when learners have to deal with larger numbers. This is well illustrated by the fact that learner performance fell off rapidly when tens, hundreds and thousands were encountered. It is clear that many learners are not making the transition to more abstract methods which depend upon a good understanding of the structure of the base 10 number system. Some learners did attempt to use shorter methods for addition of the large numbers (240 + 60), but because of their poor understanding of the number system, they wrote down either the problem or the answer incorrectly (for example, adding 6 and 2, instead of 6 and 4). The subtraction task was more difficult than the addition task for Grade 3 learners, and in some studies as many as a quarter of

Table 3.5

AVERAGE PERCENTAGE CORRECT FOR 12 ITEMS IN TASK 2, BY DISTRICT

learners were not able to subtract simple tens and units. The number of learners able to correctly solve subtraction problems decreased rapidly as the numbers used in the problems increased in size. Very few learners were able to solve problems requiring ‘borrowing’ or crossing over the number 100. Table 3.6 on page 46 shows mean percentage correct for learners by district on a sample of 10 of the 30 items in Task 1, counting and ordering. Again, the top row of the table provides the content of the item. Table 3.6 indicates that the learners found easiest those items requiring simple skipcounting. Learners found counting in twos easier than counting in fives and tens; and these items in turn were easier for learners than counting in 25s and 50s. Counting backwards was much more difficult than counting forwards, although counting backwards in 100s was easier for more pupils than counting backwards or skip-counting backwards in twos. The vast majority of learners were not able to answer questions on number lines even though these often required simple counting. Learners also had considerably more difficulty with items requiring application or interpretation of a diagram or word-problem than with ‘straight’ arithmetic problems. In the multiplication task many pupils could not distinguish between an addition and multiplication sign, so that the answer to 7 x 4 was presented as 11, and to 3 x 9 as 12. In general, pupils performed significantly

45

part I Systemic school reform and research

Study

34, 36, 38, ?

55, 60, 65, ?

80, ?, 60

90, Order 509, 250, 95, 424, 495, 300, 100, ? 516, 485 ?

DDSP

48

43

38

24

10

11

8

7

2

9

WCED

79

43

70

64

40

45

40

35

14

42

SAILI

88

83

75

71

36

51

35

25

9

31

SDU

70

66

66

55

28

35

31

30

10

35

Cadbury

76

69

712

58

39

44

41

37

14

47

CIE

73

61

57

39

38

33

24

8

4

15

Imbewu

78

75

63

56

43

42

29

31

19

10

Phalaborwa

70

61

64

41

42

29

25

13

4

25

Siyathuthuka

71

65

58

47

34

32

32

10

7

24

test. Many also used their fingers to assist in phonetically deciphering words but appeared unable to attach meaning to the sounds. The mean score in the IEA survey for Task 1 was 82.3%, and the mean score of the lowestscoring country (Indonesia) was 66.1%. It is important to remember that South African learners were given longer to complete the test and that significant numbers of them were 10 years or older when they wrote the test, compared with the norm of 9 in the IEA study.

better on the ‘straight’ multiplication problems than on the word- or application-problems. The reading test used by JET in a variety of learner assessment studies was adapted from the International Association for Educational Achievement (IEA) survey conducted in 27 countries with nine-year-old learners. However, the time limit was extended and the Grade 3 learners wrote the test in their mother tongue. Table 3.7 presents the average percentage correct for the three tasks for ten studies. The table shows that learners did best on word recognition (Task 1) and worse on sentence completion (Task 2) and comprehension of passages (Task 3). This is significant as it suggests that, in general, learners in these schools had only just begun to read. The test administrators reported that the majority of pupils had difficulty following the simple sentences used as examples in the

Variance in results between schools Learning achievement on all tests referred to in this chapter varies tremendously. For example, in the JET Grade 3 word recognition test, the highest mean percentage correct among the DDSP schools is 100 and the lowest is 15. Table 3.8 on page 47 shows the

AVERAGE PERCENTAGE CORRECT FOR THE LITERACY TASKS, BY STUDY Province Task 1 Task 2 Task 3

DDSP

KZN, EC, NC, LP

68

43

26

WCED

WC

84

66

38

SAILI

WC

82

56

30

SDU

WC

83

63

31

Cadbury

WC

83

67

38

CIE

NP

64

47

27

Imbewu

EC

80

57

37

Phalaborwa

LP

77

50

27

Siyathuthuka

KZN

86

59

34

Table 3.7

Study

46

570, 75, 1360, 98, 470, 73, 1370, ?, ?, 370, ? 71, ? 1380, ? 101

Table 3.6

AVERAGE PERCENTAGE CORRECT FOR 10 ITEMS IN TASK 1, BY STUDY

chapter 3 Lessons from learner assessment

Task

Lowest mean

Highest mean

Task 1

15

100

Task 2

6

94

Task 3

0

67

lowest and highest mean scores in the schools for the three tasks in all DDSP districts. The high variation in scores was evident not only across the schools in a study, but also within each district. This is a surprising result for these schools, which were purposely selected because they are all in relatively poor districts. Several aspects of this variation are to be noted. First, there is clearly considerable variation between schools across South Africa. Second, even within particular districts, where it is safe to assume that schools are relatively homogenous in terms of poverty and resources, there are some schools that perform twice as well as other schools. This does not happen in just one or two districts: it is a tendency in almost all the districts. It is important to emphasise the policy significance of this finding: that it may be worth assessing and disseminating what is already going on in the higher-performing schools within districts, rather than persisting with wholesale training or particular wholeschool improvement models (Vinjevold and Crouch, 2001).

Conclusion There are clear lessons for the South African research community from the experiences of assessment since 1998. The first concerns alignment between curriculum standards and the construction of test items. Tests will only validly measure systemic performance when they assess what the learners are supposed to have learnt in the curriculum. This imposes a requirement for specificity of content standards, which, as we have seen, the South African GET curriculum will only attain once the new NCS are implemented in all grades. The second lesson concerns the main threats

Table 3.8

HIGHEST AND LOWEST DDSP MEAN PERCENTAGE CORRECT, BY TASK

to reliability and validity in test format and administration. As administrators become more adept at negotiating the pitfalls of test administration, and schools get used to the regimen that testing requires, administrative threats to reliability and validity will diminish. Language will remain an obstacle to comparability whilst learners continue to display their current poor levels of mastery of the official language of instruction. Indeed, this is one of the very greatest obstacles to equitable learning, as we explain in Chapter 4. Lessons have also been learnt about learners’ competence in South Africa. The problems above notwithstanding, the local and international studies referred to in this chapter converge on a relatively clear, if disheartening, picture of systemic learner competencies. We know not only that they are performing far below the international benchmarks for their age and grade cohorts, but we also know much more clearly what it is that they know and do not know and are able to do or not do. This information must be more comprehensively and systematically monitored, and it must be more systematically used to inform systemic interventions targeted at improving performance. Finally, the most encouraging lesson from the assessment data we have at our disposal is that poor performance is not solely produced by external factors like background and resources. Schools which are otherwise the same, in terms of material and symbolic endowments, can and do have a major impact on the performance of their learners. What it is that they are doing right has yet to be systematically investigated and established. It is on this that an indigenous tradition of school improvement must be built.

47

part

II Explaining the outcomes of schooling Part I has introduced the encompassing idea of systemic school reform, and has examined the kind of research work that has been done to lend substance to our emerging knowledge base concerning the factors that have been found to have an impact on learning outcomes and learning achievement. In Part II we examine the research more closely for what it can tell us about the conditions for effective learning. Using the typology of research types developed in Chapter 2 as a framework, Chapter 4 examines what it is that large-scale descriptive and explanatory studies, as well as small-scale descriptive studies, are able to tell us about the influence of contextual, resource and educational process factors on learning. The chapter is concerned to address two questions. The first is: can schooling really make a difference to the learning of pupils from different backgrounds? The second is: what kind of information do we have, or do we need to have, to answer this question? The data available to us allows us to draw two major conclusions. The first is that class background and other contextual factors create a particular learning path for children. If this path is not deliberately shaped, it will have a major influence on the ultimate learning possibilities of children in the system. The second conclusion is that schools incontestably can and do counteract the predispositional effects of social background and con-

48

text. While generations of parents have made sacrifices in the belief that schools matter, and generations of teachers and other educators have ploughed their best efforts into schooling and made it happen, we have not had any reliable research evidence to bear it out until recently. Schools do this unevenly, and not to a degree that is reassuring, it is true. Nevertheless, we can now show that schools can and do make a difference. What is it about them that makes the difference? Chapter 4 tentatively advances the importance of language mastery, and the related dimensions of management, curriculum and pedagogy, concluding preliminarily that these must all combine optimally in order to maximize the amount, or ‘dosage’, of the intended curriculum made available to learners. What that optimal combination might be is the subject of Chapter 5. Chapter 4 shows that, given the current state of the knowledge base, it is difficult to go much beyond such broad-brush conclusions. The principal limitations here are both methodological and conceptual. Methodologically, as we also saw in Chapters 2 and 3, the smallscale descriptive studies provide us with richness and depth, but without any way of knowing how generalisable their conclusions might be, or in some cases, how reliable the data is. The large-scale descriptive and explanatory studies provide us with generalisable data, but often in categories that are of doubtful validity and with little way of

part II Explaining the outcomes of schooling

knowing whether different indicators are tapping the same construct or not. We conclude that what is needed is a theoretical framework that attempts to establish the core constructs conceptually, and, in interaction with the empirical data that we have available, to posit, by way of hypotheses, what we see to be the optimal relations between the constructs to maximize learning opportunities for all learners. This is the central task of Chapter 5. Chapter 5 is the centerpiece of the book. It looks backwards to the previous four chapters and sets out a first attempt to provide the conceptual cohesion that is absent in the school-effectiveness literature both in South Africa and elsewhere. This chapter generates a theoretical framework from four key dimensions of schooling, and concludes with a set of hypotheses derived from the framework and the existing empirical literature. These hypotheses create a tool for organising the discussions of new, and in some case existing, empirical data in the three chapter that follow, Chapters 6, 7 and 8. Chapter 5 begins by asking what it is about social background that makes a difference between learners even before they get to school. Developing the theme of language posited in Chapter 4, Chapter 5 employs the work of sociologists and the work of Basil Bernstein in particular to hypothesise that the main imprint that background leaves on the consciousness of prospective learners is a repertoire of orientations to meaning. Whereas all learners acquire a mastery of the particular ‘community code’ into which they are born and socialized, advantaged learners also acquire the rudiments of mastery of the ‘school code’, which is required for success at

school. The chapter makes an initial attempt to link the various proxies of social background to transmission and acquisition of the ‘school code’, and to link this in particular to familiarity with, and mastery of, the language of instruction. The main question that follows is: what then in schooling augments or diminishes mastery of the school code and the curriculum that it carries? Chapter 5 posits four main categories of schooling where this might happen. These are social organisation (the clarity of rules and tasks, ethos and relationships at district, school and classroom levels), knowledge and curriculum structure (the different structural forms of knowledge learnt at school and the kind of curricular form and degree of stipulation these require), pedagogy (the format and organisation of classroom learning and teaching), and evaluation (the degree of clarity with which evaluation criteria are stipulated and communicated). The chapter concludes by deriving ‘best guess’ hypotheses from these. We conclude this introduction with a caution that the synthesising efforts represented in this section are at an early phase of refinement. Our primary intention is to put something into circulation that can be confirmed, refuted, amended, refined or replaced. We are aware that, as it stands, it represents a rather crude attempt at model building. Nevertheless, crude as it is, it represents an attempt to capture the essence of the knowledge we have so far at our disposal and, in conversation with the most helpful conceptual tools at our disposal, to create a visible marker for our accumulating knowledge about schooling and its differential effects on learners from different backgrounds.

49

chapter

Factors which influence learner performance Introduction In Chapter 3 we described a wide range of variation in the results of tests designed to assess pupil performance in literacy and numeracy at the primary school level. We noted that, although the tests sampled a rather restricted portion of the total universe of South African schools, being concentrated towards the poorer, more rural part of the spectrum, they nevertheless show a very wide distribution of results. This chapter presents an overview of the types of factors that could explain these variations in learning outcomes in our sample, against the background of international findings. There is a considerable debate as to the most appropriate measures to be taken in promoting the learning that any society deems important. This debate is hardly surprising, given the huge resources expended by all societies on public schooling, but, although subject to intensive research for over three decades, agreement on its outcomes is still far from settled. In Chapter 2 we drew a distinction between three forms of design in schooling studies: descriptive, causal and process. We saw that descriptive studies can be largescale, appropriately sampled, using statistical methods to check for strength, reliability and validity of associations; or they can be small-scale, or case study in scale. We saw there too that case studies can be appropriate for investigating process questions, but that so too could larger-scale designs that check, by means of computer modelling, for example, which constellations of factors best explain the causal patterns found. Quasiexperimental designs generally make use of sampling and statistical analysis. Finally, Chapter 2 made the case that theory was needed to move beyond description to more sophisticated models with greater inferential power. An important conclusion is that different

50

4

methodologies and designs are appropriate to answer different questions in school-reform research. Unfortunately, methodology has become something of an ideological battleground, with camps that favour larger-scale work (either descriptive, causal or process), and camps who insist on the methodological primacy of smaller-scale and case study work (descriptive). Fuller and Clarke (1994) have referred to these two camps as ‘policy mechanics’ and ‘classroom culturalists’ respectively. While the former tend to search for universal determinants of effective schools, the latter are sensitive to local interpretations and the construction of meaning under particular social and political circumstances. While the validity of each perspective is dependent on the quality of their data, the two represent opposite ends of a spectrum in a number of important respects. With respect to the expertise required, statistical studies are dependent on high levels of specialised mathematical knowledge, while case studies require hermeneutical sensitivity – the exercise of judgement acquired largely tacitly through many years of experience in school and classroom situations. While the former are able to deal with very large volumes of data, correlating millions of observations across dozens of variables, case studies, because of their labour-intensive nature, generally deal with large amounts of qualitative data about a more restricted range of things. Consequently, the power of generalisation of statistical studies, if appropriately sampled, is generally high within the chosen population or sub-population, while the generalisability of case studies is usually low because the representivity of single cases is uncertain. As we saw in Chapter 2, the processes that determine the nature of the associations established by statistical studies are generally opaque, leading to the sometimes derisory

chapter 4 Factors which influence learner performance

label of ‘black box’ studies. For example, it is a long-established finding that the higher the level of education of the parents, the more successful the school performance of their children. However, the cause of this association is unclear without further investigation of the specific circumstances. What is the relative weight of factors like the fact that better-educated parents are generally higher earners, and thus able to afford better schools and additional tuition, and to limit the household demands on their offspring, enabling them to concentrate on their studies? To what extent is the association due to better-educated parents helping their children with homework tasks, or engaging them in more sophisticated levels of conversation, or structuring a more intellectually rich social and cultural environment? What is the role of genetics in the inter-generational transmission of aptitudes and attitudes, and why do some children seem to make so much more of their talents in the field of school achievement than equally gifted siblings and peers? Case studies, on the other hand, are sometimes, though not always, better at elucidating some of these linkages, albeit only on a microscopic scale. For example, while a careful case study may show how a particular teaching strategy adopted by a teacher for a specific topic in relation to her classes leads to the learning of a set of specific skills, there is no knowing, without testing generalisability further, how the same strategy would fare under different circumstances. In this chapter we analyse some of these issues with respect to a range of statistical analyses and case studies which have been performed in South Africa in the period 1998 to 2002. We then move to a discussion of research designs that attempt to combine the correlational power of statistical approaches with the insights generated by case study perspectives. We end the chapter with a set of suggestions for a comprehensive research programme aimed at better understanding the factors which influence learning in South African schools.

Large-scale descriptive studies While such methods have been utilised for decades in other countries, large-scale descriptive studies in the sphere of education are in their infancy in South Africa. Two subcategories of studies are distinguishable, according to whether their findings are generalisable to the whole population (population sampled studies) or not (targeted population studies).

Population sampled studies One of the more widely known studies of this kind in South Africa is that of Crouch and Mabogoane (2001). In a regression analysis of factors that co-vary with matric results in Gauteng and the Northern Cape, the authors found strong positive correlations with three factors: • the poverty index of the school • whether or not it was formerly administered by the Department of Education and Training (DET)8 • the qualifications of the teachers Less strong but significantly positive associations were found with what Crouch and Mabogoane term ‘strongly cognitive resources’ like books, the adequacy of media centre materials, and whether computers are used for instructional purposes. Factors such as pupil–teacher ratios (albeit a ‘cognitive resource’), conditions of the school buildings, and other resources had little or no correlation with learning in this study. After all these factors have been accounted for, some 20–30% of learning remains unexplained. The authors conclude that this component is due to a residual category to which they give the label ‘quality of management’. Implicit in this seminal paper are two of the three categories of factors which we identified in Chapter 2 as potentially affecting the quality of pupil learning: those which originate in the home and broader social and political milieu, and resource input factors. Crouch and Mabogoane did not investigate factors falling into our third category, namely, educational process factors, a category which

51

part II Explaining the outcomes of schooling

tion system that they hardly bear repeating. Nevertheless, it is important to characterise the precise nature of this inequality and to elaborate the mechanisms through which it is effected, in order to better illuminate strategies for its reduction and elimination. Anderson et al. (2001) and Simkins (forthcoming a) both describe a steady improvement in enrolment rates for all population groups since 1960, with rates for all groups now very close to 100%. However, there remains a very marked difference in terms of educational outcomes. Anderson et al. note that for younger cohorts, nearly 90% of whites have completed Grade 12, compared with only 35% of Africans. Similarly, pass rates in the matric exam are much lower for African students. In order to establish a measure for following children’s progress through school, these authors calculate the number of grades completed per year of school attended

has traditionally received little attention from the ‘policy mechanics’, and this is proving to be the case in South Africa as well. It is important to distinguish these three categories, as they are differently amenable to intervention and have very different implications for policy and practice. A number of individual factors may be recognised within each category.

Contextual factors Table 4.1 summarises the contextual factors associated with improved pupil performance identified by eight large-scale studies undertaken in South Africa. Race The fact of racial inequality, as well as its cause in the centuries of neglect in colonial times and the decades of skewed resourcing and oppression under apartheid, are such obvious features of the South African educa-

Factor

Crouch Van der and Anderson Simkins Simkins Simkins Berg and Thomas Mabogoane et al. (forth(forth(forthBerger Howie (1996) (2001) (2001) coming a) coming b) coming c) (2002) (2002)

Race Education level of parents or community Parental income or household wealth

++ ++

++ ++

+ (school) ++ (HE)

++

++

+

++

++ (school fees as proxy)

++

Settlement type

++

Family structure

+

Gender

0

Language and language of instruction

++

–

Table 4.1

CONTEXTUAL FACTORS ASSOCIATED WITH IMPROVED LEARNING

++

+ (rural location)

0

++

++

Key: ++ denotes strong positive correlation, + relatively weak positive correlation, – negative correlation found in another study, 0 no significant difference.

52

chapter 4 Factors which influence learner performance

for the different population groups. For Africans the value is around 0.80 grades per year for ages 10 to 16, while for whites the corresponding figure is around 0.94. Using an alternate measure – the change in schooling attainment with a one year change in age – Case and Deaton (1999) estimate an average advancement rate of about 0.6 per year for African children in the 10–18 age group. According to Anderson et al. these differentials are due less to African children dropping out in significant numbers than to a slower rate of advancement that begins in the earliest grades. While these descriptions mark important variations in the South African school system, they do not identify the root cause of why poor children advance so much more slowly than their more privileged peers. The problem is well illustrated by Crouch and Mabogoane’s (2001) finding that matric results correlate strongly with whether or not a school was administered by the DET. While we know that Africans still constitute the overwhelming majority in these schools, it is not clear what mix of social, educational and individual factors is responsible for their relatively poor results. We may tentatively conclude that ‘race’ is a proxy indicator for some other factor at work causing this pattern. Education level of parents or community in which school is located Using data drawn from the 1995 October Household Survey, Anderson et al. (op. cit.) found a strong positive correlation between the highest level of schooling attained by mothers and the educational progress made by their children: children at both 13 and 17 years old, whose mothers had 12 years of schooling, had advanced two grades further than children whose mothers had less than 4 years of schooling. Thomas (1996) found the following percentages of educational-attainment variance explicable in terms of parental education: 22% for whites, 35% for Asians, 35% for Africans and 37% for coloureds. Thus, the more disadvantaged any child, the greater the effect of parental education.

Surprisingly, Simkins (forthcoming a) found the association between the level of schooling attained by parents and that of their offspring to be relatively weak. An additional year of educational achievement by parents translated into about one-fifth of an additional year for Africans and coloureds, and about one-tenth of a year for whites and Asians. These figures are not necessarily in conflict with those of Anderson et al., if a mean gain is calculated for all age cohorts. Simkins (forthcoming c) found a stronger correlation at the level of higher education, where the presence of a household head with a university degree enhanced the chances of success of younger members of the household at tertiary level. Parental income or household wealth The strong influence of socio-economic status on pupil performance at school has been known since the Coleman Report was published in the 1960s (Coleman et al., 1966). These findings have been widely replicated in all countries in which they have been investigated, including South Africa. Crouch and Mabogoane (2001) found a strong correlation between family income and school performance in North-West and Gauteng provinces, and Van der Berg and Burger (2002) found a strong association between level of school fees, which they assume is a proxy for family income, and matric results in the Western Cape. Settlement type On the assumption that the type of settlement inhabited by people is a function of their mean income levels, occupational structures and institutional arrangements, Simkins (forthcoming a) investigated the relationship between settlement type and educational attainment, using data from the 1996 Population Census and the 1998 October Household Survey. He found that, for Africans, superior attainment was exhibited in decreasing order by people living in formal urban, informal urban, tribal rural, and commercial farm settings. Simkins concluded that

53

part II Explaining the outcomes of schooling

for Africans (the situation for coloureds is very similar) 30% of the variance is jointly explained in terms of settlement type, gender and age; for whites (and presumably Asians) the corresponding figure is only 6.7%. By contrast, Simkins (forthcoming c) found that, while settlement type did not exert a significant influence on Grade 12 enrolments in maths and science in the 1998 and 2000 matric exams, it had a marked effect on success rates in the overall exam results in both maths and science, with formal settlements in urban areas producing worse results than schools elsewhere, other things being equal. The seemingly contradictory conclusions reached by these two separate studies are possibly explained by the existence of more employment opportunities in the cities and the greater mobility of better matriculants from the rural areas. Or perhaps it may be that, while formal urban areas confer a distinct advantage in terms of overall prospects for educational achievement due to the higher socio-economic context, the matric results in these areas are significantly worse because of the higher levels of disruption in the high schools over at least the last two decades. Finally, it may also be that urban children have moved home more often, a factor that often correlates with poor performance in American studies (Dika and Singh, 2002). Family structure Noting that at age 12 only 55% of African and coloured children live with both parents, and that by age 17 this figure has fallen further to 50%, Anderson et al. (op. cit.) investigated the effects of family structure on schooling outcomes. Their conclusion is that children living with both parents obtain the best outcomes in terms of enrolment rates, number of grades completed, and number of years delayed (the difference between grade level attained and number of years spent at school). Those living with neither parent obtain the worst results. Those living with a single mother experience intermediate outcomes.

54

Gender Anderson et al. (op. cit.) conclude that schooling outcomes, in terms of grade-level attainment for males and females, are almost identical across all race groups, even among older cohorts. This is in contrast to the situation in a number of other African countries, but similar to the profile of Latin America. Simkins (forthcoming a) confirms this finding, although he notes that older Asian women are worse off then their male counterparts, with educational equality being a relatively recent gain in this community. Language use and language of instruction Simkins (forthcoming b) has investigated the issue of language in some detail, using the national matriculation results and the home languages of candidates. The first point he makes is that, in the matric results for 1998 and 2000, an average gap of more than 300 marks (of a maximum of 720) separated African first-language speakers from English or Afrikaans first-language speakers. This point requires two comments. First and most obviously, ‘African first-language speaker’ here is a proxy for race, with all the attendant implications of centuries of discrimination, neglect and poverty. As such, language is a social factor. Second, and almost as obvious, is that, given that the medium of instruction in all African schools is English at the matric level, African pupils have the added handicap of having to study in a second or third language. It is highly probable that the difficulties associated with studying in a language other than one’s home language are more pronounced in subjects like maths and science, which are strongly dependent on technical languages, proficiency in which is dependent upon prior proficiency in the language of instruction. Howie (2002) found a strong association between achievement in mathematics and proficiency in English. Those that spoke English or Afrikaans at home also tended to achieve higher scores, while the achievement was lower for African home-language speakers. Exposure to English (through, for

chapter 4 Factors which influence learner performance

example, the radio), and the extent to which teachers and pupils spoke English in the classroom were also significant predictors of mathematics scores. What can we conclude from this survey of contextual factors? First, although there are some variations across the studies, it is clear that race, parental income, settlement type or home area, and family structure all affect schooling outcomes in South Africa. The question is what all these factors have in common. A significant attempt to theorise this issue was made by Coleman (1988), who reasoned that they were all indicators of what he called ‘social capital’, a composite construct which comprises three forms: trust (expectations and obligations as measured by parental expectations), information channels (as measured by parent–child and sibling interaction and inter-generational closure – meaning whether parents were friendly with their children’s friends), and norms and sanctions that promote the social good over selfinterest. A number of other proxy variables were also devised by Coleman, like parental education, (lower) number of siblings, number of moves made by the family, religious participation and time spent on homework. The core idea was that social capital was a composite resource facilitating action in the social structure (see Dika and Singh, 2002). It is clear that nearly all studies using background factors, up to TIMSS, have drawn

Teacher qualifications

Resource factors Table 4.2 summarises the resource or input factors associated with improved pupil performance identified by seven South African studies. Teacher qualifications Crouch and Mabogoane (op. cit.) identified teacher qualifications as being strongly correlated with matric results. Simkins (forthcoming c) found a more mixed result: while the

RESOURCE FACTORS ASSOCIATED WITH IMPROVED LEARNING Case Van der Crouch and and Case and Simkins Simkins Berg and Mabogoane Deaton Yogo Bot et al. (forth(forthBerger (2001) (1999) (1999) (2001) coming a) coming c) (2002) ++

+ +

Facilities Pupil– teacher ratios

0

Learning materials

+

++

++

+

+

Table 4.2

Factor

heavily on Coleman’s indicators, if not on his theory. Bourdieu’s (1986) notion of social capital, with some differences, likewise accounts for social capital in terms of network resources. The nub of the theory is that social capital maximises human capital. Coleman’s and Bourdieu’s theories of social capital are resource theories. All the proxy indicators signify denser networks that amplify the ‘goods’ available through such channels. More social capital means more goods. But how do such goods translate into cognitive advantage? What is it that is ‘carried’ in the enriched networks that turns children, let us say equal in all other respects, into more able learners? In terms of the language we introduced in Chapter 2 we might ask: “What is the process factor at work here?” In Chapter 5 we will suggest an answer, which we derive from socio-linguistic studies of home and school.

+ (salary as proxy) +

+

Key: ++ denotes strong positive correlation, + relatively weak positive correlation, 0 no significant difference.

55

part II Explaining the outcomes of schooling

proportion of teachers with degrees in a school had a positive influence on the overall matric result, it seemed to have little or no effect on maths and science outcomes. Simkins speculates that this ambiguity may be due to the fact that there was no specification in the data as to whether the degree was relevant to the subject taught by the teacher or not. Facilities Bot et al. (2001) point out that there is considerable variation across the nine provincial departments on the indicators used to construct the ‘resource targeting list’ required by the legislation, which provides for the differential allocation of resources to schools on the basis of poverty (DoE, 1998b). Two of the most common include pupil–classroom ratios and condition of school buildings. While the authors make no comment on the fact that no obvious list of resource indicators occurs across the provinces, it is striking that there is such wide disagreement over which resources are most important to good schooling. Furthermore, while Simkins (forthcoming a; b) has found a moderate relationship between resources and pupil performance, Crouch and Mabogoane concluded that physical resources make little difference to the quality of learning outcomes. These mixed findings may be due to the fact that it is not resources per se that make the difference, but rather how they are utilised by school management. As we will also show later in this chapter, poverty per se is not a predictor of outcomes, and many poor schools regularly outperform richer ones (Van der Berg and Burger, 2002). Pupil–teacher ratios This is another area in which findings are not completely consistent. Case and Deaton (1999) report that lower pupil–teacher ratios have large positive effects on school quality for Africans, as measured by enrolment and school achievement, as do Baxen and Gilmour (2000), and Case and Yogo (1999) identify smaller pupil–teacher ratios as being associated with large and significant returns to education for Africans, as indicated by

56

higher rates of employment and higher earnings. Crouch and Mabogoane (op. cit.), on the other hand, found no significant correlation between these ratios and learning outcomes. Learning materials As mentioned above, Crouch and Mabogoane (op. cit.) found a moderate positive correlation between learning materials and matric performance. There is little indication in these factors as to what it is pedagogically about resources that affects learning. At this point, Crouch and Mabogoane’s surmise that cognitivelybased resources are more influential than non-cognitively-based ones is persuasive for us, and we will go on to develop that line of argument in Chapter 5.

Educational process factors Large-scale descriptive studies have far more to say about the association of contextual and resource factors with educational outcomes than they do about educational process factors. This is largely a data problem: while systematic data is collected regularly on a wide range of social indices through the Population Census and the October Household Survey, the analogous instruments in the educational field – the Department of Education’s EMIS data base, the School Register of Needs and the Education Atlas – only collect information on physical facilities and teacher characteristics. Thus there is no systematic data on two kinds of factors that are likely to have a major influence on both the quality and quantity of schooling outcomes, namely, management at district and school level, and classroom pedagogy. Large-scale descriptive studies which are population sampled are in their infancy in South Africa and a number of anomalous results have not yet been resolved. Their cardinal importance lies in their ability to be generalised to the population or sub-population under study. This feature arises from the representative nature of their samples, both for learning outcomes and for the factors which determine learning.

chapter 4 Factors which influence learner performance

Targeted population studies A second subcategory of large-scale descriptive studies may be distinguished, namely, those based on sub-populations which are not representative of the population. These take as their object of study a set of schools allocated by the province, usually for purposes of intervention, and generally from the poorest districts. A good example of such a study is the attempt to explain the variation in pupil performance in DDSP schools (Vinjevold and Crouch, 2001; Perry, 2002; Crouch and Perry, 2002; and Khulisa, 2001). In Chapter 2 we described the schools included in reform programmes such as the DDSP as predominantly poor, rural or township, and formerly falling under the DET or homeland administrations. These schools remain overwhelmingly black, and at the high school level generally fall into the bottom 70% in terms of performance on the matric exams. This is not to say that they constitute a homogenous group. Indeed, it has become a well-known feature of the South African educational landscape that a number of schools of this kind consistently over-achieve relative to their location and SES, existing side by side, under what appear to be very similar conditions, with schools of average or belowaverage performance. The starting point for Vinjevold and Crouch (op. cit.) is the observation that there is a great deal of variation in learner performance among DDSP schools. The authors illustrate this variation in a number of ways, perhaps most starkly by showing that within all but 2 of the 17 DDSP districts – spread over 4 provinces – the ratio between the average performances on the JET literacy and numeracy tests for the best and worst performing schools is at least 2:1. In 6 of the districts it exceeds 3:1. The aim of their analysis is to identify factors associated with this variability, by running regressions against five schoollevel factors: pupil–teacher ratios, teacher experience, the number of years of training of the teachers, frequency of assessment, and types of learning material used. The authors found that these factors are associated with

only about 10% of learning variation in numeracy and 7% in literacy, which puzzled them, considering that some of the factors in Tables 4.1 and 4.2 have been found to explain 60–70% of variation in learning. Vinjevold and Crouch conclude that the failure of their model to produce strikingly significant results may be due to one of two causes, or both: the model may be incorrectly specified in the sense that the most important determinants of learning were not included as factors in the analysis, or the range offered by DDSP schools in terms of the total variability across the country or even within each province is limited, a significant limitation of this kind of design. A third kind of explanation, not discussed by the authors, is that at least some of their data may be of dubious reliability. This may apply particularly to information on the frequency of assessment and the types of learning material used, which was obtained through administering questionnaires to teachers and principals. Perry (op. cit.) set out to test the first of these possible explanations, by including a wider range of factors in the regression equation. She calculated an index of school condition and socio-economic status from four factors derived from the 1996 Population Census (proportion of households in the enumerator area of the school without electricity, proportion without water, proportion functionally literate, and economic dependency ratio) and four from the 2000 School Register of Needs (availability of water at the school, availability of power, condition of school building, and pupil–classroom ratio). She found that the index accounts for less than 1% of the variance in the DDSP test scores. Perry offers the same diagnosis for these extremely weak results as Vinjevold and Crouch: It appears as if factors which we have not been able to measure in these studies (such as educator and school manager know-how as well as household factors which are more complex than average community wealth) are far more important amongst these schools

57

part II Explaining the outcomes of schooling

in determining learner achievement in Grade 3... Similarly, if all schools in the country were measured (and not just schools in poorer socio-economic districts) it is likely that school inputs and socioeconomic conditions might have more of an impact on learner performance. (Perry, 2002, 2) In a third attempt to probe this problem, Crouch and Perry (2002) cast the net wider, regressing learner performance against teacher’s subject knowledge, ascertained by means of testing the teachers, and a combination of the factors used by Vinjevold and Crouch and by Perry above. Once again the analysis failed to show any meaningful relationships, and once again the authors conclude that they have failed to include the most important factors which impact on learning. Khulisa (2001) examined the factors residing at district, school and classroom levels in the DDSP. The authors also found no evidence of the impact of school management on teaching practices, or of teaching practices on learner performance. However, they acknowledge two factors which undermine confidence in their data. First, data collection was managed by the service providers contracted to the project in the four respective DDSP provinces. Not only did the service providers have a strong interest in the outcomes of the study, hence undermining the objectivity of the data, but this practice almost certainly resulted in a lack of consistency in the training, management and quality assurance of fieldworkers. This is a difficult issue for a project of this kind and, within the available budget, these practices were probably unavoidable. Second, some of the fieldworkers used to perform the classroom observations had no education experience. This is a serious problem, given that the assessment of the quality of classroom practices is heavily dependent on the expert judgement of the observers. For example, ‘using a variety of innovative learner-centred teaching techniques’ was one of the elements in the defini-

58

tion of Good Teaching Practices. Defining good pedagogy in this way does not assist fieldworkers, especially those not practised in judging the quality of the knowledge transactions which occur between teachers and pupils, to distinguish between empty forms of learner-centred pedagogy and those which offer substantive opportunities for learning. Under these circumstances we would not expect ‘good’ pedagogy to necessarily correlate with learning. The authors are aware of this problem, pointing to what they refer to as possible researcher bias in the measurement of a number of indicators. Taken together, it is again not surprising that the effects found were weak. The Khulisa report on the DDSP constitutes the baseline of an intended impact evaluation. A baseline study of the QLP for the same purpose was undertaken by the HSRC (Kanjee et al., 2001). Much of the QLP baseline data was gathered through self-report procedures, although at district and school levels observations of key systems were conducted, while the classroom data was obtained largely through structured observations without any of the major problems identified in the DDSP study. Thus, although the QLP conclusions must be treated with some circumspection, slightly higher levels of confidence can be attributed to the findings than those of the DDSP baseline study. The QLP baseline used Hierarchical Linear Modelling (HLM) to analyse the relationships between learner performance and a number of factors at district, school and classroom levels. Table 4.3 on page 59 summarises these results. Language again emerges as a key issue, being associated with learning in three ways: • Significantly better results are obtained when the home language of the teacher is the same as that of the language of instruction. • The same effect was observed with respect to the home language of learners. • Learners whose home language was not the same as the medium of instruction obtained better scores if they spoke the medium of instruction often at home.

chapter 4 Factors which influence learner performance

EFFECT ON LEVEL District School

Learner

FACTOR Time allocated to subject Physical resources Teacher qualifications Home language of teacher Absence of teacher Home language Gender

Maths

Language

0 + + + + + + +

0 + + + + – + 0

Table 4.3

FACTORS ASSOCIATED WITH LEARNER PERFORMANCE IN THE QLP

Key: ++ denotes strong positive correlation, + relatively weak positive correlation, – negative correlation, 0 no significant difference.

No district-level effects are discernible, possibly because of very low levels of functionality of these institutions (see Chapter 8). Interestingly, learner performance in language is positively associated with higher levels of teacher absenteeism. The authors speculate that this may be due to substitute teachers allowing learners to get on with work in their own time. If further research confirms this as a robust finding then it would be a damning indictment indeed of the effectiveness of these teachers. Males generally perform better than females in maths. Simkins and Patterson (2002) supplemented the HSRC analysis, searching for relationships between social and economic factors and learning outcomes in QLP schools. Use was made of data drawn from a questionnaire administered to pupils in 102 QLP schools spread across all 9 provinces, the 1996 Population Census, matric exam results for 1999 and 2000, and maths and English tests written by Grade 9 and 11 QLP pupils.9 The data on the education of parents proved to be unusable, with the average education of mothers and fathers reported in the pupil questionnaire coming out at about double that reported in the Census, a striking illustration of the problem of self-report. The authors consider this to be a major blow to the analysis, given the known importance of parental education as a covariate of pupil performance.

However, a number of significant relationships were identified, the three most significant being: • In the area of language use, Simkins’ finding (forthcoming b), noted above, that pupils whose home language was not the language of instruction are at a considerable disadvantage, was replicated. Significantly for parents wishing to improve the school performance of their children, this disadvantage can be offset somewhat if the language of instruction is spoken sometimes at home, and it can be considerably offset if the language of instruction is spoken often at home. • Scores improve with increasing numbers of meals. • The provision of simple, inexpensive study aids by parents is also associated with improved learner performance. Interestingly, increasing household wealth, which for the sample under study occurs within a relatively narrow spectrum of the enormous variation in wealth across the South African population, does not correlate significantly with better test achievement. The authors underline the fact that these results are not generalisable to the South African population, since the sub-population of QLP schools was not drawn in any systematic manner.10

59

part II Explaining the outcomes of schooling

60

Small-scale descriptive studies

School ethos

Up until about 1997, the case study method held almost invariable sway among educationists in South Africa, with the few largescale descriptive studies that had been conducted being done largely by researchers based in other disciplines, notably economics, sociology and statistics.11 Small-scale descriptive studies generate insights which are enormously illuminating and useful, particularly in the arena of educational practice, but the problem of their generalisability, and hence of the implications of their findings for policy, is one which continues to beg methodological resolution. One example of the method is given by the research commissioned by the Department of Education in 2000 (Malcolm et al., 2000). The study set out to build on the work conducted by the then Foundation for Research Development (FRD) in 1995, which, according to Malcolm et al. (ibid.), found that ten former DET schools accounted for 20% of the total number of African students doing mathematics and science at universities. The goal of Malcolm et al.’s Feeder Schools project was to identify the characteristics of South African schools which produce strong learning performances under adverse conditions. Nine schools in four provinces were chosen, using four criteria: they were former DET schools operating in disadvantaged areas (urban and rural); they exhibited enrolments of more than 20 in Higher Grade physical science; they boasted pass rates of more than 80% in the same subject; and the sample had to contain schools from a number of different provinces. One ‘negative case’ (following Lincoln and Guba, 1985), a poorly performing school, was included in order to assist in deciding which characteristics define positive cases and which ones are more general. This research, based on interviews with principals, teachers, students and parents, draws conclusions about the characteristics of ‘the successful school’ in four categories:

The successful school sees itself as largely responsible for its own development. It succeeds through its faith in people and the human spirit. Staff and students describe themselves as ‘the family’ or ‘the team’. Students talk enthusiastically of the dedication of their teachers and principal; teachers praise the motivation and discipline of their students. The principal and staff work creatively to circumvent problems of resources, build motivation, and involve families. Their strategies are based on a vision of the school that centres on learning and success. This vision is widely shared and alive in the school. The principal is not only the administrative head, but also ‘the father/mother’ – the leader in all aspects of school life. Alongside preparation for matriculation examinations, the school actively promotes broader learning outcomes and life skills: selfdiscipline, responsibility, concern for others and communication. Discipline is important – for staff and students alike. The school promotes self-discipline through motivation and school culture, clear rules and organisational routines. Self-discipline is complemented by external discipline and achieved through monitoring systems, rewards and sanctions.

Governance and community relations The principal enjoys the view of education as a three-legged pot, where the legs represent students, staff and parents. If any leg is weak, the pot will fall. Participation is valued. It is achieved through flow of information, devolution of responsibility, consultation and negotiation.

Learning The school sees matriculation as a game. The examination is the ‘big match’. Past papers indicate how it will be played. Success depends on teaching and practice with textbooks, past papers, and class-work. Everyone wants to win, and winning requires dedication and motivation. Grade 12 students and their teachers work long hours – from 07:00 through into the evenings. Students are

chapter 4 Factors which influence learner performance

members of study-groups, and many attend extra lessons on weekends and during holidays. Teachers know their subjects. They set learning goals and communicate them to their students. They relate the current lesson to previous lessons and future lessons. Classroom rules are clear, and work is orderly and business-like. The class is cohesive and friction low. The students take pride in being proficient in English. Even so, most of them find it helpful to work also in their own African language. Code-switching occurs from time to time in their classes, according to the teacher, the context, content and purposes. Informal discussions use a mixture of English and African languages. There are at least as many girls as boys studying science and mathematics. The students and staff do not see gender as an issue in their classes, or in the curriculum overall. The school, over time and through imagination and effort, has built a strong culture of learning.

Management Management’s task it to bring all of the aspects of the school together, and to provide an environment in which teachers can teach and students can learn. As well as providing leadership, the principal ensures that effective administration is in place. Punctuality and a strong work ethic are highly valued. The attendance of teachers and learners is monitored, extramural activities take place outside of school hours, and no time is wasted between classes. If time is lost due to unforeseen circumstances (such as floods or teacher illness) then it is made up after school. These findings accord with common sense as well as with international research findings (see Hofman et al., 2002) concerning the characteristics of the successful school which not only gives priority to discipline and good exam results, but also provides a caring, nurturing environment. However, this study cannot be said to have established beyond reasonable doubt that the above features are what make for success in even the majority of well-performing schools:

• Any number of these features may be shared by poorly performing schools. Nor is the design of the study sufficiently robust to distinguish those characteristics which are pivotal to good health from those which are incidental. One ‘negative case’ is not enough to throw into relief the pivotal and the incidental. • This is a restricted sample of successful schools. There may well be many roads to success, and a broader sample may reveal a different set of characteristics associated with well-performing institutions. In our discussion of large-scale descriptive studies above we described how, even in research involving large samples, if the sample does not cover a sufficiently wide spectrum of variation inherent in the population, the relationships between the factors may remain obscure. • The sample is too small to cover these considerations. As we saw in Chapter 2, the question of sample size has no simple rules. Much depends on the degree of variability of the factor under consideration in the target population, on whether or not differences across sub-populations are to be investigated (for example, whether the factors required for success are different for DET and ‘homeland’ schools), and on the confidence level required (Crouch, 2000). In the case of the ‘feeder schools’ study, given that only one sub-population is of interest (former DET schools), an appropriate sample size would probably approach 100, depending on the degree of variability of the factors contributing to school success within this group of schools. The findings of this study provide a useful working hypothesis as to which factors contribute to school success in former DET schools. Such a hypothesis would require comparative corroboration before we could accept it with any degree of confidence. The hypothesis receives some corroboration from a very similar study conducted on behalf of the Centre for Development and Enterprise (CDE) by Onwu (forthcoming).

61

part II Explaining the outcomes of schooling

The conclusions of this latter project are almost identical to those of the ‘feeder schools’ research. The CDE project introduced further design complications, however, by including institutions from no fewer than four former departments (DET, HoD, HoR and HoA) in its sample of 13 schools, thereby significantly increasing the potential for variability in the number of possible models of success. The resultant complexity, arising from possible permutations and interactions across this more complex field of enquiry, increased the sample size required to inspire confidence in the results. Adler and her colleagues (Setati et al., 2002) attempt to establish associations between pupil learning and teacher strategies by investigating how code-switching by the teacher between the first language of the child and English, the official medium of instruction, facilitates conceptual development in their pupils, for whom English is, at best, an additional language. However, the authors are unable to demonstrate a systematic pattern over time, either within the respective sets of language practices of teachers and pupils, or in the relationship between these two variables. The same issue is pursued in another aspect of the wide-ranging research project conducted by the same team, where the authors seek to establish shifts across time in teachers’ ‘take up’ of the form and substance of learner-centred practices (Brodie et al., 2002). This study simultaneously illustrates the power of the case study methodology to illuminate specific practices, and the difficulties involved in making broader claims for the findings of such studies. It is at its most informative when describing how individual teachers deploy the empty forms of learnercentred practices without offering their pupils opportunities for substantive learning. These are insightful descriptions which significantly advance the debate on pedagogy, and we will return to them and other aspects of this research in Chapters 5, 6 and 7 below. However, no evidence is offered for the conclusion that teachers’ practices have changed

62

over time, nor how these imputed changes correlate with a number of school-level factors, nor how the claimed changes can be attributed to the programme under evaluation. Hoadley’s (1999) study of teaching practices in four Grade 1 classrooms sets out to describe just these four classrooms and not to generalise beyond the sample. In its detailed observation of the use of time, the different teaching modes, the structuring of knowledge, the social relations within these classes, and the institutional practices, this study provides a rich picture of how learning experiences for these particular children are shaped. It is a very sobering picture, which strikes a number of chords with other descriptions of teaching practices in poor South African schools (see, for example, Taylor and Vinjevold, 1999). A number of general hypotheses may be extrapolated from this study for structuring a more effective learning environment, through, for example, the better use of time throughout the school, and a more explicit structuring of the curriculum, both at school and classroom levels. These latter suppositions are supported by the work conducted by Ensor et al. (2002). The latter study concludes that what happens in the classroom is strongly affected by school management practices – such as time management and curriculum pacing – and by policy and practices which occur in the district offices and higher, such as the existence of curriculum statements in schools. In the present chapter our incomplete survey of small-scale descriptive work in South Africa is sketchier in scope than our analysis of larger-scale descriptive studies. The reason for this uneven treatment derives from the differences in purpose of the two approaches. While large-scale studies are more amenable to meta-analysis, small-scale studies, because of their idiographic intentions of understanding a particular phenomenon within its own context (Babbie and Mouton, 2001), do not lend themselves as easily to an overview analysis. This is not to say that small-scale studies do not provide important general

chapter 4 Factors which influence learner performance

insights. For example, the distinction made by Brodie et al. (op. cit.) between the form and substance of child-centred pedagogy is critically important in describing the quality of classroom practices. Similarly, work such as that carried out by Ensor et al. (2002) and Hoadley (2002) is important in pointing to the relationship between school- and classroom-level factors in providing learning opportunities for children. We return to these issues in Chapter 5.

Can schooling make a difference? The history of the international debate on the influence of school- and classroom-level factors on the performance of children has been something of a roller coaster ride since the 1960s. Creemers (1996) gives a useful summary of these developments. Following an optimistic phase in the early 1960s, the Coleman Report (1966) heralded a period of pessimism concerning the ability of schools to have a significant impact on the educational success of children, which, as we have seen, Coleman concluded to be overwhelmingly determined by social capital. The Coleman findings provided fuel for those opposed to the school-reform movement and to research into school effectiveness from both the left and the right ends of the political spectrum (Teddlie and Reynolds, 2001). This opposition has been particularly vociferous and persistent from the left, whose arguments are based on the assumption that attempts to improve schooling for the poor blame the victim, and merely divert attention away from the more urgent and deep-seated need to eliminate the root causes of inequality. Thus, according to Slee and Weiner (1998, 6), “effectiveness models favour the privileged and punish the disadvantaged”. For Thrupp (1999, 10) the key factor in differential achievement between schools lies in the ‘school mix’, or social class composition: consequently the secret to improving performance lies in “reducing SES segregation between schools, although providing substantial amounts of additional resources to low-SES schools would also help”.

However, as some of the design and methodological complexities of research on schooling began to be addressed during the late 1970s and 1980s, an increasing number of effects were identified, and confidence once more began to grow about the extent to which schools make a difference to the educational opportunities of individuals. One of the strongest lines of evidence is that pupils within certain individual schools perform considerably more successfully than some of their neighbours, which labour under what are apparently the same socio-economic and resource conditions. Thus, Scheerens (2001, 361), summarising the findings since 1970, estimates that after controlling for socioeconomic conditions, about 10–15% of the differences in learning between students are associated with school effects in developed countries, while the comparable figure for developing countries appears to be around twice this figure. Teddlie and Reynolds (op. cit.) concur with Scheerens’ estimates concerning the contribution to learning of school-level factors, although they note some significant differences between countries, with figures for the USA (15–20%) slightly higher than those for Europe (8–12%). In a review of the evidence for the existence of school-level effects, Teddlie and Reynolds quote studies which show that, after adjusting for intake factors (SES and other contextual variables), differences in student performance across primary schools in the USA and Holland may vary by the equivalent of one or two full years of study. According to Fuller and Clarke (1994) these differential effects may be due to relatively large increases in school performance being achieved through providing basic conditions for learning in highly ineffective schools, such as the provision of books and increasing the length of instructional time, while the returns on such gross measures diminish with increasing effectiveness of the school (see also Scheerens, 2001). In South Africa, Van der Berg and Berger (2002) provide similar evidence for the effects of school-level factors on educational

63

part II Explaining the outcomes of schooling

the extent to which differences in school performance across the province may be related to the availability of resources (physical condition of the school building, pupil–teacher ratios and teacher qualifications), and to what extent the differences are associated with the more efficient use of resources (the residual after correcting for SES and levels of resourcing). Van der Berg and Berger conclude that a significant proportion of the differences in performance between schools is due to factors not accounted for in their study. These conclusions are supported by a national study by Van der Berg which concludes that “regressions show that only a small portion of the remarkable differentials in performance among poor black schools can be accounted for by the socio-economic background or teaching resources, pointing to the importance of school management”(2002, 23). On the one hand, these two South African examples support the by now overwhelming international evidence that schools do make a difference, but on the other hand, they confirm our earlier conclusions concerning the lack of detail on specific factors in the areas of

performance. Thus, Figure 4.1 shows that for six of the nine provinces for which data is available, many very poor schools perform above expectations with respect to matriculation pass rates, when compared with schools in the same socio-economic bracket. Indeed, the performance of a number of these schools is indistinguishable from that of the most privileged schools in the country. Van der Berg and Berger emphasise the point by showing the pass rates for the 37 poorest schools in the Western Cape, all of which, in 1997, set schools fees at below R100 per annum, served predominantly disadvantaged communities and worked under similar conditions of historical neglect (see Figure 4.2 on page 65). Although these schools draw their pupils from the lowest SES bracket in the province and work under similar resource constraints, they display very close to the full range of variation in matriculation results, with nine scoring above the provincial mean and one registering a 100% pass rate. After standardising for socio-economic status (as measured by means of school fees and the poverty index of the surrounding area), the authors explored

120

No. of schools

100 80 60 40

Pass rate (%) Source: Van der Berg and Berger (2002, 14).

64

95-100

90-95

85-90

80-85

75-80

70-75

65-70

60-65

55-60

50-55

45-50

40-45

35-40

30-35

25-30

20-25

15-20

10-15

5-10

0

0-5

20

Figure 4.1

FREQUENCY DISTRIBUTION OF PASS RATES AMONG PREDOMINANTLY BLACK SCHOOLS WITH SCHOOL FEES OF R30 OR LESS, 1999-2000, FOR SIX PROVINCES

chapter 4 Factors which influence learner performance

100%

Figure 4.2

MATRIC PASS RATES FOR THE 37 POOREST SCHOOLS IN THE WESTERN CAPE, 1997

90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

Schools arranged from worst to best Source: Van der Berg and Berger (2002, 15).

management and pedagogy which impact on learning. Indeed, not only is Van der Berg not able to distinguish between factors at the school and classroom levels, but he is unable to separate the effects of teaching resources from issues of management and instruction.

Conclusion The international literature on school reform has accumulated a very long list of factors at all levels of the system which have been found to be associated with improved learning. We will mention the strongest of these in Chapter 5. In the present discussion, we confine ourselves to some general observations on this massive volume of work. A striking feature of our brief survey of research on schooling in South Africa is the all-pervasive and extremely powerful influence of language, which stands out as the one factor which is not only unambiguously implicated in learning, but which also offers relatively clear policy lessons. These revolve around the need for pupils to have as good a grasp of the language of teaching and learning as is possible. Pupils who attend classes conducted in a language which is not their

first language are at a significant disadvantage. This disadvantage is accentuated when the language of teaching and learning is not the first language of the teacher. However, language obstacles can be ameliorated in a number of ways: • In teacher training, teachers require intensive instruction in the language of teaching and learning. • In the home, children benefit greatly from regular and sustained practice in the official language of the school. • At the school level, there must be both a conscious focus on the importance of language, and explicit policies and programmes to develop proficiency among staff, pupils and parents in the official language adopted by the school. Large-scale descriptive studies based on a sample which represents the whole school population are beginning to identify predictors of school success. Such studies have produced their most robust findings to date with respect to social and economic factors. Race, language, parental education and household wealth are all strongly associated with learning. In these areas, work in South Africa is

65

part II Explaining the outcomes of schooling

replicating what is known in other countries, and the implication for policy is that schools should be judged not according to raw scores on indicators such as the matric exams, but according to the value they add relative to their socio-economic circumstances. With the notable exception of language, socio-economic factors are not generally amenable to short-term intervention. However, the good news in this regard is that South Africa is in the process of transition from what Simkins (2002) has termed a situation of ‘waste’, characterised by high mortality rates where families have large numbers of children in order to survive, to a situation of ‘efficiency’, where parents restrict themselves to two or three children and concentrate their resources on giving these the best life chances possible, including a good education. Unfortunately, AIDS is in the process of wreaking havoc with these hard-won gains which represent decades of development. It is also well known that certain poor schools make very much more of the social and material resources at their disposal than the majority of their peers, and conversely, that many well-resourced schools are underperforming relative to their level of privilege. Identifying the mechanisms for effecting the achievements of successful schools has become a major research focus in South Africa. Large-scale descriptive studies have to date failed to make any progress towards this goal, largely because of a lack of data at school and classroom levels. Small-scale studies, on the other hand, which take these latter aspects as the principal focus for their research, have also not made much headway in revealing this secret, in part because they are too small in scale to generalise beyond the very particular circumstances of the respective case. In addition, small-scale studies generally do not focus on systemic or institutional issues, confining themselves largely to the practices of individual teachers. Where they

66

do look at institutions, they generally concern themselves with issues of institutional culture, rather than the systems and operating procedures which might characterise well-functioning schools. A new model of school research is emerging which combines large- and small-scale purposes and approaches. These mixedmethod, multi-level models are in a very early stage of development in South Africa, and teething difficulties in matters of design and methodology have not yet been overcome to the extent where they have begun to produce insights into the determinants of learning. However, the lessons for eliminating these obstacles are emerging. Chief among these are: • The sample under study must represent the entire spectrum of variation across the system. • Self-report data must be handled with great circumspection, and triangulated with data collected by independent measures wherever possible. • Observations of management and pedagogic practices must be undertaken by researchers who can exercise high levels of professional judgement in describing and assessing the quality of these practices. • When examining effects which cross different levels of the system, such as the influence of school management on teachers, or of teachers on pupil learning, care must be taken to link the behaviour of the subjects in the respective levels as closely as is practicable. This applies also to controlling for SES, data for which must be collected at the level of the individual child. But before studies along these lines can be effectively planned, a theory of schooling is required to give direction, coherence and focus to the search for factors that might maximise the potential of all children. It is to this that we now turn.

chapter

5

A social theory of schooling Introduction A constant refrain of Chapter 4 concerns the dearth of data in South Africa on school- and classroom-level factors that may influence learning. Where data does exist, it is less than conclusive. We concluded that this may arise from problems of theoretical conception (incomplete specification of our model of schooling), research design, or research methodology. The present chapter addresses the first of these issues, deriving a theory of

schooling in order to guide the search for factors that impact on school performance. The array of research projects sampled in Chapters 2 and 4 illustrate the complexity of the school system and the variety of factors which structure and differentiate access to education, resulting in significant differences in outcomes for different children. One of the most informative schemes for identifying these factors is the model used to frame the TIMSS study. This is illustrated in Figure 5.1. Figure 5.1

OUTLINE OF THE CURRICULUM PROCESS

System

National/ regional curriculum goals Intended curriculum

School

School goals

Official teacher certification qualifications

Teacher professional organisation & environment

System characteristics 1. Tracking 2. Grade levels 3. Content decisions 4. Related characteristics (national wealth)

School course offerings & instructional support functions

Student characteristics 1. Background 2. Household economic capital 3. Household cultural capital 4. Attitudes 5. Activities 6. Expectations

Teacher characteristics Teachers’ content goals

a. Background

Implemented curriculum

What are students expected to learn?

Classroom

Instructional activities

Test outcomes

c. Pedagogical beliefs

Implemented curriculum

Attained curriculum

Who delivers the instruction?

How is instruction organized?

b. Subject matter orientation

What have students learned?

Source: Adapted from Schmidt et al. (1997, 182).

67

part II Explaining the outcomes of schooling

The power of the scheme depicted in Figure 5.1 is that it not only shows all the main categories of factors discussed in Chapters 2 and 4, but it also places them in the terrain delineated by a grid, with the main levels of the schooling system on the vertical axis, and the principal phases of the curriculum cycle largely on the horizontal axis (though partly vertical). Illuminating as TIMSS’ Outline of the Curriculum Process certainly is, however, it is too cryptic to serve our purposes adequately in a number of important areas, including what may be among the most critical influences on learning, such as the leadership and management of institutions, and the delivery of the curriculum in the classroom. Each of these latter categories consists of a nexus of factors that requires further disaggregation and elaboration before it can be operationalised into a set of research questions. Furthermore, while the TIMSS framework does provide a sense of the alignment between curriculum goals (the intended curriculum), the delivery of instruction (the implemented curriculum), and pupil performance (the attained curriculum) required by the systemic perspective of schooling outlined in Chapter 1, it has no theory for understanding the relationships between the forces which shape the learning experiences of pupils. The TIMSS scheme is a contingent collection of factors – a list rather than a coherent conceptual framework for understanding the principles that pattern schooling. Collection schemes of this kind are unable to establish causal relations between their constituent factors, and hence are unable to prioritise among competing demands on resources, or to know with any confidence whether the list is exhaustive.

Starting points Theory provides the principles for understanding how the many factors that constitute any complex phenomenon fit together and produce their effects. For Bernstein (1996), a theory identifies the object of study, builds the conceptual syntax for revealing its

68

nature, and provides a language of description for characterising the various facets of that object. In postulating principled, as opposed to serendipitous, relationships between factors, theory enables causal links to be surmised, and a prioritisation of factors to be established. A good theory confirmed by good empirical data can thus provide a guide to more effective practice. There is a cyclical relationship between the theoretical and the empirical. On the one hand, theory provides categories for collecting evidence. It thereby guides the search for evidence, which in turn is used to strengthen or elaborate the theory. The priority at the present time is for the formulation of a conceptual map to guide the search for more detailed evidence of factors that influence learning. Such a conceptual map must be based on a logical understanding of the way schooling is structured, and provide a convincing explanation of the available empirical evidence. Concerning the structure of the school system, the TIMSS outline shown in Figure 5.1 has gained wide currency, and we adopt elements of this framework in our discussion below. Concerning the evidence which requires explanation, three categories of variables were identified in Chapter 2 and used to guide our analysis: contextual factors; resource factors; and educational process factors at district, school and classroom levels. This categorisation gives rise to three parallel assumptions on which our theory of schooling rests: • The socio-economic status of a child’s family has a very powerful influence on the educational experiences of the child. The most important proxy indicators so far used are: poverty levels of the family/caregivers, education level of parents or household head, and proficiency of the family/ caregivers in the language of instruction used in the school. • Resource factors – such as pupil–teacher ratios, teacher education level and cognitive resources available at the school – generally mirror the socio-economic status

chapter 5 A social theory of schooling

of the catchment area of the school, and are variably associated with pupil and school performance. • Management and leadership factors at school, district and higher levels, and classroom instructional factors can make a critical difference to pupil and school performance. These factors offer opportunities for education officials, principals and teachers to make more or less effective use of available resources, thereby facilitating or inhibiting the progress of children through the school system. The discussion below is most concerned with the last of these assumptions, namely, exploring ways in which educational process factors may optimise the learning experiences of all children, but particularly those disadvantaged by socio-political conditions. Those factors relating to socio-economic context or resources have been detailed in Chapter 4, and, aside from speculating as to how they fit into the overall scheme, we will not disaggregate them any further here. The most fully realised social theory of schooling, which implicates knowledge, language and social class, is provided by the work of Basil Bernstein. Bernstein’s overarching interest lies in understanding how individuals learn to exercise greater or lesser degrees of control over different kinds of language, and how social class differentially distributes access to mastery of the symbol systems that we use to mediate our interaction with the world. The lynchpin of Bernstein’s theory is the notion of a ‘code’, an orientation to classifying experience and creating meaning. This work has its roots in investigations of the language patterns of young school children (1971, 1977), and, in particular, the different ways in which the language patterns which children bring to school interact with the patterns demanded by the school. His observations on children’s language use in the classroom, and in response to structured experimental tasks, led Bernstein to describe two distinct patterns or codes. All children

make use of what we call a ‘community code’, which they use in colloquial situations with family, friends or peer groupings. Community codes make sense mainly in local contexts, and their use is learned largely tacitly. A second kind of code, what we call the ‘school code’, reflects a different perspective and a different set of classification principles, which transcend local contexts. The school code is an orientation to meaning for finding commonalities across specific contexts. Where the community code provides rich access to the layers of cultural meaning in the child’s home setting, the school code enables the child to make connections between particular and general classifications, and hence to link community to more general categories of meaning. The crucial point about these codes is that, while all children master a community code at home, middle-class children seem to learn the basics of the school code at home as well. Middle-class learners thus generally come to school with two coding modalities, while working-class learners come to school mainly with one. One of the best illustrations of this is afforded by an experiment performed by Holland (1981) in which seven-year-old children were asked to sort a set of cards showing pictures of items commonly found on the school lunch menu, and which they all could therefore associate with everyday experience. This was a classification task in which the children were asked to divide the items into groups. They were free to use any principle of classification they wished. In analysing the results of the exercise, Bernstein (1990) distinguished two broad kinds of response. The responses correlated well with the class origins of the children, as indicated by factors such as the educational level and professional status of the children’s parents, the number of books in the home, and the frequency of parents reading to their children. In classifying the items, working-class children predominantly used criteria drawn from their own experiences, such as ‘things I have for breakfast at home’ or ‘things I cook for my mum’.

69

part II Explaining the outcomes of schooling

The middle-class children, in marked contrast, were far more likely to use as their first principle of classification a conceptual element that the pictures had in common, such as ‘these things all come from the sea’ and ‘those are all vegetables’. In a second phase of the experiment the children were asked to regroup the items in another way. This time a significant number of middle-class children changed their principle from one based on a general concept to criteria drawn from local context and experience, while most working-class children merely used another reason based on their personal lives. In short, middle-class children generally12 have access to two principles of classification, one conceptually specialised and the other localised and personal. For these middle-class children the school context, where this research was carried out, signals that they should first apply the school code, and only resort to a community perspective when prompted. In strong contrast, working-class children generally display mainly a community orientation to classificatory tasks. Education tends to reinforce the coding orientations that children bring to school. Middle-class children have their school code orientation reinforced and amplified. Working-class children, on the other hand, with their inclination to using their community code, have a far greater distance to travel to acquire the school language code orientation and specialised principles of classification which match the structure of school knowledge. Bernstein’s work provides insight into the mechanism through which schooling hereby reinforces inequity in literate societies, by providing differential access to formal knowledge. It seems logical that reading and writing would play an important role in the learning of the school code. For Rose (2002, 2) “reading constitutes the primary medium for engagement in formal education; the ability to read with comprehension and to learn from reading is the foundation for most other activities in schooling”.

70

Bernstein (1990) gives three reasons for the importance of reading in acquiring conceptual knowledge. Good reading skills set children free to access a far wider range of knowledge than their teachers are able to offer; they empower children to pursue learning independently; and written forms of discourse are qualitatively different from oral forms. Highly literate parents, if they do not actually teach their children to read before entering school, at least orient them to written forms of communication and to an expectation of pleasure and enlightenment from books, through reading to them regularly and through the example of their own engagement with written materials. We would also expect books to provide a key resource for orienting children to elaborated codes. This is a major, perhaps the major, disadvantage for children from illiterate and poorly literate homes, and, if not explicitly addressed throughout the school grades, particularly in the early years, becomes a mechanism for their progressive further disadvantage. It is commonly observed that there is a very low prevalence of reading and writing in most South African classrooms, and, where it does occur, it is pitched at low levels of cognitive demand (Taylor and Vinjevold, 1999; Hoadley, 1999; Schollar, 1999; 2001b; Vinjevold and Roberts, 1999; Review Committee, 2000; Adler et al., 2002; Setati et al., 2002). This would seem to present an obvious explanation for the very poor levels of reading shown in Chapter 3. It follows that if reading and writing are to be promoted in classrooms, these knowledge resources need to be procured and distributed to schools, and managed within the school in order to be accessible to children. This is another area in which institutional dysfunctionality at all levels of the South African system undermines conditions conducive to teaching and learning, with millions of Rand remaining unspent in the book budgets of a number of provinces every year (MoE, 2001; 2002). Further, as Adler et al. (op. cit.) point out, it is not enough for these resources to be available; they serve no purpose in learning if they are not used in the classroom.

chapter 5 A social theory of schooling

Closely related to the issue of reading and writing in learning the school code is the question of language, and in particular, proficiency in the language of instruction for children whose home language does not coincide with the language of the school. Children are unlikely to make progress in the acquisition of conceptual knowledge if they have a poor grasp of the language used in their classes and textbooks. Setati et al. (op. cit.) have described how English is more akin to a foreign language for many rural teachers and pupils who have few opportunities to learn the language. Since the medium of instruction for black children is generally English, these children labour under an additional layer of disadvantage, as shown all too clearly by the findings of Simkins (2002), noted in Chapter 4, that student success in the matric exams is strongly correlated with frequency of English use in the home. These results are reinforced by the hypothesis proposed by Malcolm et al. (2000) that successful schools make a point of promoting the use of English both inside and outside the classroom. Although schooling reproduces social class in general, Bernstein (1990, 6–7) insisted that his theory was not deterministic, and did not depict an inevitable state of affairs. On the contrary, by making visible the way in which the principles of coding structure vary, the theory makes visible also the choices available to change it: ... the transmission/acquisition systems the thesis projects do not create copper etchings plates in whose lines we are trapped. Nor are the systems, grids, networks and pathways embedded in either concrete or quicksand. The transmission/ acquisition systems reveal and legitimate the enabling and disabling functions of power relations which they relay and upon which they rest. Attempts to model the internal principles of such transmission do not make them eternal. Such analysis may show the poles of choice for any set of principles and the assemblies possible within those poles. It calls

attention to the selective effects of transmission, their social costs and the basis for change. In what follows, we explore what the distinction between school and community code orientations means for the policies and practices of schooling. In this analysis we follow the TIMSS structure, which locates the elements of the school system by stage in the curriculum cycle (intended, implemented and attained), and by level of schooling (system, school and classroom). We are most concerned here with the processes of the intended and implemented phases of the curriculum cycle, which may lend themselves to differential attainment by children.

The intended curriculum In Chapter 1 we emphasised that the cornerstone of a systemic approach to schooling is the notion of alignment. Curriculum and assessment standards provide the central mechanism for aligning our values and goals for schooling with learning outcomes. They constitute the formal statements of the intended curriculum, setting the targets for achievement by defining a set of cognitive and socio-affective competences. The dominant ideals of any society are reflected in the intended curriculum. Not surprisingly, therefore, following the institution of the first democratic government, South Africa adopted a curriculum model for the first nine years of schooling that was radically different from the one that had been in place under apartheid (DoE, 1995; 1997). A strong transformational stance was adopted, emphasising equity, democracy and human rights (see also DoE, no date). This proved to be the symbolic break with the past that the South African education community had been waiting for, and was greeted with widespread enthusiasm and commitment (Review Committee, 2000, 16). The Review Committee was established by the Minister of Education in 1999 to investigate the health of the new Curriculum 2005. The Report distinguishes between two kinds of patterning that any

71

part II Explaining the outcomes of schooling

curriculum must deal with: knowledge structure and curriculum structure.

Knowledge structure Bernstein (1999) distinguishes between two fundamental forms of symbolic structure. Horizontal discourse is the common sense arising out of the daily experience of living and dying, such as learning to dress, or to address people in particular ways. Its specific forms are highly dependent on context: for example, child A’s mother may cook fried fish for supper on a Friday, while child B’s mother may cook chicken breyani. Indeed, such knowledge only makes sense within a particular context: thus the form of address adopted by a child depends on whether she is speaking to her grandmother or to a close friend. It may be contradictory across contexts, but not within a context. Horizontal discourse makes sense in the home, the school playground, in peer group activities. It is generally learnt in oral communication, or tacitly by modelling. According to Bernstein (1999, 160), it entails a set of strategies for maximising encounters with persons and habitats, and it acts as a social and cultural relay for the community: “The structuring of the social relationship generates the forms of discourse but the discourse in turn is structuring a form of consciousness, its contextual mode of orientation and realisation, and motivates forms of social solidarity.” By contrast, any vertical discourse consists of a specialised symbolic structure of knowledge. Such a structure is an explicit, coherently organised set of principles. It makes sense across specific contexts. Bernstein distinguishes two forms of vertical discourse, depending on whether they are organised as hierarchical or horizontal knowledge structures. Physics is the archetypal example of a vertical knowledge structure, which attempts to create general propositions and theories which integrate knowledge at lower levels, and shows underlying uniformities across a range of specific situations. What is important in learning any hierarchical knowledge structure is mastery of the procedures of

72

investigation and instruments of observation, and understanding the theory. Vertical discourses which are organised as horizontal knowledge structures also consist of specialised languages, with specialised modes of interrogation and criteria for the construction and circulation of texts. Where they differ from hierarchically organised knowledge structures is that they do not attempt to integrate all knowledge into a single accumulated hierarchy. All too often, new knowledge entails the founding of a new, specialised language. The discreteness of these specialised languages defies incorporation into more general languages, and new languages arise because they offer fresh perspectives, a new set of questions, connections and understandings. Horizontal knowledge structures may possess stronger grammars (logic, mathematics, economics, parts of psychology), which are capable of relatively precise empirical descriptions through an explicit conceptual syntax. Horizontal knowledge structures with weaker grammars (sociology, cultural studies) have relatively weak powers of empirical description, and are relatively weakly insulated from horizontal discourse.

Curriculum structure Classification of interdiscursive relations The main point about knowledge structure is that it places two specific ordering and specification requirements on the shape of the intended curriculum. The first has to do with relations between school subjects (interdisciplinary relations) on the one hand, and between these and everyday knowledge (interdiscursive relations) on the other. In the school curriculum the different school subjects may be strongly delineated from one another – ‘strongly classified’ – or they may be integrated to a greater or lesser extent – ‘weakly classified’ in Bernstein’s terms (1996, 20). Similarly, they may be strongly isolated from everyday knowledge or integrated with it. The original C2005 exhibited very weak classification between school subjects (weak internal classification). Thus, several conventional subjects were amalgamated into three

chapter 5 A social theory of schooling

‘learning areas’ for the first three school grades. C2005 was also characterised by weak classification between school and everyday knowledges. A heavy emphasis was placed on ‘integration’, which determined that all knowledge was to be studied under the rubric of so-called programme organisers – such as ‘transport’ – which were to provide topics or themes through which school knowledge was to be integrated with the everyday experience of pupils (weak external classification). The motivation behind the weakly classified approach adopted by C2005 was that, on the one hand, it was supposed that it would facilitate the acquisition of school knowledge, by relating it to the everyday, and, on the other hand, it would facilitate the application of formal knowledge to problems of the real world. However, according to the Review Committee, which referred to this feature of C2005 as weak ‘lateral demarcation’, in the hands of teachers whose knowledge resources are not particularly strong, it all too easily results in confusion, thus inhibiting the learning of conceptual knowledge. The report quotes one of the many submissions received in support of this view: “the body of knowledge that defines mathematics is obscured or dominated by non-mathematical considerations.” (Review Committee, op. cit., 41). In view of these problems, the Committee recommended that programme organisers be scrapped and that the learning programmes be separated into distinct subject areas. The arguments advanced by the Review Committee for the decision to introduce a stronger degree of classification into the knowledge order of the curriculum concurred with the conclusions of Dowling’s (1995) study of textbooks in the UK. Dowling found that ‘lower ability’ students of mathematics are prescribed books dominated by examples intended to model everyday situations, while ‘higher ability’ students use books which foreground the vertical discourse of mathematics. The result, according to Dowling’s research, is that, while the latter students are inducted into the discourse of mathematics, those considered to be ‘lower ability’ are left

with a fragmented view of the subject as a series of isolated strategies for approaching particular problems. We have argued that such an approach “produces a result opposite to its laudable intent, visiting the exclusion on disadvantaged students it was constructed to avoid” (Muller and Taylor, 1995, 269).

Framing: relations of progression The Review Committee also took issue with a second aspect of the design of C2005, namely with the degree of specification of vertical knowledge competences, which it referred to as ‘vertical demarcation’ (op. cit., 40). In the view of the Committee, an overemphasis on the lateral integration of school subjects and of school and everyday knowledges (weak classification) was accompanied by a relative neglect of specification of progression, sequence and pacing. According to the Committee: Vertical demarcation establishes which knowledge, within each demarcated knowledge cluster, must be taught and learnt in what sequence, and at which level of competence. This involves the notions of sequence, pace and progression – what competences must be learnt before other competences can be learnt. ... The particular challenge posed here is of conceptual coherence or progression – how to ensure coherent linkage conceptually within each knowledge unit. (Review Committee, op. cit., 40) Bernstein referred to this design feature as framing: progression can be strongly or weakly framed. Strong conceptual specification is particularly important in subjects such as language, mathematics, natural science and social science, which presuppose an overt stepwise ladder of concepts and skills that must be organised in a sequential and phased way to facilitate cognitive access. (ibid., 40). As we said above, this is because these school subjects are drawn from knowledge fields with well-defined vertical structures and strong conceptual syntaxes. By under-specifying the

73

part II Explaining the outcomes of schooling

content, the levels of competence and their sequence, by grade level, for these subjects, C2005 left teachers without a conceptual roadmap for proceeding. The Committee concluded that teachers in traditionally advantaged schools were better able to cope with this situation – because of their stronger knowledge resources, facility with different teaching methods and access to learning programmes and textbooks – than were the majority of teachers, many of whom felt that, in the absence of specified content, “any content is fine.” (ibid., 47). Thus, the lack of guidance provided by C2005 resulted in lessons with little or no conceptual content, and consequently with very poor coverage of the curriculum. This was particularly marked in poor schools. For Morais and Pires (2002), the research evidence about the effect of curriculum structure on the achievement of Portuguese workingclass children is marked: a clear specification of the outcomes is strongly associated with learning performance in both the social and cognitive spheres. These views are supported by Porter and Smithson (2001) for American learners, who note that if standards are fuzzy and vague at the level of the system, their implementation in schools and classrooms is likely to be fuzzy. We may summarise as follows: if children from poor homes and hence with poorly developed school code mastery are to learn ‘powerful’ knowledge, that is, knowledge with vertical knowledge structure and strong conceptual syntax, the intended curriculum must be relatively strongly classified and framed. If it is not, learners will fail to cover the intended curriculum, fall behind as we saw in Chapter 4, and be consequently doubly shortchanged by society and the schooling system.

The implemented curriculum Schooling is one of the principal institutions through which a society transmits its ideals to the next generation. But much can go wrong in the process of transmission: ... education is a crucial device for the state whereby its dominant principles

74

may be translated into micro practices at the level of the school... However, what is selected is not necessarily transmitted, what is transmitted is not necessarily acquired and what is acquired may, for some acquirers, bear little or no relation, or indeed an oppositional relation, to the intentions of the selectors and transmitters. (Bernstein, 1986, 230) Because of the relative autonomy of the schooling system from the legislature, and of schools from the educational bureaucracy, and of teachers within the school, there are a number of points of potential slippage between the intentions of the curriculum and their realisation in the acquisition of social and cognitive competences by children in schools and classrooms. This slippage may occur because of opposition to the original intentions by transmitters or acquirers, because of inabilities or inefficiencies of transmission/acquisition, or because of differences of interpretation on the part of transmitters or acquirers. In short, there are a number of points in the knowledge cycle which offer the potential for what Bernstein called the recontextualision of knowledge.

The social organisation of schooling Any sociological theory of schooling must take account of the three sets of dimensions shown in Table 5.1 on page 75. Bernstein’s principal contribution to this debate goes back to the seventies. Bernstein (1977) identified two ideal sub-types of school code according to the organising principles which regulate the forms in which meanings may be realised. One sub-type is characterised by strong classification and strong framing, which together produce a coding principle which Bernstein called the ‘collection’ sub-type of code. When classification and framing are both weak, on the other hand, the institution is governed by an ‘integrated’ sub-type of code. In the collection code school, knowledge is organised according to strongly insulated subject hierarchies, and the organisational

chapter 5 A social theory of schooling

Dimension Social

Sub-dimension Values

Table 5.1

DIMENSIONS AND SUB-DIMENSIONS OF THE SOCIAL ORGANISATION OF SCHOOLING

Relations Task

Administrative core

Time Resources Curriculum

Technical core

Pedagogy Evaluation

features of the school are oligarchic control by the principal and subject heads, strong horizontal work relationships among senior staff across subject departments, and strong vertical work relationships among junior staff within departments. According to Tyler (1998) the tight coupling between the administrative and technical cores of the school, which characterise the collection code institution, is typical of systems with low levels of professionalism and teacher-initiated innovation, such as the school in which strong leadership, overt discipline and a concentration on basic skills feature prominently. Schools in which the integrated code dominates are characterised by weaker subject boundaries, providing teachers with greater discretion and possibilities for experimentation. This is reflected too in the organisation of time, space and roles, providing for the possibility of team teaching, a looser timetable and a more flexible approach to arranging teaching space. Senior staff provide support rather than direction and supervision, while monitoring is implicit and indirect, organised through peer networks. The radical form of the integrated code school is represented by Illich’s deschooled school, or A.S. Neil’s Summerhill, in which any form of control is seen as inhibitive of professional judgement at best and repressive at worst. Tyler (op. cit.) notes that, while integrated codes may appear to foster openness and flexibility, in practice they may rest upon closed, explicit ideologies; and while surveillance may be more overt in the collection code school, under the integrated code, peer-directed monitoring is

both more ubiquitous and more public. Similarly, while a more covertly caring, therapeutic approach is adopted to pupil discipline under the integrated code, this is often experienced as more invasive than overt forms (see Ensor, 1999). One of the besetting problems of characterising pedagogies across a single continuum like this is that debates all too easily regress towards defending this or that end of the pole, ending in an ideological debate between ‘learner-centred’ and ‘traditional’ pedagogy that is sterile and unhelpful. As depicted above, the collection code school is homogeneously strongly classified and strongly framed. The converse applies to the integrated code school. This single dichotomy is misleading on two counts. The fact of the matter is that the strengths of classification and framing can co-vary, and not all attributes of the implemented curriculum need share the same value strength. At the very minimum, this doubles the number of ideal types to four, as we see in Table 5.2 on page 76, but in reality, it allows for a wide variety of coding permutations in terms of classification and framing on a number of attributes. This scheme is useful in that it allows for the depiction of school types on a gross level, and facilitates an understanding of how certain forms of school organisation may be more or less appropriate under any particular set of social and material conditions. Thus, where a particular school community shares an ethos and a set of values, a combination of strong classification and weak framing may be ideal. However, in the absence of such an

75

part II Explaining the outcomes of schooling

Classification of spaces, agents and discourses Strong Weak Framing Weak of (personal) instructional and regulative Strong discourse (positional)

Private and religious schools (high ethos schools, with shared cultural values) The traditional school (collection code); the accountable performancebased school.

Table 5.2

A TYPOLOGY OF SCHOOL TYPES

The child-centred school ‘therapeutic’ (integrated code) Managerially run entrepreneurial schools for niche clienteles; dysfunctional, authoritarian schools

Source: Adapted from Bernstein (1996, 77) and Tyler (1999, 276).

ethos, framing may need to be stronger in order to foster a sense of common endeavour among parents, staff and students. Similarly, weak classification may be appropriate under conditions of strongly developed professional comportment among teachers, but, where this is less well developed, stronger classification may be more appropriate to ensure that pupils within any grade receive an equivalent curriculum, otherwise the learning experiences of children both within and across schools may be characterised by high levels of incoherence and hence inequity. We illustrate this variability below, and turn now to an examination of the dimensions and the mechanisms available to managers and pedagogues at all levels of the schooling system to vary the alignment between the intended curriculum and its enactment in schools and classrooms.

Social values The starting point for Morais and Rocha’s (2000) investigation into how children learn four social competences – help, respect, responsibility and autonomy – is that the transmission/acquisition of these skills is strongly influenced by pedagogic practice, as in the case of cognitive learning. This hypothesis was confirmed in a controlled observation of five classes of children, heterogeneous with respect to social background and gender. Learning outcomes, in both the cognitive and social domains, were found to correlate strongly with distinct teaching

76

styles, varying from strongly framed (positional) where the teacher drives the learning process, to weakly framed or personal pedagogy where learners exercise substantial control over learning. The authors found that not only were these social competences promoted by a personal style of pedagogic practice, but that improved social competence was also associated with more successful learning in the cognitive domain and, most significantly, by a narrowing of the learning gap between middle-class and working-class children. Although little research has occurred into the learning of social values, the work of Morais and Rocha alerts us to the importance of social competences and their relationship to the values embodied in the curriculum, on the one hand, and to the learning of cognitive competences on the other. In particular, these results indicate that learning social values and skills in the classroom entails more than creating a friendly environment: the desired competences must be identified, and explicit strategies mobilized, in order to bring them into the learning frame. It would seem likely that these principles apply equally at the levels of the district and the school, giving further support to the hypotheses drawn by Malcolm et al. (2000), summarised in Chapter 4, concerning the kind of school culture – characterised by discipline, teamwork, caring attitudes and a focus on learning – prevalent in successful schools.

chapter 5 A social theory of schooling

Administrative organisation A starting point for any institution designed to inculcate values of democracy, respect for difference, co-operation and disciplined application to learning is that these values are explicitly incorporated into the goals and culture of the institution, and constitute a conscious focus during the conduct of all institutional activities. Our emergent theory of schooling assumes that well-functioning districts (and higher level systems and institutions) facilitate the development of well-functioning schools, which in turn facilitate quality learning, both in the classroom and the school as a whole, and within its immediate community. These assumptions may turn out to be without empirical foundation. For example, the HSRC (Kanjee et al., 2001) and Khulisa (2001) studies described in Chapter 4 found no correlation between the functionality of district offices and pupil performance, although, as we pointed out in that analysis, this lack of association may well have arisen either from design or methodological flaws, or from the fact that the district offices under study exhibited such low levels of functionality that they were unable to have any influence on school behaviour (see also Chapter 8). Before statements of the curriculum can be implemented in sites of learning, they need to be distributed to those sites. While developed countries may assume good communication of curriculum standards to schools and classrooms, this is not the case in countries like South Africa, where institutional malfunction at national, provincial, district and school levels results in statements of the curriculum not being consistently available to principals and teachers. Learning in this kind of system is likely to vary greatly in type and quality with the degree of availability of curriculum and assessment standards. In addition to distribution of the curriculum statements, one of the key functions of districts and schools is to plan, quality assure and monitor coverage of all the specified competences. At the level of the school, parts of the South African system continue to be charac-

terised by very low levels of institutional functionality, with many teaching days lost to late and extended registration of pupils for the first few days and sometimes weeks of the school year; to the practice of teachers preparing and marking exam scripts during school hours; to training for athletics and choir competitions; to teacher strikes; and to in-service training programmes for teachers (Taylor and Vinjevold, 1999; MoE, 2001; 2002). Furthermore, when schools are formally in session, lack of punctuality, high rates of absenteeism and a low work rate among both teachers and pupils (Kanjee et al., 2001) would be expected to contribute significantly to low curriculum coverage and hence to low learner performance. Our theory of schooling therefore includes the degree of order and efficiency of districts and schools as factors likely to influence the quality of learning.

Pedagogic structure Pedagogic framing: planning, coverage and pacing It seems self-evident that if pupils are to learn the knowledge content delineated in the curriculum statements, they should be exposed to the full range of standards: during the course of each grade all the outcomes comprising the curriculum should be explored. It is not surprising, therefore, that a key finding of SIMSS was that “the only classroom or school variable to be significantly related to achievement growth was opportunity to learn measured as content coverage and content exposure” (Stevens, 1996, 222). The first factor affecting coverage is teacher knowledgeability. It is reasonable to assume that the degree to which a teacher is oriented to the school code and well schooled in the subject she is teaching (that is, trained in the genres of her discipline) will influence good classroom mediation. This assumption is borne out by what little research evidence is available. For example, Kennedy (quoted in Supovitz, 2001) concluded that teacher development programmes that focus on subject matter knowledge and on student learning of particular subject matter are likely to have

77

part II Explaining the outcomes of schooling

larger positive effects on student learning than programmes that concentrate mainly on teaching behaviours. Morais and Pires (2002) found that the subject matter competence of teachers explained around one-quarter of variance in pupil learning of high-level cognitive functions. For Reimers (1993), working in poor schools in Pakistan, increasing the amount of teaching time in a low-quality system does not necessarily produce learning gains for students. Improving teachers’ knowledge is a prerequisite for higher levels of productivity. However, short of directly testing teachers, and apart from inferring it from the outcomes of their students, teacher knowledge competency is difficult to ascertain through classroom-level research methods. Furthermore, a number of authors contend that, while teacher content knowledge may be a necessary condition for high-level learning, it is not sufficient. In addition to being well understood by the teacher, knowledge has to be presented in such a way as to be accessible to pupils. This latter consideration involves pedagogical knowledge, and knowledge of the pupils. Shulman (1986) coined the term Pedagogical Content Knowledge (PCK) to refer to this complex of competences required for good teaching. However, apart from its value in alerting researchers to the complexities of describing teaching practices, PCK is not easy to operationalise as an indicator of school or classroom behaviour because of its composite nature. Our list of teaching factors discussed below is an attempt to disaggregate the tools at the disposal of teachers in mediating knowledge in the classroom. After teacher knowledgeability the factor most responsible for variable coverage is pacing. This refers to the pace at which the different outcomes are explored, and whether sufficient time is allocated to the various topics without jeopardising coverage. Smith et al. (1998) describe classroom life in many innercity disadvantaged Chicago schools as characterised by repetitive cycles of low-level skills instruction, with many gaps and a consistently slow pace across grades: as a result,

78

pupils are left progressively further behind curriculum targets. This situation has also been commonly observed in South African schools (Taylor and Vinjevold, 1999; Ensor et al., 2002; Hoadley, 2002). Pacing is strongly related to cognitive demand. If pacing is too weak, and the class falls behind the intended curriculum, the appropriate level of cognitive demand will suffer. The coherence of the enacted curriculum may be analysed at a number of levels. At the level of the lesson, it may be observed in the extent to which activities, exercises and ideas are sequenced and linked. Over the course of the year, coherence refers to the extent to which topics and activities are sequenced. At the grade level, Smith et al. (op. cit.) refer to ‘grade level instructional coherence’ as the extent to which content coverage across the different classes within any grade varies. According to these authors, the more variable this content coverage within any grade, the more likely it is that students will receive incoherent instruction as they pass from one grade to the next. These considerations highlight the importance of orchestrating curriculum coverage within each grade, across grade levels within the school and across schools within the district. In South Africa this task is greatly hampered by the continuing standoff between government and the teacher unions on the nature of a system for monitoring schools and teachers. Both teacher resistance to external accountability and government reluctance to assert the right to quality assure the work of its employees has, up to the time of writing, hampered the institution of systemic accountability measures to regulate curriculum coverage and coherence, and hence systemic equity. At the classroom level, Morais and Rocha (2000) and Morais and Pires (2002) found that a relative weakening in pacing (‘internal framing’) – allowing the pupils’ rate of learning to dictate the pace of classroom activities – is an effective strategy in promoting the learning of high-level scientific concepts. They stress, however, that this must occur

chapter 5 A social theory of schooling

within strong ‘external framing’ of the intended curriculum, as we saw in the previous section: what is to be taught/learnt over the course of the year is determined by the curriculum, while on a day-to-day basis the teacher needs to exercise flexibility, within the terms of the curriculum, to ensure that all children are keeping up.

Classification of interdiscursive relations We have stressed the importance of clearly specifying the content standards in the intended curriculum so as to bring out their conceptual structure and language. This is not to say that it should be insulated entirely from everyday knowledge. While insisting that power resides in possession of the general principle, Bernstein (1996, 26) warns at the same time that if children do not recognise themselves in the curriculum, they may feel excluded. Following Walkerdine (1988), we have argued for a moderate constructivist approach at the classroom level, which consists of the selective use of everyday knowledge in order to exemplify and apply the relevant principles of formal knowledge, and the careful structuring of the relationship between the formal and the everyday, so as to clearly explicate the syntax and specialised language of the former (Muller and Taylor, 1995). Veel (1999, 209) summarises the lesson from the Australian experience as follows: ‘A more effective response would be to design teaching programmes which attempt to move backwards and forwards between strong and weak classification’, but he is careful to add that ‘it is the uncommonsense, strongly classified discursive order in school mathematics which guides the selection and expression of word problems, not everyday, commonsense experience of the world’ (ibid., 207). This is also the view taken by the Review Committee (2000).

Evaluation Much of what has been said above about the design and distribution of the curriculum applies also to assessment standards. Ideally, curriculum and assessment criteria are insep-

arable in the same statement. It is not surprising, therefore, that the Review Committee was also critical of the assessment component of C2005, on the grounds that in the absence of clearly defined qualifications, education is directionless and unfocused (Review Committee, ibid., 48). Assessment provides the primary tool for monitoring the extent to which the system is succeeding in the transmission and acquisition of the social and cognitive competences deemed desirable. The most important level of assessment is the classroom. Here it forms an integral part of knowledge mediation. According to Bernstein (1996), a key function of pedagogy is to transmit the criteria against which any discursive performance is judged as legitimate. Evaluation condenses meaning and explicates the criteria for a competent knowledge display on the part of the acquirer. Such display involves two aspects: recognition and realisation. The acquirer must first recognise the type of response demanded by the particular context. At the lowest level, this requires choosing between a community or school code, as we have seen above. Then the acquirer must correctly identify the specialised language of the particular discourse, whether this be art, mathematics or Sunday dinner with the family. Recognition does not necessarily mean that the acquirer is able to produce a legitimate text in the required discourse. ‘Text’ refers to anything which attracts evaluation: this may be something as simple as the way one sits or as complex as the way one solves an algebraic problem. Possession of the realisation rule is reflected in the ability to produce (act, speak or write) the expected (legitimate) text. Our theory of schooling describes teaching and learning in terms of knowledge mediation. For this to happen, the knowledge concepts and principles must become clearly apparent during the course of the lesson, whether these are derived inductively through investigation, or deductively through exposition. Ensor et al. (2002) have described lessons in South African classrooms in which the teacher does not assist pupils to

79

part II Explaining the outcomes of schooling

draw out the knowledge principles from a set of activities or exercises: indeed, in some of these classes the topic itself is unclear. Unless learners apprehend and engage with the concepts and principles, learning will be shallow at best. Making clear the criteria whereby a knowledge display is considered appropriate can thus not be only relegated to setting periodic formal evaluation tasks – although formal assessment is an important strategy in the repertoire of the teacher – but is a continuous process in which the teacher explicitly defines and explains the meaning of concepts, draws pupils’ attention to key aspects of the knowledge under discussion through questioning and setting assignments, and, in response to pupil knowledge displays, authorises appropriate responses and points out gaps and absences. Morais and Pires (2002; see also Morais and Miranda, 1996) refer to this complex of strategies as the ‘explication (or strong framing) of evaluation criteria’. They found this factor to strongly correlate with successful learning of complex cognitive processes for children of all social classes, and with narrowing the gap between workingclass and middle-class children. They explain the link as follows:

decades. On the issue of systemic evaluation, although the Ministry committed itself in 1998 to regular representative assessment at Grades 3, 6, and 9 (DoE, 1998a), the Department of Education has, by 2002, got no further than a first round at Grade 3 level. In the absence of progress in this area, we have little idea as to whether our policies and systems are working towards the attainment of our goals for social and cognitive learning. The learner performance data in Chapter 3, while a good indication of the health of parts of the public schooling sector, is no substitute for systematically sampled data. At the levels of the district and the school, the assessment of learning remains the most direct instrument for monitoring school performance. Tools at the disposal of district and school managers include setting policy on the frequency and levels of assessment, arrangements for quality assurance, and using individual school, class and subject results as a starting point for improving performance. Little research has been done in this area in South Africa, with the Khulisa (2001) and the HSRC (Kanjee et al., 2001) studies illustrating the current state of work in the area.

A theory of schooling Key constructs

Considering the importance of clearly telling children what is expected of them, of identifying what is missing in their textual production, of clarifying the concepts, of leading them to make synthesis and broaden concepts and considering the importance attributed to language as a mediator of the development of higher mental processes, one understands the influence of making evaluation criteria explicit. (Morais and Pires, 2002, 9) At the level of the system, standardised national evaluation has been very slow to make its appearance in South Africa, other than at the Grade 12 exit point, where existing arrangements have been in place for

80

Background factors and knowledge structure aside, four key constructs may be distilled from the foregoing discussion. These are: language, social organisation, curriculum and pedagogy, and evaluation. These are summarised in Table 5.3 on page 81, which also shows how these constructs manifest themselves at different levels of the school system. Figure 5.1 shows the extraordinary array of elements and influences comprising any public school system, which education managers and teachers need to take account of, and where possible direct and orchestrate in the best interests of their pupils. In South Africa these difficulties, faced by all countries, have been greatly exacerbated by many decades of skewed resourcing and the rapid rate of change precipitated by the advent of

chapter 5 A social theory of schooling

INDICATOR CATEGORIES THEORETICAL CONSTRUCT Social organisation

Language

SUBCONSTRUCT

District and higher

Evaluation

Classroom

Social values

Values incorporated into curriculum statements

Social relations

Style of relations between officials, principals, parents, teachers and pupils

Task

The classification of tasks

Time

The organisation of teaching and learning time

Resources

The management of resources

Proficiency in language of instruction

Language policy set and monitored

Proficiency in language of instruction promoted

Promotion of reading and writing

Policy set

Reading and Reading and writing supported writing at appropriate and monitored levels promoted Books and stationery managed

Books and stationery procured and distributed Curriculum and pedagogy

School

Planning, coverage, sequencing and pacing

Design: vertical knowledge competences and progression criteria specified. Distribution: supplied, monitored and supported

Interdiscursive relations

Design: interdiscursive relations specified

Explication of evaluation criteria

Assessment policy set, supported and monitored

Values incorporated into school culture

Curriculum planning, coverage and progression quality assured, supported and monitored

Table 5.3

EDUCATIONAL FACTORS EXPECTED TO AFFECT LEARNING

Values incorporated into lessons

Macro: entire curriculum covered over the year. Micro: pacing adjusted to cater for pupil characteristics Level of cognitive demand appropriate to curriculum statements Structuring of relations between school and everyday knowledges

Assessment quality assured, supported and monitored

Explication of evaluation criteria

81

part II Explaining the outcomes of schooling

representative government in 1994. This change has offered exciting challenges and new opportunities, but at the same time has increased pressure on the system, in the form of a veritable avalanche of new structures and policies. Under these conditions, national and provincial level officials, district managers, school principals and classroom teachers are hard pressed to demonstrate compliance with the vast checklist of obligations appearing before them daily. Prioritising and co-ordinating these into a coherent set of strategies for improving learning must often seem to present an impossible task. A similar problem faces school reformers and researchers. Any solution to this dilemma requires identifying the key levers to be prioritised in winching the system towards more efficient and equitable performance, on the one hand, and in assessing, on the other, whether any progress towards these goals is discernable amid the welter of polemic and activity. Our theory of schooling has been constructed with these needs in view: to identify those elements of practice at different levels of the system most likely to optimise learning for all children. Our theory is based, in the first instance, on theoretical considerations, and in the second, on evidence available both internationally and within the South African context. This theory is at a very crude stage of development. As such, it is not a blueprint of prescriptions for best practice, but a set of interlocking hypotheses for structuring a comprehensive research programme aimed at identifying the key levers for optimising school performance.

General assumptions The first general assumption of the theory is that the transmission/acquisition of a set of social values and a body of conceptual knowledge, both of which reflect the dominant principles of the society, constitute the core business of any schooling system. While many other activities and values may be ancillary to these goals, it is all too easy to treat these other activities as free-standing ends in themselves if the main purpose of

82

schooling is not constantly kept in view and used as a yardstick, both to prioritise among competing demands for resources and to assess the success of any school. The second and third general assumptions on which our theory of schooling rests are that socio-economic status is a critical orienting factor which differentially predisposes children of different social classes and language backgrounds to acquiring high-level conceptual knowledge, and that the kinds of resources discussed in Chapter 4 not only significantly affect learning, but may ameliorate social effects. However, and this is our final general assumption and the central hypothesis of the present chapter, while the absolute level of resources is important, the use to which the system as a whole, and the individual school in particular, puts its material and human resources is a key determinant of learning. In other words, after socio-economic origin, pedagogic practices – reflected in the orchestration of classroom activities and the management of institutions – collectively constitute the most important set of factors which structure the educational opportunity of children. Furthermore, the quality of pedagogic practice is the key mechanism for overcoming the influence of social origin. In regard to this fourth assumption, the research evidence indicates that the standoff between proponents of child-centred (competence) approaches and teacher-centred or ‘traditional’ (performance) approaches is a great hindrance to both debate and the construction of systems and practices which will benefit all children, as we said above. While the distinction may be useful for some purposes, clarifying the range of options available at different points of knowledge recontextualisation, the defence of either position on ideological grounds and the consequent willynilly espousal of all its features is bound to lead to the inadvertent subversion of at least some of the best educational intentions. At the same time – and this is the greatest strength of Bernstein’s theory and the research it has inspired – practices which opti-

chapter 5 A social theory of schooling

mise learning opportunities for children of all social and cultural situations do not simply occur by some serendipitous blend of competence and performance perspectives, but must be constructed, element by element, using a single guiding criterion: Which practice, under the particular circumstances in question, is most likely to result in children acquiring the social and conceptual competences targeted by the intended curriculum? We illustrate the last point by means of an example, drawn from Morais and Neves (2001, 210). Consider the situation in which a teacher communicates her assessment of a knowledge display to the child who offers the display. The text under assessment may be a verbal answer to a question, a written response to a test or exercise, an activity during a practical investigation, a voluntary offering or question on the part of the child, or any number of behaviours possible in the classroom. On the one hand, on the basis of their empirical evidence, the researchers conclude that learning is promoted when the teacher makes quite clear to the child which parts of the display are appropriate, and exactly what would be needed to constitute a legitimate text. Not only does this explication of the evaluation criteria facilitate the acquisition of conceptual knowledge, but it has also been found to narrow the learning gap between middle-class and working-class children. In Bernstein’s terms such action would be termed ‘strong framing of the evaluation criteria’. On the other hand, the evidence shows that the teacher’s response is more likely to achieve its desired social and cognitive effects if it is done through a personal mode of communication, as opposed to one that foregrounds the hierarchical role of the teacher. The personal mode would constitute a weak framing of the hierarchical relations between transmitter and acquirer. Thus, strong and weak values of the structuring principles of pedagogic practice are not ideologically-based articles of faith, but provide pedagogues at all levels of the system with a repertoire of strategies that they may mobilise in the interests of transmitting the intended competences.

Hypotheses With these general assumptions in place, we are in a position to summarise the hypotheses that constitute our theory of schooling.

Curriculum design We hypothesise that the first requirement necessary for a system to optimise learning gains is for the social and cognitive goals of schooling to be clearly explicated in a set of curriculum statements and content standards, which specify progression, sequence and pacing requirements by knowledge area and grade level. We expect that coherence across the system – and hence equity – will be promoted, first by placing this function at a systemic level, and second by specifying the content at a detailed enough degree so as to leave no room for ambiguity of interpretation in schools and classrooms. We expect further that, though this is crucial for all learning areas, it is particularly so for school subjects based on knowledge fields with vertical knowledge structures and strong conceptual grammars.

School organisation At the level of values, we hypothesise that the acquisition of democratic values is promoted by a personal communication mode at the individual level, and a shared ethos and shared institutional culture. At the level of organisation, we hypothesise that relatively close coupling between system, school and classroom levels of organisation best suits South African conditions. For the district, this means the distribution of knowledge resources to schools and classrooms. At the school level, this consists of ensuring the attendance and punctuality of teachers and pupils, and maximising the time for teaching and learning. This is best done by monitoring performance and by quality assuring the planning and learning coverage (strong macro framing). Within this institutional climate where the rules of performance and conduct are clear and explicit, the learning of social and cognitive competences is enhanced by relatively open, personal relations between all participants (weak micro framing).

83

part II Explaining the outcomes of schooling

Pedagogy Our theory of schooling hypothesises that it is the substance rather than the form of classroom activities that counts in promoting acquisition of the intended curriculum. Based on this, the theory expects that the most important elements of pedagogy available to the teacher are: • planning and coverage of the curriculum (strong framing of planning) • pacing that takes account of pupil capabilities (variable framing of micro pacing) • mediating the relationship between everyday and school knowledge (medium classification of inter-disciplinary relations)

Evaluation Assessment completes the curriculum cycle, on the one hand evaluating the work of pupils, teachers and institutions at all levels of the system, and on the other serving to explicate and exemplify the learning targets. Our theory expects the following to be strongly related to positive outcomes: • clear and explicit assessment standards (strong macro framing) • the presence of an evaluation culture in the school (strong meso framing) • individualised evaluational feedback in the classroom (strong micro framing), mediated, as we said above, through personalised classroom management style (weak framing of social relations) At the school level, we expect that the quality assurance of assessment practices are key mechanisms to ensure coverage, appropriate levels of cognitive demand, and coherence across all classes within any grade. Specific instruments available to school management include: setting policy in terms of the frequency and forms of assessment tasks, moderating the standards of tasks, evaluating results and using these to identify problems in teaching and learning.

84

Similarly, performance would be improved were districts to use the same instruments in all schools under their jurisdiction. At provincial and national levels, a cyclical programme of testing, at least on a sample basis but optimally whole-population testing, is essential for monitoring the health of the system.

Conclusion There is a dialectical relationship between theory and evidence. Theory provides coherence and direction to empirical investigations. In the case of school learning, it makes predictions about the kinds of factors likely to influence the kind of learning valued by any society – high-level conceptual development and democratic social values, in the case of South Africa – providing the researcher with clues as to where to search for evidence. Reciprocally, theory is always open to confirmation, refutation or elaboration by the empirical evidence. Indeed, theoretical considerations are no more than hypotheses until they have been subjected to empirical scrutiny. And even where such considerations are derived from evidence, as hypotheses and proto-theories generally are, they always remain open to corroboration or modification in the light of further observation. “A theory should generate the criteria for its evaluation, the contexts necessary for its exploration, the principles for their description, and the rules for interpretation” (Bernstein, 1990, 98). It follows that the theoretical framework shown in Table 5.3 and spelt out in the hypotheses above is open to elaboration in the light of the evidence. In Chapters 6, 7 and 8 we test the hypotheses comprising our theory of schooling against a sample of the district-, school- and classroom-level data identified in Chapter 2, and investigate whether any correlations exist between these descriptions of pedagogic practice and the learning outcomes reported in Chapter 3.

part

III Implications of the model for classrooms, schools and districts The theoretical model derived in Chapter 5 serves as a two-way lens. On the one hand, it provides a frame – a set of hypotheses – which guides our interrogation of empirical evidence in the search for factors which influence school learning. Simultaneously, the model focuses the conclusions of the research back onto the theory, in order to elaborate our understanding of schooling and how to improve its processes and outcomes. The logic of our model commences with the formulation of a set of curriculum standards which embody the social and cognitive values espoused by the society, and follows the distribution and recontextualisation of these ideals, through successive institutional levels and networks of the schooling system, to their manifestation in pedagogic relationships and learned competences in schools and classrooms. There is an argument to be made for following the same logic in analysing the new evidence, tentative and sketchy as it is, which this book brings into the terrain of scholarly and public debate. This logic would entail first looking at the district and higher-level data for indications as to how these institutions shape and direct the work of schools, followed by an examination of the school-level evidence for signs of how school life influences learning, and

finally sifting through the classroom data in order to better understand transmission/acquisition relationships at this level. However, in Chapters 6–8 we reverse this order, principally for pragmatic reasons. Our evidence is more substantial at the classroom level than it is for schools, and sparser still for districts. Consequently, what we know about classrooms is more likely to illuminate both what we can gain and what is missing from our knowledge of schools, than the converse. The same holds for our quest to better understand the relationship between schools and districts. Hence, we look at the classroom-level evidence in Chapter 6, turning our attention to schools in Chapter 7, and finally to districts in Chapter 8. The analysis of the evidence at our disposal in these chapters is an ex post facto exercise. Although the intuitions which guided the collection of this data represented an emerging form of the model summarised in Table 5.3, the fit is far from perfect, and consequently we have no data on a number of indicators which our theory predicts to be important in assessing the quality of schooling. In addition, the questions which structured the collection of the evidence available were generally not as precisely defined as the indicators derived from the constructs shown in Table 5.3.

85

part III Implications of the model for classrooms, schools and districts

As a result, our data illuminates the indicators rather haphazardly at times. Our organisation of this evidence uses a mixed-method approach. The intention is to combine the insights afforded by the case study perspective with the generalising power of statistical methods. As we pointed out in Chapter 4, case studies offer the potential of opening the ‘black box’ of schooling, providing a view of the processes which structure the learning outcomes issuing from the collective practices of the system. However, case studies have no generally accepted method for revealing systematic patterns in the objects of their gaze. Thus, while the excerpts quoted in such educational studies generally provide profound insights into the behaviour of principals, teachers and pupils, it is often difficult to know to what extent this data represents a fair picture of what happens even in one classroom, or reveals systematic patterns across several classrooms or schools. We attempt to circumvent this problem by quantifying the observations, and illustrating the scores obtained in this way by means of a series of systematic quotes across the full spectrum of performance. The method is illustrated in Chapter 6, where our data is sufficiently complete to provide a relatively full picture of the majority of the constructs and subconstructs in our model. Our data is more sketchy at school and district levels, and in Chapters 7 and 8 we therefore make no attempt to delineate patterns of behaviour through quantification of the observations, confining ourselves to illustrative quotes. The results of this exercise are inconclusive. We are unable to link pedagogical practices at the three levels of schooling with each other or with the learning outcomes

86

described in Chapter 3. Although our method is designed to facilitate the search for associative links between educational process factors and pupil performance, the collection of the data at our disposal transgresses one or more of the principles of rigorous research design which we derived from our discussion in Chapter 4. Hence the possibility of subjecting it to a correlation analysis of any kind is precluded. In view of the less than satisfactory state of the data used in Chapters 6 to 8, the knowledge gains made by these analyses are rather modest. Nevertheless, we offer these descriptions for two reasons: • Although based on a small and unrepresentative sample of schools, they enlarge our knowledge store of pedagogical practices in South African districts, schools and classrooms, corroborating and elaborating the insights produced by other studies. While the design weaknesses of the research studies from which this data is drawn do not permit any rigorous interrogation of the hypotheses which constitute our model of schooling, the conclusions of Chapters 6 to 8 are by no means incompatible with the model, and encourage us to pursue a more carefully designed research programme within the framework outlined in Table 5.3. • The approach used to process the data in Chapter 6 is an example of the kind of mixed method advocated by Tashakkori and Teddlie (1998). We offer a detailed account of our method in the interests of furthering the debate on the kind of technique most appropriate to the description of educational process factors, and the analysis of their influence on pupil learning.

chapter

Curriculum delivery in the classroom Research question The central research question at the level of the classroom is: what factors of classroom design and organisation best promote learning in both the social and cognitive domains? This is a composite question which our theory allows us to disaggregate into nine dimensions (subconstructs) of classroom life, and finally into eighteen indicators for collecting and analysing the evidence. Together these elements comprise a model of classroom learning, summarised in Table 6.1 on page 88. The model thus enables us to ask 18 distinct sub-questions, which are proxies for what are hypothesised to be the principal facets of the central research question.

Design The present chapter describes an ex post facto analysis of data collected during the baseline components of the Phalaborwa and Siyathuthuka evaluation studies. Table 2.1 shows the kind of data that was collected during each of these studies. The design of the original baseline studies consisted of pupil testing in literacy and numeracy in a sample of project schools, and case studies conducted in a sub-sample of the latter. The case studies collected data at school and classroom levels. Tables 3.4, 3.5, 3.6 and 3.7 show the results of the literacy and numeracy tests conducted in these schools. The purpose of Chapter 6 is to use the framework depicted in Table 6.1 to analyse the classroom component of the data from these evaluation studies. There are a number of limitations on what can be concluded. The biggest shortcoming in the research design of these evaluation studies is the lack of linkage between the classroom-level data discussed in the present chapter, and the learner-performance data described in Chapter 3. We alluded to this issue in Chapter 4, under our discussion of the attempt made by Vinjevold and Crouch (2001) to correlate

6

learner performance with teacher knowledge in the DDSP study. We speculated there that one of the possible explanations for this lack of correlation was that there was only a small overlap between the classes from which the learner performance data was derived, and the classes taught by the teachers whose subject knowledge was assessed. Thus, the study was attempting to compare the mean pupil test results at two grade levels, with the knowledge of an ‘average teacher’ as measured in a small sample of teachers across several grades in the school. The link between teacher knowledge and pupil performance was further attenuated in the Crouch and Vinjevold study when the authors were constrained to aggregate the teacher knowledge data to the district level because of the small sample size. While our problem in the present chapter is not of the same order of attenuation, it is similar in nature. In order to derive the learning-performance data given in Chapter 3, 40 pupils were sampled across the Grade 4 and Grade 6 classes in each school. The results therefore represent an average for all pupils within each grade. The pedagogic data described below was obtained, in general, from observations made on one teacher per subject per grade. Given the fact that performance across classes within each grade is likely to vary significantly, and given that individual pupils cannot be linked to specific teachers in our data set, statistical correlations between the pedagogic practices of teachers and learning performance are unlikely to produce any meaningful conclusions. Therefore, we did not attempt any such correlations. Nevertheless, indicative comparisons between pedagogic practices and performance are compatible with the tenets of our model, as we shall see below. Aside from the problem described above, any ex post facto research design faces a number of potential limitations, chief of which is

87

part III Implications of the model for classrooms, schools and districts

THEORETICAL CONSTRUCT Social organisation

SUBCONSTRUCT

INDICATORS

Social values

1. To what extent are the social values embodied in the curriculum included as specific content components of the lesson?

Social relations

2. How open are relations between teacher and pupils? 3. How open are relations among pupils?

Language

Proficiency in language of instruction

4. How proficient are teachers in the language of instruction?

Promotion of reading and writing

7. Quantity of writing: how much writing is done by pupils (all writing tasks are counted, including classwork, homework and written assessment tasks – tests, exams, projects, assignments)?

5. How proficient are pupils in the language of instruction? 6. Are explicit attempts made to promote development of the language of instruction?

8. Quality of writing: do individual pupils engage in extended writing activities? This may be done during group activities, but each pupil in the group needs to be engaged. 9. Quantity of reading: do individual pupils engage with written material? This may be done during group activities, but each pupil in the group needs to be engaged. The material may consist of textbooks, worksheets, magazines. Curriculum and pedagogy

10. Is there evidence that the teacher has planned the delivery Macro-level of the curriculum over the year so as to ensure: pacing – planning, • coverage of all the outcomes of the curriculum coverage and standards? sequencing • regular formal assessment tasks? • logical sequencing of topics? 11. Is there evidence that the curriculum plan is being followed? 12. Is there evidence that the teacher has planned the lesson under observation, with sufficient detail to show: • knowledge content? • class activities? • homework (if appropriate)? 13. Is the daily lesson plan up to date with the curriculum plan? Micro-level pacing 14. Is the teacher ascertaining at what level the learners are, and engaging them at that level? Cognitive demand 15. Is the knowledge object of the lesson clear? 16. Is there a discrepancy/gap between the intended and enacted curriculum at the level of the contents/cognitive competences?

Evaluation

88

Interdiscursive relations

17. Does the teacher intentionally and appropriately link everyday knowledge with curriculum knowledge?

Explication of evaluation criteria

18. To what extent does the teacher: • ask leading questions so as to draw out conceptual principles? • give feedback to learners’ questions, and verbal and written answers, authorising correct answers and correcting misconceptions and gaps? • ensure that the knowledge principles arising out of investigations and other activities are clearly and explicitly stated?

Table 6.1

CLASSROOM-LEVEL INDICATORS DERIVED FROM THE THEORY OF SCHOOLING

chapter 6 Curriculum delivery in the classroom

the degree of fit between the framework used to collect the data, and the model which guides its analysis. Fortunately, our data was collected within a framework which was, implicitly at least, informed by the arguments presented in Chapter 5. Hence there is a relatively good fit between the indicators used to collect the data and those used in the analysis below. Nevertheless, it is by no means a perfect correspondence, and this misalignment has resulted in some limitations in the fitness of our data for answering the research question. Chief among these limitations are:

Missing indicators The conception which guided the collection of data took no explicit account of social values, and hence there is no data on indicator 1: the extent to which the social values embodied in the curriculum are included as content components of the lesson. Although indicators 2 and 3 were not specifically targeted during the fieldwork, we were able to collect data on these two components through a secondary analysis of the field evidence. Indeed, all the data used in the analysis below were collected in this manner. This means that the findings are indicative, and have yet to be systematically established. The most serious gap is in the area of language, where, apart from some sketchy observations on indicator 6 – attempts to promote development of the language of instruction – there is no information on indicators 4, 5 and 6.

Insufficient detail Fieldworkers were given no explicit instructions to collect information on indicators 2, 3 and 17, and, although we are relatively confident that our reconstruction presents a credible picture of these pedagogical aspects, there is no doubt that we would be more confident still had this data been collected directly.

Distance between indicators and pedagogic dimensions In addition to the limitations discussed above, which are specific to our data set, and arise out of the research design, a further potential weakness is, to a greater or lesser extent, likely to be generic to most research studies. This concerns a potential gap between each indicator and the particular aspect of the pedagogic relation for which it stands proxy. For example, we have used the quality of daily lesson plans (indicator 12), and the degree to which this fits an annual curriculum plan (indicator 13) to ascertain the extent to which the teacher is pacing the coverage and sequencing the curriculum standards. If a significant number of teachers, say, are working closely from textbooks, without annual or daily plans, then these indicators may be misleading as proxies for these dimensions of pedagogic practice.

Sample A total of 65 classes, 37 in Grade 4 and 28 in Grade 6, were observed in 20 schools, as shown in Table 6.2. Table 6.2

LESSONS OBSERVED Lessons observed Project

Schools sampled

Siyathuthuka

10

Phalaborwa

10

TOTAL

Grade

Maths

Science

English

TOTAL

4

7

5

5

17

6

5

7

7

19

4

9

11

–

20

6

5

4

–

9

4

16

16

5

37

6

10

11

7

28

20

Source: JET (2001d; 2001e; 2001f; 2001g).

89

part III Implications of the model for classrooms, schools and districts

Both projects are located in rural areas, one in Limpopo Province and one in KwaZuluNatal. Poverty, unemployment and illiteracy are high among parents. Their living conditions are what Simkins (see Chapter 4) has described as ‘other rural’ (in contrast to formal urban, informal urban, and commercial farming). Many teachers live in the town nearest the school, travelling between 50 and 80 km to and from school daily. Most schools are in possession of basic equipment such as pupil desks, chairs and blackboards, although it is relatively common for there to be insufficient chairs and/or desks for all children (around 25% across the three projects). While pupil/teacher ratios are generally under 40:1, it is not uncommon for class sizes to exceed 50 or 60 because of a shortage of classrooms. None of the schools has a library. Lack of running water is a widespread problem, and the state of toilets for both teachers and pupils is often very poor. We will discuss school conditions more fully in Chapter 7.

Method Procedures and instruments Classroom observations constituted one component of case studies that were carried out in a sample of each set of project schools. The evaluation design assumed that between four and six lessons would be observed in each school. The researcher was to observe a Grade 4 and a Grade 6 mathematics, science and English lesson in the Siyathuthuka schools; and maths and science classes in the Phalaborwa schools. The observations were conducted using a structured observation schedule, which captured data on classroom conditions, activities, the content of the lesson, and levels of pupil participation and interaction. Observers also completed a narrative report on the lessons observed: these proved to be invaluable during the ex post facto analysis, allowing the reconstruction of much of the data required by the present study which did not form part of the original evaluation study. Classroom observations included a review of teachers’ lesson plans, schemes of work (yearly teaching pro-

90

grammes), learner workbooks, and assessment tasks and records. The protocol for the observation process began with a group interview to inform teachers about the purpose of the visit. The intention of these interviews was to find out the perspectives of the teachers regarding the interventions, as well as their interpretations of their own practices, and to assure the interviewees of the confidentiality of the data collected. Group interviews were conducted with the School Management Teams (SMTs). The studies also examined the ways in which teachers articulated the purpose and relevance of their lessons through a post-lesson interview. These articulations were meant to shed light on the teachers’ understanding of the structure of the subject, their awareness of changes and debates that occur in the profession, and their awareness of the context and circumstances in which they are immersed.

Limitations, and efforts to reduce their potential effect The following methodological limitations of the study must be noted: • Inter-rater reliability. In a study where multiple researchers are responsible for collecting data, there is a danger that different people will interpret information and events differently. In order to reduce the possible effects of having more than one researcher responsible for data collection, a pilot study was used to standardise data collection and recording methods. After school visits, the researchers would meet with their counterparts who had visited a different school and review the data collection schedules. Where information was unclear, contradictory or not completed in a standard way, this would be rectified. • Potential subjectivity inherent in observation methodologies. When data is gathered through observations, there is a danger that the recording and interpretation of this data may be subjective. In order to limit this, a structured schedule was used which collected a large amount of data on factual

chapter 6 Curriculum delivery in the classroom

elements, and provided a framework for capturing other elements. This may in turn affect validity, as we noted in Chapter 2. • Low reliability of self-report data. Efforts were made in the construction of interview schedules to avoid questions which would elicit data that could not be verified either through direct observation or crossvalidation between interview groups. Where information was reported in interviews (e.g. the development of yearly schemes of work) it was corroborated with data gathered by the classroom observer (structured review of schemes of work) and reviews of the documents mentioned or practices reported. A number of logistical limitations also hampered the studies. In several schools it was not possible to observe the number of lessons as planned. The following factors limited the number of lessons which could be observed. A number of these reflect the low levels of functionality of the school. • Absence of teachers responsible for teaching key subjects. This problem was common and it was rare to visit a school and find all the teachers teaching maths, science and English to Grades 4 and 6 present on a given day. Where possible, Grade 5 lessons were substituted for the lessons that could not be observed. If this was not possible, fewer observations than planned were conducted. • Design of school timetables so that each lesson lasts one hour. This meant that in some schools not all the lessons which we wished to observe were being taught on the day when the research team visited. Teachers were often accommodating and rearranged their personal timetables to ensure that as many observations as possible could take place on the day of the visit. • Lack of adherence by schools to their timetables. In some cases schools finished early and classes did not take place as planned. In these instances, learners were either engaged in cleaning classrooms or sports activities.

• Inability to conduct interviews with principals in three schools. In two schools this was due to the fact that principals were attending a principals’ meeting arranged by the Department of Education. In a third school the principal was not present, with no explanation given for his absence. In these three schools, interviews were conducted with deputy principals and other members of the school management team. • Lack of preparation by some schools for our visit, although all schools were aware of our visit. Documents such as teacher preparations, requested in letters sent to schools prior to the visits, were therefore not always available. Many teachers defended this situation by reporting that their union had instructed them that preparations for teaching are personal and should not be released to anyone.

Data analysis Each indicator was operationalised by means of the scoring criteria defined in the third column of Table 6.3 on pages 92–93. The last three columns of the table show the mean percentage score, for each school, on each indicator. These percentages are a measure of the degree to which the score was maximised on each indicator. No significant differences were found between the Grade 4 and Grade 6 scores within each of the three subjects, hence scores for the different grades were aggregated by subject. However, some interesting differences were noted across subjects for scores on the same indicator. These are discussed below. Since 14 indicators were scored, and the maximum for each indicator is 2, the maximum total score for any lesson is 28. In the analysis which follows we attempt to use a mixed method, as described by Tashakkori and Teddlie (1998). The scores for each indicator reveal the level of performance, by subject, on the respective indicator. At the same time, the excerpts quoted give insights into the range of classroom activity classified under each indicator, and reveal how the scores were derived. In general, the excerpts used to illustrate the different pedagogic

91

92 41

22 17

13 27

26 30

2: 20% or more of class time spent writing 1: 10–19% of class time spent writing 0: less than 10% of class time spent writing 2: paragraphs/whole computations; maths sentences 1: sentences/part computations; maths phrases 0: no writing; only single words/numbers 2: 20% or more of class time engaged with reading 1: 10–19% of class time engaged with reading 0: less than 10% of class time engaged with reading 2: written plan available: shows coverage, assessment, logical sequencing 1: written plan available: insufficiently detailed 0: no written plan available

7. Quantity of writing: how much writing is done by pupils (all writing tasks are counted, including classwork, homework and written assessment tasks – tests, exams, projects, assignments)?

8. Quality of writing: do individual pupils engage in extended writing activities? This may be done during group activities, but each pupil in the group needs to be engaged.

9. Quantity of reading: do individual pupils engage with written material? This may be done during group activities, but each pupil in the group needs to be engaged. The material may consist of textbooks, worksheets, magazines.

10. Is there evidence that the teacher has planned the delivery of the curriculum over the year so as to ensure: • coverage of all the outcomes of the curriculum standards? • regular formal assessment tasks? • logical sequencing of topics?

Promotion of reading and writing

Macro-level pacing – planning, coverage and sequencing

47

54 36

Not scored

4. How proficient are teachers in the language of instruction? 5. How proficient are pupils in the language of instruction? 6. Are explicit attempts made to promote development of the language of instruction?

Proficiency in language of instruction

34

69

31

19 32

2: more than half the pupils talk to each other about tasks 1: a significant number of pupils, but fewer than half, talk to each other 0: little engagement; talk is mostly not task-related

3. How open are relations among pupils?

34

31 36

2: teacher adopts personal style; more than a quarter of pupils ask questions 1: teacher adopts positional style; only the most assertive pupils ask questions 0: teacher adopts imperative style; pupils hardly ever ask questions

2. How open are relations between teacher and pupils?

MEAN % S E

Social relations

M

Not scored

SCORING CRITERIA

1. To what extent are the values embodied in the curriculum included as specific content components of the lesson?

INDICATOR

CLASSROOM-LEVEL INDICATORS OF PEDAGOGIC PRACTICE

Social values

THEORETICAL SUBCONSTRUCT

Table 6.3

44

38

25

63 63

53 48

34 59

31 25

2: explicit, remains clear throughout the lesson 1: mostly clear, but significant parts of lesson not clearly related 0: not apparent 2: knowledge level appropriate to grade 1: knowledge level in range specified by curriculum, but one grade too low or too high 0: knowledge level two or more grades too high or too low 2: most of the time 1: around half the time; teacher misses significant opportunities 0: seldom or never 2: most of the time 1: around half the time 0: seldom or never

15. Is the knowledge object of the lesson clear?

16. Is there a discrepancy/gap between the intended and enacted curriculum at the level of the contents/cognitive competences?

17. Does the teacher intentionally and appropriately link everyday knowledge with curriculum knowledge?

18. To what extent does the teacher: • ask leading questions so as to draw out conceptual principles? • give feedback to learners’ questions, and verbal and written answers, authorising correct answers and correcting misconceptions and gaps? • ensure that the knowledge principles arising out of investigations and other activities are clearly and explicitly stated?

Cognitive demand

Interdiscursive relations

Explication of evaluation criteria

28

22

35 29

2: more than half the class keeping up, without subject being too easy 1: lesson pitched too high or too low for more than half the class 0: lesson pitched too high or too low for all but a few; teacher makes no effort to ascertain whether level is appropriate

14. Is the teacher ascertaining at what level the learners are, and engaging them at that level?

0

Micro-level pacing

9

34

0

2: lesson plan correctly positioned wrt curriculum plan 0: lesson plan not correctly positioned wrt curriculum plan; no plan

0

2: written plan available: shows content, activities, home- 27 39 work (if appropriate) 1: written plan available: insufficiently detailed 0: no written plan available

0

MEAN % S E

12. Is there evidence that the teacher has planned the lesson under observation, with sufficient detail to show: • knowledge content? • class activities? • homework (if appropriate)?

0

M

2: completion dates filled in 0: completion dates not filled in; no plan available

SCORING CRITERIA

11. Is there evidence that the curriculum plan is being followed?

INDICATOR

13. Is the daily lesson plan up to date with the curriculum plan?

Macro-level pacing – planning, coverage and sequencing

THEORETICAL SUBCONSTRUCT

Table 6.3 cont’d.

93

part III Implications of the model for classrooms, schools and districts

dimensions comprising our model are not quoted in isolation, but embedded within at least a significant part of the lesson description, so as to show each dimension in relation to the lesson context. A number of the excerpts quoted constitute the entire description available.

Social values Indicator 1. To what extent are the values embodied in the curriculum included as specific content components of the lesson? This indicator was not scored as it was not part of the framework within which the data was collected, and there is insufficient information in the raw data to reconstruct a picture of this aspect of classroom practice.

Social relations Our model assumes that the nature of the relations between teacher and pupils and among pupils is a key mechanism for structuring the learning of both cognitive and social competences. Morais and her colleagues (Morais and Rocha, 2000; Morais and Neves, 2001; Morais and Pires, 2002), whose work is discussed in Chapter 5, have shown that these relationships may be charted by means of a number of indicators. Two indicators were used to describe this dimension of pedagogy: the degree of openness between teacher and pupils (indicator 2) and the degree of openness among pupils (indicator 3). We have defined these indicators by means of the style adopted by the teacher and the degree of spontaneous pupil questioning, in the case of teacher/pupil relations, and the degree to which pupils talk to each other about the tasks they are engaged in, in the case of pupil/pupil relations.

Indicator 2. How open are relations between teacher and pupils? The mean percentage scores on indicator 2 are shown in Table 6.3, and summarised below:

94

M

S

E

31

36

34

Low scores across all three subjects shows the persistence of top-down (imperative) pedagogic styles in these classrooms, despite the fact that since 1997, C2005 has championed more open relations between teachers and pupils. It would seem that the use of group activities is interpreted by teachers as a proxy for this dimension. However, while group work is a common feature of life in these classes, scores on both indicators 2 and 3 are very low. Although the forms of learner-centred pedagogy are regularly instituted, the substance – the promotion of pupil initiative and interaction – is largely missing. Some illustrations of how indicator 2 was scored follow: Score 0 Lesson 5 (maths, Grade 4, total score 4) The teacher used an imperative style, calling individual pupils to perform tasks on the board: those who the teacher considered to have performed adequately were applauded and those who did not provide an adequate response were sent back to their seats and someone else called forward (indicator 2 = 0), without explanation as to what was lacking in the response (indicator 18 = 0). There was no communication among pupils (indicator 3 = 0). Score 1 Lesson 8 (maths, Grade 4, total score 12) The topic of the lesson was ‘days of the week’. This was clear throughout the lesson (indicator 15 = 2). In asking questions like: “how many days in 2, 4, 6 weeks?” the teacher did push the level of cognitive demand into the range for Grade 4, but the whole lesson was spent on these questions and considerably more progress could have been made (indicator 16 = 1). No attempt was made by the teacher to ascertain the extent to which the learners were engaging with the content (indicator 14 = 0). Pupils worked in groups and then reported their answers to the class: incorrect answers were merely labelled

chapter 6 Curriculum delivery in the classroom

‘incorrect’ by the teacher, without explanation (indicator 18 = 0). While ‘days of the week’ is a very appropriate everyday topic for exploring the cyclical nature of our measurement of time, and while the 7-day cycle was implicitly understood by most pupils, the teacher did not use the opportunity to explicate the principles of the weekly cycle (indicator 18 = 0). This aspect could have been explored, for example, through questions like: “if Tuesday is the 4th of April, give the dates of all the other Tuesdays in April,” etc. The teacher was quite successful in promoting pupil participation and engagement with each other on the tasks, but could have done more to promote the development of initiative among pupils (indicators 2 = 1, 3 = 1). Although pupils did spend a significant proportion of the lesson writing (30%) (indicator 7 = 2), the quality of writing did not progress beyond the extent of sentences (indicator 8 = 1). Score 2 Lesson 41 (English, Grade 4, total score 15) The topic of the lesson was ‘capital letters’, done partly in groups and partly as a whole class lesson (indicator 15 = 2). The teacher used everyday knowledge effectively, through the use of the names of famous people – Mandela, Mbeki, Thobela, Nomvete (indicator 17 = 2). The children were enthusiastic (indicator 2 = 2) and they discussed the tasks in their groups (indicator 3 = 2). Unfortunately, too much time was spent on everyday knowledge, preventing a movement towards an appropriate level of cognitive demand: the use of capitals was confined to proper names (indicator 16 = 1). This proved to be a relatively easy principle for the pupils to grasp and the teacher did not stretch them (indicator 14 = 0). The teacher also missed many opportunities to correct grammar and spelling (indicator

18 = 0). A considerable quantity of writing was done in groups (indicator 7 = 2), where the children wrote whole sentences, putting in capitals and full stops (indicator 8 = 1).

Indicator 3. How open are relations among pupils? M

S

E

19

32

31

These scores are even lower than those for indicator 2, indicating that pupils exhibit very low levels of interaction in the classroom. Score 0 Lesson 12 (maths, Grade 7, total score 8) Pupils worked in groups and reported their work to the rest of the class. The purpose seemed to be to promote group work without any cognitive purpose. Thus, while there was a great deal of animated discussion in the groups, much of this was not constructively focussed on the task, particularly among the boys who spent a great deal of time arguing about petty matters such as who was to use which crayon. (indicator 3 = 0). Neither was there a focus on conceptual knowledge, with no guidance on the knowledge tasks being supplied by the teacher (indicator 15 = 0). Score 1

See Lesson 8 above

Score 2

See Lesson 41 above

Language of instruction Indicator 4. How proficient are teachers in the language of instruction? Indicator 5. How proficient are pupils in the language of instruction? Indicator 6. Are explicit attempts made to promote development of the language of instruction?

95

part III Implications of the model for classrooms, schools and districts

No scoring was done on indicators 4–6. However, two excerpts illustrate some of the problematic language issues common to these classes. Lesson 9 (maths, Grade 4, total score 14) The lesson was spent investigating the following problem: Sipho has R25 to spend on rulers and pens. A ruler costs R4 and a pen R3. He must use all the money. Find how many rulers and how many pens he can buy. While problems of this type are appropriate for Grade 4, the entire lesson was spent on it (indicator 15 = 2), and much more progress could have been made (indicator 16 = 1). Manipulatives in the form of 25 counters were handed to each pair of pupils to assist in solving the problem. Each pair was to find a solution, write a mathematical expression for the problem, and then present their solution to the class: this worked well (indicators 2 = 2, 3 = 2). Thus, an explicit link between the problem and its formulation in the language of maths was forged (indicator 17 = 2). The teacher commented in detail on each solution (indicators 18 = 2). More efficient use could have been made of the time if the pupils had not been required to use the counters, which in many cases proved a hindrance to finding a solution. Also, there was much repetition and much time was wasted in getting every pair to report on their method. Many pupils had difficulty understanding the problem and the teacher took considerable pains to explain it to them, resorting to Zulu whenever difficulties arose (indicator 14 = 2). Lesson 45 (English, Grades 5 & 6, total score 7) This was a comprehension lesson in which the teacher read a story and pupils were required to answer questions, first verbally and then by means of multiple choice written answers.

96

During the verbal session the pupils were allowed to answer in Zulu, thus missing one of the major goals of any English lesson: to learn to speak English. Code switching between children’s first language and the language of instruction has been shown to promote conceptual learning under certain circumstances (Setati et al., 2002). However, as the second excerpt shows, code switching can hinder development of the language of instruction, and thus, in the longer term, undermine conceptual development.

Promotion of reading and writing Indicator 7. How much writing is done by pupils (all writing tasks are counted, including classwork, homework and written assessment tasks – tests, exams, projects, assignments)? M

S

E

54

36

69

Children in the classes surveyed undertook almost twice as much writing in English as they did in science, with maths occupying an intermediate position. While this may be understandable on the grounds that ‘language’ requires more writing than ‘practical’ subjects like science, our model predicts that learning to write any discourse genre is critical to learning its realisation rules. Score 0 Lesson 11 (maths, Grade 7, total score 8) Apart from the work of individual respondents on the board – which was to shade the columns of a bar graph, no writing was done. The classwork books showed that the only writing that had been done during the 2 weeks of the term was the short exercise comprising the present lesson (indicator 7 = 0). Three very short tests had been written since the beginning of the year (this was early June): these consisted of a few 1-,

chapter 6 Curriculum delivery in the classroom

2- and 3-digit additions, or 3 2-digit multiplications (indicator 8 = 0). Score 2

The material may consist of textbooks, worksheets, magazines.

See Lesson 46 below

M

S

E

Indicator 8. Quality of writing: do individual pupils engage in extended writing activities? This may be done during group activities, but each pupil in the group needs to be engaged.

13

27

47

M

S

E

22

17

34

This is one of the lowest scoring indicators, with scores almost exactly half those of indicator 7. Score 0

See Lesson 5 above

Score 1 Lesson 42 (English, Grade 4, total score 14) An examination of the pupils’ test books (in May) showed that one test had been written in each of the previous three months. The first consisted of a 10-word spelling test, the second of short, one sentence answers to a comprehension exercise, and the third of a sentence-completion exercise which required filling in single words (indicator 8 = 1). Score 2 Lesson 46 (English, Grades 5 & 6, total score 7) Only one test had been written (by May), although exercise books contained a considerable quantity of written work, which included exercises on comprehension, grammar, and sentence construction (indicator 7 = 2). Children also had separate composition books, which contained extended writing exercises (indicator 8 = 2).

Indicator 9. Quantity of reading: do individual pupils engage with written material? This may be done during group activities, but each pupil in the group needs to be engaged.

This is another very low scoring indicator, particularly in maths and science, which gives much cause for concern, considering the importance given by our model to reading in the development of conceptual competences, and in developing independent learning skills. Furthermore, indicator 9 gives an indication only of how much time is spent reading: assessing the quality of reading, an even more critical indicator, would require more intense observational techniques than were utilised in this study. Score 0 Lesson 28 (science, Grades 3 & 4, total score 10) The lesson consisted of following an experiment with water and air from the textbook. Books were only available for each group of 6 children, of whom one was the designated reader. Only the reader read from the book, during which time a number of other members of the group were obviously not paying attention (indicator 9 = 0). Score 1 Lesson 44 (English, Grade 6, total score 6) The topic of the lesson was ‘interviewing’. Children worked in groups and asked each other questions about their personal lives (name, age, name of father, place of abode, etc). There was no explicit or implicit focus on any component of the vertical discourse of school English (hence indicators 14, 15 and 16 all scored 0). The pupils discussed the task in their groups (indicator 3 = 2). The answers were written on loose sheets handed out by the teacher, apparently because the children had no exercise books. Considerable time was spent on the writing (indicator 7 = 2),

97

part III Implications of the model for classrooms, schools and districts

although this did not progress beyond simple sentences (indicator 8 = 1). Each pair was then called forward to read their ‘findings’ to the rest of the class (indicator 9 = 1). According to the observer, it was clear that a number of children were very poor readers, stumbling over the most elementary words. Score 2

No examples

Macro-level pacing – planning, coverage and sequencing Indicator 10. Is there evidence that the teacher has planned the delivery of the curriculum over the year so as to ensure: • coverage of all the outcomes? • regular formal assessment tasks? • logical sequencing of topics? M

S

E

26

30

41

Only 2 teachers, 3% of the 65 observed, produced adequate year plans, while 33 (51%) could produce no plan; the remainder produced what were considered inadequate plans according to our scoring criteria. This means that the overwhelming majority of teachers surveyed are either working on intuition or following the textbook which might be excellent or inadequate. This situation leaves much space for inadequate coverage of the curriculum, and poor coherence across classes, grades and schools. Score 0 Lesson 5 (maths, Grade 4, total score 4) The teacher claimed that, as a member of SADTU, she would not show her annual or daily plans to anyone, as it is SADTU policy that all preparation is the private responsibility of the individual teacher. (This argument was repeated by two other teachers). The lesson was taught directly out of the textbook. While examining the pupils’ classwork books, the researcher noticed that the

98

lesson had been done before as a class exercise (indicators 10–13 = 0). Score 1 Lesson 42 (English, Grade 4, total score 14) Although a learning programme was produced, the goals were phrased in terms of the SO, AC language of C2005, making it difficult to decipher the meaning of what was intended, and whether these intentions were realised in the classroom (indicator 10 = 1). Score 2 Lesson 26 (science, Grade 4, total score 9) The teacher followed the learning programme supplied by the project.

Indicator 11. Is there evidence that the curriculum plan is being followed? M

S

E

0

0

0

Indicators 11 and 13 are the only two that were scored on a yes/no basis, scoring either 2 or 0. Of the 32 teachers who did produce an annual plan of one sort or another, none could demonstrate how they were keeping track of coverage of the curriculum standards.

Indicator 12. Is there evidence that the teacher has planned the lesson under observation, with sufficient detail to show: • knowledge content? • class activities? • homework (if appropriate?) M

S

E

27

39

34

Score 0 The majority of teachers were unable to produce a lesson plan.

chapter 6 Curriculum delivery in the classroom

Score 1 Lesson 11 (maths, Grade 7, total score 8) The teacher had a curriculum plan which covered work for 2 weeks: this was inadequate as a curriculum plan to ensure coverage for the year, and contained insufficient detail to serve as a daily plan to guide the lesson (indicators 10 and 12 = 1). Score 2 Lesson 13 (maths, Grade 7, total score 12) The lesson plan specified the outcomes (converting from common to decimal fractions), activities (whole class exposition; paper cutting, written work in groups), and exercises (page references to text book) (indicator 12 = 2).

Indicator 13. Is the daily lesson plan up to date with the curriculum plan? M

S

E

0

9

0

Score 0: Only two teachers were able to demonstrate that their daily lesson plans were appropriately co-ordinated with an annual plan. The remainder scored 0 in this indicator. Score 2: Lesson 21 (Science, Grade 4, total score 11) Although the year plan showed only a section of the work (and thus scored 1 for indicator 10), the lesson plan was properly located within this section of the annual plan, indicating that it was conceived as part of a larger plan (indicator 13 = 2).

Micro-level pacing Indicator 14. Is the teacher ascertaining at what level the learners are, and engaging them at that level? M

S

E

35

29

22

The low scores indicate that teachers generally follow their own pace during lessons, making little or no attempt to ascertain whether the children are keeping pace or finding the level of cognitive demand too easy. Score 0 Lesson 6 (maths, Grade 4, total score 6) Most of the lesson was spent in groups trying to work out how many 10c pieces make up 110c (indicator 15 = 2). While the level of cognitive demand of this problem would have been appropriate if the purpose of the lesson had been to investigate division, this was apparently not the focus. Nevertheless, the cognitive demands of working with such quantities of money were more or less appropriate to the grade. However, a score of 1 was allocated for indicator 16, rather than 2, as very little progress was made during the lesson, with most groups not even managing to get the right answer for this one problem. No attempt was made by the teacher to ascertain the extent to which the learners were engaging with the content (indicator 14 = 0). There was much confusion and most groups seemed unable to make progress: many did not seem to understand the question. Eventually the teacher intervened but the explanation was not clear to many pupils (indicator 18 = 0). Score 1 Lesson 35 (science, Grades 5 & 6, total score 11) Although the children sat in groups this was a whole class lesson, with a demonstration of filtration performed at the front by pupils volunteering and coming up to carry out individual parts of the experiment. The object of the experiment was clear: the filtration of permanganate solution (indicator 15 = 2). However, the teacher did not make the most of her opportunities to clarify the concepts (indicator 18 = 1), nor to stretch childrens’ understanding (indicator 16 = 1). Learners were all able

99

part III Implications of the model for classrooms, schools and districts

to follow the teacher but no attempt was made to extend their conception (indicator 14 = 1). Score 2

See Lesson 9 above

Cognitive demand Indicator 15. Is the knowledge object of the lesson clear? M

S

E

63

63

44

It is noteworthy that the scores for maths and science on this indicator are almost 50% higher than the English score. It was sometimes difficult to discern the knowledge object of a number of lessons in all three subjects. This problem was particularly acute in English. This may be because the knowledge standards for vertical discourses with a strong grammar – such as maths and science – are more clearly defined than they are for discourses characterised by a weak grammar – like English. The move towards whole language teaching, as opposed to a more formally structured curriculum, in the last decade or so may have aggravated this situation. Score 0 Lesson 10 (maths, Grade 4, total score 2) In the lesson plan the topic was listed as ‘measuring length’. However, during the presentation, which the teacher followed from a resource book, she constantly switched between approximating numbers – e.g. 82 mm is about __ cm – and converting between mm and cm. She herself got confused a number of times, and was clearly not too sure about the correct conversion procedure. For example, she got 32 mm for the sum of 2 cm and 3 mm. Thus, not only was the knowledge focus unclear (indicator 15 = 0), but the level of cognitive demand was inappropriate (indicator 16 = 0), and the evaluation criteria were confusing (indicator 18 = 0). This lesson

100

may well have done the pupils more harm mathematically than good. Score 2 Lesson 31 (science, Grade 7, total score 5) This lesson illustrates the fine line between a good lesson and a confusing one, in which pupils learn very little. The focus of the lesson was the conduction of heat and the good conductivity of metals in particular, and this was clear throughout the lesson (indicator 15 = 2). It started with a teacher demonstration using an apparatus containing rods of 3 different metals, which were heated from the bottom. Children were then required to determine the order of conductivity by touching the rods. After completing half of the experiment, the teacher then asked groups of pupils to complete it. This turned out to be chaotic, as there were too many children and too few sets of apparatus for more than a few to participate, and those that did were not clear as to what was expected. Thus, the evaluation criteria were not explicated (indicator 18 = 0) and as a result the level of cognitive demand was inappropriate (indicator 16 = 0), and pacing was poor (indicator 14 = 0). Nevertheless, the relationship between everyday metals and the scientific concept of heat conduction was apparent (indicator 17 = 2). This was corroborated by an examination of learners’ workbooks, which showed that a great deal of integration of school and everyday knowledge did occur. For example, one question was about the energy crisis in South Africa and how it could be addressed.

Indicator 16. Is there a discrepancy/gap between the intended and enacted curriculum at the level of the contents/cognitive competences? M

S

E

53

48

28

chapter 6 Curriculum delivery in the classroom

In only eight of the lessons (12%) was the level of cognitive demand found to be appropriate to the curriculum standard. The situation was considerably worse in English than in maths and science. If children are not exposed to conceptual knowledge at the appropriate level then they cannot be expected to keep up, and, since this practice was found at both grades observed, it is probable that pupils are falling progressively behind in their acquisition of conceptual knowledge competences with each successive grade. Score 0 Lesson 3 (maths, Grade 4, total score 8) The focus of the lesson was the number line. Only single digit numbers were used: this was 3 grades below the appropriate level of cognitive demand. Score 1

See Lesson 9 above

Score 2 Lesson 32 (science, Grade 7, total score 12) This was a whole class lesson which followed the textbook closely. Thus the knowledge focus – amphibians – was clear (indicator 15 = 2) and the level of cognitive demand with respect to the curriculum was appropriate (indicator 16 = 2). The teacher asked questions and the learners were required to find the answers by reading the book (indicator 9 = 2). This generated much enthusiasm among members of the class, who competed for being asked to respond (indicator 2 = 2, 3 = 2). However, there was no authorisation from the teacher as to the appropriateness of children’s answers (indicator 18 = 0). This failure to explicate the evaluation criteria adequately was corroborated by an examination of the test books, which showed incorrect answers marked correct. For example, 10 m x 5 m = 50 m received full marks. The teacher seemed driven by a predetermined set of questions rather than any sense of whether the children were following or not (indicator 14 = 0).

Interdiscursive relations Indicator 17. Does the teacher intentionally and appropriately link everyday knowledge with curriculum knowledge? M

S

E

34

59

38

These scores indicate that it was easier for the teachers in our sample to bring everyday knowledge into science than into maths or English. This is understandable in the case of maths, since the discipline is more abstract than science, which at the Intermediate Phase level can be seen as an attempt to build a conceptual understanding of a number of phenomena that children are familiar with in their surroundings: plants, animals, inanimate matter, etc. The case as to why concepts in the study of English should be more difficult to relate to the lives of pupils is harder to make. Perhaps this is related to the comments made under indicators 15 and 16 above. It may be that, with-in the present understanding of the school subject English, the difficulty is not so much bringing the everyday to bear on formal knowledge, as the reverse: it may be that school English has come to be seen as the study of text largely from the perspective of the everyday, with conceptual knowledge being relegated very much to the background. Score 0 Many lessons scored 0 on this indicator, with no attempt being made to link everyday knowledge with curriculum knowledge. Score 1 Lesson 4 (maths, Grade 4, total score 9) The pupils engaged with the mathematical concepts in a hands-on way, using scissors and paper to cut out and examine the properties of various shapes. However, the teacher did not make the most of her opportunities to induct the children into the discourse of school geometry. Consider, for example the following extract:

101

part III Implications of the model for classrooms, schools and districts

‘T: “Compare a square and a rectangle.” L: “The one is tall and the other is short.”’ This was a useful ‘commonsense’ starting point for introducing a more formal definition of rectangles and squares in terms of the lengths of their sides. In failing to do so adequately, the teacher missed an opportunity to a build a bridge between the everyday knowledge of the pupils and academic geometry (indicator 17 = 1), thus leaving the latter at an inappropriately low level of cognitive demand (indicator 16 = 1). Score 2 Lesson 21 (science, Grade 4, total score 11) Through the use of seeds, which the pupils had collected and brought to class, the teacher guided them through a discussion of how ‘people and seeds need each other’: many edible plants require human and other agents for the dispersal of their seeds. This was then extended into a broader discussion of environmental issues. Thus, a solid bridge was built between the everyday world of the children and scientific concepts (indicator 17 = 2). Score 2 Lesson 33 (science, Grade 7, total score 13) The lesson consisted of an experiment performed in groups, in which each group had a magnet and a variety of both magnetic and non-magnetic materials. The teacher directed the experiment from the centre, talking the groups through the activity and asking questions, taking them carefully through common-sense notions – some materials ‘can be picked up’ – to the scientific concept of magnetic materials (indicator 17 = 2).

Explication of evaluation criteria Indicator 18. To what extent does the teacher: • ask leading questions so as to draw out conceptual principles?

102

• give feedback to learners’ questions, and verbal and written answers, authorising correct answers and correcting misconceptions and gaps? • ensure that the knowledge principles arising out of investigations and other activities are clearly and explicitly stated? M

S

E

31

25

25

The scores for all three subjects are uniformly low on this critical dimension. If children are not told whether their knowledge displays or social competences are appropriate, and if not, how a more suitable text may be constructed, their learning is likely to be haphazard at best. Score 0 Lesson 27 (science, Grade 4, total score 6) The topic of the lesson was ‘filtration’. Each group of learners was given a beaker, filter funnel and filter paper, which they used to filter water. However, the purpose of filtration was never discussed with the class (indicator 18 = 0): instead of serving as a means to understanding a scientific principle and perfecting a technique, merely getting through the exercise appeared to be the sole object of the activity. Score 0 Lesson 42 (English, Grade 4, total score 14) Lack of clarity in the explication of evaluation criteria may occur through overt mistakes made by the teacher, or through not giving adequate feedback, or any at all, on children’s responses to tasks. The following extract illustrates both kinds of problem. The topic of the lesson was ‘teaching children to construct sentences’, which remained in focus throughout (indicator 15 = 2), although this topic was somewhat below the cognitive level at which Grade 4 English lessons should be

chapter 6 Curriculum delivery in the classroom

pitched (indicator 16 = 1). The lesson started with the teacher handing out a hand-written sheet listing ‘sentence starters’: I, we, they, you, it, she, etc. The children were sitting in pairs and each pair was required to read the words silently. Volunteers were then called to describe a verb. After various attempts, the teacher defined a verb as ‘that part of a sentence that makes it a sentence’ (indicator 18 = 0; see below for corroboration). Each pair of children was then asked to construct a sentence from each of the 8 verbs put up by the teacher, and the ‘sentence starters’ contained in the list. The construction of the sentences was discussed by the pairs and entered into their notebooks (indicator 3 = 2). Each pair was then required to come to the front of the class and read the sentences they had constructed. Although this process enabled the teacher to ascertain how each pair was coping with the task (indicator 14 = 2) each of the 20 pairs took significant time to read their 8 sentences (indicator 9 = 2). In total this resulted in what the observer described as “not the most effective use of time”. The teacher offered no comment on the work of the pairs (indicator 18 = 0). The assessment record of pupils was grouped into categories: ‘picture’, ‘drawing’, ‘spelling’, ‘oral’, ‘language’, ‘key concepts’ and ‘learning skills’, and a mark of qualitative assessment given for each category. However, pupils’ books showed no evidence of communication of these criteria to the children (indicator 18 = 0). Score 1 Lesson 16 (maths, Grade 6, total score 14) The teacher obviously knew the names of all the children in the class and made a point of calling them by name. The observer notes that the class had a very warm atmosphere (indicator 2 = 2). On the face of it the level of

cognitive demand was appropriate for the grade (indicator 16 = 2). The topic was ‘division by 10, 100, 1000’, and examples like 730/10, 3700/100 and 53 000/1000 were explored. However, the topic was dealt with instrumentally, according to the rule ‘when you divide you knock off as many zeros in the number you are dividing into as there are in the number you are dividing by’. While the children enjoyed the lesson and were able to get the answers right, they did not learn the mathematical principle underlying this operation (indicator 18 = 1). Score 1 Lesson 43 (English, Grade 5, total score 1) This was the lesson which deviated most from the ideal predicted by our theoretical model, scoring 0 on every indicator except one. Although the teacher was very articulate, he adopted a highly imperative style (indicator 2 = 0). The lesson consisted of a story which the teacher read to the class, stopping to explain the theme, narrative structure and certain grammatical constructions, and to ask questions. These digressions did make available to the pupils evaluation criteria for a range of vertical knowledge elements, but this was not done systematically (indicator 18 = 1). It was very difficult for him to elicit any kind of answers from the pupils, who remained painfully passive throughout (indicator 3 = 0). It took continuing questioning for some 5 minutes, and a number of ‘don’t knows’, before someone correctly responded to a question about who the subject of the passage was: landowners, referred to as ‘they’ for much of the story. This indicated that the pupils did not follow much of the story (indicator 14 = 0). Score 2

See Lesson 9 above

103

part III Implications of the model for classrooms, schools and districts

The relationship between classroom practice and learner performance The performances of the project schools on the JET literacy and numeracy tests are shown in Tables 3.4–3.7. The results indicate that pupils in these schools are performing well below the standard at which the tests are set. This conclusion is particularly sobering in view of the discussion in Chapter 3, which describes how, in order to obtain results in which the majority of pupils registered any score at all, the Grade 3 tests were constructed at the Grade 2 level, and the Grade 6 instruments at the Grade 4–5 level. Thus, the mean scores reflected in Chapter 3 indicate the performance of Grade 3 children to be around the Grade 1–2 level, and that of Grade 6 children around the Grade 3–4 standard. Unfortunately, because our research design precludes a direct link between the performances of children and the pedagogic practices of specific teachers, no meaningful statistical correlation between the learner performances described in Chapter 3 and the pedagogical practices described in the present chapter could be performed. However, these two data sets are entirely consistent. The model predicts that the classroom practices reflected in Table 6.3 would result in the pupil test results shown in Tables 3.4–3.7. Care must be exercised in interpreting this statement. There is no substitute for a regression analysis to establish the strength of association between particular pedagogic factors which affect performance, and we reserve judgement until such an analysis has been performed. The present chapter indicates that the model derived in Chapter 5 provides a promising framework for investigating the influence of classroom-level factors on learner performance. However, a statistical analysis may indicate that only a small subset of the factors listed in Table 6.3 is associated with learner performance, which would entail a modification of our model. Or the analysis may show that our 18 factors are associated with only a small proportion of variation in the performance scores, in which case our model would require substantial respecification.

104

With these caveats firmly in view, it is instructive to discuss the links between the pedagogic practices described above, and the test performances exhibited by the children in our sample, as predicted by our model.

Social values This is an under-researched dimension of schooling. In the light of our theoretical considerations, which suggest that explicit value orientation may play a role in positive outcomes, we recommend that this aspect be included in a comprehensive investigation into the part played by individual pedagogic factors in learning.

Social relations Our analysis shows the persistence of topdown teaching styles in these classrooms, despite the widespread use of group-based pedagogic forms. Although the forms of learner-centred pedagogy are almost ubiquitous, the substance – the development of pupil initiative and interaction – is largely missing. The model predicts that learning in both the social and cognitive domains will be improved by a more personal style of communication between teacher and pupils, and more interaction among pupils. In order to achieve these goals, teachers must encourage and reward the exercise of initiative and responsibility, and communication among pupils during engagement with class tasks, where appropriate. Some examples of these practices in our sample schools are quoted above.

Language of instruction Congruent with the research evidence, the model predicts that a dedicated drive to improve proficiency in the language of instruction, whether or not this is the first language of the teachers and children, will improve proficiency, which in turn will have a powerful positive influence on cognitive and social learning. This is another area in which the present analysis suffers from a paucity of data. The very limited evidence at our disposal hints that code switching may inhibit development of the language of

chapter 6 Curriculum delivery in the classroom

instruction, where this is not the first language of pupils, although this does not preclude a role for code switching practices. The identification of effective strategies for developing proficiency in the language of instruction is another area of school research that would benefit from attention.

Promotion of reading and writing While the quantity of writing is low, particularly in maths and science, the quality in the overwhelming majority of cases is way below the level intended by the curriculum in all three subjects. This is likely to reinforce the effects of our findings concerning the low level of cognitive demand. The quantity of reading would appear to be inadequate for the development of what our theory premises to be possibly the single most important skill to be learned in the Foundation and Intermediate Phases. This prediction is supported by the observations made in Chapter 3, that many of the children in the sample have barely learned to read or write.

Planning, coverage, sequencing and pacing (macro and micro) The model assumes that learning will be promoted by strong framing of macro-level pacing of the curriculum, that is, by covering all the standards specified for the grade level. At the same time, pacing should be relaxed at the micro level, with the teacher remaining sensitive to the progress of individual pupils, and adjusting her pace accordingly. The evidence quoted above reveals that very little attention is paid in the classes sampled to: • planning and monitoring coverage of the curriculum, either at the macro level of the school year, or at the daily micro level • pacing daily learning activities so as to ensure that individual pupils are keeping up Under these circumstances it would be surprising indeed if these children completed the required curriculum during the course of the year. Thus, pupils are likely to fall further behind with each grade. Furthermore, there is

likely to be poor coherence across classes, grades and schools, with children receiving schooling of widely different quality, even within the same school.

Cognitive demand and evaluation Lack of a clear knowledge focus in many classes and the establishment of a very low level of cognitive demand as a matter of routine are likely to reinforce the effects of poor planning and pacing noted above, in systematically depriving these children of the learning experiences intended by the curriculum. It seems highly likely that the test results summarised in Tables 3.4–3.7 are at least in part explained by the composite effects of poor planning, slow pacing and low cognitive demand. The way in which evaluation criteria are conveyed to children is likely to further add to the cumulative effects of poor planning, coverage, pacing and inadequate knowledge focus in providing weak learning experiences for these children. Thus, even where there is a knowledge focus to the lesson, and where the level of cognitive demand is somewhere approaching the curriculum specifications, the conceptual principles are not made clear to the pupils, either through faulty formulations on the part of the teacher or, more commonly, through giving little or no feedback to the knowledge displays offered by pupils.

Interdiscursive relations Our observations show that teachers are maintaining a reasonably good relationship between everyday knowledge and conceptual knowledge in science. This is understandable in view of the common presence of the objects of school study at this level in the daily environment of children. It is obvious that showing links between horizontal discourse and school maths is more difficult, with a number of teachers attempting to explore this interdiscursive space with mixed success. Our theory points to the exercise of caution in this process so as not to obscure the conceptual principles.

105

part III Implications of the model for classrooms, schools and districts

Without a firm grasp of the conceptual base of maths, teachers are unlikely to be able to do this well. In English it would seem that the opposite problem requires attention. There is a paucity of conceptual knowledge in the content of these lessons, and it seems likely that this would slow the acquisition of principled knowledge of the language. The same principle would apply to any language of teaching and learning.

Conclusion The description of the pedagogic practices in our sample schools, in the light of learnerperformance data from the same schools,

106

confirms the usefulness of the model derived in Chapter 5 for guiding the search for classroom-level leverage points to improve learning. On questions of design and methodology, the investigations conducted during the present chapter confirm the recommendations made in Chapter 4. Our analysis of classroom behaviour in this chapter indicates that several pedagogic practices can be significantly improved at the classroom level, through the creation of an appropriate climate and the maintenance of curriculum management systems at school and district levels. We turn to an analysis of these issues in Chapters 7 and 8.

chapter

7 Organising schools That part of school life which falls outside the classroom is every bit as complex as the set of practices and relations characterising the classroom-level pedagogies described in Chapter 6. Resources are available to principals to orchestrate the work of schools into a variety of possible institutional cultures, leadership and management styles, and learning outcomes. In this chapter we bring the hypotheses comprising our model to bear on the school level, with a view to better understanding those permutations of pedagogic practice most likely to improve the life chances of children. Consistent with a founding assumption of the model, that schooling offers an inherent advantage to children from privileged homes, a particular interest of our quest is to identify school-level practices which increase the probability of poor children succeeding at school and gaining access to a wider variety of post-school choices.

Research question The central research question at the level of the school is: what factors of school design and organisation best promote learning in both the social and cognitive domains? Since these activities will be aimed at creating a climate which facilitates classroom learning, and directing, supporting and monitoring the work of teachers, the same pedagogic dimensions derived in Chapter 5 and used to investigate classroom-level practices in Chapter 6 are used to disaggregate our research question at the school level into 24 sub-questions. These are shown in Table 7.1 on page 108.

Design and method As discussed in Chapter 6, the present study is an ex post facto analysis of evaluation studies conducted prior to the formulation of our model. Although the collection of data was implicitly guided by the assumptions underlying the model, little data was collected about

values and social relations (indicators 1–6), on the organisation of tasks within the school (indicator 6), or on the management of resources other than books (indicator 11). In addition, the data informing the remaining indicators is far sparser than the classroomlevel data reflected in Chapter 6. Consequently, no attempt is made to score the quality of performance within each indicator, and once again, no statistical correlation was attempted between pupil test scores and the pedagogic practices reflected by our indicators. The principal purpose of this analysis, therefore, is to measure the assumptions of the model against the available data. Any incompatibilities will alert us to potentially weak components of the model, while any suggestive congruence between theory and evidence will increase our confidence in the respective hypotheses. For detail on the sample and method, see Chapter 6. In addition to the data from Siyathuthuka and Phalaborwa schools (JET, 2001d; 2001e; 2001f; 2001g) we also used information from the Mahlahle baseline study (JET, 1999; 2000) summarised in Table 2.1.

Data analysis Social values Indicator 1. To what extent are the values embodied in the curriculum included in the school mission? The curriculum in operation at the time of the collection of the data, which is the subject of our analysis in Chapters 6 to 8, was the original C2005, described in some detail in Chapter 5. The values around which that curriculum was built have been retained in the Revised National Curriculum Statements (DoE 2002a), which arose out of the 2000 Review of C2005. These are listed in the seven Critical and five Development Outcomes published by the South African Qualifications Authority (Government Gazette, 1995). They describe young citizens able to:

107

part III Implications of the model for classrooms, schools and districts

Social organisation

SUBCONSTRUCT

INDICATORS

Social values

1. To what extent are values embodied in the curriculum included in the school mission?

Social relations

2. How open are relations 3. How open are relations 4. How open are relations outside the classroom? 5. How open are relations

Task organisation

6. To what extent are roles and responsibilities of all role players – principal, SMT, subject heads, teachers, pupils and parents – clearly defined?

between school and parents?

7. Does the school maintain systems for maximising the number and length of working days? 8. Does the school monitor attendance of teachers and pupils? 9. Does the school monitor punctuality of teachers and pupils? 10. Does the school monitor time spent in the classroom?

Resource management

11. Does the school maintain systems for management of finances and resources other than books?

Proficiency in language of instruction

12. Does the school have a programme for developing proficiency in the language of instruction?

Promotion of reading and writing

13. Does the school have a policy on reading and writing? 14. Is implementation of the policy monitored? 15. What is the state of school systems for procurement and management of books and stationery?

Curriculum and pedagogy

Planning, coverage, sequencing and pacing

16. Do subject/phase heads lead collective planning of curriculum coverage – including regular formal assessment tasks – for the year/quarter, by grade level? 17. Is coverage of the curriculum monitored? 18. Does the school have a programme of in-school INSET (including subject meetings, workshops on content/ pedagogy, team planning/discussion)? 19. Does the school encourage teachers to attend out-school INSET courses, conferences? 20. Does the principal/subject head/phase head support and monitor teachers in the classroom?

Evaluation

Explication of evaluation criteria

21. Do subject/phase heads quality assure content of formal assessment tasks against curriculum standards? 22. Are results of formal assessment tasks evaluated by subject/phase heads? 23. Are results of formal assessment tasks used to improve teaching/learning, through planning, monitoring and support? 24. Are results of formal assessment tasks reported to parents?

• identify and solve problems and make decisions using critical and creative thinking. • work effectively with others as members of a team, group, organisation and community. • organise and manage themselves and their activities effectively. • collect, analyse, organise and critically evaluate information.

108

between principal and teachers? between principal and pupils? between teachers and pupils,

Time organisation

Language

Table 7.1

SCHOOL-LEVEL INDICATORS DERIVED FROM THE THEORY OF SCHOOLING THEORETICAL CONSTRUCT

• communicate effectively using visual, symbolic and/or language skills in various modes. • use science and technology effectively and critically, showing responsibility towards the environment and health of others. • demonstrate an understanding of the world as a set of related systems by recognising

chapter 7 Organising schools

• • • • •

that problem-solving contexts do not exist in isolation. reflect on and explore a variety of strategies to learn more effectively. participate as responsible citizens in the life of local, national and global communities. be culturally and aesthetically sensitive across a range of social contexts. explore education and career opportunities. develop entrepreneurial opportunities.

Something of a mix of generic competences and specialist detail, the SAQA Outcomes are quoted in full here as they provide a core to the South African school curriculum, having survived seven years of curriculum change, and no summary can adequately capture their breadth and suggestiveness. One of their most important features, from the perspective of our model, is the way they encapsulate the interrelationships between social and cognitive values. Our data set contains no information on indicator 1, and we turn for assistance to the Feeder Schools study described in Chapter 4 (Malcolm et al., 2000), which has much to say on this issue. In describing the organisational culture prevalent in the ten schools sampled, the study concluded that the following are common to the ‘successful’ school: • The focus of the school is learning, in both the social and discursive terrains. • Regarding the cognitive dimension, the chief indicator of success is pupil performance in the annual matric exam; learning in the lower grades is principally directed toward this purpose. • Dedication, self-discipline and a sense of responsibility on the part of staff and students alike are seen as key to achieving the goals of the school. A combination of inspirational leadership, monitoring of performance and the application of rewards and sanctions by school management are important elements in the culture of these institutions.

Social relations Indicator 2. How open are relations between principal and teachers?

Indicator 3. How open are relations between principal and pupils? Indicator 4. How open are relations between teachers and pupils, outside the classroom? Indicator 5. How open are relations between school and parents? Here too our data set is unable to shed light on these components of the model, and we again rely on the findings of the Feeder Schools study. The nurturing of a strongly cooperative atmosphere was found to be common to all nine successful schools. In comparing the successful school to a family, one principal emphasised the importance of a personal mode of communication between all the actors. Another principal used the metaphor of a three-legged pot, with staff, pupils and parents each playing an essential role in maintaining a climate conducive to learning.

Task organisation Indicator 6. To what extent are roles and responsibilities of all role players – principal, SMT, subject heads, teachers, pupils and parents – clearly defined? The strong classification of roles and tasks – or the clear delineation of functions within the school – is a prerequisite for the efficient functioning of the institution. No information is available on this indicator.

Time organisation Indicator 7. Does the school maintain systems for maximising number and length of working days? Indicator 8. Does the school monitor attendance of teachers and pupils? Indicator 9. Does the school monitor punctuality of teachers and pupils? Indicator 10. Does the school monitor time spent in the classroom? The discussion of indicators 1–10 above reveals gaps in our data set, drawn from the evaluation studies described in Chapter 2. What this means is that, while these

109

part III Implications of the model for classrooms, schools and districts

evaluations paid close attention to charting instructional factors in the schools studied, they paid scant regard to social organisation. Conversely, while the Feeder Schools research provides an indicative correction to the gaps in our own data set, this work provides little substantive information about the organisation of instruction. While it mentions a number of instructional elements, it gives no information on which systems are effective in regulating the work of teachers and pupils, nor how such regulation is best effected. Thus, the contrasts between our own evaluations and the Feeder Schools study reveal the strengths and blind spots of research perspectives, which lean heavily towards performance and competence approaches respectively. Our theoretical model provides an integrated frame for using both perspectives to give a three-dimensional view of schooling. Almost all of the research studies reviewed investigated whether schools had timetables. Across the various studies it was shown that, with few exceptions, all the schools surveyed had timetables – no matter how dysfunctional they were or how poor other management systems were. It was only in exceptional cases that schools reported that they had not completed the timetable or required assistance in order to do so. The fact that a school has a timetable is not sufficient cause to regard it as prioritising effective time usage. Schollar’s report on the Imbewu evaluation (Schollar, 2001b) notes that, while most schools had timetables, it was commonplace to see teachers not in their classrooms, learners wandering around the school during lesson time, and in some cases, teachers appearing unprepared to teach on the day of the visit to the school. In order for the timetable to play a meaningful role in the school it must be displayed prominently and then operationalised, with teachers knowing which classes to teach when, and at what time periods start. A few of the studies reviewed investigated the processes leading up to the production of the timetable. The Mahlahle project baseline studies (JET, 1999; 2000a) showed that, in

110

both primary and secondary schools, up to two weeks of teaching time may be lost or disrupted while teachers struggle to complete the timetable. In 80% of the primary schools and most of the secondary schools surveyed, it was reported that timetables are constructed during the first two weeks of the year. Some schools reported that they use a ‘temporary timetable’, often based on the previous year’s timetable during this time. However this is not a foolproof method of ensuring that teaching takes place, as teachers may have changed and the numbers of classes may have changed from one year to the next. In the Siyathuthuka study (JET, 2001d; 2001e) a similar pattern was found, where 60% of schools surveyed took two weeks or more to finalise the school timetable. Similarly, in the Mahlahle study it was found that, because many schools spend the first two weeks of the school year finalising the school timetable, schools only begin operating normally in the third week of the first term (JET 1999; 2000). The introduction of Curriculum 2005 was cited as one of the reasons why schools struggled to construct timetables. It was not uncommon for researchers to indicate that schools operated two timetables simultaneously – an ‘old’ timetable based on a five-day cycle, and an ‘OBE’ timetable utilising notions of flexitime and operating on an eight-day cycle. Very few studies investigated the quality of the timetables, investigating whether sufficient time had been allocated to key subjects such as mathematics, science and English. None of the studies reviewed investigated the extent to which the timetables were utilised in the schools. Some anecdotal evidence was provided making it clear that schools were not making use of their timetables. Schollar’s research (quoted in Taylor and Vinjevold, 1999) reported that it was not unusual for around half of available teaching days to be taken up by non-teaching activities, including the setting and marking of exam papers, cultural and extramural activities, funerals and teacher absenteeism. The school day is also eroded when learners are expected to clean classrooms or partic-

chapter 7 Organising schools

ipate in sports activities towards the end of the day during scheduled teaching time (JET, 1999; 2000). The QLP report (Kanjee et al., 2001) indicated that teaching time was also lost due to lessons starting later than their scheduled times, and teacher and learner absenteeism. Another common reason given for teaching time being lost was teachers’ participation in development activities. While these activities aim to improve classroom practice, opportunities to learn are lost when teachers attend these programmes. In the QLP baseline study it was reported that the schools surveyed taught, on average, for 152 days each year. The secondary schools surveyed are thus losing about 40 teaching days each year. Few of the studies investigated whether schools had formalised systems for catching up lost teaching time. One exception to this was the Siyathuthuka baseline study (JET 2001d; 2001e), where some interviewees indicated that they extend the school day by half an hour or an hour, or that classes are held on weekends or during school holidays. Some teachers indicated that they left work for learners to complete in their absence. However, these practices could not be verified, and their implementation seemed to rely on the willingness and initiative of individual teachers. No standard systems for catching up lost time were found, nor did it appear that it was easy to alter the schools’ operating times, as many teachers (and learners) in rural areas rely on public transport which arrives and departs at set times. Only two studies investigated whether schools had systems to monitor teacher attendance. The QLP report indicates that 15% of educators lose about 25 working days each year. The report does not provide further evidence on how this problem is dealt with by school managers or on the extent to which regular defaulters are recorded and action is taken against them. In the Siyathuthuka baseline study, it was found that all 10 schools maintained attendance registers for teachers, although the researchers note that the usefulness of the system was undermined when

teachers did not record the correct departure times and where managers did not assess the veracity of information recorded. In one school, several hours before departure time, it was noted that teachers had already indicated their departure times.

Resource management Indicator 11. Does the school maintain systems for management of finances and resources other than books? Our model postulates that the effective management of resources is at least as important, in maximising learning, as the level of resourcing available to the school. No information is available on this indicator.

Language of instruction Indicator 12. Does the school have a programme for developing proficiency in the language of instruction? The Feeder Schools study (Malcolm et al., 2000) found that successful institutions in their sample strongly encouraged, in and out of the classroom, the use of the language of instruction – which in all cases was English, and not the home language of pupils – and an explicit focus on the development of proficiency by pupils and teachers, who took great pride in their ability to speak the language of instruction. This is in agreement with the other findings on language described in Chapter 4.

Promotion of reading and writing Indicator 13. Does the school have a policy on reading and writing? Indicator 14. Is implementation of the policy monitored? Indicator 15. What is the state of school systems for procurement and management of books and stationery? The existence of systems for setting and monitoring policies on reading and writing was not investigated in any of our studies, although several investigated it indirectly by looking at the ways in which schools manage their learning resources, including the procurement, storage, dissemination and

111

part III Implications of the model for classrooms, schools and districts

retrieval of books and stationery. Schollar (2001b, 11) noted that “many schools have chaotic storage and administration systems and loss [of learning materials] is common”. This statement summarises the findings of the other studies, which found that schools do not have efficient procurement, management, dissemination and recovery systems. The Mahlahle and Siyathuthuka baseline studies (JET, 1999; 2000; 2001d; 2001e) found that most schools had some system for recording the delivery of books. In most cases these could not be described as efficient inventory systems, since year-on-year totals of books were not recorded, nor were records organised in a systematic manner. Records were often difficult to access, with many schools indicating that the teacher responsible for maintaining stock records was not available on the day of the visit, which explained their inability to provide any evidence of learning-material management systems. Few schools maintained accurate or systematic inventories where entries were organised either by grade or by subject. Many schools had inventory books, but few had been updated, some for more than 5 years prior to the studies, despite the regular delivery of new textbooks to support the implementation of Curriculum 2005. Delivery notes were used as proxy inventories, with delivery notes for books being stored with delivery notes for cleaning materials and other items. These delivery notes were seldom filed systematically. Sound procurement systems are necessary in order to ensure that teachers select and order books which they will use. The studies showed that procurement systems were haphazard, with teachers selecting books without having reviewed them. Teachers reported that they selected setbooks and textbooks from a catalogue produced by the Department of Education, without having had an opportunity to look at the books. Teachers indicated that they selected books on the basis that their titles indicated that they were ‘OBE-compliant’. The ‘blind’ ordering of books leads to massive losses when teachers find that the language

112

levels in the books are too complex, or discover later that a more appropriate book is available. This means that books that have been ordered lie in storerooms unused, often not unwrapped from their plastic wrappers. Once books have been ordered and supplied, schools need systems to regulate the storage and dissemination of materials. Studies conducted in rural primary schools found that many schools lack basic storage facilities, which means that books are often stored in cupboards in the principal’s office, the staff room and classrooms, or are stacked in piles in these various rooms. Where schools have storerooms, the books are stored alongside maintenance materials, cleaning materials and any other items which the school wishes to pack out of sight, like damaged furniture and cooking pots. The availability of storerooms appears to vary from province to province, with 80% of the schools in the KwaZulu-Natal-based Siyathuthuka study having storerooms, while only two of the Mahlahle schools in the Northern Province had burglar-guarded storerooms. Even where books are stored in cupboards in classrooms, the researchers report that the books shared space with rats, termites, ants and bats. The researchers noted that books were not stored systematically, with books for different grades being placed in the same piles and different subjects being mixed together. One of the worst cases reported was of a school that had chosen to throw all books not currently in use into a covered area between two buildings. The main reason given for this approach was that the books were not OBE compliant and had therefore been ‘thrown away’. The Mahlahle primary schools study found that only 3 schools (20%) had stored books by grade and subject. The reports indicate that poor storage and recording systems make it difficult to manage the distribution of books to teachers and learners. The Mahlahle baseline study indicated that few schools had stock control systems. Even where teachers reported that they kept ‘distribution registers’, these were

chapter 7 Organising schools

often unsystematic and did not indicate which learners had received which books (book stock numbers were often not recorded). Teachers indicated that they did not distribute books to learners except in the senior primary grades as they feared that the learners would not look after them and that distributing books would lead to greater stock losses. The Mahlahle secondary schools study showed that even where books were distributed, this was not done systematically, which made it difficult for teachers to account for the books which they had received. Teachers indicated that where there were insufficient books for all learners, they did not distribute books to the learners, preferring to distribute materials at the start of each lesson. A significant amount of time is wasted during lessons on fetching, distributing and collecting materials. The national Ministry of Education made it a priority for schools to retrieve textbooks at the end of the school year and halt the attrition of materials. Only three studies investigated the extent to which schools had systems in place to retrieve textbooks. The studies found that, due to poor stock control and inaccurate inventories, the schools could not track what materials they had and what had been lost. In a number of instances in both the Mahlahle and Siyathuthuka studies, teachers complained that they were unable to apply any sanctions to learners who failed to return books or indicated that books had been lost. The teachers blamed the fact that they were no longer permitted to withhold learners’ results for their inability to ensure that learners returned materials which had been given to them. In some cases teachers indicated that they expected learners to replace books which they had lost, but because many rural villages were far from book stores or book distributors, parents preferred to give them money to replace the books. This money was deposited into the general school fund and no schools could provide evidence of replacement books being purchased. Such indirect measures do not of course

speak directly to the issue of fostering reading and writing. What they do, at a crude level, is display the obstacles to fostering this essential skill, and they are also a graphic symptom of its absence.

Planning, coverage, sequencing and pacing Indicator 16. Do subject/phase heads lead collective planning of curriculum coverage – including regular formal assessment tasks – for the year/quarter, by grade level? Indicator 17. Is coverage of the curriculum monitored? The studies which did investigate the management of curriculum planning found that, despite school managers professing an awareness of the importance of overseeing and monitoring the long-term planning of the curriculum, there was no evidence that it had any meaningful effect on their management practices. Some studies investigated the extent to which school managers insisted that teachers produce long-term teaching plans and then reviewed and commented on the quality of these plans. In general, these studies found that little systematic curriculum planning takes place, and that systems are often not in place in schools to ensure that planning takes place. In the Mahlahle baseline studies in primary and secondary schools, the QLP baseline study and the Siyathuthuka Baseline evaluation, school managers indicated that teachers are expected to produce long-term teaching plans. However, on average only half of the teachers observed in each study were able to produce evidence that they had developed plans which would ensure that the curriculum was covered during the course of the school year. Low levels of curriculum planning appear to be endemic in both primary and secondary schools. The primary school studies found that only 30% to 50% of the teachers observed were able to produce any form of curriculum plan or lesson plan. The QLP baseline study, which focused on secondary schools, found that although 80% of teachers surveyed said that they had curriculum plans, only 40% of mathematics

113

part III Implications of the model for classrooms, schools and districts

educators and 27% of language educators could actually produce these plans. These observations give support to the more detailed analysis of these indicators in Chapter 6. Where teachers do prepare long-term teaching plans (also known as macro plans or schemes of work), the quality of these plans is poor. The Phalaborwa study reports that only 3 teachers (8% of those observed) produced curriculum plans that were more detailed than a simple list of topics to be covered. The Mahlahle secondary schools study also reported that only 8% of teachers’ plans were sufficiently detailed to guide and inform teaching. Several studies raised concerns at the fact that curriculum plans were little more than lists of content topics by week. In most cases the topics to be covered were listed in very general terms (e.g. fractions or comprehension) and there was no indication of the actual content to be taught or the skills that the teacher wished to develop. In extreme cases teachers simply listed the specific outcomes and assessment criteria to be applied during the course of a particular week, with no indication of the related subject content, assessment techniques or materials to be used. Planning systems were also found to be inconsistent with respect to the level of detail in curriculum plans and length of planning cycles. A study conducted in one region in KwaZulu-Natal found that the length of planning cycles varied from two days to a year. Even within schools, different planning cycles exist at different grade levels and in different subjects. The lack of uniformity in planning cycles should not be assumed to be unique to this area, as the Mahlahle baseline study found that some teachers planned for a few months at a time while others produced curriculum plans for the entire year. Studies also found that the level of curriculum planning varies between different subjects. The Siyathuthuka baseline study showed that levels of preparation were lower in Grade 6 (in English, mathematics and science) than in Grade 4. Similarly the

114

Mahlahle secondary schools study showed that levels of preparation were lower among Grade 8 teachers than Grade 9 teachers. In these schools lower levels of preparation were attributed to the introduction of Curriculum 2005 in each of these grades. Teachers reported that they were unsure of how to develop C2005-aligned plans, or were awaiting more guidance before embarking on planning. Often it appeared to the observers that the ‘waiting for OBE’ phrase became a catch-all for poor administrative and delivery systems. The process of producing curriculum plans does not appear to be well managed. In some schools teachers took between oneand two-and-a-half months to produce their annual curriculum plans. Data collected in the Mahlahle primary schools baseline study showed that teachers who produced quarterly plans used the last week of the school term to plan the following term’s work. The Mahlahle secondary schools baseline study showed that some teachers only produced their annual plans by the end of the first term. The fact that such poor quality planning exists suggests little involvement by school managers in ensuring that teachers plan the delivery of the curriculum. Teachers are not held accountable for ensuring curriculum planning and the coverage of the syllabus. The Mahlahle baseline study in primary schools found that teachers submitted their curriculum plans to the management team in only half of the schools visited. The baseline study conducted in secondary schools showed a more dismal picture, with teachers having incomplete plans in 9 of the 15 schools visited. In these schools teachers intimated that they no longer submitted their plans to the management team. Teachers consistently reported that when they submitted their curriculum plans to principals they did not receive qualitative or developmental feedback. The submission of plans to school managers appeared to be little more than a bureaucratic exercise, with one teacher saying that it was only a ‘formality’. In most cases managers simply signed and

chapter 7 Organising schools

stamped the submitted plans without giving teachers feedback on the quality of the plan, the logical sequencing of topics or how the curriculum was to be enacted. Where teachers did indicate that they received feedback on their plans it usually related to administrative errors, such as incorrect dates or the correction of spelling. The effect of not monitoring curriculum planning is clearly illustrated in the following school. One teacher – attempting to integrate across learning areas and to teach using a programme organiser – had been teaching Grade 5 learners about the water cycle for eight months. The learners had been taught the same content all this time. Better monitoring systems would have identified that very little content was being covered, and that learners were not mastering the skills and knowledge necessary for progression to the next grade.

Indicator 18. Does the school have a programme of in-school INSET, including subject meetings, workshops on content/pedagogy, team planning/discussion? Indicator 19. Does the school encourage teachers to attend out-school INSET courses, conferences? Indicator 20. Does the principal/subject head/phase head support and monitor teachers in the classroom? Quality assurance, that is, ensuring that all practices and services conform to an agreedupon specification, is one of the key mechanisms through which teachers can be both supported and held accountable for discharging their primary function: the delivery of the curriculum. This function of the principal, and indeed of officials at all levels of the system, has been eroded in South African schools, with inspection being stigmatised and some teacher unions indicating that any persons performing a supervisory function are ‘banned’ from the classrooms. This attitude has spread to other areas of quality assurance, with three teachers in one study, described in Chapter 6, indicating to the researcher that the union had informed them

that their lesson preparations were their personal property, and that they were not obliged to show lesson plans or any teaching plans to anyone else. These practices strongly undermine the authority of the school management team to implement systems to maximise curriculum delivery. Monitoring and quality assuring the delivery of the intended curriculum can take place in several ways, including (i) reviews of curriculum plans and learner workbooks, (ii) class visits and classroom observations and (iii) the implementation of formal appraisal systems. As with many other management practices, school managers reported an awareness of the need to monitor curriculum delivery, but interviews with teachers and a lack of evidence of monitoring systems indicate that these systems are not implemented regularly or consistently. There was little agreement between school managers and teachers within schools on the existence and implementation of monitoring systems. Some schools openly reported that no monitoring or quality assurance systems were in place. One form of monitoring reported by principals in the studies reviewed was a comparison between the intended curriculum, as expressed in teachers’ workplans, and the enacted curriculum, as evidenced by the work recorded in learners’ workbooks. Only three studies reported on monitoring practices in any detail, indicating that this practice is often honoured in the breach. Fewer than five schools in the Mahlahle baseline study reported that they tried to monitor the delivery of the curriculum by reviewing learners’ books. The secondary schools baseline study showed that in seven schools (45%) teachers submitted their schemes of work and learner workbooks for review each quarter. In these same schools 66% of managers reported that learner workbooks were reviewed, although teachers disagreed with their principals on this issue, reporting that these systems were not implemented regularly. Conflicting evidence of this kind is not uncommon with self-report data.

115

part III Implications of the model for classrooms, schools and districts

Many principals indicated that classroom visits by members of the school management team were stigmatised and that, as a result, they did not undertake classroom visits. Managers also indicated that they were unable to conduct regular classroom visits because they lacked the person power to do this, as they carried full teaching loads, giving them little time for supporting and monitoring teachers in their classrooms. Classroom visits were only reported by two of the ten schools visited during the Siyathuthuka baseline study. Given some of the difficulties mentioned above, these school managers indicated that they were rarely able to observe a full lesson, preferring to ‘pop in’ to classes and observe portions of lessons. In none of the studies did teachers report that they received any formal feedback on their lesson delivery. None of the studies reported that any of the schools visited were implementing formal appraisals or assessments of teachers’ abilities using the Developmental Appraisal System.

Explication of evaluation criteria Indicator 21. Do subject/phase heads quality assure content of formal assessment tasks against curriculum standards? Indicator 22. Are results of formal assessment tasks evaluated by subject/phase heads? Indicator 23. Are results of formal assessment tasks used to improve teaching/learning, through planning, monitoring and support? Indicator 24. Are results of formal assessment tasks reported to parents? Schools reported a range of practices with respect to how they quality assured assessment tasks. The researchers who reviewed teachers’ schemes of work reported that assessment tasks were often not indicated in their curriculum planning. Data gathered in the Mahlahle studies showed that schools seldom adhered to assessment policies, both policies developed by the schools, which regulated the frequency of assessment and the submission of mark schedules, and assessment policies developed

116

by the Department of Education. Despite schools reporting that learners wrote monthly tests in all subjects, researchers reported that learners had written far fewer tests than was expected at the time when the observations took place. There was little agreement between teachers and managers regarding the frequency of assessment and the nature of assessment practices in schools. This made it very difficult to obtain reliable data, particularly without independent observations by which self-report data could be confirmed or triangulated. The extent to which schools reported that they quality assured assessment practices varied markedly. In the Mahlahle primary schools only 20% of schools indicated that they quality assured tests and examination papers before they were administered to learners, while 73% of secondary schools reported that they moderated assessment tasks. A similar trend emerged in QLP secondary schools where 87% of educators reported that assessment tasks were moderated. However, the report contains no indication that this selfreport data was verified. In contrast to the practices in the Mahlahle primary schools, all primary schools in the Siyathuthuka study reported that they quality assured tests. This was reportedly done through various means, including regular submission of tests to senior teachers or subject-based HODs. In one school, teachers had established a ‘testing committee’ which met to determine a schedule for the submission of performance tasks for review, and which also met to review the results of learner performance assessments. It was not possible to corroborate or verify the practices which were reported.

Conclusion The school-level data at our disposal indicates that the monitoring and support systems which our theoretical model predicts to be most important for improving the quality of learning outcomes are very poorly maintained in the small number of schools in the sample. In particular, no standard systems exist for:

chapter 7 Organising schools

• improving proficiency in the language of instruction • promoting reading and writing • procuring and managing books and stationery • maximising teaching time • ensuring curriculum coverage • quality assuring assessment

On the question of testing the postulates of our model regarding the influence of these school-level pedagogical practices on pupil learning, our data is too sketchy to be anything more than sporadically illustrative. Perhaps the most that can be said is that the model is not contradicted by the data. We take this as further encouragement for using the model to guide a more comprehensive research programme.

117

chapter

8 Why districts matter The case for districts There is broad agreement that the district is the most appropriate level of government to initiate and sustain school reform, in the case of failing schools, and to monitor and support the routine maintenance needs of wellfunctioning schools. This agreement exists both in the international literature (for example, Elmore, 1993; and Lofton et al., 1998), and in South Africa (Malcolm, 1999; Mphahlele, 1999; 2002; Roberts, 2000; de Clercq, 2001; Narsee, 2002). The argument is simple and compelling: • None of the provincial departments of education, which, according to the authority delegated by both the Constitution (Government Gazette, 1993) and the South African Schools Act (Government Gazette, 1996), are responsible for the administration of schools, is able to interact directly with the large number of schools under its jurisdiction. • As the lowest level of government and the one closest to schools, district management is best placed to grasp and act on the local conditions under which schools operate. District management is thus the crucial link between school operation and management of this all important public service. How well it performs this linking function is consequently a question of some importance.

The South African context In Chapters 1 and 5 we pointed to a number of systemic problems in the South African school sector, which inhibit the work of schools through inefficiencies in providing essential resources such as textbooks, and through exercising only the crudest kind of monitoring and support functions at the school level. Much has been written in recent years describing how the majority of school districts in South Africa are manifestly ill-

118

equipped to carry out the most routine maintenance functions, let alone undertake the demanding work of turning dysfunctional schools around. The discussion that follows highlights a number of issues relevant to our theory of schooling which arise out of these South African studies. In 1999/2000 Mphahlele (1999; 2002) undertook a survey in which questionnaires were sent to all 178 districts, accompanied by case studies of 27 districts spread proportionately across the country. He identified the following problems: • The absence of a coherent legislative framework at national level results in confusion concerning the functions of district offices. • The absence of job descriptions and lines of accountability results in confusion concerning the roles and responsibilities of district officials. • The ratio of professional to administrative posts in the staff establishment (1:2.8), and a preponderance of low-level administrative posts leaves districts ill-equipped to undertake a professional development role. • A high proportion of vacant posts (47% among professional staff and 39% among administrative staff) significantly inhibits the functionality of district offices. • Low levels of essential resources – such as the paucity of vehicles for visiting schools – jeopardises the efficient link between districts and schools. • The absence of electronic information systems (EMIS) renders any monitoring function virtually impossible. In other words, in large parts of the country, the key systemic functioning of accountability and support breaks down at the district link. These conditions have motivated all provinces to restructure their district systems

chapter 8 Why districts matter

over the last six or seven years. Not much research information concerning these efforts exists. One exception is Fleisch’s (2002) analysis of the various restructuring initiatives undertaken by the Gauteng Department of Education (GDE) since the installation of the new government in 1994. This is an insider’s view from someone who worked as a district manager for the first five years of the GDE. The difficulties faced by the nine provincial departments in establishing entirely new systems around a fundamentally different orientation from that adopted by their predecessors cannot be overestimated. In this regard it should be remembered that the schooling system, as one of the instruments of apartheid, had been a principal target of the struggle against apartheid, and one of the first tasks of the GDE had been to stabilise a highly volatile situation within a milieu almost entirely devoid of institutional coherence or respect for authority. Under these circumstances, it is not surprising that improving the outcomes of schooling tended to be a secondary priority. With regard to improving school performance, Fleisch concludes that little fundamental change has occurred as a result of the numerous restructuring initiatives undertaken by the GDE, and poses the question: “Is restructuring primarily about buffering, creating structures and activities that appear to be about change, without actually touching the core business of the organisation?” (Fleisch, 2002, 196). For Fleisch, one of the principal problems inhibiting thoroughgoing change is the absence of indicators and systems for measuring progress in the core business of the department. In particular, there is no attempt to measure the outcomes of schooling in terms of learner performance (aside from the Grade 12 level), and, as a result, the largest sector of the schooling system falls outside the loop of performance management. The general systemic lack of focus of the GDE, identified by Fleisch, stands in strong contrast to the two clear and measurable successes achieved over this period: the improvement in matric exam results through the EAZ

programme during 2000–2002, and the stabilisation of the matric exam crisis of 1996. We have discussed the first of these issues at some length in Chapter 1. The matric exam crisis of 1996 was precipitated by the sale of question papers to students prior to the examinations. The leaking of this information to the press resulted in a media storm criticising the GDE. The response from the department was swift and decisive. An independent commission of enquiry appointed by the MEC revealed hopelessly inadequate systems for the safekeeping and distribution of question papers, underpinned by a naïve approach to accountability in which senior managers considered themselves to be collectively responsible for all decisions taken by the department, with no conception of individual responsibility for carrying out specific functions. Within a year the department had established a new exam section, including a new head and a revised set of operating procedures, the success of which, Fleisch notes, has been amply demonstrated by the disappearance of the issue from newspaper headlines. We would argue that these two success stories of the GDE had three aspects in common, which provide valuable lessons for improving the system as a whole. First and foremost, the core business of the respective section responsible for the activity in question was clearly defined: improve the exam results in the case of the EAZ, and formulate and process question papers to the required standard, without leaks, in the case of the exam section. Second, clearly defined, measurable indicators were linked to each core business. Finally, discussion of both the nature of the indicators and performance against them was a matter of intense public interest. Thus, the criteria for success or failure were apparent to all, and lines of responsibility traceable to teams and individuals. In short, the minimum conditions for ensuring transparency and accountability, much vaunted concepts in the new South Africa but little achieved in practice in the schooling system to date, were in place in the case of these two initiatives. A more detailed analysis of the work of districts in the GDE, undertaken by the

119

part III Implications of the model for classrooms, schools and districts

Education Policy Unit (de Clercq, 2001), reveals that the first of these conditions, a clearly defined core business, does not exist, hence the possibility of achieving coherence in the activities undertaken at this level of the provincial system cannot yet arise. Between September 2000 and March 2001 the team undertook a survey of the Teaching and Learning Units in each of the 18 district offices, and a more detailed case study analysis of what appeared to be the two most successful Units. The picture which emerges is one in which the Units are generally overstretched and often overwhelmed by demands to implement a steady barrage of new policies emanating from the provincial level. The heart of the survey consisted of a request to the Units to list their four most important work priorities. In all, 24 topics were listed, 9 of these being mentioned three times or more, and the remaining 15 only once each. Most significantly, each topic was viewed by the respondents as a free-standing project. Thus, the Senior Secondary Intervention Programme (listed most frequently at 12 mentions) appears to be run independently of the Common Exams project (second most frequent at 9 mentions), with no coherent plan linking these project-like activities or directing them towards a common goal of improving the quality of teaching and learning. The impression of atomised organisation is reinforced by the fact that 60% of activities were mentioned only once each across the 18 districts. The authors conclude that: On the whole the survey reveals a picture of districts which struggle with their support work without being able to be very proactive or showing a willingness to use their feedback evaluation to strategise their support work more effectively. What came out more clearly was the picture of frustrated district officials who did not take responsibility for their ineffective support work, preferring to blame their lack of impact on the ground on their difficult, demanding and under-resourced work environment. (de Clercq, op. cit., 26)

120

The two Teaching and Learning Units selected for in-depth analysis were chosen on the basis of their high levels of focus and commitment, on the assumption that, under generally low levels of functionality, the best performing Units were most likely to provide systemic lessons. The characteristic which most distinguished these two relatively successful Units from their peers was the capacity of their leadership to interpret policy and be proactive and directive in implementating the numerous and diverse demands emanating from the provincial office. The research team argues that Unit activities would be more effectively integrated if they were driven by a vision focused on the improvement of the core school business of curriculum delivery and professional support and development. The relative capacity advantage enjoyed by the two most effective districts notwithstanding, even these institutions were not close to addressing their main problems and priorities, and the report concludes that the overall impact of their activities was negligible. Given the state of the GDE district system described above, it is not surprising that provincial-level leadership tends to by-pass district structures and run high priority programmes such as the EAZ directly from head office. And if the situation described in the GDE is anywhere near a reflection of the national picture, then the tendency of the Minister and the National Department of Education to intervene directly in schools, as they have been doing since 1999, would seem to be unavoidable. In the latter case, the only tools in the hands of the Minister and his officials are management by ‘walking around and shouting’, and tough talk through the media. These are very blunt instruments and, although they appear to have been instrumental in squeezing some 20% of slack out of the system in terms of improved matric results from 2000 to 2002, we predicted in Chapter 1 that the improvements generated by these methods are likely to reach a rather low ceiling of diminishing returns. Further efficiency gains are most likely to be achieved only by systematically instituting standard

chapter 8 Why districts matter

operating procedures at provincial, district and school levels for providing accountability and support measures to schools, and building the capacity to run these systems. Furthermore, the present crude methods for improving the quality of learning outcomes are directed only at the Grade 12 level, while every indication is that the problem of poor learning has its seeds in the early grades, where the reading progress made in the majority of primary schools is significantly slower than the available resources would allow. If this is true, it follows that, at the present time, the greatest obstacle to improving the educational opportunities of all children is systemic inefficiency, which, to quote Hirsch, represents the major source of avoidable injustice in our schools (Hirsch, 1996, 33).

How can our theory of schooling assist in focusing the work of districts? The theory derived in Chapter 5 assumes that the core business of schooling, and of every component institution comprising the system, is to foster in children the social and cognitive competences formulated in the intended curriculum. We proceeded to argue that the goals of the intended curriculum would best be met by focusing attention on four key constructs, translated into policy levers, which could serve to align accountability and support measures, both within individual institutions and across the different levels of the system. These are summarised in Table 8.1 on page 122, and disaggregated into subconstructs and indicators designed to assess district performance in working towards achieving the core business. Our theory further postulates that the resource conditions pertaining in the schools under the jurisdiction of any particular district office predict the classification and framing values most appropriate for steering schools towards higher levels of performance. Classification here refers to the degree of detail to which policies and procedures are specified (strong classification indicates high

levels of explication), while framing refers to the degree of control which resides at the district level in regulating relations between districts and schools (strong framing indicates high levels of control by district officials). The theory predicts that in schools in which the principal and teachers have internalised the values of the curriculum and possess high levels of knowledge and material resources, classification and framing may be relaxed by the district, in order to provide school-level personnel space in which to exercise professional judgement in recontextualising the curriculum and mediating the process whereby targeted competences are transmitted to/acquired by their pupils. Schools characterised by lower levels of knowledge and material resources, on the other hand, are more likely to benefit from stronger degrees of classification and framing by district officials, at least in the short to medium term.

Sample Little evidence exists to systematically illuminate the indicators listed in Table 8.1. The best sample available is the QLP study conducted by the HSRC into the state of 17 district offices spread across the nine provinces, which forms part of an analysis using hierarchical linear modelling to investigate the effects of district-, school- and classroomlevel practices on learner performance in mathematics and language at the Grade 9 and 11 levels (Kanjee et al., 2001). Data from this study therefore forms the core of the analysis below, supplemented with information from the Mahlahle baseline study (JET, 1999; 2000a), which included two districts in Limpopo Province; from de Clerq’s GDE study (2001); from the DDSP baseline reported in Chapter 4 (Khulisa, 2001); and from case studies of two Eastern Cape districts undertaken by Roberts (2000). As in the case of the data at classroom and school level described in Chapters 6 and 7, what follows is a post hoc analysis of data collected within a very early, broad-brush version of the framework depicted in Tables 5.3, 6.1, 7.1 and 8.1. As a result there is nothing to report against a

121

part III Implications of the model for classrooms, schools and districts

SUBCONSTRUCT

Social organisation Social values

Social relations

3. How open are relations among officials within the district office? 4. How open are relations between district officials and their counterparts in schools? 5. How open are relations between district officials and their superiors in the provincial office?

Task organisation

6. To what extent are roles and responsibilities of district officials clearly defined, according to the demands of their functions? 7. What proportion of staff vacancies are filled? 8. What is the state of the data system used to assist in managing district functions? 9. What is the state of the performance management system within the district office, to assist district managers to fulfil their functions?

Time organisation

10. Is there evidence that activities are planned and directed towards achieving the core business of the district? 11. Does the district maintain systems for maximising number and length of working days of schools under its jurisdiction? 12. Does the district monitor attendance of the principal and other personnel in its schools?

Resource management

13. Does the district maintain systems for monitoring management of finances and resources other than books, in its schools?

Proficiency in language of instruction

14. Does the district have a programme for developing proficiency in the language of instruction in its schools? 15. Does the district monitor and support implementation of this policy at school level?

Promotion of reading and writing

16. Does the district have a policy on reading and writing for its schools? 17. Is implementation of the policy monitored? 18. What is the state of district systems for procurement and management of books and stationery?

Curriculum and pedagogy

Planning, coverage, sequencing and pacing

19. Does the district have a policy for planning curriculum coverage, including regular formal assessment tasks, in its schools? 20. Is coverage of the curriculum monitored? 21. Does the district have a programme of INSET for its principals and teachers? 22. Do relevant district officials support and monitor the work of their principals? 23. Do relevant district officials support and monitor teachers in the classroom?

Evaluation

Explication of evaluation criteria

24. Do district officials quality assure frequency and content of formal assessment tasks against curriculum standards in their schools? 25. Are results of formal assessment tasks monitored at district level? 26. Are results of formal assessment tasks used to improve teaching/learning, through planning, monitoring and support at school level?

Language

122

INDICATORS 1. To what extent are values embodied in the curriculum included in the district mission? 2. Are curriculum statements and associated knowledge resources (eg pace setters) distributed to schools?

Table 8.1

DISTRICT-LEVEL INDICATORS DERIVED FROM THE THEORY OF SCHOOLING THEORETICAL CONSTRUCT

chapter 8 Why districts matter

number of the indicators. Nevertheless, we offer the following analysis as an illustration of the method, in order to bring the material available into the public domain and to subject it to public debate. In the QLP analysis, researchers visited the 17 district offices, where they interviewed the 17 district managers and gave questionnaires to a further 70 circuit managers, 11 mathematics-learning-area specialists and 19 language-learning-area specialists. Data was collected on the capacity and resources of the district and circuit offices, and the monitoring and support they provide to schools. As noted in Chapter 2, the 17 districts allocated by the provinces to the QLP are among the most poorly performing in the country, with mean matric scores for the schools under their jurisdiction lying 10–20% below the respective provincial means, except in the case of Limpopo Province, where the mean for QLP schools lies within 2% of the provincial mean. Many of the district offices are in remote rural areas, and the remainder in townships.

Data analysis Social organisation Indicator 2. Distribution of curriculum statements to schools Sixteen district managers (88%) interviewed in the QLP baseline study report having distributed curriculum statements to schools. These figures are confirmed by school-level data, where 90% of principals report using curriculum statements to monitor the implementation of the curriculum, and between 68% and 77% of teachers surveyed report being in possession of Grades 9 and 11 mathematics and language statements. However, during the classroom observations and post-lesson interviews, only 40% of teachers were able to produce these, putting these figures in question.

Indicator 6. Definition of roles and responsibilities Our theory assumes that a clear division of responsibilities among staff, and the co-ordination of such responsibilities, is crucial for

the effective functioning of any organisation, but particularly so in the case of a highly complex area of work such as schooling. The QLP report argues that the most practical way of achieving these twin organisational principles is through development and implementation of organograms and job descriptions. In the QLP baseline study only 7 out of 17 districts (41%) were able to provide evidence that they had organograms. A further 7 districts claimed to have organograms but could not produce them, indicating either that the documents did not exist, or that they were inadequately used. The QLP baseline study found that only 2 districts (12%) could produce job descriptions. A further 6 claimed to have them but were not able to provide proof. The Mahlahle baseline study found that officials had only duty lists instead of full job descriptions. However, these did not stipulate reporting channels or provide detailed descriptions of performance expectations. The two EC case studies found that job descriptions were not uniformly available and, where they were given to the researcher, they were so vague and so wide ranging that the report concluded that they were not realistic.

Indicator 7. Staff vacancies The QLP baseline study found that in 7 of the districts surveyed (41%) there was no learning-area specialist for mathematics, and in 5 of the districts (29%) there were no languagelearning-area specialists. The report gives no indication of the number of schools serviced by each professional staff member present, although the figures quoted earlier by Mphahlele would indicate that they are spread so thinly as to be weakly effective at best.

Indicator 8. Data management systems The QLP study did not investigate the presence of EMIS, but the Mahlahle baseline study found that the data management systems were very cumbersome at district level. Officials reported that they travelled more than 600 km per week in order to enter data

123

part III Implications of the model for classrooms, schools and districts

electronically at the provincial head office. The two EC case studies revealed that district officials struggled to provide basic information such as the numbers of primary and secondary schools served by the district: data management was done manually with information being placed in school files which were stored in filing cabinets scattered throughout the district office.

Indicator 10. Planning and directing of activities When district-level planning processes were investigated, it was found in the QLP baseline study that only one of 17 district managers was able to produce evidence of having a three-year strategic plan for the district. A further 11 claimed to have plans, but were unable to produce them, casting doubts on the extent to which these reported plans actually influenced the district’s daily and monthly operations. Similarly, only one district manager could produce a plan for the improvement of mathematics, which is hardly surprising given the paucity of learning-area specialists noted above. No district was able to provide a copy of a budgeted year plan. A similar situation was reported in the Mahlahle study, where officials reported that co-ordinating structures were in place, but that they had not produced year plans which would guide the activities of officials. A recent pilot study conducted for the QLP found that in the district visited, officials produced monthly plans, although the connection between these activity plans and the district’s strategic plan was unclear. The structural configuration of the district also influences planning systems: districts often develop strategic plans which are a drawing together of plans developed by different units, often with little cross-referencing and little common focus.

Language Indicator 18. Procurement and management of books and stationery Unfortunately the QLP study did not collect district-level data on this indicator. However,

124

the Mahlahle baseline study found that no systems had been established at the district level to monitor school resources of any kind. The case studies conducted in two Eastern Cape districts showed that standard resource management systems were not in existence within one region of the same province. In one district there were systems for recording which schools had collected materials, while in the other district there were no systematic records of what had been distributed to schools.

Curriculum and pedagogy Indicator 21. In-service training In the GDE study described above (de Clerq, 2001), the principal form of support provided to schools by districts was found to be in the form of training workshops, which teachers generally rated as ineffective. Similarly, the QLP baseline analysis reports a negative rating by teachers of district-level training: of the 405 teachers surveyed in 105 schools, 156 (39%) reported that they had not attended any district-run workshops in the past year. Only 20–25% had attended these workshops quarterly or annually. Those who had attended workshops indicated that these workshops often focused on new policies (such as Curriculum 2005). The QLP study reports that workshops for principals were held more frequently than those for teachers, although very few principals indicated that they had received any training on outcomes-based education or C2005. This is an important point, as it goes to the heart of why management and monitoring systems were reported to be so poor in many schools – teachers indicated that they felt principals were unable to monitor and assess their curriculum planning or classroom practices as they were unfamiliar with new curriculum policy. The DDSP baseline study found that attendance at district training and the frequency of district office visits was ‘moderately correlated’ with educator knowledge on literacy and numeracy. This report also noted that school visits by district officials were directed towards policy compliance.

chapter 8 Why districts matter

Indicators 22 and 23. In-school monitoring and support of principals and teachers None of the research reports which were consulted provided any indication that districts had set targets for how frequently schools should be visited or monitored. Data on the frequency with which officials visited schools varied dramatically between provinces, and often appeared to depend on resource levels within districts, principally the availability of cars. Data from the DDSP baseline showed that, on average, districts visited schools once every six months, although this varied significantly between provinces, with schools in Limpopo Province receiving more frequent visits than schools in the other DDSP provinces (Eastern Cape, Northern Cape and KwaZulu-Natal). The QLP study found that of the 396 teachers who responded, 311 (79%) reported that learning-area specialists never came to observe lessons in classes, which was understandable given the small numbers of these personnel in district offices noted by Mphahlele (op. cit.). Approximately 10% of these teachers reported that such visits took place annually, and another 10% reported a frequency of three to four times per year, while 75% indicated that individual tutoring and consultation by district professional staff never occurred. It would seem that the extent to which districts monitor school functionality and performance is largely determined by each district. In some districts standard instruments have been developed by which the district officials assess performance in certain areas. For example, circuit managers may visit schools and conduct an assessment of management systems and documents using structured schedules, and then provide the school with written feedback and advice on how to strengthen these systems (de Clerq, 2001). Officials located in the curriculum section of the district office may utilise similar instruments when observing classrooms. However this is not standard practice across all districts. De Clerq also notes that when officials visited schools they checked whether educa-

tors had produced learning-area programmes and phase organisers, whether they were familiar with outcomes and whether learners had attained these outcomes. However, she notes that officials did not have any benchmarks against which to make judgements. One of the report’s recommendations was that officials needed training in the use of monitoring instruments and that the instruments needed to be clearer and easier to use. De Clerq also notes that the monitoring of schools was not quality assured by senior officials. This appears to have affected the reliability of the data collected by officials: when head office officials visited schools they often found practices at odds with those reported during the monitoring visits. Officials surveyed as part of the Mahlahle baseline study (JET, 1999; 2000) indicated that there was no set target for the frequency with which they visited schools. They also noted that visits often did not take place as planned and were somewhat haphazard. The DDSP baseline study reported that officials visited schools less than once every six months. Although officials reported visiting schools in several of the studies reviewed (Mahlahle, DDSP and QLP baseline studies) the nature and focus of these visits was often unclear. Where studies described officials’ activities when visiting schools, the most common activities of officials responsible for improving school management were to: • drop off or collect circulars and forms • notify principals of training programmes • drop off or collect EMIS forms • process requisitions • assist principals with management issues • ‘chat’ to principals and discuss any problems that they might be experiencing • monitor learner and teacher attendance (JET, 1999; 2000; Khulisa, 2001) Very few officials reported that they conducted management-related training when visiting schools. Schools in the DDSP baseline study reported that district officials seldom appraised staff when visiting schools.

125

part III Implications of the model for classrooms, schools and districts

The mention of accountability measures is sometimes associated with the stigma of oldstyle inspections. Schools do not always react positively to visits conducted with the express purpose of monitoring policy implementation and compliance, especially when this is done in a top-down and heavy-handed way (Narsee, 2002). The culture of opposing inspection has severely eroded the authority of the district office with respect to conducting monitoring visits. In 2000, when case studies were conducted in two Eastern Cape districts, officials noted that they had been banned from visiting classrooms and could only hold meetings with the principal when visiting schools. Some of the principals and teachers interviewed in de Clerq’s study (2001) criticised the officials’ approaches when interacting with schools, noting that they were inclined to be controlling and utilised ‘policing’ tactics. Some schools also noted that the districts should ‘negotiate’ their relationships with them. These responses show how debased versions of the notions of democratisation and devolution of authority have undermined districts’ ability to hold schools accountable. The new Whole School Evaluation (WSE) system is designed to establish standardised instruments and procedures for monitoring school performance and establishing the support needs of schools. However, although the system was developed and tested in schools in 2001, full-scale implementation has been paralysed by opposition from the teacher unions. Apart from these political difficulties, the WSE also suffers from a design problem: the system has been designed to be administered by teams which are quite independent of district offices. In our view, this method of implementation may work against building capacity at district level to adequately monitor and support schools, although given the state of the districts, this may, in the short term, be unavoidable.

126

Evaluation Indicators 24 and 25. Monitoring and quality assuring of assessment practices According to our theory of schooling, the most important mechanisms for monitoring the extent to which schools are performing in terms of their core business are the quality assurance of formal assessment tasks, by subject and grade level, and the tracking of the results of these tasks. Yet it is in this area that districts are most poorly equipped. The paucity of professional staff, the almost complete absence of EMIS systems and the opposition from unions to school monitoring all militate against the institution and maintenance of these essential procedures. On a positive note, the QLP study found that the districts surveyed had started to set and write common trial matric exam papers: 47% of districts reported setting common papers in mathematics, 37% in English, and 31% in Afrikaans.

Indicator 26. Use of assessment data to improve the quality of instruction The QLP study also investigated the extent to which districts had developed plans which specifically aimed to raise performance levels in mathematics and English (the predominant language of instruction in the district). They found that only one district had an improvement plan for mathematics, while 7 (of 17) had improvement plans for English. The differences in the availability of plans between the two learning areas mirror the availability of specialist staff in these areas.

The impact of districts on school performance The HLM analysis, performed as part of the QLP baseline study, found that district effects on learner performance in mathematics and language were non-existent. Although data from this research on district-level functionality is sketchy, the information available is fully consistent with the HLM finding. What emerges from the QLP district study is a picture of institutions which lack focus, capacity, resources, and the political climate necessary

chapter 8 Why districts matter

to act with professional authority in providing for the routine maintenance needs of schools, let alone undertaking the very demanding work necessary for improving the performance of the many poorly-functioning schools under their jurisdiction.

Conclusion Over the last eight years the district has been relatively neglected in terms of setting the policy direction and building the capacity required for playing its appropriate role in improving the quality of schooling. This is mirrored, on the research front, by a lack of systematic information on the efficacy of districts in performing this function, and any conclusions regarding the work of this key level of the schooling system must be regarded as tentative at this stage. However, if our picture of districts is anywhere near accurate, then it is highly likely that they have little effect on the work of schools. Furthermore, the prognosis for improving this situation does not appear to be promising. At the national level, policies for monitoring schools appear to be designed to circumvent any significant role for districts. If this is the case, then in separating monitor-

ing and support functions, they are likely to perpetuate incoherence and lack of alignment of key systems. Narsee’s study shows that districts tend either to act as inefficient accountability conduits for an endless stream of memoranda from the province, or to throw all their efforts into an equally inefficient form of bottom-up support. The systemic approach to school reform outlined in Chapter 1 postulates that accountability and support measures are two sides of the same coin, which are most effectively administered as part of a co-ordinated strategy. As matters stand at present, the policy has the effect of bifurcating and de-coupling them, rather than aligning them. While district-level restructuring is a focus of intense activity at the time of writing, if this does not proceed from the corebusiness starting point of improving the acquisition by pupils of the social and cognitive values inscribed in the intended curriculum, and aligning with cognate mechanisms at provincial and national level, then they are destined to continue to generate activity without achieving significant progress, vindicating Fleisch’s rather pessimistic prognosis quoted on page 119.

127

Conclusions What have we learnt? The single most important lesson learnt about schooling by researchers, non-governmental organisations (NGOs) and government, in a decade of activity in schooling, is that it is a social phenomenon of immense complexity, opaque to the best-intentioned interventions based on the most self-evidently righteous explanations. The schooling system is both the site and reproducer of inequities wrought by apartheid policies and practices, but serious attempts to redress inequities at both system and programme level, with a few exceptions, notably the improvement in matric results in recent years, seem to have had little or no effect. It would seem that no one knows quite why the best efforts have produced so little change, or quite why schooling outcomes at levels other than matric, despite our best efforts, seem to have declined even further. Since the early 1990s we have seen a plethora of both governmental and nongovernmental activity in schooling reform, which have taken the form of both programme interventions for improvement, and research into the nature of schooling and the effects of programme interventions. It is surely too much to say that we are no wiser at all, but only the foolhardy will claim that we have any firm answers. At least part of the problem has been the lack of co-ordination between governmental and non-governmental activity, between the research and the programmatic reform efforts, and among the various research efforts themselves, most of which, with a few exceptions, have worked as if in a social vacuum, referring to few prior studies done in the country, and thereby eschewing the possibility of a collective growth in knowledge about schooling. The main reason for this state of fragmentation, this book has argued, has been the absence of a unifying theory able to produce a metaanalytical conceptual synthesis that makes sense of all the conflicting views, conse-

128

quences and findings. The result has been that most of the well-meaning attempts to ameliorate or shed light on the problem have simply added to the inassimilable information and compounded it. This is without any doubt an objective legacy of the fragmented and fragmenting history of our country, which has left indelible patterns in our policy and intellectual capacities and practices. Our primary aim in this book has not been to give an account of this past, but rather to make a contribution to a theory of schooling, which might help to create a unifying intellectual roadmap for collective use in locating our endeavours. Chapter 2 constructed a partial overview of reform initiatives from the non-governmental sector, and reviewed the lessons to be learnt from research into their success and practices. We saw there that the lessons were almost all circumscribed in their generalisability because of features of design, on the one hand, and their non-accumulative intents on the other. Of course, much of this research did not set out to be replicable and generalisable, and there is a sense in which judging it by these criteria is unfair. As Shavelson and Towne (2002, 73–79) point out, many ‘humanist’ sub-traditions in the educational research community eschew replication and generalisation. But as they also go on to say, replication and generalisation are two key hallmarks of scientific research. It is this form of research that we believe holds the greatest promise for contributing to systemic reform. It is worth spelling out the guiding features of this kind of research in a little more detail. Shavelson and Towne (2002, 51) characterise it as ...seeking conceptual (theoretical) understanding, posing empirically testable and refutable hypotheses, designing studies that can test and rule out competing counterhypotheses, using

Conclusions

observational methods linked to theory that enable other scientists to verify their accuracy, and recognising the importance of both independent replication and generalization. From this they distil six principles for guiding all systematic enquiry: • pose significant questions that can be investigated empirically • link research to relevant theory • use methods that permit direct investigation of the question • provide a coherent, explicit chain of reasoning • replicate and generalise across studies • disclose research to encourage professional scrutiny and critique It is our view that the adoption of these principles would provide greater focus for governmental and non-governmental endeavours alike, and would also provide conditions for the burgeoning pool of research into effective schooling to have a greater impact on evolving systemic policy and reform. This does not mean that we favour one style of research only. Rather, what we have urged in this book is a closer rapprochement between what Fuller and Clarke (1994) once called the ‘policy mechanics’ and the ‘classroom culturalists’. We should say again, as Shavelson and Towne (2002) make abundantly clear, that such principles for systematic enquiry do not distinguish between qualitative and quantitative research, both of which can and do contribute to systematic enquiry. Chapter 3 addresses the dearth of systematic information about learner performance at levels below that of the matric exam, especially in the foundation phase. The seriousness of this dearth is only now, with new research information at our disposal (e.g. Van der Berg, 2002), becoming clear. The most important contribution to our knowledge of the data provided in this chapter is that, at the end of the foundation phase, learners have only a rudimentary grasp of the principles of reading and writing. When assessed, it

is clear that they are unable to perform satisfactorily at either the level expected by the new National Curriculum Statements (NCS), nor at the level of internationally accepted benchmarks (see TIMMS-R and MLA). When considered at the level expected by the NCS, they are performing 1 to 2 years below by Grade 3 level, and 2 to 3 years behind by Grade 6. It is very hard for learners to make up this cumulative deficit in later years, though through targeted intervention it is not impossible: Eric Schollar’s evaluation of reading gains in Transkei schools following three years of READ interventions shows that learners in READ schools were performing 2 grades higher than those in non-READ schools (Schollar, 2001a; Seekings, 2001b, 163). But particularly in those subjects that the Review Committee (2000) identified as having vertical demarcation requirements (especially mathematics and science), the sequence, pacing, progression and coverage requirements of the high school curriculum make it virtually impossible for learners who have been disadvantaged by their early schooling to ‘catch up’ later sufficiently to do themselves justice at the high school exit level. The implications are both moral and economic. Morally, failing to provide foundation competencies in the foundation phase is disturbing enough, but the schooling system does not pick this up until the matric exit exam at Grade 12, when it is far too late to do anything about it. Economically, the system continues through the middle years to spend money on educating learners in a way that leaves them with an increasingly slim chance of success. No more than a third of any given age cohort passes matric (Crouch and Mabogoane, 2001). In 1997 in the Western Province, arguably the most efficient province (at least according to their matric track record), 80,000 learners patiently sat out the middle years of their schooling until the Grade 9 exam. Only 40,000 of them (50%) were able to proceed to Grade 12 (Fiske and Ladd, 2002). Chapter 4 investigates the range of factors that might explain the lamentable level of

129

Conclusions

learning in our schools. There we saw that the state of research knowledge in South Africa is little different from that of the rest of the world: we know far more about the input or resource factors of schooling, and about the social background of learners, than we do about the management and instructional factors that might make a difference. The first thing we know with relative assurance is that poverty has a central differentiating effect on success at school. The second thing we know is that resources do have an effect on schooling outcomes, but the effect is not uniform, nor is it linear. In other words, some schools make different and better use of the same resources than do other schools working under the same conditions. Or, to put that even more graphically, some of the poorest and poorest-resourced schools regularly produce schooling outcomes equivalent to those of better-resourced schools. While we do not yet know quite why this is, it tells us at least, as Reynolds and Creemers (1990, 1) put it: “that schools matter, that schools do have major effects upon children’s development and that, to put it simply, schools do make a difference”. On this difference, all the available evidence points to the differential impact of managerial and pedagogical factors in the system, in the school and in the classroom. It is here that we know the least. As we have said before, lack of progress in South African research has been hampered by limitations of design, methodology, and above all, by the lack of a comprehensively articulated theory to account for how and why certain factors rather than others might produce schooling effects. It is this third issue which Chapter 5 addresses. The question this chapter seeks to answer is: how do factors of class and background, resources, and management and pedagogy (the domain of human intervention and the core business of schooling, after all) interact to produce schooling outcomes? We should say that in the theoretical model discussed in Chapter 5, we do not systematically consider the question of

130

resources. This is for two reasons. The first is that, under normal (or perhaps ideal) conditions, one should be able to assume in a state system that a certain common baseline of resources has been reasonably equitably distributed. We know that this is not so: though the DoE allocates budget according to national norms, the money is not always distributed as intended, and some provinces do not spend all their allocation. This is undoubtedly a management issue. But we do not address it in our model because our primary interest is to define those factors that optimise the use of resources so as to maximise learning and opportunity to learn. Secondly, educators see resources primarily as necessary aids to learning, where more resources generally mean better learning. Economists see them as cost factors, and ask, in the first place, whether they indeed do assist learning, and in the second place, whether they do so cost-effectively. For example, in the famous Tennessee class-size study (see Ritter and Borusch, 1999), smaller class sizes did indeed lead to stable learning gains. But because the unit gains were small, and because the costs of increasing the teacher corps in sufficient numbers to make worthwhile learning gains across the system would have been so huge, Tennessee did not implement the finding.13 In South Africa, as we saw in Chapter 2, class size is not uniformly correlated with outcomes. Years of teacher qualifications, however, are more strongly correlated, as are textbooks (see Crouch and Mabogoane, 2001; Vinjevold and Crouch, 2001). Which would be the more costeffective investment? Such studies have yet to be done in South Africa. Indeed, of all the research gaps that there are in South Africa, cost-benefit and cost-effectiveness studies are the most conspicuous. We have only the crudest data for the true price of educational factors in South African schools. In addition, we tend to concentrate on official costs only. Current international studies are careful to calculate in the ‘hidden’ or ‘subtle’ costs of schooling (Hummel-Rossi and Ashdown, 2002), which would include school fees of course, but also extra money raised – either by the schools

Conclusions

themselves or by large provincial intervention projects, private extra lessons, NGO activity in schools, parental participation (a particularly hidden cost) – as well as re-scheduling costs which attend many extra-curricular activities. In other words, properly costed, South Africans would probably be astounded at the high costs to the country of providing schooling, let alone adequate schooling.14 Nevertheless, as we said above, the focus of the model developed in Chapter 5 has to do primarily with which educational features maximise the potential for learning. The essence of the model can be summed up in the following way. We take it as given that there is a cyclical relation between social background, attainment of symbolic mastery (measured as success at school), and position in the labour market, and that there is a certain cyclical tendency in the system to reproduce the poverty cycle of society. But we also take it as given that schools can make a significant difference in maximising the acquisition of symbolic mastery, and hence breaking the poverty cycle. A key feature of some of the research reported here shows that some schools regularly do just that (see Chapter 4). The model then goes on to examine how, at the different levels of the system, the factors that make a difference vary. What we try to do here, in accordance with Shavelson and Towne’s principles, is to generate a coherent, explicit chain of reasoning that will show the logical linkages from empirical indicators (for example, ‘cover entire curriculum’ as evidenced by examining learners’ workbooks through the year), back through its conceptual construct (here, ‘pacing’), back to the formal feature by which, according to the model, it is claimed to vary (here, ‘strong or weak framing’; see definition in Chapter 5 and the discussion below). The closer to the empirical object we get, the more recognisable the factor will be; the closer to the constructs and theoretical features we get, the more its characterisation will depend upon the specific ‘theory’ or language of description we are using, which here is mainly derived from Basil Bernstein. When we refer

to our ‘theory of schooling’, then, we mean to denote the factors, constructs and formal features which are related to each other by the explicit chains of reasoning. The entire ensemble is the theoretical model, not only the terms we derive from Bernstein. The aim of this conclusion is not simply to re-state the model, but to make the chains of reasoning – or hypotheses of the theory – as clear and visible as possible.

Background What is it about ‘background’ that makes the difference to learners even before they get to school? What exactly is it about ‘class’ that constructs advantage and disadvantage? We incline to the cognitively oriented set of theories, of people like Halliday, Hymes, Cazden, Bernstein, Delpit and others, rather than only to the ‘capital’ oriented theorists like Bourdieu and Coleman, that advantage lies in a particular orientation to meaning, to which learners from poor homes, because of the pressures placed on the family by poverty, just do not get exposed. Let us be clear: every child learns a ‘community code’ at home. But middle-class learners, in addition, have a statistically better chance of imbibing the groundwork of a second code – let us call it a ‘school code’ – whose associative principles are different from those of community codes. Community codes are unquestionably also resources for learners at school. But in order to progress in school subjects like language, maths and science, that have what the Review Report called ‘vertical demarcation requirements’ (Review Committee, 2000), learners are expected to progressively master the school code, and to be able to switch appropriately between the school and community codes. Middle-class learners are better prepared at home to do this than are working-class learners. The composite orientation to meaning that learners get from home sets them each on a particular course or school trajectory. This course or trajectory does not, of course, set up any deterministic inevitabilities, but it weights the statistical probabilities. There is no finding in the sociology of schooling more stable than this.

131

Conclusions

How would one measure this school code or orientation to meaning? The experiment into coding reported in Chapter 5 is one way (see Holland, 1981). Another index is the size and complexity of vocabulary middle-class children have when they enter school as compared to that of working-class children (Hart and Risley, 1995). Because of this coding orientation, we could surmise that middle-class children come to school with a distinct cognitive advantage over their poorer compatriots, although there is no empirical data in South Africa to directly back up this supposition. The data that we do have shows that African learners whose mothers have 12 years of schooling are approximately 2 full grades more advanced at ages 13 and 16 than those children whose mothers have less than 4 years of schooling (Anderson et al., 2001). As social capital theory reminds us, this is not a one-off advantage but a continuous one, since the source of the advantage lies in the networks that the children continuously inhabit. Thus, in answer to the question ‘what is it about home background that makes a difference?’ the model proposes that it is a particular orientation to meaning, the possession of a particular code. Two policy questions then arise. The first equity question is: how does a society ensure that disadvantaged learners gain proficiency in the code, in the language of instruction, to provide them with the tool they need to access conceptual learning? The most thoughtful reform efforts globally have realised that this requires a systemic policy effort. Coleman’s initial study led to the policy of bussing poor children into first-language better-off schools, a policy that might sound bizarre to us, but has been able to show long-term cognitive gains. More recently in the USA, there have been proposals for a nationwide programme for training foundation phase teachers in the teaching of reading. In the UK, as we saw in Chapter 1, a national literacy policy has led to the Literacy Hour, a policy that has also shown dramatic effects in a short time (see Fullan, 2001). It is inappropriate to make specific recommendations in a book of this kind, but we can say with some

132

assurance that without a determined national push towards providing disadvantaged learners with symbolic mastery as soon as possible at school, all subsequent efforts to improve schools will struggle to overcome a fundamental and self-perpetuating handicap. The second policy question that arises is the following: given that the majority of learners are disadvantaged by entering school with poor levels of code mastery, what forms of curriculum, pedagogy and management attenuate or ameliorate that disadvantage? This is where the model must prove its explanatory mettle. The model asserts that forms of curriculum, pedagogy and management vary as to how they configure power and control relations, that is, as to their classificatory and framing values: • Classification is the measure of the distinctiveness of categories. For example, school subjects may be more or less clearly defined and distinguished from one another (C+); or they can be integrated (C–). Similarly, roles in the school can be clearly marked and their responsibilities clearly defined (C+: this is what an accountability system must seek to do); or there can be a putative ‘democracy’ (C–; putative, because this is nearly always more apparent than real). Schools are institutions of modernity, and hence presume an advanced division of labour. They thus intrinsically repel a simplifying division of labour, or else they begin to malfunction. The main aim of the EAZ project in Gauteng can be said to have tried to move role and rule definition from C– to C+ (Fleisch, 2001). • Framing is the measure of control over communication over time. For example, if learners proceed at their own pace, and hence stagger the temporal attainment levels in the class, then pacing is weak (F–); if the learners proceed at the pace set down by the curriculum or the teacher, pacing is strong (F+). It may on the face of it seem perverse to treat management practices, the intended curriculum and classroom pedagogy as varying along

Conclusions

the same parameters, but in this we follow Bernstein’s view that, at a high level of abstraction, the difference between them disappears. They are all integrally part of the same device that regulates consciousness, and are therefore all part of the broader notion of pedagogic discourse theorised by Bernstein.15

Curriculum Turning first to the curriculum: what is it about curricular form that makes a difference to learning? Following the Review Committee Report (2000), the model assumes that knowledges from which school subjects are drawn differ in terms of vertical and lateral demarcation. Some knowledge areas, like maths, science and language, have extended vertical demarcation: that is, they have long interconnected conceptual chains of increasing abstraction. Other knowledge areas, like Life Skills for example, have knowledge clusters that relate to one another laterally and functionally rather than building conceptually on one another. These differences have direct implications for the school curriculum. Subjects drawn from knowledge fields with extended vertical demarcation require clear curricular signposts, with clear sequencing, clear progression paths, and clear signs to the teachers as to the pace they should maintain in order to cover the material in the time available. In contexts where there are teachers with a well-internalised roadmap of the terrain to be covered and the pace, depth and sequence in which the material must be handled to cover the curriculum satisfactorily, the curricular requirement for explicit clarity is less crucial. South Africa is, unfortunately, not in that lucky position, and as evaluations of the implementation of Curriculum 2005 all showed, the vast majority of teachers are simply lost without such a roadmap (ibid.; Jansen, 1999). In other words, the model suggests that if teachers are less than proficient in their knowledge and teaching of extended vertical demarcation subjects, then the curriculum must be very clearly stipulated, sequenced and paced (called strong macro

framing in Chapter 5). Where this is not the case, as with the first version of Curriculum 2005 (weak macro framing), the curriculum structure simply contributes to the situation of inadequate curriculum exposure, hence amplifying the disadvantage of the already disadvantaged. Subjects drawn from knowledge fields with rich lateral relations are able to relax sequence, progression and pacing requirements without threatening opportunity to learn, since the specific cognitive content to be covered is less specified. These subjects are also able to have thematically-driven momentum without impairing their progression and coverage requirements (weak classification in terms of the model). For the more vertical knowledge subjects, the model suggests that the classification must be stronger, which means that themes (or programme organisers) derived from everyday life should not drive progression at the expense of the conceptual structure of the subject, nor should the requirement to ‘integrate’ material with other subjects cloud the progression sequence.

Pedagogy Turning next to classroom teaching or pedagogy: what is it about pedagogical form that makes a difference to learning? What patterns of professional behaviour optimise the opportunity to learn? Following on from the discussion above, it depends first on knowledge structure and then curriculum structure. Making integrative links with other subjects and everyday examples is pedagogically desirable as long as these do not impede progression. Once again, poorly trained teachers find this more difficult to do adequately than their better trained peers.16 To mandate a strongly thematic style of pedagogy, as we still find in policy documents, especially for vertical extension subjects with teachers less than fully proficient, is to embed a direct threat to opportunity to learn in the policy. Again, the model suggests that thematic latitude is far less problematic in lateral extension subjects.

133

Conclusions

Certain features of the learning programme, the model suggests, require moderate to strong stipulation (or framing). This includes pre-eminently the need for clear evaluation rules and unambiguous feedback. Here, positive feedback is as important as, and in the earlier grades possibly more important than, negative feedback (Hart and Risley, 1995). The sequencing, or macro pacing, must also be moderately strong, to ensure that the ‘effective learning time’ is maximised and the content specified by the curriculum is covered. The evidence we have shows that poor coverage, hence weak macro pacing, is widespread in our schools (Taylor and Vinjevold, 1999; Hoadley, 1999). However, at the level of day-to-day teaching (micro pacing), there are often good grounds for variation, to cater for the specific needs of the specific learners in the class and to give all learners the chance to meet the macro standard. Proficient teachers are able to vary the pace of the learning course to pick up the variable learning paces of individuals in the class without compromising coverage (macro pacing). This is the true meaning of learnercentred teaching. Weakly proficient teachers are often not able to do so, because their poor training leaves them unattuned to the subtleties of partial comprehension on the part of different learners. With such teachers, a learner-centred approach can easily mean loss of control over pacing and coverage altogether. Under these circumstances, the curriculum guidelines should direct the pedagogic tempo. Good textbooks and a carefully developed set of curriculum exemplars to illustrate the various conceptual rungs of the curriculum are essential to make guidance possible where teachers are less than proficient, and district offices unable to offer regular learning programme guidance. The findings in Chapters 6, 7 and 8 suggest that, in the sample of schools examined in these chapters, learning is likely to be significantly enhanced through the tighter regulation of pacing, and this is best achieved through co-ordinating efforts at district, school and classroom levels.

134

Management The management and regulation structure, what Bernstein called regulative discourse, is, after pedagogic discourse (curriculum and pedagogy), the second great power and control structure regulating schooling. We saw above that in Bernstein’s terms the two discourses form a systemic unity, with regulative discourse (management) playing the determining part. This suggests that without a functioning regulative order at the management level, no amount of attention to the instructional discourse (curriculum and pedagogy) can have any systematic effect. Pouring money into classroom improvement when schools and districts are not working, however well intentioned, is money ill-spent. This point has become conventional wisdom in world school improvement literature (Hopkins and McGilchrist, 1998). This suggests in turn the following: at the broad management level, at district, school and classroom level, there should be clear and overt specification and monitoring of roles and responsibilities (moderate to strong classification and framing). The issue here is clarity and explicitness, not arbitrariness or enforcement. Clarity means that everyone knows what is expected of whom. This is greatly facilitated by a strong, common values framework, and indeed, gives it effect. An effective evaluation and accountability system requires classificatory and framing clarity. As with the pedagogy, explicitness at the broad management level does not preclude more relaxed interpersonal relations, that is, personalised relations among colleagues and parents, principal and staff, teachers and learners. Indeed, goal (or value) clarity, rule clarity, and the effective use of monitoring and evaluation (Teddlie and Reynolds, 2000), together with convivial and personalised interpersonal relations, seems to differentiate private and religious schools, which consistently do better than public schools in the USA, UK, Australia and Holland (Hofman et al., 2002). Organisa-tional and regulatory setting is widely seen as the factor responsible for the sector effect (Bryk et al., 1993). What our model suggests

Conclusions

then is that ‘organisational and regulatory setting’ is optimally specified by clarity of role and accountability at the macro level, combined with relaxed interpersonal relations at the face-to-face level. It should be clear that until provincial and especially district officials have clear role expectations which are effectively monitored, provincial services to schools, from books and stationery to managerial and learning support, will continue to be variably and indifferently delivered. As our findings in Chapter 8 show, non-functional districts effectively impede classroom teaching because of nondelivery of clear curriculum guidelines and poorly distributed textbooks and stationery. An evaluation and monitoring system at this level is urgent. The same holds for the internal school-management level. Together these constitute what is commonly called ‘administrative climate’ (Hofman et al., 2002, 252). The international literature backs up the point: high outcomes are regularly correlated with community, parental and teacher participation (frequency of meetings and contact), rule clarity, and an achievement-oriented educational culture backed by monitoring and evaluation, together with a shared school ethos of high expectations and teacher satisfaction. At the classroom level, the optimal requirements seem to be ‘clarity in classroom rules’ (ibid., 264), efficient organisation, teacher collaboration, regular homework set and checked, and high expectations. Above all, performance expectations must be clear, a factor which closes the circle with the evaluation requirements we found when considering pedagogy. Hofman et al. (2002, 269) summarise what they see as the combined systemic, school and classroom management ensemble that optimises learning outcomes: A positive educational climate, parents’ educational involvement and effective school-based management are found to be prerequisites for an effective schooling process in countries all over the world...For schools to develop an effective educational climate and instructional processes and to

make sure that teachers know how to deal with each and every pupil, frequent monitoring of the educational growth (cognitive and socially) of pupils seems important. We have found that schools that work efficiently, that have a monitoring policy for measuring, following and improving pupil achievement, and that focus their monitoring policy on activities at the different levels in the school system seem to achieve better results. Chapters 6, 7 and 8 use the model to interrogate some of the data described in Chapter 2, with respect to pedagogy and management at district, school and classroom levels. The aim of these chapters is to test the robustness of the model in a preliminary way. Although the data does not allow strong conclusions, there is nothing in the findings to contradict our suggestions and hypotheses. We are quite aware of the limitations of post hoc analyses that do not allow us to make strong causal confirmations or disconfirmations. As we have said, the data is too limited in sampling scope and richness to be anything but broadly illustrative of the potential promise of the model. The main value of the data presented in these three chapters is to provide some empirical description of what is happening in some of the key nodes of the model.

Concluding comments The research-based data produced on South African schooling since the early 90s, read in the context of international school-improvement research, has left us with few sure-fire certainties, but it has left us with a clearer, richer and more grounded picture of schooling. Above all, we now have a set of well-supported suppositions about what makes schooling tick, what is going right or wrong, and why. It is hard to doubt the importance of the following: • Language and early exposure to reading and writing. These key elements provide young learners with the foundational learning platform for attaining symbolic mastery at school. The schooling system has a special responsibility to those children

135

Conclusions

who come to school without effective exposure to literacy at home, especially to literacy in the language of instruction. • Mechanisms which ensure coverage of the curriculum – like clear pacing signals, sequence signals, good exemplars and textbooks. It may seem obvious to state it so baldly, but it is only since the recent generation of testing that we have come to appreciate that if teachers do not cover the curriculum, the learners will not learn it. It is not unreasonable to suppose that this might become a constitutionally claimed right in future. • Explicit, regular and systematic evaluation and assessment. • An administrative and management climate that values and monitors high attainment for all. What this book shows is that each of these factors depends for its success on effective educational action, not only at one level, but across various levels. The jeremiads that blame the teacher, the principal or some discredited pedagogy have all got hold of one part of the story only. What this book has tried to make clear is the importance of co-ordination across levels, and the crucial importance of a monitoring and accountability system that gets things working in concert. This is so not only for management, but as we have seen above, for curriculum and pedagogy as well. In this book we have also tried to make the case for evidence-based policy, and the importance of careful and rigorous research in providing it. Here we must claim relatively modest gains only. As Chapters 2 and 3 show, much of the research the community has done over the ten years since the early 1990s has had methodological limitations, which most researchers are now trying to correct. The research that we report on in Chapters 6, 7 and 8 is likewise limited, consequently narrowing the conclusions we are able to draw. We have claimed, nevertheless, that none of the data at our disposal persuades us that the

136

model developed in this book is fundamentally wrong in any crucial respect. What we believe is now required is some well-planned, well-funded team research into details of the model. We are clear that the model at present is no more than a suggestive set of ‘best guesses’, and requires some targeted research to affirm, disconfirm or otherwise modify it. In the process, we are certain that the ‘best guesses’ will become refined, more nuanced and, in short, better guesses. We have also come to see that an absolutely indispensable component of the research now needed is a theoretical account of the causal factors that produce the results we are seeing. We have made an effort above to produce a first version of a theoretical model. We are aware that, even as we write, there are researchers working with some of these theoretical resources at highly detailed conceptual and empirical levels. This is as it should be. The model is an attempt to benchmark a certain level of conceptual consensus, which must change as the scientific activity in the educational research community grows in confidence, sophistication and rigour, as we are sure it will. What we are certain of is that, just as evidence matters, so equally does theory. As Hirsch (2002, 14) recently observed: Without greater theoretical sophistication we are unlikely to achieve greater practical results. With it, educational research could begin to earn the high gratitude and prestige that it currently lacks but which, given its potential importance, it could some day justify. It is as well to end with a reminder of just why such research is important. Nine years after our political transition, the best evidence we have shows that learners at school are not learning what they are supposed to be learning, indeed, what they are entitled to be learning. The research community has produced some highly suggestive information about why that is. It is time now to lift this endeavour to the next level.

Notes 1.

2.

3.

Van der Berg (2001b) distinguishes between discrimination and inequality. While government has not only eliminated discrimination in the distribution of public resources but also begun to institute redress measures, significant inequalities remain. These arise from access to significantly larger sources of private funding on the part of privileged schools – largely through school fees – and the fact that better qualified and hence better paid teachers prefer to work in suburban, more privileged schools. Five provinces reported receiving donor support to the total extent of some R600 million in 2001 (MoE, 2002). These figures are distorted in a number of ways, resulting in a significant underestimation of donor funding. First, Gauteng and the Western Cape were obviously of the view that the school support provided by a number of programmes – such as GETINSET, SAILI, the QLP and EQUIP – did not qualify, since it was not passed through the provincial treasury. Second, a number of provinces did not report at all, including Limpopo Province, one of the largest recipients of aid, and known to have received support for some 60 projects in 2000, affecting an estimated 25% of schools in the province. Third, the reporting is clearly very selective: for example, only the Eastern Cape reported the contribution of the QLP, a R25 million p.a. programme across all nine provinces. There are sceptical voices about the efficacy of the NLNS (see, for example, Brown et al., 2001), but our reading of the literature is that there is a preponderance of agreement with Fullan’s conclusions by other independent evaluations (see Beard, 2000).

4.

5.

6.

7.

8.

9.

Randomised trials should not be confused with random sampling. The former is a design feature that regulates strength of causal inference. The latter is a feature that allows the researcher to control for questions of generalisability and can be a feature of all three kinds of research design: descriptive, explanatory or process. Random sampling is one way to ensure valid generalisation to other people, times and settings. This is a standard list. Porter (2002), for example, uses five of these as his measure of ‘cognitive demand’ in mathematics: memorise, perform procedures, communicate, solve non-routine problems, conjecture/generalise/prove. The ‘Top 10% benchmark’ is the number above which lie the scores of the highest-performing 10% of pupils; similarly the ‘Upper Quarter benchmark’ is the number above which the scores of the highest-performing 25% of pupils lie. 60% of pupils from Japan, Hong Kong, Korea and Chinese Taipei reached this benchmark as did 75% of pupils from Singapore. Prior to 1994 schooling in South Africa was administered by 19 distinct authorities, including the DET for black students who resided in ‘white’ South Africa, the HoA for whites, the HoD for Asians, the HoR for coloureds, and a number of ‘homeland’ departments. These tests were constructed specifically for the QLP with the assistance of the Association of Mathematics Educators of South Africa, based on the curricula currently in use at the Grade 9 and 11 levels. Simkins and Patterson found a significant but not perfect correlation between school achievement in these tests and the matric exam,

137

Notes

10.

11.

12.

13.

14.

138

concluding that the two sets of measures are assessing approximately the same competencies. The fact that QLP districts were allocated to the project by each of the provincial authorities, and that the poorest districts in the province were allocated in almost every case, hardly constitutes a scientific sampling procedure. Professor Jansen, Dean of the Faculty of Education at the University of Pretoria, laments this situation, noting that it is a great handicap to the systematic and rigorous study of education in South Africa that it is very rare to find a student at the Masters or PhD level who possesses even the most elementary statistical literacy (Jansen, 2001). These are statistical generalisations, and social class cannot be assumed to fix the orientation to language of any individual. Bernstein was at pains to emphasise that code orientation does not reflect cognitive competence, but a habituated orientation to particular ways of making meaning. Seekings (2001b, 112–113) estimates, for example, that in order to reduce the ratio in South African schools from 32:1 to 29:1 would require 10% more teachers at about R5 billion more a year. Crouch and Mabogoane show that this would barely impact on aggregate learning gains. Crouch and Mabogoane (1997) estimated that it takes on average 18 years

15.

16.

to produce one matric pass, which is three times in time and money what the ‘efficient’ normal twelve years would be. ‘Often people in schools and in classrooms make a distinction between what they call transmission of skills and the transmission of values. These are always kept apart as if there were a conspiracy to disguise the fact that there is only one discourse. In my opinion, there is only one discourse, not two...Most researchers are continually studying the two, or thinking as if there are two: as if education is about values on the one hand, and about competence on the other. In my view there are not two discourses, there is only one’ (Bernstein, 1996, 46). Bernstein goes on to talk about ‘regulative discourse’ and ‘instructional discourse’ at a lower level of abstraction, and we will distinguish between regulation (values and management practices) and instruction (curriculum and pedagogy) while understanding their essential identity at the level of pedagogic discourse proper. ‘The collection code is capable of working when staffed by mediocre teachers, whereas integrated codes call for much greater powers of synthesis and analogy, and for more ability to both tolerate and enjoy ambiguity at the level of knowledge and social relationships’ (Bernstein, 1975, 108).

References Adler, J and Y Reed (eds) (2002). Challenges of Teacher Development: An Investigation of Take-up in South Africa. Pretoria: Van Schaik. Adler, J and Y Reed (2000). ‘Researching teacher’s “take-up” from a formal in-service professional development programme’. Journal of Education, 25: 192–226. Adler, J, Y Reed, T Lelliott and M Setati (2002). ‘Use of resources: A dual challenge for teacher education’ in Adler, J and Y Reed (eds) (2002). Adler, J, L Slonimsky and Y Reed (2002). ‘Subject-focused INSET and teachers’ conceptual knowledge-in-practice’ in Adler, J and Y Reed (eds) (2002). Anderson, K, A Case and D Lam (2001). ‘Causes and consequences of schooling outcomes in South Africa: Evidence from survey data’. Social Dynamics, 27(1): 37–59. Babbie, E and J Mouton (2001). The Practice of Social Research. Oxford: Oxford University Press. Bassey, M (1999). Case Study Research in Educational Settings. Buckingham: Open University Press. Baxen, J and D Gilmour (2000). ‘Baseline study of Western Cape schools: Final research report’. Mimeo. Beard, R (2000). ‘Research and the National Literacy Strategy’. Oxford Review of Education, 26(3,4): 422–436. Bernstein, B (1999). ‘Vertical and horizontal discourse: An essay’. British Journal of Sociology of Education, 20(2): 157–173. Bernstein, B (1996). Pedagogy, Symbolic Control and Identity: Theory, Research, Critique. London: Taylor and Francis. Bernstein, B (1990). Class, Codes and Control, Vol IV: The Structuring of Pedagogic Discourse. London: Routledge. Bernstein, B (1986). ‘On pedagogic discourse’ in Richardson, J (ed), Handbook of Theory and Research for the Sociology of Education. New York: Greenwood Press.

Bernstein, B (1981). ‘Codes, modalities and the process of cultural reproduction: A model’. Language in Society, 10: 327–363. Bernstein, B (1977). Class, Codes and Control, Vol III (revised). London: Routledge and Kegan Paul. Bernstein, B (1975). Class, Codes and Control, Vol III: Towards a Theory of Educational Transmissions. London: Routledge and Kegan Paul. Bernstein, B (1971). Class, Codes and Control, Vol I. London: Routledge and Kegan Paul. Bhorat, H and J Hodge (1999). ‘Decomposing shifts in labour demand in South Africa’. South African Journal of Economics, 67(3): 348–380. Bot, M, D Wilson and S Dove (2001). The Education Atlas of South Africa. Johannesburg: The Education Foundation. Bourdieu, P (1986). ‘The forms of capital’ in Richardson, J (ed), Handbook of Theory and Research for the Sociology of Education. New York: Greenwood Press. Brodie, K, T Lelliott and H Davis (2002). ‘Developing learner-centred practices through the FDE programme’ in Adler, J and Y Reed (eds) (2002). Brown, M, M Askew, V Rhodes, H Denvir, E Ranson and D Wiliam (2001). ‘Magic bullets or chimeras? Searching for factors characterising effective teachers and effective teaching in numeracy’. Presentation to the Symposium ‘Pedagogy and Educational Policy: Modernising teaching or narrowing the agenda?’ University of Bath. Mimeo. Bryk, A, V Lee, and P Holland (1993). Catholic Schools and the Common Good. Cambridge, MA: Harvard University Press. Business Day (2002). ‘Asmal, unions agree to stop surprise visits to classrooms’. Business Day, 20 August. Case, A and A Deaton (1999). ‘School inputs and educational outcomes in South Africa’. Quarterly Journal of Economics, 114: 1047–1084.

139

References

Case, A and M Yogo (1999). ‘Does school quality matter? Returns to education and the characteristics of South African schools’. NBER Working Paper No. 7399, October. CEPD (1995). The National Teacher Education Audit: The NGO Sector. Johannesburg: Centre for Education Policy Development. Chinapah, V, et al. (2000). With Africa for Africa; Towards Quality Education for All; 1999 MLA Project. Pretoria: Human Sciences Research Council. Christie, F (ed) (1999). Pedagogy and Shaping of Consciousness. London: Continuum. Cohen, D (1990). ‘A revolution in one classroom: The case of Mrs Oublier’. Educational Evaluation and Policy Analysis, 12(3): 327–345. Coleman, J (1988). ‘Social capital in the creation of human capital’. American Journal of Sociology, 94 (issue supplement): 95–120. Coleman, J, E Campbell, C Hobson, J McPartland, A Mood, F Weinfeld and R York (1966). Equality of Educational Opportunity. Washington, DC: US Government Printing Office. Creemers, B (1996). ‘The school effectiveness knowledge base’ in Reynolds, D, R Bollen, B Creemers, D Hopkins, D Stoll and N Lagerwej (eds), Making Good Schools: Linking School Effectiveness and School Improvement. London: Routledge. Crouch, L (2002). Personal communication. Crouch, L (2001). ‘Turbulence or orderly change? Teacher supply and demand in the age of AIDS’. Mimeo. Crouch, L (2000). ‘Some remarks on sample-based studies in South African education’. Mimeo. Crouch, L (1999a). ‘Educational indicators in South Africa: Who needs them? What for? Some theoretical and practical considerations’. Mimeo. Crouch, L (1999b). ‘Will South Africa be able to afford to educate her children into the 21st century?’ Mimeo. Crouch, L (1997). ‘Will things or will knowledge fuel South Africa’s growth?’ Mimeo. Crouch, L and T Mabogoane (2001). ‘No magic bullets, just tracer bullets: The role of learning resources, social advantage, and education management in improving the performance of South African schools’. Social Dynamics, 27(1) 60–78.

140

Crouch, L and T Mabogoane (1997). ‘Aspects of internal efficiency indicators in South African schools: Analysis of historical and current data’. Edusource Data News, 19: 4–28. Crouch, L and H Perry (2002). ‘Determinants of learning and learning change in DDSP schools Grade 3’. Mimeo. Daniels, H (2001). Vygotsky and Pedagogy. London: Routledge. De Clercq, F (2001). ‘Research study into the departmental interventions regarding scholastic performance’. Prepared by the Wits Education Policy Unit for the Gauteng Department of Education. Mimeo. Dika, S and K Singh (2002). ‘Applications of social capital in educational literature: A critical synthesis’. Review of Educational Research, 72(1): 31–60. DoE (2003). ‘Report to the Minister: A Review of Financing, Resourcing and Costs of Education in Public Schools’. Pretoria: Department of Education. DoE (2002a). ‘Revised National Curriculum Statement Grades R to 9 (Schools) Policy’. Pretoria: Department of Education. DoE (2002b). ‘Foundation Phase Systemic Evaluation 2001 (Mainstream): National Report’. Pretoria: Department of Education. DoE (2001). ‘Draft Framework for Systemic Evaluation’. Pretoria: Department of Education. DoE (2000a). ‘The National Policy on Whole School Evaluation. Final’. Pretoria: Department of Education. DoE (2000b). ‘Whole School Evaluation: Evaluation Guidelines and Criteria’. Pretoria: Department of Education. DoE (2000c). ‘Whole School Evaluation: Instruments for Whole School Evaluation and School Self-Evaluation’. Pretoria: Department of Education. DoE (1998a). ‘Assessment Policy in the General Education and Training Band. Grades R to 9 and ABET’. Government Gazette, Vol 402, No. 19640. Pretoria: Government Printer. DoE (1998b). ‘National Norms and Standards for School Funding’. Pretoria: Department of Education.

References

DoE (1997). ‘Curriculum Framework for General and Further Education and Training: Discussion Document Developed by the Consultative Forum on Curriculum’. Pretoria: Department of Education. DoE (1995). ‘Curriculum 2005: Learning for the 21st Century’. Pretoria: Department of Education. DoE (no date). ‘Manifesto on Values, Education and Democracy’. Pretoria: Department of Education. DoE/ELRC/SACE (2001). ‘National Professional Diploma in Education’. Pretoria: Department of Education. Dowling, P (1995). ‘Discipline and mathematise: The myth of relevance in education’. Perspectives in Education, 16(2): 209–226. Elmore, R (2002). ‘Unwarranted intrusion’. Education Next, Fall 2002. Available on http://www.educationnext.org/20021/30.html. Elmore, R (1993). ‘The role of local school districts in instructional reform’ in Fuhrman, S (ed), Designing Coherent Educational Policy. San Francisco: Jossey-Bass. Elmore, R, C Abelmann and S Fuhrman (1996). ‘The new accountability in state education reform: From process to performance’ in Ladd, H (ed), Holding Schools Accountable: Performance-based Reform in Education. Washington, DC: The Brookings Institution. Elmore, R and D Burney (1999). ‘Investing in teacher learning: Staff development and instructional improvement’ in DarlingHammond, L and G Sykes (eds), Teaching as the Learning Profession: Handbook of Policy and Practice. San Francisco: JosseyBass. Ensor, P (2002). ‘Module Two, JET Methods Course’. University of Cape Town. Mimeo. Ensor, P (1999). ‘A study of the recontextualising of pedagogic practices from a South African University pre-service mathematics teacher education course, by seven beginning secondary mathematics teachers’. Collected Original Resources in Education. No. 24. Ensor, P, T Dunne, J Galant, F Gumedze, S Jaffer, C Reeves and G Tawodzera (2002).

‘Teachers, textbooks and pedagogy: Studying maths for all in primary classrooms’ in Lubisi C and C Malcolm (eds), Proceedings of the 10th SAARMSTE Conference. Pietermaritzburg: University of Natal. Fedderke, J, R de Kadt and J Luis (1998). ‘Uneducating South Africa: The failure to address the need for human capital – a 1910–1993 legacy’. Mimeo. Finn, J, and C Achilles (1999). ‘Tennessee’s class size study: Findings, implications, misconceptions’, Educational Evaluation and Policy Analysis, 21(2): 97–109. Fiske, E and H Ladd (2002). ‘Financing schools in post apartheid South Africa: Initial steps towards fiscal equity’. Paper presented to the Conference: Education and Decentralisation: African experiences and comparative analysis. Johannesburg, 10–14 June. Fleisch, B (2002). Managing Educational Change: The State and School Reform in the New South Africa. Johannesburg: Heineman. Fleisch, B (2001). ‘Prescriptive school improvement model study: Interim report’. Mimeo. Fuhrman, S (ed) (2001). From the Capitol to the Classroom: Standards-based Reform in the States. Chicago: University of Chicago Press. Fullan, M (2001). The New Meaning of Educational Change. New York: Teachers College Press. Fuller, B and P Clarke (1994). ‘Raising school effects while ignoring culture? Local conditions and the influence of classroom tools, rules, and pedagogy’. Review of Educational Research, 64(1): 119–157. Garden, R (2002). ‘Design and development of the TIMSS-R mathematics tests’. Mimeo. GDE (1999). ‘Annual Report to the Legislature, Financial Year 1998/99’. Johannesburg: Gauteng Department of Education. Gilmour, D (2001). ‘Training the trainers: Doing educational research in South Africa’. Mimeo. Godden, J (1996). ‘Issues in devolving responsibility for education to local/district level’,

141

References

in Coombe, C and J Godden, Local/District Governance in Education: Lessons for South Africa. Johannesburg: Centre for Education Policy Development. Gouws, A and J Gibson (2001). ‘The study of political tolerance in the South African context’. Social Dynamics, 27(2): 109–133. Government Gazette (1996). Act 84 of 1996. Pretoria: Government Printer. Government Gazette (1995). Act 58 of 1995. Pretoria: Government Printer. Government Gazette (1993). Act 200 of 1993. Pretoria: Government Printer. Greaney, K (2001). ‘Review of Elley, W.B. (2000) STAR supplementary test of achievement in reading in New Zealand’. Educational Studies, 36(1). Hart, B and T Risley (2000). Meaningful Differences in the Everyday Experience of Young American Children. Baltimore: Paul H Brookes Publishing Co. Hasan, R (2002). ‘Semiotic mediation, language and society: Three exotripic theories – Vygotsky, Halliday and Bernstein’. Presentation to the Second International Basil Bernstein Symposium: Knowledges, pedagogy and society. University of Cape Town. Hirsch, E (2002). ‘Classroom research and cargo cults’. Policy Review, 115: 1–15. Hirsch, E (1996). The Schools We Need: And Why We Don’t Have Them. New York: Doubleday Books. Hoadley, U (2002). ‘Time to learn: Pacing and the external framing of teachers’ work’. Mimeo. Hoadley, U (1999). ‘An analysis of transmission in four Grade 1 classrooms’. Mimeo. Hofman, R, W Hofman and H Guldemond (2002). ‘School governance, culture, and student achievement’. International Journal of Leadership in Education, 5(3): 249–272. Holland, J (1981). ‘Social class and changes in orientations to meaning’. Sociology 15(1): 1–15. Hopkins, D and B MacGilchrist (1998). ‘Development planning for pupil achievement’. School Leadership and Management, 18(3).

142

Howie, S (2002). ‘English language proficiency and contextual factors influencing mathematics achievement of secondary school pupils in South Africa’. Published PhD thesis, University of Twente, Enschede, Netherlands. Howie, S (2001). Mathematics and Science Performance in Grade 8 in South Africa 1998/1999: TIMSS-R 1999 South Africa. Pretoria: Human Sciences Research Council. Howie, S and C Hughes (1998). Mathematics and Science Literacy of Final-year Students in South Africa. Pretoria: Human Sciences Research Council. Hummel-Rossi, B and J Ashdown (2002). ‘The state of cost-benefit and cost-effectiveness analyses in education’. Review of Educational Research, 72(1): 1–30. Jacklin, H (2001). ‘The learner progress and achievement study’. Mimeo. Jansen, J (2001). ‘Changing institutional cultures: Lessons learned from recent experience’. Quarterly Review of Education and Training in South Africa, 8(2). Johannesburg: Education Policy Unit, University of the Witwatersrand. Jansen, J (1999). ‘Why outcomes-based education will fail: An elaboration’ in Jansen J and P Christie (eds), Changing Curriculum: Studies in Outcomes-based Education in South Africa. Kenwyn: Juta. JET (2002). ‘Western Cape Education Department Grade 3 Learner Assessment Baseline Study’. Mimeo. JET (2001a). ‘Cadbury School Development Initiative: Baseline Study’. Mimeo. JET (2001b). ‘Cadbury Project: Grade 6 Learner Assessment Study’. Mimeo. JET (2001c). ‘SAILI Project: Grade 3 and Grade 6 Learner Assessment Study 2001’. Mimeo. JET (2001d). ‘Siyathuthuka Project: Baseline Evaluation Study’. Mimeo. JET (2001e). ‘Siyathuthuka Project: Baseline Evaluation Report’. Mimeo. JET (2001f). ‘Phalaborwa Foundation Mahlasedi Project Evaluation Report’. Mimeo. JET (2001g). ‘Phalaborwa Project: Grade 4 and Grade 7 Learner Assessment Baseline Study’. Mimeo.

References

JET (2001h). ‘Strategic priorities for initial teacher education’. Report prepared for the Department of Education. Mimeo. JET (2000). ‘Mahlahle: Report on the Phase Two Baseline Study’. Mimeo. JET (1999). ‘Mahlahle: Report on the Phase One Baseline Study’. Mimeo. Jubber, K (1998). ‘The home and family environment and its impact on school achievement’, South African Journal of Education, 8: 287–297. Jubber, K (1994a). ‘The early home environment and its influence on Standard 4 and Standard 10 school performance’. South African Journal of Education, 14(3): 135–140. Jubber, K (1994b). ‘Early home influences, reading performance and Standard 4 and Standard 10 school performance’. South African Journal of Education,14(4): 169–175. Kanjee, A, A Patterson, C Prinsloo, G Khosa, A Moore, M Kivilu and C Pheiffer (2001). Improving Learning in South African Schools: The Quality Learning Project Baseline Evaluation. Pretoria: Human Sciences Research Council. Khulisa Management Services (2001). ‘Summary: Report on interrelationships amongst indicators in the DDSP Baseline research’. Johannesburg: Khulisa Management Services. Mimeo. Lincoln, Y and E Guba (1985). Naturalistic Inquiry. New York: Sage Publications. Lofton, G, E Chad, F Hill and S Chauvin (1998). ‘Five years after implementation: The role of districts in maintaining an ongoing school improvement process’. School Effectiveness and School Improvement, 9 (1). Malcolm, C (1999). ‘Districts and management of educational change: A review of the international literature’. Mimeo. Malcolm, C, M Keane, L Hoohlo, M Kgaka and J Ovens (2000). ‘Why some “disadvantaged” schools succeed in mathematics and science: A study of “feeder” schools.’ Mimeo. Martin, M, I Mullis, E Gonzales, K Gregory, T Smith, S Chrostowski, R Garden and K O’Connor (2000). TIMSS 1999 International Science Report: Findings from IEA’s Repeat of the Third International Mathematics and Science Study at the Eighth Grade. Chestnut Hill: Boston College.

MoE (2002). ‘Fifth Report to the President from the Minister of Education’. Pretoria: Ministry of Education. MoE (2001). ‘Fourth Report to the President from the Minister of Education’. Pretoria: Ministry of Education. Moore, R and J Muller (2002). ‘The growth of knowledge and the discursive gap’. The British Journal of Sociology of Education, 23(4): 627–637. Morais, A and C Miranda (1996). ‘Understanding teachers’ evaluation criteria: A condition for success in science classes’. Journal of Research in Science Teaching, 33(6): 601–624. Morais, A and I Neves (2001). ‘Pedagogical social contexts: Studies for a sociology of learning’ in Morais, A, I Neves, B Davies and H Daniels (eds), Towards a Sociology of Pedagogy: The Contribution of Basil Bernstein to Research. New York: Peter Lang. Morais, A and D Pires (2002). ‘The what and the how of teaching and learning: Going deeper into sociological analysis and intervention’. Presentation to the Second International Basil Bernstein Symposium: Knowledges, pedagogy and society. University of Cape Town. Morais, A and C Rocha (2000). ‘Development of social competences in the primary school: Study of specific pedagogic practices’. British Educational Research Journal, 26(1): 91–112. Motsomame, E, T Nyabanyaba and J Roberts (2001). ‘Mahlahle Secondary School Report’. Human Sciences Research Council, Mimeo. Mphahlele, L (2002). ‘District development for systemic change’. Mimeo. Mphahlele, L (1999). ‘District development: What the research says. Part 1’. Mimeo. Mseleku, T (2002). ‘Department of Education efforts to secure school effectiveness’ in Roberts, J and J Muller (eds) (2002). Muller, J (2002). ‘Research for systemic reform’ in Roberts J and J Muller (eds) (2002). Muller, J (1998). ‘The well-tempered learner’. Comparative Education, 34(2). Revised version reprinted in Muller, J (2000). Reclaiming Knowledge: Social Theory, Curriculum and Education Policy. London: RoutledgeFalmer.

143

References

Muller, J and J Roberts (eds) (2002). Balancing Support and Accountability to Improve School Performance. Johannesburg: JET Education Services. Muller, J and J Roberts (2000). ‘The sound and fury of international school reform: a critical review’. JET: Mimeo. Muller, J and N Taylor (1995). ‘Schooling and everyday life: Knowledges sacred and profane’. Social Epistemology, 9(3): 257–275. Mullis, I, M Martin, E Gonzales, K Gregory, R Garden, K O’Connor, S Chrostowski and T Smith (2000). TIMSS 1999 International Mathematics Report: Findings from IEA’s Repeat of the Third International Mathematics and Science Study at the Eighth Grade. Chestnut Hill: Boston College. Narsee, H (2002). ‘And what about districts? Districts and their role in educational service delivery’. Paper presented to the Kentonat-Muldersdrif Conference, November. National Council of Teachers of Mathematics (2000). Principles and Standards for School Mathematics. Reston, Virginia: NCTM. Neves, I and A Morais (2001). ‘Knowledges and values in science syllabuses: A sociological study of educational reforms’. British Journal of Sociology of Education, 22(4): 531–556. Onwu, G (forthcoming). ‘Maths and science project: Case study composite report’. Johannesburg: Centre for Development and Enterprise. Perry, H (2002). ‘The impact of socio-economic status and school condition on Grade 3 learners’ literacy and numeracy results’. Mimeo. Porter, A (2002). ‘Measuring the content of instruction: Uses in research and practice’, Educational Researcher, 31(7): 3–14. Porter, A and J Smithson (2001). ‘Are content standards being implemented in the classroom: A methodology and some tentative answers’ in Fuhrman, S (ed) (2001). Reeves, C and C Long (1998). ‘Focus on four: An investigation into Grade 4 mathematics teaching and learning’. Mimeo. Reimers, F (1993). ‘Time and opportunity-tolearn in Pakistan’s schools: Some lessons

144

on the links between research and policy’. Comparative Education, 29(2): 201–213. Review Committee (2000). ‘A South African Curriculum for the 21st Century: Report of the Review Committee on Curriculum 2005’. Pretoria: Ministry of Education. Reynolds, D and B Creemers (1990). ‘School effectiveness and school improvement: A mission statement’. School Effectiveness and School Improvement, 1(1): 1–3. Reynolds, D and C Teddlie (2001). ‘Reflections on the critics and beyond them’. School Effectiveness and School Improvement, 12(1): 99–113. Riddell, A (1997). ‘Assessing designs for school effectiveness research and school improvement in developing countries’. Comparative Education Review, 41(2): 178–204. Ritter, G and Borusch, R (1999). ‘The political and institutional origins of a randomized controlled trial on elementary class size: Tennessee’s Project Star’. Educational Evaluation and Policy Analysis, 21(3): 111–125. Rivett-Carnac, L (2002). ‘Education Action Zones in action: Four case-studies of schools under tight bureaucratic control’. Mimeo. Roberts, J (2001). ‘Mapping school reform initiatives in South Africa: An overview of 12 school reform projects’. Mimeo, available on http://www.jet.org.za. Roberts, J (2000). ‘District development: The new hope for educational reform’. Mimeo. Roberts J and J Muller (eds) (2002). Balancing Support and Accountability to Improve School Performance. Pretoria: JET Education Services. Rose, D (2002). ‘Sequencing and pacing of the hidden curriculum: How indigenous children are left out of the chain’. Presentation to the Second International Basil Bernstein Symposium: Knowledges, pedagogy and society. University of Cape Town. Scheerens, J (2001). ‘Monitoring school effectiveness in developing countries’. School Effectiveness and School Improvement, 12(4): 359–384.

References

Scheerens, J and B Creemers (1989). ‘Conceptualising school effectiveness’. International Journal of Educational Research, 13(7): 691–706. Schindler, J (2002). ‘An analysis of the 2001 Senior Certificate examination’. Edusource, 36: 1–9. Schmidt, W, C McKnight, G Valverde, R Houang and D Wiley (1997). Many Visions, Many Aims. Vol 1: A Cross-national Investigation of Curriculum Intentions in School Mathematics. Dordrecht: Kluwer Academic Publishers. Schoenfeld, A (2002). ‘Making mathematics work for all children: Issues of standards, testing, and equity’. Educational Researcher, 31(1): 13–25. Schollar, E (2001a). ‘A review of two evaluations of the application of the READ primary schools programme in the Eastern Cape Province of South Africa’. International Journal of Education Research, 35: 205–216. Schollar, E (2001b). ‘Final report of the evaluation of the Imbewu project’. Mimeo. Schollar, E (1999). ‘Final report of the evaluation of the INSET component of the Mpumulanga Primary School Improvement project’. Mimeo. Sedibe, K, J Roberts, P Vinjevold (2000). ‘Mahlahle: Report on the Phase Two Baseline Study’. Mimeo. Seekings, J (2001a). ‘The uneven development of quantitative social science in South Africa’. Social Dynamics, 27(1): 1–36. Seekings, J (2001b). ‘Making an informed investment: Improving the value of public expenditure in primary and secondary schooling in South Africa’. Report to the Parliament of South Africa, commissioned by the Standing Committee on Public Accounts. Setati, M, J Adler, Y Reed and A Bapoo (2002). ‘Code-switching and other language practices in mathematics, science and English language classrooms in South Africa’ in Adler, J and Y Reed (eds) (2002). Shalem, Y (2002). ‘Do we have a theory of change?’ University of Witwatersrand. Mimeo.

Shavelson, R and L Towne (eds) (2002). Scientific Research in Education. Committee on Scientific Principles for Education Research. Washington, DC: National Academy Press. Shepard, L (2001). ‘Protecting learners from the harmful effects of high stakes testing’. Paper presented at the 2001 annual meeting of the American Educational Association. Seattle, Washington. Shulman, L (1986). ‘Those who understand: Knowledge growth in teaching’. Educational Researcher, 15(2): 4–14. Simkins, C (forthcoming a). ‘The determinants of educational attainment’ in Simkins, C (forthcoming d). Simkins, C (forthcoming b). ‘Mathematics and physical science results in the Senior Certificate examinations of 1998 and 2000’ in Simkins, C (forthcoming d). Simkins, C (forthcoming c). ‘School and community determinants of mathematics and physical science results in the Senior Certificate examinations of 1998 and 2000’ in Simkins, C (forthcoming d). Simkins, C (forthcoming d). South Africa’s Human Capital. Simkins, C (2002a). ‘The social and economic determinants of performance in the language of instruction and mathematics in Quality Learning Project schools’. Mimeo. Simkins, C (2002b). ‘The jagged tear: Human capital, education, and AIDS in South Africa, 2002-2010’. CDE Focus, March. Johannesburg: Centre for Development and Enterprise. Mimeo. Simkins, C and A Patterson (2002). The Social and Economic Determinants of Performance in the Language of Instruction and Mathematics in Quality Learning Project Schools. Johannesburg: JET Education Services. Slee, R and G Weiner (1998). ‘Introduction: School effectiveness for whom?’ in Slee, R and G Weiner with S Thomlinson (1998). Slee, R and G Weiner with S Thomlinson (1998). School Effectiveness for Whom? Challenges to the School Effectiveness and School Improvement Movements. London: Falmer.

145

References

Smith, J, B Smith and A Bryk (1998). ‘Setting the pace: Opportunities to learn in Chicago elementary public schools’. School Research Report: Consortium of Chicago. Online at http://www.chi-challenge.org/home.htm Stevens, F (1996). ‘Opportunity to learn science: Connecting research knowledge to classroom practices’. Journal of Teacher Education, 48(3): 222–232. Strauss, J (1999). ‘Results of the Monitoring Learning Achievement project’. Research Institute for Education Planning. Mimeo. Supovitz, J (2001). ‘Translating teaching practice into improved student achievement’ in Fuhrman, S (ed) (2001). Tashakkori, A and C Teddlie (1998). Mixed Methodology. Thousand Oaks, California: Sage. Taylor, N (2002). ‘Accountability and support: Improving public schooling in South Africa: A systemic framework’ in Roberts, J and J Muller (eds) (2002). Taylor, N (2001). ‘Outcomes, effort and values in schooling’. Presentation to the conference: New Millenium Business. Wits University Business School, 15 May. Available on www.jet.org.za Taylor, N, and P Vinjevold (eds) (1999). Getting Learning Right. Johannesburg: Joint Education Trust. Teddlie, C and D Reynolds (2001). ‘Countering the critics: Responses to recent criticisms of school effectiveness research’. School Effectiveness and School Improvement, 12(1): 41–82. Teddlie C and D Reynolds (2000). The International Handbook of School Effectiveness Research. London: Falmer Press. The Teacher (2002). ‘A new guide to standards is ready to roll’. 6 May 2002. Online at http://www.teacher.co.za Thomas, D (1996). ‘Education across generations in South Africa’. American Economic Review, 86: 330–334. Thrupp, M (1999). Schools Making a Difference: Let’s Be Realistic. Buckingham: Open University Press.

146

Tyler, W (1999). ‘Pedagogic identities and educational reform in the 1990s: The cultural dynamics of national curricula’ in Christie, F (ed) (1999). Tyler, W (1998). School Organisation: A Sociological Perspective. London: Croom Helm. Van der Berg, S (2002). ‘Education, poverty and inequality in South Africa’. Paper to the Conference: Economic Growth and Poverty in Africa. Centre for the Study of African Economies, Oxford, March. Van der Berg, S (2001a). ‘Resource shifts in South African schools after the political transition’. Development Southern Africa, 18 (4): 405–421. Van der Berg, S (2001b). ‘Redistribution through the budget: Public expenditure incidence in South Africa, 1993–1997’. Social Dynamics, 27(1): 140–164. Van der Berg, S and R Burger (2002). ‘Education and socio-economic differentials: A study of school performances in the Western Cape’. Paper presented to the Conference: Labour Markets and Poverty in South Africa, DPRU/FES, Johannesburg, October. Veel, R (1999). ‘Language, knowledge and authority in school mathematics’ in Christie, F (ed) (1999). Vinjevold, P (2000). ‘Cadbury School Development Initiative (CSDI): Grade 3 and Grade 6 Learner Assessment Baseline Study’. JET: Mimeo. Vinjevold, P and L Crouch (2001). Literacy, Numeracy and Policy: Towards Assessing Learning in Grade 3. Pretoria: Research Triangle Institute. Vinjevold, P and J Roberts (1999). ‘External evaluation report: Provision of learning materials. Grade 7 pilot project’. JET: Mimeo. Walkerdine, V (1988). The Mastery of Reason. London: Routledge. Webb, P, G Boltt, P Austin, M Cloete, V England, N Feza, J Isley, R Kurup, M-L Meires and K Wessels (1999). ‘The nature and impact of accredited inservice education of under-qualified science and mathemetics teachers: Which factors generate best practice in the classroom?’ University of Port Elizabeth. Mimeo.

Index Page numbers in italic refer to tables or figures. A accountability 5–6 in districts 118, 126 measures 10–15 in South African system 9–10 administration and management 136 administrative organisation 77 assessment 34–37 practices 126 scope of in SA 35–36 using data for 126 B background of learners 131 ‘black box’ studies 51 books 70, 130, 136 budget and expenditure for 15, 15 procurement and regulation of 112–113 and stationery 124 storage and recording of 112 C case studies 24 generalisability of 50 methodology of 62 child-centred (competence) approach 82 classification 132 of interdiscursive relations 72–73, 105–106 ‘classroom culturalists’ 50, 129 classroom learning, model of 87, 88, 89 classroom practice and learner performance 104–106 class size 23, 130 closed reporting mode 31–32 code ‘collection’ and integrated sub-types of the school code 74, 75, 76, 138n community and school codes 69, 71, 74 orientation 70, 137n code-switching 62 cognitive competence 138n cognitive demand 100–101, 105, 137n

cognitive resources 15, 51, see also books; stationery collection code 74–75, 138n community code 69 comparative studies 41 competence and performance models of pedagogy 4, 5, 82 competency specification, lack of 37 conceptual knowledge 70, 82 conditional grants, expenditure on 13, 13 conditions of schools 40 construct reliability 32–33 context and input indicators 21 contextual factors 28–29, 38, 52–55, 68 and improved learning 52 costs of schooling 130 Cronbach’s Alpha 40–41, 41 curriculum 133 coverage 77, 78, 105, 113–114 design 83 implemented 74–80 intended 71–74 and pedagogy 124 process, outline of 67 structure 72–74 underspecification of content 37 Curriculum 2005 110–113 D data analysis 91, 94–103, 107 dependent 31 and districts 123–126 independent 31–32 data management systems 123–124 demarcation, vertical 73–74 dependent data 31 descriptive studies 24 design features of studies 22, 23–25 and method 107 of schooling theory 87–89 development appraisal system (DAS) 9, 15 discourse, horizontal and vertical 72 discrimination 137n distributional equity 3

147

Index

District Development and Support Project (DDSP) 8–9, 21 district-level indicators 122 district case for 118 impact of 126–127 management 138n and South African context 118–121 and theory of schooling 121 donor(s) local and offshore 3 and research 20 support 137n E early childhood educare 13 education, access to 67 Education Action Zone (EAZ) 7–8, 14, 21, 119 educational factors and learning 81 educational level of parents and community 53 educational process factors 31–33, 51–52, 56, 68 Education Management Information System (EMIS) 19 educator development 15–16 enacted (implemented) curriculum 31 enquiry, principles for 129 enrolment rates of population groups 52–53 ethos of school 60 evaluation 126 and assessment 79–80, 84, 136 criteria 102–103, 116 evidence-based policy 136 expenditure, absorptive capacity 13 explanatory studies 25 external framing 79 F facilities 56 family structure 54 financial management 13 framing 73, 76, 131, 132 external 79 internal 78–79, 113–115, 131 micro-pacing 99–100, 105 macro 84 macro-pacing 98, 105

148

meso 84 planning and coverage 77, 78, 98 progression 73–74 sequence 105, 113–115 funded interventions 20 G gender factors 54 generalisation 50, 128 governance and community 60 government-initiated projects 3 H Hierarchical Linear Modelling (HLM) 58 higher education institutions 20 HIV/AIDS 13 home background factors 28–29 household effect index 29 wealth 53 human capital 55 human resources 29–30 I Imbewu Project 21 independent data 31–32 inequality 137n information base for research 19–21 input factors 29–31 in-service education and training (INSET) 7, 124 in-school 115–116 out-school 115–116 inside-out (school-focused) approach 4–5 in South Africa 6–7 institutional culture 15 instructional and regulative discourse 138n integrated code 75 interdisciplinary relations 72–73 interdiscursive relations 72–73, 105–106 indicators 101–102 internal framing (pacing) 78 International Association for Educational Achievement (IEA) 46 international comparative studies 20, 41 inter-rater reliability 32, 90 intervention studies 26

Index

J JET Education Services 3–4 JET Research on School Development 4 K knowledge mediation 79–80 structure 72 L language 135–136 of instruction 95–96, 104–105, 111 and learning 58–59, 65 and school code 71 of tests 39, 43–44 use and instruction 54–55 large-scale descriptive studies 51–59 learner assessment studies 36 background data 29 performance levels 38, 59, 104 testing 34, 35 learner achievement 41 grade level benchmarks 43 variance in results 46–47 learner-classroom ratio 30–31, 56 learner-teacher ratio 30–31, 56 learning materials 56 maximising potential for 131 and motivation 60–61 Life Skills tests 35 literacy JET tests 57 low levels of 40 tasks study 46 tests 35 logistical limitations 91 M macro framing 84 macro-level pacing 98, 105 Mahlahle project baseline study 9, 21, 107, 110–114, 121 management 134–135 at district and school level 56 and leadership 69 quality of 51

systems 13–15 task 61–63 mathematics and science studies 35 and settlement type 54 teaching and learning 34 tests 42, 42, 43–44 meso framing 84 micro framing 83, 84 micro-level pacing 99–100, 105 mixed-method, multi-level models 66 monitoring in-school 125 performance 10–13, 115 and quality assuring assessment practices 126 Monitoring Learner Assessment (MLA) study 42, 43 mother’s educational level 29 multiple content specifications 37 N National Curriculum Statements (NCS) 10, 129 National Literacy and Numeracy Strategies 6 NGOs 24 and INSET 16 intervention studies by 24 Norms and Standards for Funding Schools 3 numeracy 42 JET tests 57 and MLA test 43 and reading tests 44–46, 44, 45, 46 numeracy/mathematics tests 35 O observed data 32 observational research potential subjectivity of 90–91 see also quasi-experimental design off-budget expenditure 3 organisational development 13–15 orientation to meaning 132 outside-in (standards-based) approach 4–5 in South Africa 7–8 P pacing (internal framing) 78–79, 113–115, 131

149

Index

parents education variance of 53 income 53 pass rates black schools 64 poorest schools 65 Pedagogical Content Knowledge (PCK) 78 pedagogic pallet 5 pedagogic practice 82 classroom-level indicators 92–93 pedagogic structure 77 pedagogy 84, 133–134 performance models for measuring 4–5 monitoring of 10–13 Phalaborwa Project 21, 107, 114 physical resources 30–31 planning of curriculum 105, 113–115 and directing activities 124 ‘policy mechanics’ 50, 52 population sampled studies 51–52 targeted studies 57–59 poverty 130 principal qualifications 30 process factors data collection 31 indicators 21 studies 24 procurement systems 112, 124 progression (relations of) 73 provinces, variations across 11, 11–12 Q quality assurance of assessment practices 126 framework for 35 Quality Learning Project (QLP) 8, 21, 111, 121, 123 quasi-experimental design 23–24, 25 R racial inequality 52–53 racial prejudices 34 randomisation 23 randomised trials 137n READ Business Trust Intervention 9, 21, 129

150

reading tests 34 and writing 70, 96, 105, 111–113, 135–136 reform, approaches to 4 regulative discourse 138n reliability construct 32–33 inter-rater 32, 90 of self-report data 91 of tests 39–41 replication 128 research questions 23, 107 resource(s) factors of 55–56, 55, 68, 69 management of 111 optimising use of 130 Review Committee Report 133 Revised National Curriculum Statements 107 roles and responsibilities 123 S SAILI Primary School Project 9, 21 sampling 25–28 and districts 121–123, 122 and schools 89–90 size 28 for TIMSS-R in South Africa 26 school code 69, 74, 131–132 condition index 31 organisation 83 performance 126–127 structure of system 68 types, typology of 76 school-based social capital 30 schooling and performance 63–65 see also theory of schooling school reform approaches to 4–6 in South Africa 6–9 School Register of Needs (SRN) 30–31, 57 science and settlement type 54 test 42, 42 scientific research 128–129 self-report data 91

Index

senior certificate examination results 10 maths results 12 pass rates by province 11 sequencing of curriculum 105, 113–115 settlement type and income 53–54 Siyathuthuka 21, 107, 110, 112 small-scale descriptive studies 60–63 social capital 55 school-based 30 social organisation and districts 123 of schooling 74–76, 75 social relations 94, 104, 109 social theory of schooling, see theory of schooling social values 76, 82, 94, 104, 107 socio-economic context 69 socio-economic status (SES) 28–29, 53, 66, 68, 82 socio-political conditions 69 South African Qualifications Authority 107, 109 staff vacancies 123 stationery budget and expenditure for 15, 15 see also books statistical literacy 138n statistical studies generalisability of 50 subjectivity, potential 90–91 sub-populations 57 success of schooling indicators 21 support development appraisal 15 in districts 119 measures 5–6 of principals and teachers 125–126 in South African system 9–10 symbolic mastery 131 symbol systems, mastery of 69 Systemic Evaluation 12 systemic reform 5–6 key nodes of 17 in South Africa 8–9

teacher education 30 knowledgeability 77, 78 motivation 11 qualifications 55–56, 130 teacher-centred (performance) approach 82 Tennessee Project STAR 23 test(s) construction of 36–38 format of items 39 instruments 34, 40–41 length of 39–40 results of 41–47 validity and reliability of 34, 39–41 textbooks, see books theory and hypotheses 83–84 need for 68 theory of schooling 80–84, 81, 88, 131 data analysis 91–103 design 87–89 district-level indicators 122 school-level indicators 108 starting points 67–71 time organisation 109–111 TIMSS-R (Third International Maths and Science Study-Repeat) 21, 24, 25, 41–42, 129 mean scores 42 sample 126 transmission costs 5 of skills 138n of values 138n U Upper Quarter benchmark 137n V validity of tests 39–41 W WCED Learner Assessment Project 25–26 Whole School Evaluation (WSE) 126

T targeted population studies 57–59 task organisation 109

151