| Archive - these pages are part of the continuing record of Executive Government - for the current Administration, see www.beehive.govt.nz |
The essential documents on which Reviews are based are the National Education Guidelines which include the National Education Goals, the National Curriculum Statements and the National Administration Guidelines. (Appendix 5) Schools and Early Childhood Education Centres are required to have a Charter under the Education Act (1989) s.61-64 and s.312-313. These documents are integral to a self-managing system because they encapsulate the vision of the institution and the local circumstances which influence it, including Maori in the community and special features. Charters may be amended after consultation with the community and with the approval of the Secretary for Education who is also empowered to enforce the Charter but not without consulting the Chief Review Officer. Notwithstanding the intention of the Act that the Charter is a contract for educational delivery, the Panel has concluded that Charters are, by and large, no longer functional documents that guide the schools’ and centres’ operations, despite a number being presented to the Secretary for modification during 1997.
For the Early Childhood Education sector there is also provision for Statements of Desirable Objectives and Practices (DOPs) to be set by Regulation. The 1996 revision of the Statements will become operational on 1 August 1998 (as stated in the New Zealand Education Gazette 13.10.97). The purpose of the DOPs is to establish national criteria for the provision of quality early childhood education and care. They are deemed to be part of the Charter of every provider.
Effectiveness reviews, therefore:
| Stakeholder | Performance Interest |
|---|---|
| Crown (purchaser) | The consistency of service delivery with purchase/ contractual requirements (e.g. as may be set out in charters) |
| Crown (owner) | The quality with which service providers manage the Crown’s ownership interests i.e.
|
| Students and Parents (consumer) | The ability of governing/ managing bodies to:
|
From Education Briefing notes to the Panel. (Further details on Accountability Reviews are set out in Appendix 9.)
The Panel is also aware that the Office recognises the need to be more firmly focused on education effectiveness, while retaining the requirement for schools to meet legal compliance hence the introduction of Accountability Reviews in 1998. The Panel does, however, have serious reservations about the transition from Effectiveness Reviews to Accountability Reviews. These concerns are largely to do with effective communication.
Schools do not appear to have had the same exposure and experience of Effectiveness Reviews as they have had with Assurance Audits. Many schools report that, in the past seven years, they have had two Assurance Audits and as recently as in the past 12-18 months had their first Effectiveness Review. A small number of schools report that they are due to have their first Effectiveness Review this year although the 1996/97 Education Review Office Annual Report indicated that 359 of 1243 reviews were Effectiveness Reviews (28%) the timetables for reviews in each region for the first part of Term 3 1997 which were provided for the Panel show that, of 123 reviews to be undertaken, only 22 (18%) were Effectiveness Reviews. Only the Central South Region showed a preponderance of Effectiveness Reviews (10/15). This is surprising when the Office is aware of the degree of compliance which exists and reinforces the need to move rapidly to Accountability Reviews.
Submissions repeatedly referred to schools being diverse communities and collectives of learners that frequently acted in synergy though not always exclusively so. While academic achievement is unequivocally an important component, a range of other significant factors and complex interactions contribute to a student’s achievements, many of which are not tangible and, consequently, are not directly measurable. That this is so should not preclude the value of such factors being considered in the evaluation process. Indeed, it is the Panel’s opinion that qualitative data is as important and as valid as the quantitative data which is gathered to provide a profile on a student, programme or school. The value of any piece of data will, however, be determined by a range of factors, including context and relevance.
The Panel further subscribes to the view that it is not a matter of either/or but that a combination of both would produce the richest and most satisfying results for all interested parties. It should of course be noted that either data base is of little value to a school if it is only being collected for the purposes of meeting the requirements of the external evaluation agency rather than to inform the school, the students and their caregivers.
"All OECD countries have made tremendous economic efforts during the past twenty years to invest in the material provision of schools and to carry out sweeping structural, organisational and curricular reforms. These efforts have brought considerable success. So far, however, success has been measured largely in material terms." The OECD (1989) warns that "the next phase will call for emphasis on less tangible improvements which will necessarily prove more difficult to achieve than the fulfilment of quantitative targets. It will require educational authorities at all levels to consider now what compulsory schooling should look like in ten or twenty years’ time. What will society then be wanting from the schools and what will schools be wanting from society?"
When the Ministers of OECD countries met in Paris in 1984, " Quality in Basic Schooling" figured as one of three major items on their agenda. The Ministers concluded their deliberations by issuing a communiqué in which they declared that a priority for future OECD work in the education field should be:
Writers of the 1990s such as Hargreaves & Hopkins (1991) Southworth (1993) and Reynolds, Hopkins and Stoll, (1993) offer advice on linking knowledge of effectiveness with practices designed to result in "better" schools and centres. The contemporary improvement movement assumes, e.g. that the individual institution should be the centre of change; that improvement should be carefully planned and managed; that educational goals are more than test scores; that leadership support and staff energy are both important; and that change succeeds when it becomes part of natural behaviour.
The Panel further develops this approach in the report.
"...teachers themselves must depend on their observation of students to determine their effectiveness on a day by day basis. Research has shown that teachers depend on available signs, involvement and motivation to determine whether students are learning or not. Since teachers have no direct access to the minds of students, and they must adjust their teaching to the apparent level of involvement of students in the learning process, teachers have to use all available indirect signs of their effect on students’ minds...." (Nuthall, submission 104.)Nuthall goes on to remind us that observation of student behaviours in the classroom is not a reliable substitute for direct assessment of student learning even though such evaluation is always relative. He comments that "the observational expertise has been translated into a myth and that research has shown it is rarely possible to tell by observation when students are learning." In Nuthall’s view, observations need to be regular and set in context to allow for meaningful analysis and conclusions to be drawn.
These comments constitute some of the reasons for reviewers requiring assessment data, its aggregation and analysis in making judgements about the effectiveness of programmes delivered by teachers. There are no shortcuts. This does not mean that descriptive information should not be kept. The Panel has already indicated that both quantitative and qualitative information are desirable.
Submissions from Principals and Trustees from the low decile schools were particularly vociferous on the need for review reports to include comment on the contextual issues that impacted on their schools. This was supported by others who added that comment should also cover government social sector policy generally, while others believed it should be restricted to the Ministry of Education’s policies and practices.
Essentially the message the Panel received is that schools want contextual issues to be included in the review process as an acknowledgement that there are factors external to the school over which they have no control; but that it is likely these factors have an impact on student achievement levels and the effectiveness of the school’s performance.
Even education researchers have varying views on the impact of external factors on a school’s effectiveness. The researchers commissioned by PPTA to Review the Education Review Office (in an informal communication with the Panel) refer to Metz (1990) who observes:
"To the degree that the educational reform movement sets aside class differences as unimportant, it brackets and overlooks one of the major influences on schools. It consequently relies for all of its impact on attempting to change patterns that exert much weaker influence. To ignore the most forceful influences in a situation is rarely a prescription for effective reform."
On the other hand, Thomas, Simmons and Mortimore (1995) comment that:
"Recent research has shown that data relating to factors outside the control of the school (for example, gender and social class, and neighbourhood characteristics of pupil’s homes) are useful only in fine-tuning schools’ value added results."Results from two major American studies on socio-economic status as a context variable, one by Hallinger & Murphy and one by Teddlie and Stringfield (in Reynolds et al 1994) confirm that there were some characteristics which distinguished effective from ineffective schools, regardless of the socio-economic status (SES) of the schools. They included: clear academic mission and focus, orderly environment, high academic engaged time on task, and frequent monitoring of student progress.
Teddlie further reports that the differences in characteristics associated with effectiveness in middle and low SES schools revolve around six areas:
According to Silver (1994), schools, the philosophers, and providers of schooling, have always been concerned with outcomes.’ Indeed quality educational outcomes are at the very heart of this review. In attempting to define contextual issues the Panel believed that a range of variables might be considered including the effects of the school’s culture on the learning outcomes of their students. It is the examination of this issue that has inspired considerable research in the last 20 years, notably within the School Effectiveness Movement.
Mortimore notes that, in the USA, ‘the research on the potential differential effects of individual schools has been closely bound up with concern about the plight of disadvantaged children and the limited opportunities available to them.’ In the United Kingdom, detailed research (Reynolds 1976, Rutter 1979, Mortimore 1988, Tizard 1988, Smith & Tomlinson 1989), found clear evidence of school differences in students’ outcomes. Each of the research teams used this evidence to argue that these differences were not simply due to the effects of schools receiving different types of students but, rather, that they were associated with differences in the way the schools, through management and the quality of the teaching and learning environment, promoted achievement.’
