It is important to establish that the visit is one part of the review. Documentation sets the school in perspective for the reviewers who will then visit the school for a number of days. This is followed by analysis of the information gathered in order to make judgements and to form opinions before reporting formally to the Board of Trustees within 15 or 20 working days (depending on the type of review) of the completion of the on-site visit. The Board has a further 20 working days to respond to the unconfirmed report and to discuss errors of fact and to suggest amendments. The report is then confirmed and becomes a public document.

At the conclusion of the on-site visit an oral report is given to the Board and Principal, although staff may also be involved. The oral report has come to be regarded as an indication of the likely contents of the unconfirmed report when the reality is that it is a means of initiating discussion and providing feedback to those present before the team leaves the school. It is an exit meeting for reviewers at the conclusion of the on-site visit. The confusion over the differences between the oral and written reports has lead to accusations of inconsistency and lack of credibility. It is clear to the Panel that a great deal of effort must go into clarifying the purpose of the oral report meeting.

While teachers are crucial to effective curriculum delivery their individual competence is not being examined by reviewers. It is the task of the Principal to evaluate teacher competence by using the Performance Management System as the basis on which to advise the Board of Trustees on the nature of professional development which may be required. Teachers are responsible for classroom management, motivating their students and monitoring student progress in a supportive school environment. The reviewers will be asking what steps the school takes to support the teacher, whether or not the principal provides professional leadership, how often a teacher’s classroom has been formally visited and if there are good working relationships in the school. They can also investigate the degree to which the school is providing the resources needed to meet teachers’ needs, and they can inquire, for example, about whether the teacher has been trained in information technology and how the internal communication system works.

The Panel concluded that much of the feeling about to the Education Review Office is based on lack of knowledge and understanding about a review’s nature and purpose in schools and centres as well as ineffective communication from the Office. It is an issue which must be addressed as a matter of urgency by all the parties involved. There is a dual responsibility for information to be provided by the Office and to be understood by people in schools and centres.

When an on-site visit is scheduled and conducted, it is essential to the credibility of the Education Review Office that reviewers adhere to the timetable and give their undivided attention to the school or centre during the time allocated.

Submissions from the Early Childhood Education sector called for greater frequency of reviews. They believe, and the Panel concurs, that a 4-year interval is too long between reviews. Early Childhood educators welcome reviewers and want the "spot" reviewing to continue, although concern was expressed that the Ministry of Education had moved into this area as well. (See Section 7 of Terms of Reference 4.) There was also a call for greater attention to be given to curriculum development and delivery, whether Te Whariki was being implemented and the quality of adult-child interactions. The Panel is reluctant to specify the frequency of reviews of early childhood centres but believes it must be reduced to a 3-year interval as a minimum. Four years could be the entire life of a child in a centre, which is too long, particularly if there are risk factors which may go undetected.

The introduction of Accountability Reviews should allow the Education Review Office to rethink their scheduling of on-site visits.

Where a school has already established a high level of legal compliance with effective curriculum delivery and reporting of student achievement, with a robust self-review system in place, the frequency of review can be extended to 4 years. This would allow more frequent reviews where there are indicators which raise concerns about a school’s performance (e.g. a change of Principal, high staff turnover, a transient population, poor student achievement, high truancy, having to co-opt a number of Board of Trustees members or other risk factors). The panel believes the Office should advise the School or Centre of the appropriate timing of its next review in the confirmed report.

One argument, heavily supported, which the Panel found persuasive, was the need for a realistic time-frame to be put in place to allow schools and centres to respond to the recommendations and to develop an action plan to implement the changes required to improve their effectiveness. Accordingly, we are recommending changes to the period between receipt of the unconfirmed and confirmed written reports so that the actions taken by the school and centre can be included in the final report. Our proposal is more in line with the Picot and Tomorrow’s Schools plan and extends the present time-frame for the confirmed report. We believe the proposal will be seen as fair and reasonable and also will give greater incentive to respond positively to the recommendations.

Once the unconfirmed report is received, and any errors of fact in it are negotiated to a satisfactory conclusion, then, a period of three months should elapse to allow the school to develop an action plan to address the review report recommendations. This would allow time for the Board to seek advice (see page 50-51) and the Education Review Office to alert the Ministry of Education. Plans developed would be incorporated in the report which would be confirmed and become a document in the public domain.

In the interests of natural justice, the Panel concluded that, in the first instance, there is an onus on the parties involved to find solutions where conflict emerges between a school or centre and the Education Review Office over matters of fact or the interpretation of them in the unconfirmed report. This would probably involve the Regional Manager of the Educational Review Office. If the purpose of the oral report was clearly understood and reviewers were more direct in their presentation then such events would be rare. The Education Review Office should make available to a school or centre the evidence they hold in support of their statements in the report, when requested to do so.

In the event of a dispute being unresolved, the view of the Panel is that it is the responsibility of the Education Review Office to resolve the matter with the National Manager, Reporting Services, making a judgement on the basis of evidence provided by the parties.

This approach assumes that this manager would have no part in reporting on individual schools or centre. The final authority for dispute resolution must rest with the Chief Review Officer. The key to reducing differences of opinion and the potential for disputes is transparency. The final element in the dispute resolution process is that all disputes requiring intervention of the National Manager Reporting Services, should be reported to the Advisory Council on Quality in Education of the Education Review Office in the interests of transparency.

"when we invest more into understanding the thinking of teachers, and helping them to be more aware of their own thinking as they teach, we will begin to unlock many of the mysteries of what makes the difference between effective teachers and less effective teachers.’ We want ‘teachers who will approach their teaching with an eagerness and commitment that they not only can make a difference, but they do." (Education Review 19.9.97)

The Panel is convinced that the barriers of socio-economic status and culture cannot be an excuse for not making a difference and, indeed, Harker and Nash (1996) show that some schools are significantly more effective than others when School Certificate results in Mathematics, Science and English are considered and that variability is substantially reduced when the initial ability of students, their gender, SES and ethnicity as well as school mix are taken into account. The value-added estimate reveals a high level of success among many schools which are regarded as not performing. As David Lange (1995) commented,

"We never talk about success in terms of the numbers of children whose English vocabulary has expanded by 50 per cent in two years. Yet (many poor) schools would beat any other school in the country on that count."

It seems that administrators and educators in New Zealand are not persuading teachers to engage in, and commit to applying indicators which reveal student learning and not just observer or teacher beliefs about them. In his submission to the Panel, Professor Nuthall (University of Canterbury) advocated that there is a substantial body of research to guide how the quality of teaching services and student achievement should be evaluated. In isolation from other factors, observation is not a good tool and neither are checklists. Direct assessment is essential and related to data-bases of normative data about what can or ought to be expected of students in specific circumstances. It is not acceptable, as many teachers have told the Panel, that their professional experience tells them whether or not their students are making progress at school.

It is against this background that the Education Review Office work must be based, not just on the questions, "What do you expect your students to achieve?" and "What assessment practices are used to determine learning?" but also on results of assessment, analysis of performance against expectation and how the information is used to improve teaching and learning. If teaching is to be a truly "research informed profession" then a great deal of effort must be invested in improving teacher belief in applying the indicators which will give them greater certainty about student learning.

Education in New Zealand is highly regarded by the international community for its innovation and responsiveness as well as for its willingness to lead and to take risks. Examples would include curriculum development, self-management, direct resourcing and the qualifications framework. One of the great strengths of New Zealand teachers is their independence, and their ability to develop teaching strategies which suit them and the children’s learning style. Such teachers think for themselves and reflect on their practices. Within the existing accountability framework, the Panel would welcome greater acknowledgement of the diversity of approaches within the present system. Diversity is celebrated within the early childhood sector where the different approaches have developed collaborative mechanisms for developing and implementing guidelines for improving the quality of education and care. The prime example for this is Te Whariki, the Early Childhood Education Curriculum statement. The same ought to be true for the compulsory sector where there is a need for better dissemination of best practice models.

Boards of Trustees, Principals and Centre Managers understand that accountability and transparency go together and that the public have a right to know what the external evaluator has found in reviewing a school or centre. There is an acceptance that the Education Review Office is not responsible for the media but there is also concern that media reports may sometimes be less than accurate in the way the report is presented, especially if there are a number of problems to be addressed. As was observed earlier, the vast majority of media reports in newspapers have cast schools and centres in a positive light with about ten percent getting negative headlines. Most reports are condensed summaries of findings and appear to be both fair and balanced. There are some clippings which show that schools have acknowledged that the publication of the report was a positive stimulus to addressing their problems and in turning the schools into effective learning environments. The Panel readily acknowledges that this is not always the case.

The Education Review Office has no powers to enforce its recommendations. In making its reports publicly available, it concedes that it is using them as an incentive for action by the school or centre concerned. Transparency is regarded by the Office as an instrument of accountability. Its powers are limited to returning to the institution for a follow up review to determine the extent of changes and to report to other agencies on the findings in the expectation of action. The Panel believes that its recommendation to extend the time-frame for action, with subsequent inclusion of the implementation of the recommendations in the confirmed report, will add an incentive and reduce the threat which can accompany the early release of review reports.

Some schools have developed a good relationship with their local media and are proactive in informing local papers about their activities. The School Trustees Association has acknowledged that managing publicity has become an issue to which it is directing attention in its Boards of Trustees training.

It is claimed by Boards of Trustees, Principals and Centre Managers that the reviewers are among the few in the Education Sector who have an overview of practice in the sector and they are looked to for advice and guidance. Some reviewers, the Panel heard, will respond to requests orally but most will not. The Panel has formed the view that the Education Review Office should encourage reviewers to point to examples of best practice where there is a need and a request is made to do so.

The availability of advice, guidance and support appears to be variable and, in a country where many schools and centres are isolated in small communities, the absence of professional colleagues can cause acute loneliness. Many teachers do not know how to access advice and guidance, peer support is minimal and subject associations may not be as active as formerly.

The Panel acknowledges that the gap is a very real one and believes there is an urgent need for both proactive and reactive advice, guidance and support systems to be put in place. As a first step, it is suggested that the Education Review Office act as broker by inserting a page in its reports to schools and centres providing information about the sources of advice which are available in the district in which the school or centre is located. This would include the Advisory Services at Colleges of Education; the Early Childhood Development Unit; consultants in the field of governance, management, curriculum, assessment, personnel, finance, property and evaluation; possible resource personnel who would give peer support and relevant Evaluation publications.

The suggestions we have made would neither challenge the principle of self-management nor the independence of the Education Review Office. They would meet a demand from the professionals in the schools and centres who say, "It would be nice to know that there is support there, when we want it."

The position of small and rural schools is of particular concern. School clusters working together seem to the Panel to be a way of securing improvement in educational opportunity. They would be able to produce solutions to identified needs with the advantage of peer support. The consequence would be an increase in confidence and reduced workload for those involved. There are important issues. Some schools, principals and teachers may be unwilling to participate or may feel that undue advantage may be taken of their willingness to do so or it is believed it is the Ministry of Education’s responsibility to provide the support. However, the Panel recommends that, wherever possible, clusters of schools work together on the issues of concern to them. It may be necessary for the Ministry of Education to provide facilitators to assist the clusters.

We endorse the practice of publishing Evaluation Reports which are based on best practice on topics of relevance to schools and centres.

Wylie (1997) identifies advisers as the primary sources of information and advice on curriculum, assessment policy and advice and staff development. Other major sources are other teachers in the school and curriculum contracts. Consultations with leaders of the Advisory Services in Dunedin, Christchurch and Hamilton revealed a variety of responses. It was reported that, after an Effectiveness Review, a Principal often calls in advisers to help to deal with the non-compliance issues. In the Waikato, the service has been renamed the "Teacher Support Service" aiming for school development, offering leadership training and facilitating school development through exploring questions such as, "Where are you at, where do you want to be, when do you want to get there, how can we help you to get there, and how will you know when you’ve arrived?" Initiatives such as this are very important in moving through self-review to self-improvement.

The model to which the Panel has responded positively has been developed by the Christchurch College of Education Advisory Team and the Education Review Office. A draft protocol is in place to ensure that each service has an accurate understanding of the other’s work and to allow the easy access and exchange of knowledge and expertise from one service to the other. In practice, it means the Advisory Services can be alerted where a review highlights issues where the Service may need to work with a school or centre. The Panel recommends that this initiative be followed in other regions as a most productive way of involving the Advisory Service in a proactive role.