A small number of submissions advocated that the evaluation of schools and centres should be a contestable service. The Office for Standards in Education (OFSTED) in the United Kingdom was cited as a possible model. There, teams of independent inspectors are invited to tender for the inspection of individual schools by OFSTED; if they meet a minimum quality threshold, contracts are awarded on the basis of value for money. Every inspection team must include one "registered inspector" who takes ultimate responsibility for the satisfactory completion of the contract and one "lay inspector" who should not be professionally involved in education. The Panel has considered the proposal and has resolved that there is no groundswell of support for it.
Some of the reasons for this conclusion include: the small size of the New Zealand market; the difficulty of covering the country, particularly its more remote parts; the need to achieve national consistency where there are disparate entities conducting reviews (OFSTED’s principal problem); the need to ensure that the teams have the required knowledge and background to evaluate effectively; and the training costs where there is a constant or periodic changeover of personnel.
Reviewers require a substantial background of experience and knowledge and, even when competent educational professionals are recruited, they still need extensive training before going into the field and, even when they do, further support is required.
The Education Act s.62 and s.63 make provision for policies and practices that "reflect ...the unique position of Maori culture" and that
"all reasonable steps to ensure that instruction in tikanga Maori and Te Reo Maori are provided for full time students whose parents ask for it".
Great concern about Maori underachievement was reflected in submissions received about Maori education and the education of Maori. These concerns are shared by the Panel.
It is evident that the Education Review Office has a responsibility to investigate and report on the policies and procedures adopted by schools and centres and that the rigour with which the Education Review Office approaches this task needs to be intensified. Doing so would signal to schools and centres that responsiveness to Maori and tangata whenua is a priority that is critical to improving the effectiveness of educational outcomes and results that are of an enduring nature. Rigorous, robust review must occur of the total learning experience of Maori students and must not be restricted to Maori language and custom. The review process would take into account a range of features which could include some or all of the following: resourcing that facilitates responsiveness; the policy framework that sets direction and accountability; data collection and analysis that informs and improves; professional development; appropriate cultural practices; stakeholder satisfaction; pastoral care provision; representation; liaison and consultation; and attendance, attrition and retention rates of Maori students.
It is important to acknowledge that the wide range of performance indicators set out in Appendix 10 also apply to Maori Education.
The Panel noted the emerging trend for students to move to and from immersion and mainstream programmes. Anecdotal information suggests that the numbers of learners in this category are significant and growing. The implications of this flow needs to be examined, analysed and evaluated to ensure that no barriers to learning are developing. A systematic collection of data in this area is required to establish the extent and nature of any problems, the risk factors which are apparent and how schools and centres are currently responding as well as what may be required in the future by students who are making these transitions.
The Panel makes a number of recommendations which are believed to be first steps in addressing some of the issues for Maori Education.
The Education Act (1989) s.21 provides for parents to apply to the Secretary of Education for long-term exemption from enrolment. The Secretary must be satisfied that the ‘person will be taught at least as regularly and as well as in a registered school’. A registered school is a State School, or is an independent school (to which the National Curriculum Guidelines do not apply). The latter however, are required to provide a suitable curriculum. To comply with these conditions the Secretary must have evidence of the routines to be followed in the home-schooling environment and the approach, planning and balance in the curriculum adopted.
Since 1994, when Government withdrew funding for reviewing home schooling, the Ministry of Education notified all home schoolers that they must provide a detailed written report annually on all aspects of their home schooling programmes. Mathematics and Language and Reading are to be commented on fully in 1997 against the following questions:
- What knowledge, skills and attitudes have you concentrated on during the year?
- What activities have you used to build up the knowledge, skills and attitudes?
- What progress has your child made and how has this been revealed?
- Are there areas where progress was unsatisfactory and what do you plan to do about this?
- What does your child enjoy most?
At present some 5,500 young people of school age are being home-schooled. Given this level of activity, a decision was made to reinstate home schooling audits in the 1997-98 financial year, with 940-980 reviews to be undertaken. These reviews are a form of Accountability Review and will concentrate on quality of programmes, their effectiveness and the educational outcomes.
Strong submissions were made by home schoolers that the current restriction on access to private homes should be retained. This has the disadvantage of preventing reviewers from observing the educational programme being provided unless access is volunteered by parents. It can also mean that there is no access to the student which is more serious. While the Panel accepts the prohibition on entry without consent to private dwellings, the view was formed that, in the interests of the student, entry should not be denied without valid reason. A reviewer is, however, entitled under s.327(b)(ii) and (d) and (e) to meet with, inspect the work of, and make copies of any work documents of a student. The Panel believes that the Ministry should continue to require home schoolers to provide documented evidence of curriculum provision through their annual written reports on specified areas of knowledge.
A number of New Zealand schools own and manage student hostels and, so, they are in effect ‘boarding schools’ where the hostel is regarded as an integral part of the educational provision. At present, the Educational Review Office has no powers to audit the hostel where it is possible the safety of students may have a material influence on the quality of their educational experience. While some independent schools have requested the Education Review Office to include the hostel in their review, some School Trustees have rejected the possibility that the Education Review Office should have any interest in student accommodation. The Independent Schools Council made a strong submission that the boarding component of schools had been neglected; yet it is critical to the health, safety and culture of a school. The Panel has reached the conclusion that the Education Review Office ought to have powers of entry to hostels where they are an integral part of the educational provision of the school.
The Education Act (1989) s.8 and 9 explicitly refer to the equal rights of those with disabilities to enrol and receive education at state schools. It was brought to the Panel’s attention that negative attitudes to children with disabilities persist in many schools and parents are often dissatisfied both with the reception they receive when applying for enrolment and the educational provision their children receive. The Education Review Office refers to special education in its reports but it remains clear that an in-depth review of school and centre policies, enrolment procedures, education provision and its effectiveness needs to be undertaken to assure parents and the community that the intentions of the Act are being adhered to.
The National Administration Guidelines are explicit in their intention to ensure that the nation’s young people receive a balanced curriculum covering the areas of knowledge, skills and attitudes detailed in the National Curriculum Statements along with monitoring progress and achievement and overcoming barriers to learning. They also require Boards of Trustees to provide appropriate career information and guidance for all students. To do so, Boards are expected to have strategies and policies, appropriate to the background and age of the students that include goals, processes and reporting procedures. Services to students such as guidance, counselling, careers awareness and advice, transition services and health and well-being advice are all important aspects of school life and on which the Education Review Office should comment in its reports.
Questions have been raised regarding the size and costs of the Corporate office of the Education Review Office in Wellington. What at first sight appears to be a top heavy structure emerges as a very lean operation in practice. The Office has three Outputs with $14.871M (90 percent) of its allocated budget for the 1997/98 financial year devoted directly to Reporting Services, that is (Outputs 1 and 2) Accountability Reviews in Early Childhood Services and Accountability Reviews in Schools and other education providers.
The remaining $1.603M is allocated to (Output 3) the Quality of Education Reports and Services, which includes Evaluation Reports Policy Advice and Ministerial Services.
The Corporate Office includes the Chief Review Officer’s Unit, Legal and Financial Services and Evaluation Services. Also located at Corporate Office are Te Uepu a Motu, the National Manager Reporting Services, and the Development and Human Resources Unit.
Many of these are services to the organisation as a whole and are part of a strategy which brings the management team together for greater efficiency and effectiveness. Given this breakdown, it is clear that the Education Review Office is an extremely lean operation.
Nevertheless, if in the future the Education Review Office were to be allocated an increase in funding resource, then the Panel believes priority must be the appointment of reviewers rather than expansion of the Corporate Office.
- That there be no change in the arrangements for recruiting reviewers to the Education Review Office.
- That the Education Review Office focus on Maori Education in 1998 and publish an Evaluation Report on its provision and effectiveness in mainstream schools and centres as well as the entry and exit circumstances of students to Kura Kaupapa and Te Kohanga Reo.
- That the Statements of Performance Indicators proposed in this review include targets for Maori education and education of Maori.
- That the Education Review Office ensure that review teams include a speaker of Te Reo wherever appropriate and, if not possible, a reviewer with an appreciation of Tikanga Maori.
- That the Education Review Office work with Te Puni Kokiri to establish an appropriate protocol for the review of the education of Maori and Maori education.
- That home schooling caregivers be reviewed on an ongoing basis and continue to be required to provide a written annual report to the Ministry of Education as the exempting authority.
- That the Education Review Office prepare protocols for the audit of home schooling providers, including access to the learner and the learning environment, where caregivers are willing to give access and make them available to those included in the review each year.
- That the Education Act be amended to give the Education Review Office powers of entry to school hostels where these are an integral part of the education provision of the school.
- That the Education Review Office take particular note in its Accountability Reviews of the provision of education for students with disabilities in schools and centres.
- That the Education Review Office focus on Special Education in 1998/99 and produce an evaluation report on the provision and practices in relation to students in schools and centres.
- That the Education Review Office include and comment on the range of guidance services available for students and their effectiveness in its reports on schools.
- That any increase in funding for the Education Review Office should be allocated to the appointment of additional qualified reviewers.
METHODOLOGY FOR EVALUATING THE IMPACT OF POLICY AND DELIVERY OF SERVICES OF OTHER STATE EDUCATION AGENCIES
The Mission Statement of the Education Review Office is "High quality evaluation contributing to high quality education." It follows that, to fulfil this mission, the Education Review Office should sit at the table when strategies for education are being developed. The feedback which the Education Review Office can provide will be based on its knowledge of and experience in the field, its links with a wide range of educational agencies (the Education Review Office consulted with 31 different agencies and working parties in 1996/97 and, in the first six months of 1997, with 16 major education stakeholders) and its independent Evaluation Reports.
The Ministry of Education recognises the importance of the Education Review Office’s contribution and acknowledges that there have been communication problems in the past. Policy development and implementation depends to a large extent on the effective use of people and resources. That there has been ‘stand-off’ in the past is regrettable and may have been the result of the separation of the Ministry of Education from the education service agencies like the Education Review Office which were part of the Tomorrow’s Schools development. There has been a lack of accountability to one another. Communication was not well developed and resulted in poor relationships. These may have been inhibiting factors in making good use of people with a wealth of knowledge, experience and expertise from the field and who could make a useful contribution to policy development. The Ministry of Education was perceived by the Education Review Office, Boards and Principals as being unreceptive and unresponsive.
It may be that the linkages between the independent agencies were inadequately detailed in the Education Act (1989). Given the emerging goodwill and evidence of better communication and involvement in policy development, the Panel suggests that the greater co-ordination and coherence within the education sector which has always been necessary, is now possible. Contributing factors to this improvement have been the appointment of a Minister responsible for the Education Review Office, separate from the Minister of Education, regular combined meetings of Chief Executive Officers together and with the Minister of Education and Minister responsible for the Education Review Office, a conscious acceptance of the Education Review Office’s ability to alert the Ministry to real and potential problems and the establishment of the Advisory Council on Quality in Education by the Education Review Office.
The Panel has already called for a communication strategy to be developed between the Education Review Office and the schools and centres. Expanding this strategy to include the Ministry of Education and Te Puni Kokiri and other agencies such as the Teacher Registration Board, New Zealand Qualifications Authority and Children and Young Persons Service would be welcomed by educators.
The Education Review Office must remain as an independent Department of State actively making inputs into policy through formal consultation with the Ministry of Education. Further, the Panel is satisfied that initiatives which have been taken already will have the effect of ensuring that protocols are in place and that formal and informal lines of communication are open and operating. It is the Ministry of Education which has the primary responsibility for policy development and its implementation once the approval of Cabinet is given. It is incumbent, therefore, on the Secretary for Education to continue to develop the consultation protocols with the agencies delivering education services or which have a direct impact on education. These include Te Puni Kokiri, the Education Review Office, New Zealand Qualifications Authority, the Early Childhood Development Unit, Te Kohanga Reo National Trust, the Education Training and Support Agency, the Careers Service, the Special Education Service, the Teacher Registration Board, the Colleges of Education through the Advisory Services, Children and Young Persons Service and the Ministry of Health.
The Panel believes that the Crown and other stakeholders need to be assured that consultation occurs among these agencies and with the Ministry of Education in the development and implementation of policy.
As the evaluator, the Education Review Office has no direct or primary role in designing or implementing the law, regulations and official education policies against which it must review schools, centres and other educational providers. Its input is confined to information, advice and consultation; this is appropriate, given the Office’s function.
The Education Review Office has direct lines of communication with the Government through its Minister and with the Ministry of Education. The Minister may direct the Office to initiate action or to produce an Evaluation Report on some specific topic. The Office’s reports to the Ministry may result in action being initiated as in the case of the Mangere-Otara project. The open lines of communication with other agencies should result in increased confidence among Boards, Principals, teachers, parents, the community and the education interest groups.
The Education Review Office has had a significant impact through its reports on schools and centres which are all sent to the Ministry of Education. Evaluation Reports, Annual Report to Parliament and comment from time to time on issues which are emerging as a result of its work are all conveyed. The Education Review Office draws the attention of the Minister to areas which may become priorities for work to be done.
Over the last seven years respect for the Education Review Office has grown. The Office is acknowledged as an independent evaluator with integrity and with a staff of committed people. Some important achievements include:
- Its significant impact in ensuring that strategies were developed in 1996 to address the pressing and long-standing issues in Mangere and Otara schools,
- It was instrumental in the development of the Performance Management Systems in schools following its report on Managing Staff Performance in Schools in 1995, and
- Arising out of the Office’s concerns about Boards of Trustee expertise, the Ministry has contracted trainers to work with Boards in specific areas of need.
Recently, the Education Review Office has drawn attention to emerging issues which include:
- The quality of professional leadership in New Zealand schools and the importance of training as a pre-requisite for principal and senior management positions,
- The exemption of teachers in Kura Kaupapa Maori from compulsory registration or carrying a Limited Authority to Teach,
- The potential for the Limited Authority to Teach to be used inappropriately in a time of teacher shortage, and
- The teacher supply problem and parent and community demand for high quality teaching.
Other questions to which attention is drawn are recorded in the Education Review Office’s Annual Report (1997) and include:
- What are regarded as core competencies of capable teachers?
- What do policy makers regard as a ‘balanced curriculum’?
- What are the critical stages of students’ learning and how might these best be assessed?
- What are regarded as tolerable levels of inconsistency in students’ school experiences?
In identifying these issues and questions, the Education Review Office is exercising the function of ‘diagnosis’ of problems in the system. It is outside the scope of the Review to do anything other than list these issues in the expectation that there will be appropriate reaction from the Minister and the Ministry of Education.
The Panel believes that the diagnosis function should be extended to identifying the causes of performance problems in schools and centres and on which the Education Review Office would be expected to report. This is quite separate from the Education Review Office having a role in providing advice and guidance or in enforcing compliance which are correctly included in the functions of the Ministry of Education.
Early childhood centres - short notice visits
The Early Childhood Sector has drawn to the Panel’s attention the problems which have emerged due to both the Education Review Office and the Ministry of Education being involved in short notice visits commonly called spot checks of Early Childhood Centres. There is universal support within the sector for spot-checking to continue. The Panel was given examples of centres who, when notified of a visit, will transfer equipment and staff to the centre for the duration of the visit in order to get a good report. The Education Review Office claims that they were not briefed prior to the implementation of the policy by the Ministry while the Ministry says it is liaising with the Office at the local level. The Ministry report that its spot checks were instituted because of a major risk of non-compliance with the Regulations or Charters due to the Education Review Office extending out its cycle of visits from three to four years. This ‘double checking’ has created confusion and annoyance within the sector because the practitioners are unaware of the motivation for this action. While the Ministry has responsibility for issuing licences and approving Charters, it is encroaching on the role of the Education Review Office in instituting spot checks without consultation and direct liaison with the Chief Review Officer. The Ministry concede that, because the Education Review Office has increased its activity in the Early Childhood Centres, (935-980 reviews in 1997/98 as opposed to 610-650 in 1996/97) the need for their involvement is no longer necessary.
Assessment in schools
The school sector has reported its confusion over the expectations of assessment that are part of the National Curriculum Statements. Submissions have been made that the Education Review Office expects each objective to be assessed and records kept of student achievement against all objectives. The Ministry of Education has stated (New Zealand Education Gazette March 1995) that ‘it is concerned that assessment does not dominate the teaching and learning process, rather it should be an integral part of teaching and learning.’ They say, ‘The schools should aim for assessment procedures which are manageable for teachers, non-intrusive for students and focused on promoting learning.’
The Ministry of Education advocates the ‘key’ as being for ‘schools to plan in advance what they intend to report on and how they might best collect and aggregate the information.’ Too many teachers are engaged in over-assessment and the planning advocated by the Ministry is frequently not in evidence.
Assessment is an integral part of teaching and learning and, therefore, must be included in every teacher’s learning plan. It is the Panel’s view that the process it has recommended, of a 3-year Strategic Plan and an annual Statement of Performance Indicators, will greatly assist teachers to plan, monitor and report effectively against stated targets. With these benchmarks in place and the Education Review Office evaluating against them, there will be greater certainty of expectation along with manageable assessment practices and reduced workload for teachers and principals.
- That the Education Review Office and other agencies are to be commended on their improved working relationship.
- That protocols be developed between the Ministry of Education and the Education Review Office and Te Puni Kokiri that will establish relationships, functions and involvement in policy development which will survive changes in personnel and operate effectively on a continuing basis.
- That the Education Review Office continue to have a direct input into policy development through the Ministry of Education.
- That the Education Review Office include diagnosis of the causes of performance problems in its review reports on schools and centres.
- That the Ministry of Education withdraw from spot-checking in the early childhood sector.
- That the Education Review Office continue its practice of spot-checking early childhood education centres and reporting any non-compliance to the Ministry of Education for action.
