Save Search

Note: Where available, the PDF/Word icon below is provided to view the complete and fully formatted document
Joint Committee of Public Accounts and Audit - 02/11/2011 - Auditor-General's reports Nos 16 to 46 (2010-11)

ARTHUR, Dr Evan, Group Manager, National Schools and Youth Partnerships Group, Department of Education, Employment and Workplace Relations

BLOOR, Ms Rhyan, Branch Manager, Department of Education, Employment and Workplace Relations

McPHEE, Mr Ian, Auditor-General, Australian National Audit Office

ROGALA, Mr Mark, Senior Director, Performance Audit Services Group, Australian National Audit Office

SIMPSON, Mr Mark, Acting Executive Director, Performance Audit Services Group, Australian National Audit Office

TURNBULL, Mr Stuart, Executive Director, Performance Audit Services Group, Australian National Audit Office

Committee met at 12:17

CHAIR ( Mr Oakeshott ): I declare open today's public hearing for the inquiry into Audit report No. 30 2010-11: Digital Education Revolution program—National Secondary Schools Computer Fund. I now welcome representatives from the Australian National Audit Office and the Department of Education, Employment and Workplace Relations. Given the short time available, statements and comments by witnesses should be relevant and succinct. Although the committee does not require you to give evidence under oath, I advise you that these hearings are formal proceedings of the parliament and warrant the same respect as proceedings of the respective houses. The giving of false or misleading evidence is a serious matter and may be regarded as a contempt of parliament. The evidence given today will be recorded by Hansard and attracts parliamentary privilege. Before we proceed to questions, do any of the witnesses wish to make a brief opening statement to the committee?

Mr McPhee : I have a brief opening statement. The $2.4 billion DER program is a key policy under the government's education reform agenda, which seeks to better prepare students for further education and training and to live and work in a digital world. The major component of the DER program is the $2.2 billion National Secondary School Computer Fund, which provides for new information and communications, equipment and associated costs for all secondary schools with students in years 9 to 12.

The ANAO examined the effectiveness of the department's administration of the DER program, focusing on the National Secondary School Computer Fund. The ANAO assessed whether DEEWR had established sound administrative and payment arrangements, properly managed administrative and payment arrangements, and effectively monitored delivery and outcomes for the program. The audit concluded that the department's administration of the DER program had been effective in supporting progress through a partnership approach towards the computer fund's objective of increasing the computer-to-student ratio for all Australian students in years 9 to 12 to one computer per student by 31 December 2011. Nevertheless, there were some aspects of the department's oversight of implementation that could have been strengthened, and the audit made three recommendations directed to the department: strengthening future program funding agreements with non-government education authorities; establishing a balanced set of portfolio budget statements, key deliverables and performance indicators to measure the effectiveness of the program; and increasing assurance over schools' achievement of the computer-to-student ratio of one to one through an audit of a sample of schools in early 2012. The department agreed with these recommendations.

At the time of the audit, education authorities had reported solid progress in the installation of computers purchased through the funding, although there remained a relatively short time frame for schools to transition to a computer-to-student ratio of one to one. Subsequent to the audit, in September 2011, the Minister for School Education, Early Childhood and Youth announced that the rollout of computers was on track, with 75 per cent of the total number of computers installed at the end of June 2011. Thank you, Chair. We would be pleased to respond to any issues the committee wishes to raise with us.

CHAIR: Dr Arthur, do you have anything to add?

Dr Arthur : I wish to make a very brief statement—just to say that the department very much appreciated the opportunity of working with the Audit Office on the administration of the program and very much valued the contribution which the office has made to continuous improvements of our administrative processes. In terms of the physical recommendations that have been made by the Audit Office, as indicated by the auditor the department agreed with those recommendations. Two of those recommendations have been implemented; the third applies to the 2012 calendar year and we have made a decision that we will implement that recommendation when that particular time comes around. I can provide further detail of those decisions to implement the recommendations and I can respond to questions if that is appropriate.

CHAIR: My first and obvious question goes to the progress on implementation of the recommendations. Any further detail on any or all three would be appreciated.

Dr Arthur : Certainly. The first recommendation was that DEEWR establish for future DER program funding agreements an obligation for non-government education authorities to provide an annual acquittal of program funds, including an independent audited statement that the funding was expended for the purpose of achieving deliverables and performance benchmarks in accordance with the agreements. In fact, since the audit report there have been no more funding agreements because the basis of the program has changed to that of a national partnership. However, under the DER funding agreements, which are part of the national partnership arrangements, non-government education authorities are, in fact, required to submit six-monthly progress reports, outlining their progress to reach a computer-to-student ratio of one to one by the end of 2011. All non-government education authorities have complied with these requirements, and the most recent progress report was received by us on 15 July 2011.

Recommendation No. 2 was that, in order to strengthen external reporting and help steer program direction, DEEWR establish a balanced set of portfolio budget statement key deliverables and performance indicators to measure the effectiveness of the Digital Education Revolution program. This was essentially taking some information that was held by the department as part of its program monitoring and to increase overall public scrutiny; to place some indicators from that within the principal reporting mechanism of departments, the portfolio budget statements; and to make those milestones and progress against those milestones explicit in the budget statements—which we have done. So the 2011 DEEWR portfolio budget statements contain the performance indicators recommended by the Audit Office and specifically contain performance information on the number of schools assisted and on the number of computers installed under the computer fund. Obviously, the full detail of that is available for scrutiny in the portfolio budget statements. I have an extract here and, if the committee would like me to table that, I would be very happy to do so.

CHAIR: That would be good.

Dr Arthur : The third recommendation went to assuring ourselves that with respect to the key deliverable of the program—namely, the installation of computers—we should have a process of getting out a heightened assurance of that. There was a recommendation that we conduct an audit of a sample of schools funded under the DER, in early 2012, to assist in providing assurance on the accuracy of the information reported by education authorities on computer installations, confirming whether the schools had achieved the one-to-one computer to student ratio and identifying any reasons for schools not achieving the one-to-one computer to student ratio, including any funding deficiencies. We are advancing that as part of our overall evaluation strategy for the fund, and I or Ms Bloor can provide further details of that. But a key element of that is that we will institute a mid-program review in 2012 to cover a number of aspects and, as part of that mid-program review, we will incorporate an audit in conformance with the recommendations of the Audit Office.

CHAIR: Thanks for that. I have a few more questions before handing over to others. The first is that, as a local member, anecdotally, there seemed to be just a flood of computers land last month. Do you have any recent data on the progress of the rollout, compared to today, and how close you are going to be to hitting the target on 31 December?

Dr Arthur : We do have data; however, I would need to take your question on notice in that, to date, the government made a decision that it would only publish information derived from the six monthly consolidated reports from all the education authorities, the most recent one of which was halfway through this year and which contained the figures which the Auditor has already provided to you. As a matter of program management, we do have more recent information. We have a computer system whereby schools are able to update the installation of computers as they are installed either at school level or in some cases at the system authority level, so we do indeed have more recent figures available to us. I would just need to consult with the government on whether they wish to vary the policy they have had to date in order to provide information to this committee. So I would need to take it on notice for that reason.

CHAIR: Anything you can provide, either on notice or now, would be appreciated. My third and final question is on something that the member for Lyons and others have been raising as to whether you want to comment on the record with regard to any reflection on the relationship between the noble goals of this program and the frustrations of trying to roll out such a program either via COAG or with state based education services that have a whole number of different requirements attached and their willingness or otherwise to participate in the hunt for those noble goals in this program.

Dr Arthur : I am happy to do so, Chair. In my experience—and I have a very detailed set of experience over a long time in Commonwealth-state rolling out of programs—there have been very few frustrations in this program and it has gone forth in a very cooperative way with all players. At the start of the program there were disagreements and those disagreements are fully public and are reflected in the audit reports and were the subject of specific COAG decisions. Those disagreements centred around responsibility for the payment of costs, in addition to the cost of the actual computer. As a result of those discussions this government made a decision to change the funding basis of the program and provide an additional $807 million. It reached specific agreements with states and territories via COAG on funding responsibilities for the program. Since then, there have been no substantive difficulties within the administration of the program. There have, as always, been questions of detail that have arisen through the program. However, those have all been handled in an entirely cooperative fashion with all our partners.

I would say that, during the phase of disagreements at the most senior decision making level on the funding basis, the operation of the program proceeded with a very cooperative approach with the states and territories. That can be demonstrated by the fact that, at the same time as the discussions about funding responsibility were being played out, we reached complete agreement on how to administer the program and on the role of the Commonwealth, the states and the territories and the Catholic independent education authorities. On the deadline for the program, including the commitment of dollars within the first 100 days of the program, the agreement on funding guidelines within that 100 days and the commitment of dollars by the end of the first financial year of the program were all achieved. Indeed, at no point has the program failed to meet any of its preexisting timetable commitments.

Mrs D'ATH: It is nice to hear how well the program is going and the positive outcomes. I am interested in the collection of data—and I do not know whether the department is collecting this sort of detail—identifying what the schools have invested in with regard to ICT, whether they are going for PCs, whether they are providing laptops and whether the laptops are being made available to students outside of school hours. That is just to get an understanding of the extent of the benefit from this technology being introduced into the schools.

Dr Arthur : I will make some general comments and we could take on notice what further information we are able to provide. The administration of the program is being done, as I said, in cooperation with the states and territories or with education authorities. Wherever definite decisions are made about technology deployment, in the case of NSW government schools, which authorise quite a centralised approach, the education department has made a number of technology choices. At the other end of the spectrum with independent schools, largely it will be the independent school itself. Some authority within the school will make those decisions, and you will have every possible variation between that. In Victoria we have government schools—self-managing government schools—and they will be making those choices. So a very wide range of choices have been made. Overall, I would say that the majority of deployments have been what are called Netbooks, which are, if you like, slimmed down versions of laptops. People have also been funding the program to buy desktops for particular environments and every configuration in between. Of late, there has been increasing investment in slates, whether that be in iPads or some of the alternative providers of that form of device. They are still, I would say, in the minority. Likewise a number of different choices have been made about how those things are made available. Again, the New South Wales government have a program where at the start of the year every year 9 student is issued with their own laptop. They have that access all the time, and they keep that laptop when they leave school. Other educational authorities have made quite different choices.

Also, the investment that has flowed through this program has made significant changes to the ICT environment within schools. In schools around Australia what you would have found prior to this program is a whole variety of, and often many inadequate, examples of networking inside the schools. The ability to have a computer in a school and to actually use it in the classroom was incredibly varied. What you largely find as a result of the program is that there has been an entire rehash of those activities, with complete rollouts of very high specification wireless networks and a whole range of support structures to make it work, including the effective management of the identity of students. Now the process of installing a computer in, say, a NSW government high school is such that essentially you need the logon ID of the student, and when you give them the computer it is installed—it works. All the stuff has been done on a standardised basis because of the very high level of one-off funding injection through the on-cost funds. So there has been significant progress. We would not like to claim that it is perfect but there has been significant progress to the absolute precondition for the effective use of technology in education, which is technology that just works. You do not have to worry. As a teacher you do not have to be an expert on computer configuration. There is a network there and there are devices there. They work to each other and you can get on with the business of trying to do interesting things with the technology above the neutral platform. I would not claim that it is perfect everywhere but we have made major strides from where we were before to achieving that precondition for the effective use of technology in education.

Mrs D'ATH: One thing I would like to give the department the opportunity to comment on, because it has been the subject of many media reports in recent times—certainly in Queensland—is that there has been commentary around parents having to pay fees because of the federal government's program to introduce computers.

Dr Arthur : Indeed.

Mrs D'ATH: Would you like to put on the record whether those fees are in any way associated with this program or any deficiencies in the funding being available to the schools?

Dr Arthur : Sure. We have been through this issue in some detail and there is a very clear position from the governments on this issue, which is that there should not be any fees associated with the provision of computers funded by the Commonwealth. The Commonwealth, as a result of the COAG agreements that I mentioned previously, is, to use a technical term, meeting the total cost of ownership of the device for four years. In the budget there are estimates for the Commonwealth contribution to the sustainment of that. However, there are a range of other activities which schools can be involved in in terms of using computers. There are three specific kinds of circumstances where we recognise that, essentially, the Commonwealth government has no role in restricting the choices that schools make and how they fund their range of activities. So, if the school wishes to buy devices which are more expensive than the notional price of the computers that we are paying for, that is their choice to do so. In some cases they may want to deal with very high-end multimedia work, and they may wish to choose particularly expensive devices. If they wish to do so, fine. If they wish to cover years other than years 9 to 12, that also is appropriate. I have just acquired a mental blank on the third condition.

Ms Bloor : The third area where they can charge fees is to support their own sustainment of the 30 per cent of computers that they had in place at the time that the fund was introduced. The fund was introduced in 2008. There was a preliminary survey at that time and it was estimated that there were about 30 per cent of the devices in place that would be needed to get to a one-to-one computer-to-student ratio. That meant that the program would be funding 70 per cent. Some schools had a practice of charging a parental co-contribution for the computers that were in place as part of that 30 per cent—particularly non-government schools come to mind there. If it is to fund their ongoing maintenance, we recognised that the Commonwealth could not prevent that, but we do stipulate that it has to be with the agreement of the parent body and with full discussion.

Dr Arthur : Essentially a point there is that if a school chooses to seek fees for those activities then it would be unreasonable for the Commonwealth to say you have to go through administrative complexities of exempting those students and those parents who have a Commonwealth laptop in years 9 to 12 and only charge the parents whose students are in earlier years. That would be an unreasonable piece of Commonwealth red tape going into the management of schools if the school chose, after appropriate consultation with the parent body, to charge fees across the parent body from, say, years 7 through to 12, saying that that was a perfectly reasonable thing for the Commonwealth to do. The Commonwealth should not seek to interfere in the internal economy of schools to that extent, but there should be no components of those fees which are a cost associated with the provision of the Commonwealth computers.

Mrs D'ATH: I have one more question on that point and on your first reasoning of why fees may be charged, which is where a school has elected to purchase or install a higher level of equipment that is more expensive than the funding that is being provided. Is there any case to say that the funding that has been allocated does not provide for technology that is at a suitable level for what is needed to be provided in the classroom and for the various activities that that technology might be used for—for example, if there is a complaint that, 'We're only getting this much and so we're obliged to try to top it up to get a decent computer'?

Dr Arthur : In broad terms I would say no. There are specific cases which I will go to in a second but, in broad terms, given the nature of the way the technology works in this place now, it is quite clear that you can get a very highly capable device and maintain that device within the funding envelope for the program. I might note, in general terms, that these devices are universally sourced from overseas, and there has been a moderately significant appreciation of the Australian dollar over this period, so we are entirely confident that the overall budget for the program is adequate to provide computing technology suitable for use.

However, there can be specific instances. If you are running a music class or a graphic design class, you may well require equipment of very high specifications and very specialised software, which takes you outside the envelope of the program. If a school chooses to purchase devices for that purpose, notwithstanding the fact that there are entirely capable devices within the particular price range of the function, and goes out and buys equipment which is more expensive, because they have their own reasons for doing so, the Commonwealth does not seek to interfere. It is just that, quite appropriately in terms of the stewardship of taxpayers funds, there is a ceiling on what the Commonwealth provides per device.

Mrs D'ATH: I want to confirm the second point you made in relation to graphic design. In that case we are talking about programs that the school may want to install, which is separate from the physical infrastructure equipment that is being provided.

Dr Arthur : Yes, but there is a connection in that, if you have a particularly memory-hungry piece of software, that will have an effect on the capacity you need and the underlying device.

Mr FRYDENBERG: On the $807 million funding gap at the start, what advice did you give to the government in terms of how that would be funded? I know you tried to go to COAG. Did you advise on the likely success of COAG funding? Why wasn't it actually provided at the start?

Dr Arthur : All of that advice is on the public record.

Mr FRYDENBERG: Just remind me.

Dr Arthur : The Commonwealth made a decision to request Dr Paul Grimes, at that time the deputy secretary of the department of finance, to prepare reports on those issues to go to COAG, which would look at legitimate additional costs associated with the installation and maintenance of computers over four years. Dr Grimes carried out the task, and I was part of the group that worked on that. We looked at a whole range of issues. In the end the government published the Grimes report and COAG took a decision collectively to agree with the analysis of Dr Grimes and to agree with the funding figure that arose from the culmination of his identification of the per-device cost that was appropriate to be allocated to meet the legitimate additional costs.

Mr FRYDENBERG: I am sorry; you are talking about the costs. I am asking: what was the recommendation to the government as to how it was to be funded?

Dr Arthur : The recommendation was made in the context of the Grimes report. The recommendation was that the funding should be made available to meet these costs.

Mr FRYDENBERG: Who was to fund it? I am not familiar with the Grimes report. Are you saying that the Grimes report said that COAG was to fund it? What was the department's recommendation? When the government said, 'We are doing computers in schools,' and the department said, 'Okay; it's going to cost X number of dollars,' how did you go about suggesting you were going to fund the $807 million?

Dr Arthur : I will need to go back a little. Immediately after the election of the Rudd government, there was a COAG meeting which agreed to the rollout of the computers, but there was a reservation, as I recall, in terms of funding of the legitimate additional costs. At a subsequent COAG meeting—and I do not have the date in my head but we can provide it for you—the Commonwealth agreed that it would meet legitimate additional costs for installation of the computers. That was a decision which the Commonwealth made and communicated to COAG and was set out in a COAG communique. A process was then agreed by which we would quantify what those legitimate additional costs were. That was the process which was managed by Dr Grimes. The results of that process went back to, I think, the third COAG meeting of the Rudd government—I can check that on notice. On receipt of the Grimes report, the government decided to accept the quantification of the costs contained within the Grimes report.

Mr FRYDENBERG: What I am trying to ascertain is, when the government took to COAG this proposal, what was the government's advice at that point as to how it was to be funded?

CHAIR: The Audit Office might want to chime in at any point on this or on any of the evidence that has come in.

Dr Arthur : The events are indeed reflected in the audit's reports and were canvassed in some detail in the conduct of the audits. The second COAG meeting was when the Commonwealth government agreed that it would meet those costs, quantify them and then agree the specific amount of dollars in time for the third COAG meeting.

Mr FRYDENBERG: Does 75 per cent delivered mean operational?

Dr Arthur : Yes.

Mr FRYDENBERG: The target was the end of 2011. Obviously that has been extended. What is the predicted date they will be delivered and operational?

Dr Arthur : The agreement is that they need to be available and operational for the start of the school year. This has being canvassed previously, including at an estimates hearing. Operationally, the thing to do was to make sure they will be available for the start of the school year. It is not exactly the same date in every state and territory. For example, in New South Wales the year 9s who start in 2012 will be the ones who are issued with the new computers and then everyone will have a computer all the way through. The current year 12s in New South Wales are moving through. They were postdated. They will not be issued with computers this calendar year. The year 9 class, when they come in, will have their computers at the start of the academic year next year. That is when they all have to be in place and functioning.

Mr FRYDENBERG: It was talked about how opening the first applications in the first hundred days of the government created some problems because that coincided with the summer holidays. What was your advice to government about the timing of the opening of those first applications?

Dr Arthur : We did not provide advice to government on that issue, as I recall. It was a commitment of governments that that would occur.

Mr FRYDENBERG: But you did not tell them there was going to be a problem because school was on holidays and there needs to be consultation? Was 100 days an arbitrary deadline?

Dr Arthur : That was a decision of the government at the time. I cannot really comment beyond that.

Mr FRYDENBERG: So there was no advice. Is that what you said? Is that correct?

Dr Arthur : The sequence was, if I recall, the then Prime Minister made that announcement at a very early stage of the government.

Mr FRYDENBERG: But there was no advice from the department?

Dr Arthur : I cannot comment beyond that. I have to say that my own involvement in the program postdated that by some weeks, so I am not in a position to make any particular comment about occurrences that occurred at that time.

Mr FRYDENBERG: You have mentioned in your opening comments there will be a review in 2012. Let us say the government comes out and announces all computers are being delivered in March 2012. Given that there have been some historical discrepancies in the data, how do we know that that will be an accurate statement given that your review will be subsequent to that, or is it going to be prior to that?

Dr Arthur : It will be subsequent to that date. We need to be clear about the question of data. There are certainly matters canvassed in the audit report about whether there are mechanisms which could be employed to give a better fix of the data. We do not have information that there are discrepancies in the data. We have collected this data in the same way as we collect data for all activities of this type for schools—that is, we ask schools to provide us with the information. If you think for a while about the actual mechanics of that, that is to provide us with information on this particular day that there were X number of computers of particular types alive and functioning. There is no way that you can do anything other than collect that information and use that information. If you go back three weeks later with a very large audit team to check those figures, you will not get a better handle on the figure—

Mr FRYDENBERG: But didn't your internal auditor say, of the 2,929 schools, 16 per cent of those did not match?

Dr Arthur : That is indeed the case—

Mr FRYDENBERG: So that is accurate.

Dr Arthur : It is set out in our response and our commentary in the audit documents, which is that there were four figures—as I recall, and I am not going to get this totally right. We decided to rely on two sets of figures in order to make decisions about allocation of the funds. We did not bother to check some calculations to check the other figures, because we did not use those figures in our decision making. We also went through a number of processes, a number of iterations with the state and territory authorities, to check that the figures which we were using to make decisions were accurate. We went through a number of iterations saying, 'Please check again that those figures are correct,' and no discrepancies arose.

CHAIR: We will have an opportunity for questions on notice if there are any questions you want to put in.

Mr FRYDENBERG: That is fine.

Mr ADAMS: In the matter of what it was worth for every student in the country to have a computer as opposed to every school having a flagpole, we can check that in the future. But I have a couple of serious questions. One is: Was the number of computers agreed to be supplied based on the number of students in a school the year before? Or was it the number of students that were registered in the school?

Dr Arthur : We used the figure that the Commonwealth uses for most of its programs which involve calculations of individual student numbers. Every year there was a census taken of students in a particular month. There was an agreed figure of how many students are in the Australian school system and our recurrent funding is based on that. That is the figure we used as the denominator in the equation.

Mr ADAMS: The figure was set on a monthly basis in each school, so if a school was arguing with its state department about how many numbers and they would say, 'The Commonwealth will only give us X,' it is right to say that that monthly figure was the figure?

Dr Arthur : It is not a monthly figure in that sense. It is an annual figure. It is a particular month at which the census is conducted and there is a high degree of process around that particular figure, but that is the figure. It is perfectly true that it is possible for a school to consider that that figure was not entirely accurate. The only thing I can say about that is that overall this is a $2 billion-plus program. There has been very clear evidence of states and territories being able to make very significant savings in the price paid for equipment. There have also been the aforementioned increased purchasing capacity for equipment caused by exchange rate movements. There are more than enough dollars provided in this program for any issues around the margins of those figures to be addressed.

Mr ADAMS: So that should not be a problem.

Dr Arthur : It should not.

Mr ADAMS: You were talking about the IT backbone put into schools, whether wireless or cable. I take it that the connection to that backbone from the outside source—Telstra, Optus, whatever—is met by the state government?

Dr Arthur : It is met by the school authority, whatever it happens to be—state government, Catholic Education Office or independent school. That is true.

Mr ADAMS: So that is the agreement.

Dr Arthur : That is the agreement.

Ms O'NEILL: Thank you very much for your evidence so far. I do not know that you get to see the faces of the students who receive them or to knock on doors and have a mother come to you and say, 'My son has changed his whole attitude to education because this is the first new thing he has ever had in his life.' That is the sort of testimony to this program that I have experienced in my electorate and particularly in the suburb of Kariong where many families have been very advantaged by this. Their kids have got the advantages they need to progress into the future. There is something about how we measure this that I want to ask. I know there are numbers and I know there are dollars, but in my world the dollars serve human needs; they do not exist as of value in themselves. Of course the management of the money is important but how are we measuring the outcomes? I know the KPIs for outcomes are something that are a little more difficult to come to some agreement about, but ultimately the KPIs need to deliver the outcomes which are there to meet this statement of objective that we are enabling students to prepare for further education, for training and to live and work in the digital world. I would like to know a little more about how that is going.

The other thing that I note is that the annual acquittal of use of funds for non-government sector schools is a little different from that of the government sector schools. It would seem to me that it would be fair to have annual reporting from everybody who has received them, and so I would like a bit of feedback on that.

The other one is intermediate progress milestones for how things are going so that there can be a timely response if there are problems. What are your plans for that, and what is your forward program in continuing your work to ensure that there is adequate access and excellence in delivery?

Dr Arthur : To begin with, we agree with you very much that it is the educational outcomes which are truly important in this space. The technology is only a means to that end. Indeed, it has been agreed that the national partnership now governs the operation of this. If you look at that document you will find that what is stressed about the outcomes of the program are the educational outcomes, not the specific provision of computers. That is covered in monitoring arrangements, but it is made perfectly clear that is not the purpose.

How will we measure that? As I mentioned briefly at the beginning we have an overall evaluation strategy. It is largely in the area of evaluation, in some cases qualitative evaluation, that you can get to those issues. Ms Bloor may like to speak about that in a second.

In terms of your other questions: the acquittal process between the government and non-government sectors partly reflects the architecture of national partnerships, where we tend to work through the intergovernmental agreement processes for acquittal of funds with the state and territory governments. There are certain architecture issues with that which means there is a difference between the responsibility for provision of precise acquittal documents. Having said that, this program is actually a far more concrete program than many, and the mechanism of having very regular reporting on pure installations provides us with a very definite indication of expenditure of funds against the program.

In terms of the third issue, I think that that is just part of our overall program-monitoring activity. We need to understand that, and it is built into our program-monitoring arrangements. Ms Bloor might like to talk more about evaluation.

Ms Bloor : Yes. The DER National Partnership builds on the strategic plan which was developed in consultation with education authorities at the commencement of the program. It stresses the need to progress improvements against not only infrastructure, which is where the devices come in, but against leadership, teacher capability and also digital resources.

It also recognises that in the context of this program the Commonwealth and education authorities have a shared role to play. I think that the ANAO in their report mentioned that an evaluation that picks up those sorts of different roles is necessarily sometimes complex.

We have asked education authorities in their six-monthly reports not only to report to us on the installation of computers but also to report to us on those other themes in the national partnership, and also to provide us with case studies that can be used and built on in the evaluation of good practice in the classroom. In developing the evaluation strategy we spoke to the Commonwealth, state and non-government committee, the Australian Information and Communications Technology in Education Committee, which was responsible for the oversight of the program from the outset about an evaluation strategy. It has been agreed that it should be thematic; that it should pick up mechanisms that are qualitative as well as quantitative; that there would be a mid-program review in 2012 which will provide us with the capacity to pick up on the audit recommended by the ANAO; and that it needs to be something that occurs over time to look at these things that go to the education outcome.

CHAIR: Just to clarify that, the status of that evaluation is ongoing?

Ms Bloor : It is. We are in the process at the moment of identifying a service provider to undertake the mid-program review and the audit in 2012.

CHAIR: Thanks.

Mr Simpson : Chair, if I may: if you look at the report in chapter 5, at the time the audit was completed we indicated that it has taken some time to get the evaluation framework in place and that work was continuing in this direction, as Ms Bloor has just indicated. But we also mention in chapter 5 that education authorities are also undertaking longitudinal studies, and there is an example there from New South Wales in partnership with the University of Wollongong, which is looking at issues and effects from the program in relation to pedagogy, student engagement and outcomes. So there is work going across which has been outlined in the audit report itself.

CHAIR: Thanks.

Ms O'NEILL: I would like some clarity on the KPIs; you started to address those there. In terms of outcomes, the distinction between outputs and things being achieved and outcomes and students' demonstration of those is kind of the area that I am really most interested in. The program can be delivered and we can have a few studies that show us that some people are doing good things. I would like to know if there are any KPIs being developed to indicate that students are actually significantly improving their performance across a range of subject areas because of their engagement with new technologies.

Dr Arthur : That is a very fraught area. There have been any number of studies that have attempted to tease out that relationship. The key problem is that the number of inputs into good or bad educational outcomes, as I am sure you are very well aware, is very, very large, and we would need to make an attempt to isolate out the influence of technology and then to sub-isolate out people who are using the technology effectively in the classroom and not using it effectively in the classroom—because we recognise that even a program such as this cannot guarantee that individual teachers will use the technology effectively in the classroom; what we can do is create preconditions, but we cannot guarantee outcomes. So, at the moment, I am not aware of studies that are capable of doing that. I know that some people at the former BECTA in England produced a number of studies at a reasonably high level, trying to simply do correlations between introduction of technology and results, and they produced some positive results, but I would not want to oversell them. It is a very, very fraught methodological area.

CHAIR: I know the Courier-Mail is watching, and it had a very good article last month on Doomadgee State School; their NAPLAN results have gone through the roof with some of the work that the principal there is doing in regard to ICT generally. So I think there are some tools out there that could do the longitudinal work by default other means.

Dr Arthur : Indeed. The problem in all of those studies is to isolate out the influence of inspirational principals. We know from all the studies that principals, leadership and quality teaching are the dominant variables in those results. What I would say at a personal level is that, if you combine those strengths with the kind of potential that technology offers, I personally am sure that you can get exceptionally good results from that. I am just being cautious in the sense that I would not want to claim that we can demonstrate that to a level of proof which would satisfy academic peer review rigour. But we understand that this is a very important issue, and trying to get a better handle on the data and allow that data to be sufficiently sophisticated so we can do some of those correlations is certainly one of the things that we need to keep working on overall.

CHAIR: We are over time. We can do questions on notice.

Ms O'NEILL: The question was really at the end point. We have further educators. We have skills service providers for students in postsecondary contexts. How much are they engaged in helping identify what they would see as performance indicators that would show that there is some development in technology capacity? I would like to see if they are involved, what their feedback is and whether they are consulted.

Dr Arthur : We could take that on notice, certainly.

Ms SMYTH: Thanks very much for your evidence today. The question that I really had was around the residual funding incentive that was contemplated in the report. I just wondered if you had any particular comments on that, particularly around whether you would use that mechanism in future or refine that mechanism for future similar programs.

Dr Arthur : I do not quite understand your question; my apologies.

Ms SMYTH: The funding incentive whereby schools could use any residual funds for complementary equipment.

Dr Arthur : Yes, that certainly is the policy. Overall, we are definitely not attempting to micromanage the expenditure of the funds at the school level, so schools certainly do have the flexibility to apply the funds for a range of complementary purposes and in many cases are doing so.

CHAIR: I have two questions to put on notice. One is around professional development. Please come back to the committee with any policy work that is happening in parallel with what is effectively an infrastructure rollout on teacher development and professional development—anecdotally noticeable as something that is desperately needed—and whether that parallel work is happening. You made some comments before about the school now being well-connected via wireless and any other means as a consequence of this program. In regard to linking policy, has any policy work been done or is any proposed to be done on the nine-year rollout of the National Broadband Network, and, regarding the school hub, is any policy work on that becoming a community hub for internet technology and ICT improvements generally? Any work in that regard would be appreciated.

I am conscious you have taken a fair few questions on notice. I think one of them requires executive approval for potentially releasing information. Josh Frydenberg also has three questions he would like on notice. Without pushing you too hard, we are on a pretty skinny timeframe to report back, so any information you can get back to us in the next 1½ weeks to the secretariat would be appreciated, particularly regarding the one that requires ministerial approval because I know that slows the process down at times. Any effort you can use in that regard would be appreciated. Thank you for attending and thank you for assisting the committee with its inquiry. We apologise for the short timeframe for those questions on notice, but it will help us with our report deliberations by the end of the year.

On behalf of the committee I would like to thank all the witnesses who have given evidence at the public hearing today. I thank the ANAO for their work, once again, and also thank Hansard and Broadcasting for their assistance.

Resolved (on motion by Ms Smyth):

That this committee authorises publication, including publication on the parliamentary database, of the transcript of the evidence given before it at public hearing this day.

Committee adjourned at 13:07