Note: Where available, the PDF/Word icon below is provided to view the complete and fully formatted document
Education and Employment Legislation Committee
01/03/2017
Estimates
EDUCATION AND TRAINING PORTFOLIO
Australian Research Council

Australian Research Council

CHAIR: Welcome.

Ms Harvey : We do not have an opening statement.

CHAIR: Senator Carr.

Senator KIM CARR: So, no opening statement?

Ms Harvey : No opening statement.

Senator KIM CARR: Ms Harvey, you are acting?

Ms Harvey : Yes.

Senator KIM CARR: You have been acting in the job for a while. How long has it been?

Ms Harvey : Our previous CEO left in about June or July last year. Sorry, he did take some leave. Technically he left us at the end of August and so I have been acting since then.

Senator KIM CARR: Minister, I noted in your answer to question 16000999 you said the position would be advertised for a month from September to October. I take it that has all been concluded. How many applications were there?

Senator Birmingham: Dr Bruniges may be able to indicate application numbers or she may need to take that on notice.

Senator KIM CARR: Presumably you will be able to take on notice how many international applications there were?

Dr Bruniges : Yes, I can do that too.

Senator KIM CARR: Can you tell me how many interviews were conducted?

Dr Bruniges : I think we conducted three interviews.

Senator KIM CARR: I presume there is a shortlist.

Dr Bruniges : There would have been. A process was carried out with the panel of shortlisting and then interviews.

Senator KIM CARR: Therefore, if there were three interviews I presume the shortlist was three?

Dr Bruniges : That would be right. I will just clarify that in terms of the number of interviews. I need to cast my memory back to last year.

Senator KIM CARR: When were these interviews conducted?

Dr Bruniges : I can tell you that. It was in December.

Senator KIM CARR: How long is the appointment for? What is the period of the contract?

Dr Bruniges : I think most of them are up to five years unless they were advertised quite differently. It is normally a five-year period.

Senator KIM CARR: That is my memory, yes. Is it correct that the report from the selection panel was due with the government in early January?

Dr Bruniges : We would have done that report, yes. That would have preceded the interviews. There would have been some follow-up and referee checks as a normal part of the process and then a report to government.

Senator KIM CARR: Was the report received in early January?

Dr Bruniges : I would have to check on the exact date.

Senator KIM CARR: I take it that was forwarded to government in early January.

Dr Bruniges : We would have set it up. It was late January.

Senator KIM CARR: There was a question on notice that referenced early January. That is the basis of my question. So, you are saying late January?

Dr Bruniges : Yes.

Senator KIM CARR: That is neither here nor there, but it has been nine months since Professor Byrne announced his intention. When do we expect to hear of an announcement, Minister?

Senator Birmingham: You are correct. The report of the selection panel was provided to me. I have now had a couple of conversations with the recommended applicant from the selection panel. I can advise that the necessary appointment processes around that recommended applicant have all proceeded in accordance with government requirements, that final contract negotiation and settlement arrangements are being undertaken and I would anticipate an announcement being imminent.

Senator KIM CARR: There was a figure of $78,980 spent on the search for the new CEO. Is that the current figure? Has there been any revision to that number?

Ms Harvey : I would be happy to answer that, because the ARC pays for the search firm. Yes, that is approximately correct.

Senator KIM CARR: So, it remains—

Ms Harvey : Yes.

Senator KIM CARR: How much is the CEO to be paid?

Senator Birmingham: On that matter, unless somebody has the figure at the table, we may have to take that on notice.

Ms Harvey : Under the Remuneration Tribunal there is a specific band that it is paid within. There is a band involved in that.

Senator KIM CARR: What is the band?

Ms Harvey : I do not have that figure in front of me. I would have to go off memory.

Senator KIM CARR: Surely it is not too difficult for an officer to tell you that.

Ms Harvey : Yes. We can get that for you.

Senator KIM CARR: Does someone else have that figure here?

Ms Harvey : I do not have it behind me. We could take it on notice.

Senator KIM CARR: I am actually interested in getting it now. I do not think it is a difficult piece of information to extract from the system. So, if there is someone who could find that. My recollection here was that the old rate was $580,302. Certainly that was the amount published in the annual report in 2013-14. Where would I find the equivalent figure in today's annual report?

Ms Harvey : I would query that amount, because of the band under the Remuneration Tribunal. From memory, the current upper band of that is about $470,000, but that is what we will clarify.

Senator KIM CARR: So, my information is wrong?

Ms Harvey : I am not saying it is wrong. It depends on what is included in that. What year was it in?

Senator KIM CARR: 2013-14. That is the annual report figure.

Ms Harvey : Yes, but there might have been a transition period with regard to the changeover of CEOs back then, if I think about Professor Sheil and Professor Byrne.

Senator KIM CARR: So, you will be able to confirm for me what the figure was for that time if you are challenging the number I have given?

Ms Harvey : Yes, and what it included.

Senator KIM CARR: If it is higher than that, what else is included in the stated figure in the annual report?

Ms Harvey : Yes. I am happy to do that.

Senator KIM CARR: The real issue I am looking for is: where do I find that information in the current annual report?

Ms Harvey : Ms Emery is looking up the financial statements part. There have been some changes in the way that salaries are reported by the Department of Finance. From memory—and I will just confirm—there has been a collapsing in how that is reported, and so there is less detail than there used to be.

Senator KIM CARR: You are right on to me there. I am familiar with this matter. I am seeking to know why there had been a change in the reporting requirements and you have said that it is the Department of Finance.

Ms Harvey : Yes. From my understanding, you would have to ask that portfolio.

Senator KIM CARR: Yes, I know and I have. I understand there was a change to the Public Governance, Performance and Accountability Act that led that to happen. This government's change in policy led to that happening. I am going to go to the questions that arise from that. You would no doubt be familiar with the recent media attention to the government's announcements about correspondence between Australia Post and the Senate Environment and Communications Legislation Committee. Are you familiar with that matter?

Senator Birmingham: From the media coverage.

Ms Harvey : From the media, yes, absolutely.

Senator KIM CARR: Yes, that is right; I refer to the media report.

Ms Harvey : I was trying to think about how to phrase that.

Senator KIM CARR: You would be familiar with that.

Senator Birmingham: And Senator Paterson's questioning and work.

Senator KIM CARR: Yes. It is very good work, too.

Senator PATERSON: Thank you.

Senator KIM CARR: I am not arguing the toss about the quality of the work. I would like to know: has the ARC has received any correspondence from the finance minister in relation to executive remuneration and the presentation of that information in your annual report?

Ms Harvey : Only with regard to how that would fit within the financial statements as we report them. We would follow the instructions of the Department of Finance.

Senator KIM CARR: Yes, I know that. I am not disputing that you would follow the instructions.

Ms Harvey : Absolutely.

Senator KIM CARR: My question is: have you received any correspondence from the Minister for Finance to change those instructions?

Ms Harvey : Not to my knowledge, from the ones that we have got here now, because that would have been under the memorandum that these were prepared under at that point.

Senator KIM CARR: Can you take that on notice, please?

Ms Harvey : Absolutely.

Senator KIM CARR: You are relying on memory. Maybe there is something that has come in that you have not seen. The current report for 2015-16 refers to seven executives.

Ms Harvey : Yes, that is right.

Senator KIM CARR: What are the positions that attract executive status of the seven. What are the seven people, and then I will obviously want to know who they are?

Ms Harvey : There are the senior executive service officers. We have, for example, the branch manager and we have the executive general manager. We also have our executive directors, which are drawn from the academic sector which form part of the SES cohort, of which this is the senior executive remuneration that sits within this.

Senator KIM CARR: And those persons are all on your organisational chart?

Ms Harvey : Yes, on our organisational chart. Would you like me to—

Senator KIM CARR: Yes. If you could provide that, it would be helpful so I know that I have got the right ones.

Ms Harvey : Yes. I am happy to do that.

Senator KIM CARR: You do not pay executive bonuses, do you?

Ms Harvey : No, we do not.

Senator KIM CARR: The government has made it very clear that it expects the pay of our senior public servants to match community expectations. Does that apply to the ARC?

Ms Harvey : If that is the government's position, it definitely applies to the ARC.

Senator KIM CARR: It applies to the ARC?

Ms Harvey : Yes.

Senator KIM CARR: You have every expectation that you will be required, but it is a question of whether or not you have received any directive to that effect?

Ms Harvey : No. To my knowledge there are no directives to that effect.

Senator KIM CARR: The government has also made it clear that it expects its business enterprises to disclose that information. Do you expect that you will have to provide that information?

Ms Harvey : I will wait to see the advice that we get from the Department of Finance with regard to how we present the information in our annual reports.

Senator KIM CARR: Can you provide this committee with that information now?

Ms Harvey : With regard to the individual remuneration?

Senator KIM CARR: For the seven executives that are listed that you have already indicated to me, can you provide the details of their remuneration?

Ms Harvey : One of the challenges with being a small agency is with regard to making sure about the Privacy Act. We would be happy to have a look at it. I am happy to tell you about the positions.

Senator KIM CARR: You used to provide it. There was no difficulty in providing it in 2014-15. What is the difficulty in providing it now?

Ms Harvey : Just the way that we are required to report that information, but I would be happy to take that on notice.

Senator KIM CARR: The only restriction is the finance department directive on this. I am asking whether there is any restriction on you providing that information, not the method of your reporting, to this committee as you have done in the past?

Ms Harvey : I will take that on notice. I now have the band from the Remuneration Tribunal's website.

Senator KIM CARR: Yes.

Ms Harvey : It is band C and it is with regard to up to $470,840.

Senator KIM CARR: And there are no other allowances associated with that?

Ms Harvey : No, it is a total remuneration amount.

Senator KIM CARR: Four-seven-eight?

Ms Harvey : It is $470,840.

Senator KIM CARR: The figure I have quoted of 5,832 must be wrong; is that what you are saying?

Ms Harvey : No. I am saying that there may have been people acting in that position. There may have been a changeover of people being CEOs at that point. I would have to check.

Senator KIM CARR: You are acting in the position. You are not telling me that you earn more than that?

Ms Harvey : No, I do not.

Senator Birmingham: I think what Ms Harvey alluded to is she thought that there might have been a year where there was a changeover where, I assume, if entitlements were paid out for the departing CEO you could have an overlap in the sense of CEO salaries.

Senator KIM CARR: I see. It refers to others.

Ms Harvey : Yes.

Senator KIM CARR: All right. Can I turn then to the question of the changes to the impact measurement that you are using. You have undertaken a series of consultations in regard to the ERA arrangements, have you not, on impact?

Ms Harvey : On engagement and impact?

Senator KIM CARR: Yes.

Ms Harvey : Yes, we have.

Senator KIM CARR: Have they been concluded?

Ms Harvey : Yes. We have concluded the round of consultation with regard to that. We have prepared for the pilot and we will be opening for the pilot in the next month or so.

Senator KIM CARR: What has been the level of direct industry participation in those consultation exercises?

Ms Harvey : I will just get some details for you on that.

Senator KIM CARR: By that what I am saying is apart from the business representatives on the steering committee itself.

Ms Harvey : We conducted a range of different and varying consultation processes. We did what we call an end user survey with regard to people who have been partners on linkage grants to look at what their views were and how they thought that we could measure impact. There has been a range of direct meetings by the then CEO, which is Professor Byrne, and meeting with a range of different industry representative bodies as well.

Senator KIM CARR: Can you specify what those contacts have been? You have indicated a range of them.

Ms Harvey : Yes.

Senator KIM CARR: What exactly are you talking about?

Ms Harvey : Professor Byrne has met with the groups, including the Australian Industry Group, the Australian Chamber of Commerce and Industry, Business Council of Australia, Council of Small Business Australia, Australian Human Resource Institute, the Competitive and Consumer Commission and a range of others. They are the high-level ones.

Senator KIM CARR: Yes, predominantly peak organisations?

Ms Harvey : Yes.

Senator KIM CARR: Not too many individual businesses?

Ms Harvey : We did the direct approach to those with regard to the consultation and sent them information that we were keen to hear their views about. We did direct consultation with that through our end user survey.

Senator KIM CARR: At what point were the peak organisations consulted in the consultation process?

Ms Harvey : They were consulted while the steering committee was still meeting, or in the lead-up to looking at how we could implement this and what we should take into account.

Senator KIM CARR: I just want to get this clear. Are you saying that this is independent of the steering committee or separate from the steering committee?

Ms Harvey : No. We provided any advice that we receive through the consultation back to the steering committee as well. This is independent of the steering committee. But, of course, Professor Byrne was the chair of that steering committee.

Senator KIM CARR: I am just trying to get my head around the proposition. Professor Byrne met with these industry groups. Did he write reports or what was the form of transmission of the consultation findings?

Ms Harvey : He made notes and then discussed them at the steering committee.

Senator KIM CARR: I see, and they were all done through Professor Byrne?

Ms Harvey : Yes, and I attended with him on some and sometimes other people attended as well depending on where they were.

Senator KIM CARR: Can we have the dates on which these meetings occurred?

Ms Harvey : Yes. With the Australian Industry Group, I do not have an actual date, but it was May 2016. Council of Small Business Australia was 16 May. Australian Human Resources Institute was 17 May. The Australian Competition and Consumer Commission was also 17 May. There was a range of meetings in August, including the Small Business Ombudsman and different ombudsmans there, as well as the Business Council of Australia, which was in July.

Senator KIM CARR: Were the consultations done at the level of a draft set of indicators?

Ms Harvey : No. It was asking their views on what would be good and what would work. Of course, Professor Byrne sometimes went into some ideas that others had floated and discussed beforehand, but there was no discrete set of indicators that were given that people gave feedback on. It was about gathering information.

Senator KIM CARR: Is it correct that the peak organisations did not see the indicators?

Ms Harvey : Some of them would have been discussed with them. The ones that I was with, with Professor Byrne, are the only ones that I can discuss. He did actually talk about what were some of the indicators that were being considered and whether they thought they were useful or not.

Senator KIM CARR: At what point were the draft set of indicators shown to the peak organisations?

Ms Harvey : I do not think they have been shown to them.

Senator KIM CARR: That is the reason I am asking the question. The consultations consisted of discussion?

Ms Harvey : Yes.

Senator KIM CARR: And a report back by Professor Byrne. What were the responses from business? Are there any written responses from business anywhere?

Ms Harvey : We have the information that we collected from the end user survey.

Senator KIM CARR: So, from the survey; is that it?

Ms Harvey : Yes, and Professor Byrne's notes at the time and the information for the steering committee.

Senator KIM CARR: Do you think it is adequate that business not see the draft indicators?

Ms Harvey : They were still being developed at that point.

Senator KIM CARR: Had they been concluded?

Ms Harvey : The indicators were finalised just before Christmas and we have been dealing with looking at having the pilot this year. The actual full exercise is not until 2018. We have a pilot approach that we are doing in 2017 to test the validity, whether it works or does not work.

Senator KIM CARR: I might say to you that it seems to me that they really have not had a chance to look at this yet. That is the nature of the evidence you presented. What about the universities? When do they see the draft indicators?

Ms Harvey : The indicators went out to those that indicated they will be participating. They received the documentation in December 2016. As I said, we were finalising them.

Senator KIM CARR: What was the date in December?

Ms Harvey : The 16th.

Senator KIM CARR: 2016.

Ms Harvey : 16 December 2016.

Senator KIM CARR: 16 December 2016. My experience with universities is that 16 December is not a great time to be talking to people. How many responses did you get from universities?

Ms Harvey : The vast majority of universities are participating in the pilot.

Senator KIM CARR: No—responses to the draft indicators.

Ms Harvey : We did not actually ask them to provide us with a response, but I can tell you they have all received them. They got them then. They were still working, if that is what you are saying.

Senator KIM CARR: As I say, not many people are about at that time. So, you did not even ask them for a response?

Ms Harvey : No, because we just released the documentation. They do not submit until May.

Senator KIM CARR: How many businesses responded to the survey?

Ms Harvey : I would have to take that on notice.

Senator KIM CARR: Is it many?

Ms Harvey : Yes. We will get that for you.

Senator KIM CARR: Thank you.

Ms Harvey : Just under 200 responded to the survey.

Senator KIM CARR: Two hundred?

Ms Harvey : Yes.

Senator KIM CARR: Would you be able to provide us with the details of all correspondence on this matter between the universities, industry representative groups and individual businesses around the indicators or the metrics for the engagement or impact?

Ms Harvey : Yes.

Senator KIM CARR: I understand that the pilot arrangements are in place, as you say, but when do you expect the pilot to actually be undertaken or I presume you mean trialled? The pilot is a trial?

Ms Harvey : Yes.

Senator KIM CARR: What is the date of that application?

Ms Harvey : The universities will be submitting in May 2017—early May. Assessment will take place from then until the end of June. Then we will do a review of the pilot, which we will publicly release in the third quarter. But individual university ratings will not be made public.

Senator KIM CARR: Why not?

Ms Harvey : We have found that very successful when we did that, when we introduced ERA in 2009. What we found is that it is a good chance to trial things, have a go at things, give really good feedback about how things work and then finalise and critique what we need to do to tweak it.

Senator KIM CARR: I have to concede that worked very well.

Ms Harvey : That is why we thought we would replicate the methodology.

Senator KIM CARR: Fair enough. So, have you got a copy of the draft indicators that we could see?

Ms Harvey : Yes.

Senator KIM CARR: Can that be tabled?

Ms Harvey : Yes, I would be happy to. I have only got one copy. Just before I hand them over, I would point out, though, that they are a non-limited set of indicators. We have pulled out where we have had agreement amongst what would be good indicators for a vast majority of things, but they are also encouraged to supply additional indicators in the narrative that they give with regard to the engagement indicators.

Senator KIM CARR: I see. Did you say that all the universities are participating?

Ms Harvey : No. I said nearly all the universities.

Senator KIM CARR: Which ones are not participating?

Ms Harvey : There are two universities that are not participating.

Senator KIM CARR: Which ones are they?

Ms Harvey : That would be up to them to say.

Senator KIM CARR: Why?

Ms Harvey : Because they have made a choice. They have discussed with me they are not participating for other reasons this time, but have indicated that they will be participating next time.

Senator KIM CARR: If you could take that on notice.

Ms Harvey : I am happy to.

Senator KIM CARR: I would be surprised as to why they would not want that information public. Anyway, you will establish that. Were there any indicators that were actually proposed and rejected?

Ms Harvey : There were lots of indicators that we looked at with regard to what could be used and what could not be used, whether they could be verified and whether they could be replicated. We have looked at a whole range of indicators. Things were suggested to us. But looking at the validity of them, whether they can be replicated and collected reliably is all taken into account with what we need to do. We have principles that underpin the indicators, very similar to the ERA indicator principles, which is that they must be transparent. You must be able to verify them. You must be able to replicate them.

Senator KIM CARR: How many of the indicators were actually suggested by industry?

Ms Harvey : I think the vast majority were actually suggested by both university and industry. I cannot think of a unique one that just industry came up with.

Senator KIM CARR: All right. We will have a look at those.

Ms Harvey : Yes.

Senator KIM CARR: When I see them I can perhaps follow that through. My understanding is a number of vice-chancellors have indicated their concern about the use of registered patents as a suitable metric, given the way in which patents as an indicator have been manipulated in the past. Has that been considered by the steering committee?

Ms Harvey : Yes. One of the things we have looked at are patents and patent citation indicators. They are one of the ones we will be looking at in the pilot. They are included, where it is applicable, to the discipline that is being assessed.

Senator KIM CARR: Who is on the steering committee?

Ms Harvey : Professor Byrne is co-chair. David Learmonth is also co-chair, who is at the table. Graham Whickman, President and CEO of Ford. Shanny Dyer from Wavefront Biometric Technologies. Ken Boal, Vice-President, Cisco Australia and New Zealand. And we have Emeritus Professor Lesley Johnson, Prof Ian Jacobs, Ms Belinda Robinson, Professor Scott Bowman, Professor Anne Kelso, from the NHMRC, Mr Mark Cully, who is the Chief Economist, Department of Industry, Innovation and Science, Dr Alan Finkel and Professor Shearer West, who is Deputy Vice-Chancellor, University of Sheffield.

Senator KIM CARR: In the past Professor Byrne himself has highlighted concerns about the way in which ERA could be managed or manipulated. Have those considerations be made in regard to the use of these metrics as far as gaming is concerned, for instance?

Ms Harvey : One of the things that we do, whether it is for ERA or preparing for Engagement and Impact, is look at, as I said, the reliability of how they can be collected and how they can be used. We look at all of those things. If we believe there is some concern over the use of those, we look at what internal controls we could put in place to look at how that is done and we discuss with the university sector about how that can be collected. I included some other information on there that might be useful.

Senator KIM CARR: All right. I will have a look at that. As you know, I have been concerned for some time about the question of rorting in this matter. I know this is not a matter that I alone share. You are saying you are going to rely upon the universities to tell you. What is the method that you are going to use to counter the rorting and gaming of this impact measurement?

Ms Harvey : Depending on what the indicator is and how that is collected, there are a number of controls you can put over the top of that. In particular, what we will be doing, not for the pilot but in 2018, is making all of the information public that is transmitted to us, unless it is of commercial or a sensitive nature.

Senator KIM CARR: But you cannot tell me who is not going to participate off the bat?

Ms Harvey : Not in the pilot, but all universities have indicated they will participate next year.

Senator KIM CARR: All right. So, you are going to make it public. Now, let us deal with some really basic stuff. Have you considered the structure and the length of narrative statements for the various case studies?

Ms Harvey : Yes, we have.

Senator KIM CARR: What will that be?

Ms Harvey : If you have a look at the information that I have just given you, if you are talking about the impact study—and bearing in mind that the indicators actually form part of the engagement assessment—the impact studies will deal with what the research is that underpins the impact, what the approach is to impact that is supported by the universities and then what the detail of the impact itself is. You can include different indicators or measures in that if you wish to.

Senator KIM CARR: We will have to look at that in the pilot itself. What is the funding envelope for this exercise?

Ms Harvey : As part of the National Innovation and Science Agenda across five years, in total it is $11.2 million to the ARC, because we have the system and have the assessors.

Senator KIM CARR: Eleven-five?

Ms Harvey : Yes, $11.2 million over the five years.

Senator KIM CARR: Have you looked at what has happened in the United Kingdom? Recently you undertook a comparison of their impact arrangements.

Ms Harvey : Yes, we have.

Senator KIM CARR: You have?

Ms Harvey : We deal quite closely with the UK as a jurisdiction that also looks at things. Previously we have shared our information and they have shared theirs. A number of people who were on their assessment panels volunteered to get together and assist us with the learnings that they had out of that as well as HEFCE, the Higher Education Funding Council of England, which also helped us with what learnings they had. It was similar to what we did when we looked at our metrics, so we do not have to relearn the same lessons over again.

Senator KIM CARR: In the British case they found that it was costing them $246 million in 2014 alone, of which $212 million was to be borne by the higher education community, presumably the universities. What is the breakdown expected in Australia?

Ms Harvey : We would not be able to give an indication of that until we complete the pilot.

Senator KIM CARR: You must have some assessment?

Ms Harvey : We are looking at a different approach to try to make sure that the burden is not just passed across to universities, including reusing a range of data that we already hold in the ARC. We are looking at the additional work that they need to put back. For example, in the pilot we will derive a whole range of things and give it back to the university to verify as opposed to having the university pull that together.

Senator KIM CARR: You must have done some assessment if you have a costing here of $11 million over five years. That is your share of the costs?

Ms Harvey : Yes, that is right.

Senator KIM CARR: Presumably you must have some assessment of the universities' share of the costs of this exercise?

Ms Harvey : No, because when we were doing the assessment of how much this would be, it was about what could be delivered, what we needed to build to make that appropriate, what we could reuse and to look at how to do that. We did not have the final shape at that point, as we still do not because we have a pilot.

Senator KIM CARR: So, you do not know?

Ms Harvey : I would not like to say.

Senator KIM CARR: Ms Harvey, have I understood that correctly; you do not know?

Ms Harvey : We have an estimate when we looked at the regulatory impact which is across all of the higher education sector of just under $3 million per annum.

Senator KIM CARR: Three million?

Ms Harvey : When we have an assessment year.

Senator KIM CARR: When was that?

Ms Harvey : When?

Senator KIM CARR: Yes. When did you do that assessment?

Ms Harvey : When we were looking at the regulatory impact last year when we were doing this.

Senator KIM CARR: So, that is a 2016 figure?

Ms Harvey : That is an estimate, yes. It is very much an estimate.

Senator KIM CARR: Three million a year?

Ms Harvey : For an assessment year.

Senator KIM CARR: For an assessment year?

Ms Harvey : Yes.

Senator KIM CARR: And it is across all of the universities?

Ms Harvey : Yes.

Senator KIM CARR: Who else does it include? Is it just the universities?

Ms Harvey : Just the universities, yes, because of the regulatory burden.

Senator KIM CARR: Do you anticipate that figure to rise? In your calculations, have you presumed that will remain static?

Ms Harvey : As I said, that is an estimation as we were looking at how we would build the approach and that is something that we would reassess afterwards.

Senator KIM CARR: Have you done any assessment of what the consequences will be for researchers in regard to changing behaviour?

Ms Harvey : I think that is actually one of the things that we are looking at—and I am not sure whether the minister might want to comment—which is with regard to looking at how you look at the value of research outside of academia. So, one of the challenges—and they have learnt one of the lessons from the UK—is looking at how researchers explain what they are doing, in a way, and looking at how we have these discoveries and what they mean, both for Australia and the world. So, you would look at anything that is measured, similar to when we introduced ERA, that has resulted in an improvement in the quality of research that is conducted in Australia. It has been very successful. That is one of its objectives.

Senator KIM CARR: Yes, and of course you would not be doing it unless you wanted to change a behaviour.

Ms Harvey : That is right.

Senator KIM CARR: That is what ERA did. I did not expect you to say anything else on that. Let us go back to the question of costs. In the British case it is said that there was an increase in costs of 133 per cent. You are saying you can contain it here?

Senator Birmingham: Increase in costs off what base?

Senator KIM CARR: I will just read directly from the report. It estimated the total cost to the UK of running RIF was $246 million. The report found that the cost of submitting to the REF was 133 per cent more than that of the 2018 RAE. So, from 2008 to 2014, in terms of the change in the use of this particular question, there was an increase of 133 per cent in the cost of submitting. Are you saying you can avoid that here?

Ms Harvey : It is important that we actually look at apples and apples. The RAE was about quality. The REF is broader. We have looked at the methodology that the UK used. We are requiring fewer case studies. You are also talking about a larger research sector in the UK. We have actually tailored it, similar to what we did with ERA, to what suits the Australian environment.

Senator KIM CARR: So, it is subjective?

Ms Harvey : All measures are subjective. It is looking at what works. That is why we do the consultation. What will work for Australia? What are we trying to achieve? How do we best do that?

Senator KIM CARR: The overwhelming bulk of the costs in the United Kingdom were to be borne by the universities. Are you saying that will not happen here?

Ms Harvey : The way that the REF is undertaken is different from the way that we are doing the impact and engagement assessment. There are far fewer case studies, and it is looking at how that works. The way they assess quality is different, too, because it is not all in as is ERA, where we require all research outputs to be part of the assessment.

Senator KIM CARR: I take it that you have studied the British report on the question of gaming. How are you going to avoid what they refer to in the United Kingdom as extensive behavioural changes, which is a polite way of talking about the gaming of the system in the United Kingdom? How are you going to avoid that in Australia?

Ms Harvey : That is what we have looked at. It is about how many case studies you have. What do you require to be in them? What are you trying to achieve? The behaviour change that we are looking at is continuing to have high-quality research done in Australia and looking at how we also encourage that applied research as well. So, we looked at how we put that together. We have looked at that. They have helped us with that, but we have also looked at what controls need to be put in place around that.

Senator KIM CARR: Can you be specific? Perhaps you can take on notice the specific measures you will take to stop the rorting of the system that occurs in the United Kingdom?

Ms Harvey : I would be happy to take it explicitly on notice. One of the great things is having the case studies published and machine readable. It is a really good way of making sure that when people are making assertions they can stand by those assertions.

Senator KIM CARR: The other major criticism that arises from these exercises is the impact that such studies have on interdisciplinary work. What measures are you taking to protect interdisciplinary work, given how important that is in modern research?

Ms Harvey : Absolutely. In ERA, as I am sure you know, one of the things we look at is being able to make sure that interdisciplinary research is able to be assessed appropriately. It is one of the things that we are looking at in the pilot as well. An example in the pilot is that we are going to look at Indigenous research. It does not fit underneath a direct two-digit field of research code. It will be assessed against all of the same benchmarks as the other disciplines, but it does not fit within the Australian Bureau of Statistics code. We are looking at how you do that, how you refer it and how you have an interdisciplinary panel that assesses that. We have had a lot of experience from doing that with ERA and making sure that it works again for the impact and engagement.

Senator KIM CARR: On the question of avoiding short-termism, which of course comes about as a result of gaming in these matters, what action will you take there to prevent that type of work being undertaken?

Ms Harvey : It is about having things in a balance. We continue to do the assessment of ERA. That is why they will be done in conjunction. It is about having high-quality research and it is about being able to choose what is put forward. There is absolutely fantastic fundamental research done in Australia and that will continue to be recognised and rewarded under different mechanisms. It is not just about applied. The focus is not just flicked straight over to applied.

Senator KIM CARR: What assurance can you give this committee that you will not undermine the integrity of ERA as a result of these measures?

Ms Harvey : We are looking at doing them in conjunction. We are looking at what data we are collecting, what those internal controls are that I have talked about, which is about making sure, and we have over 10 years worth of data in the ARC of everything to do with research in Australia with regard to world datasets, and we can then look at how that all fits together.

Senator KIM CARR: I think you will need to do a bit better than that.

Ms Harvey : Without talking about the different indicators and the controls that we have under those ones.

Senator KIM CARR: We will obviously need to come back to this.

Ms Harvey : I look forward to the next estimates.

Senator KIM CARR: Yes, with a new CEO to answer the questions.

Ms Harvey : I am pretty sure I will be there.

Senator Birmingham: Ms Harvey will still be here.

Senator KIM CARR: Can you explain what the relationship is between the ARC and Elsevier?

Ms Harvey : Yes. They are a supplier that we have used for citation provision.

Senator KIM CARR: For some time?

Ms Harvey : Yes, but we have had appropriate procurement arrangements that they have been successful under.

Senator KIM CARR: When was the last time they secured a contract with you?

Ms Harvey : They secured the contract for the ERA 2015. Having said that, we have arrangements with other citation providers as well.

Senator KIM CARR: What is the value of contracts that the ARC has with them?

Ms Harvey : Currently?

Senator KIM CARR: Yes.

Ms Harvey : I would have to take that on notice.

Senator KIM CARR: Of course.

Ms Harvey : It would be on Aus Tender.

Senator KIM CARR: Can you provide the detail of the other citation reference groups that you have as well? What is the value of the contracts with them?

Ms Harvey : Yes. I am happy to do that.

Senator KIM CARR: What is the point of engaging them? What do you get out of it?

Ms Harvey : For the ERA 2015, if I can use that as an example, we require a stylised world data set of all of the journals, which are stylised to the Australian and New Zealand research codes, so we can look at, for every paper that has an Australian author, how many citations they have for that paper. We have it normalised by year and by field. For example, people in the health disciplines, in particular the medical disciplines, cite often, whilst other disciplines do not cite at quite the same pace with regard to what they are doing. You have to normalise it, because you cannot say that because somebody has 15 citations they are better than somebody from a different field that has 10. You have to normalise it so I can compare, for example, my citations in a particular field against Ms Emery's citations in a particular field. That is what we get out of that. We also have the metadata to do with all of the research outputs in Australia that are published. We get that as well. We use that for policy work as well, where we actually look at what we are looking at if we have areas.

Senator KIM CARR: Thank you for that. Have you been consulted about the second tranche of NISA?

Ms Harvey : There have been a number of conversations about what could happen.

Senator KIM CARR: You have been consulted on the second tranche?

Ms Harvey : What would you determine as consulted?

Senator KIM CARR: I want to know whether or not you have been asked about what the next variation of NISA should look like.

Ms Harvey : I am not sure about what it should look like. Obviously, as part of the Public Service, we have been asked about whether we have any ideas, views, what has worked and what has not worked.

Senator KIM CARR: Have they concluded?

Senator Birmingham: No.

Senator KIM CARR: The consultations have not concluded?

Senator Birmingham: The finalisation and obviously release of the next steps in NISA have not concluded.

Senator KIM CARR: Can I ask you about the ARC. Is it working on an update for its 2013 policy on open access for the publication of research?

Ms Harvey : I will pass that across to Ms Emery. She looks after that area.

Ms Emery : We certainly are.

Senator KIM CARR: What are you doing in that regard?

Ms Emery : We have just recently been out for consultation on a revised policy. We have made it clearer what are our research outputs, and there was no change to the timeframe for when publications need to be made open access.

Senator KIM CARR: When do you expect the new policy to be released?

Ms Emery : That should be quite soon. I think the consultations closed either yesterday or today. We had probably less than 15 submissions to date and we should be able to finalise that policy quite soon.

Senator KIM CARR: Are the submissions public?

Ms Emery : No.

Senator KIM CARR: Why not?

Ms Emery : No particular reason. We did not say that we would make them public.

Senator KIM CARR: Can you provide them to the committee?

Ms Emery : I can certainly take that on notice.

Senator KIM CARR: Who in particular have you consulted with?

Ms Emery : We went out to the universities to consult with them. We have also been consulting with the peak body for librarians and similar bodies.

Senator KIM CARR: Are you going to the question of fair use provisions?

Ms Emery : I do not believe so.

Senator KIM CARR: In terms of the question of open access, have you settled on a date? Is it 12 months or six months that you are proposing?

Ms Emery : Currently we have 12 months. Publications must be made open access within 12 months. But if that is not possible because of the publishing rules, they need to explain that to the ARC in terms of their final reports when they finish their funding.

Senator KIM CARR: Is that one of the measures that you are looking at as a change in policy?

Ms Emery : No, we are not.

Senator KIM CARR: It is not?

Ms Emery : No.

Senator KIM CARR: So, you expect it to stay at 12?

Ms Emery : Yes.

Senator KIM CARR: That concludes my questions of the ARC. I thank you very much for your assistance.

CHAIR: Thank you very much.

Dr Bruniges : Can I just correct something I said earlier for Senator Carr?

CHAIR: Yes.

Dr Bruniges : Senator Carr asked me a question about the number of candidates we had interviewed.

Senator KIM CARR: Yes.

Dr Bruniges : We shortlisted three. We called three to interview. We actually interviewed two, because one dropped out post being called.

Senator KIM CARR: Thank you.

Ms Harvey : And a correction: Professsor Byrne left as CEO on 8 September, to be precise.

Senator KIM CARR: Thank you, but he did notify ARC on 24 May, did he not? He did indicate his resignation?

Ms Harvey : I would have to check what date he actually formally notified.

Senator KIM CARR: Yes, whatever it is.

Ms Harvey : He gave a long notice period.

Senator Birmingham: It was, as I recall, during the caretaker period, because I remember ringing you about it.

Ms Harvey : Yes.

Senator KIM CARR: That is right.

Ms Harvey : That is why I had to check it.

Senator KIM CARR: My concern goes to the length of time it has taken to get a replacement. I understand there was the caretaker period there for a little while, but it is a long time to be without a notified CEO. Very well represented by the acting CEO, I would readily concede as well.

Ms Harvey : Thank you.

Senator KIM CARR: But it is a long period. It has been put to me by universities that it is far too long. There is some concern in the system about the length of time that has been taken.

Senator Birmingham: Thank you. As I said, an announcement hopefully is imminent.

CHAIR: Are there any other questions for the ARC? Thank you very much. I will just check if there are any other senators with questions for TEQSA. We do know that Senator Lambie does. Senator Lambie, if you are in the building and can hear this message, please make your way to 2S3. TEQSA is waiting for you.