

- Title
Environment and Communications References Committee
17/11/2023
Optus Network Outage
- Database
Senate Committees
- Date
17-11-2023
- Source
Senate
- Parl No.
47
- Committee Name
Environment and Communications References Committee
- Page
2
- Place
- Questioner
Grogan, Sen Karen
Roberts, Sen Malcolm
Hughes, Sen Hollie
Cadell, Sen Ross
Pocock, Sen David
Henderson, Sen Sarah
- Reference
- Responder
Hanson-Young, Sen Sarah (The CHAIR)
Mrs Bayer Rosmarin
CHAIR
Mr Kanagaratnam
- Status
- System Id
committees/commsen/27530/0001
17/11/2023
Optus Network Outage
BAYER ROSMARIN, Mrs Kelly, Chief Executive Officer, Optus
KANAGARATNAM, Mr Lambo, Managing Director, Networks, Optus
Committee met at 09 : 00
CHAIR ( Senator Hanson-Young ): I declare open this hearing of the Senate Environment and Communications References Committee's inquiry into the Optus network outage. I begin by acknowledging the Ngunnawal and the Ngambri peoples, the traditional owners of the land on which we meet, and pay our respects to the elders past and present.
On behalf of the committee, I welcome everybody here today. This is a public hearing and a Hansard transcript of the proceedings is being made. The hearing is also being broadcast by the Australian Parliament House website.
Before the committee starts taking evidence, I remind all witnesses that in giving evidence to the committee they are protected by parliamentary privilege. It is unlawful for anyone to threaten or disadvantage a witness on account of evidence given to a committee and such action may be treated by the Senate as a contempt. It is also a contempt to give false or misleading evidence to a committee. The committee generally prefers evidence to be given in public, but under the Senate's resolutions witnesses have the right to request to be heard in private session. If a witness objects to answering a question, the witness should state the ground upon which the objection is taken and the committee will determine whether it will insist on an answer having regard to the ground on which it is claimed. If the committee determines to insist on an answer, a witness may request the answer be given in camera. Such requests, of course, may be made at any other time. I would also ask witnesses to remain behind for a few minutes at the conclusion of the evidence in case the secretariat needs to clarify any of the terms of reference.
Welcome. Thank you very much for being here today. I understand that we've got a bit of media interest today and you're okay with the media presence in this hearing, and the committee membership is as well. I understand that information on parliamentary privilege and the protection of witnesses and evidence has been provided to you. I invite you to make a short opening statement, and then we'll go to some questions.
Mrs Bayer Rosmarin : I'm grateful for the opportunity to assist the Senate with its inquiry. Firstly, I would like to acknowledge that the extent and duration of the outage last Wednesday had a significant impact on the lives and livelihoods of our valued customers and the broader community. We understand the intrinsic, essential role we play in Australian lives, and we take our responsibilities to provide continuous connectivity seriously. As a business, we are nothing without our customers who put their faith in us, and it is indisputable that on that day our performance was not acceptable. We let you down, and for that I am deeply sorry.
I want to make it clear that we have taken immediate and ongoing steps to rectify any shortcomings, and today I want to assure you, my teams and our customers that we are committed to regaining trust and rebuilding faith in our brand. We have communicated directly to every Optus customer and, as you know, offered them not just a heartfelt apology but additional data as a gesture of thanks for their ongoing support and patience, and have committed to talking to any customer or small business who has special circumstances they would like us to consider.
I understand there has been much commentary about aspects of our response to the outage, and I really welcome the opportunity this morning to answer all of your questions as openly as we can and hopefully clarify some misconceptions as well as be frank about some of our mistakes. We have provided a detailed written submission to assist with specific time lines and technical aspects.
I appear today with the Optus head of Networks, who can assist the committee with any of the more technical questions it may have around why the key Optus routers disconnected from the network in response to a change in routing information that resulted in the shift from an alternative peering router during a scheduled upgrade on the Singtel international peering network. The reality is that our network should have coped with this change, but on this occasion it did not. I can confidently assure the Australian public that we have made immediate changes to our systems to ensure that this specific issue will not cause another outage of the nature that we experienced last Wednesday.
Optus has in place a very clear process for responding to network outages. There is an allocated set of personnel available 24 hours a day, on duty to form an incident response team and follow the process of recovery, restoration and communications. The process is designed to empower the team to quickly manage the critical components of the issue and escalate as the severity and timings change. Last Wednesday, that process was enacted. Our teams were working hard on a series of assessments to isolate and identify the root cause, including rolling back recent updates on our network and disconnecting from various links.
These actions did not resolve the issue, and so we effectively performed a hard reboot of the network, which required a complex, coordinated set of both virtual and physical activity across the country. This commenced at around 10.30 am, with the vast majority of customers recovered by 2 pm and 99 per cent by 4 pm. The actions we took were a brute force resuscitation of the network. We had not yet identified the cause of the issue, and so whilst the crisis was over for our customers by 4 pm it was not over for our teams, who needed to immediately shift focus to understanding what had happened so preventative action could be taken on the network to ensure it wouldn't happen again. This work continued for many days—and, with your indulgence, I would like to acknowledge and thank the team for their tireless recovery efforts under enormous pressure.
While I believe wholeheartedly that we did everything we could to provide timely, accurate and credible information, I acknowledge that there is always more we could have done. There is no question that the outage itself initially adversely affected our ability to more effectively communicate with each other, our consumers, media and government in the early hours of Wednesday morning. I want to be very open about that.
Some have criticised me for not appearing publicly sooner. The key reason for this was twofold. Firstly, I prioritised the team's actual crisis response, which included not only a clear focus on immediate recovery of the network but also decisions around how to handle our call centre being down, whether to open our stores, adding security and support for our frontline staff and assessing the impact on other IT systems, and, most importantly, we needed to identify if the outage was a result of malicious or ongoing attack. This was critical to ensure we could re-establish the network safely.
Secondly, I wanted to ensure that before I spoke, and given how little information we had about the cause and potential restoration time, we could at least rule out the possibility of malicious activity to reassure our customers and the nation. As soon as our cyber specialists, who were in contact with the Australian Cyber Security Centre—whom I thank for their support—ruled this out, I began publicly fronting the issue on behalf of my team.
The initial incident response and crisis management team first issued a broad statement at 6.33 am, and then eight additional media statements throughout the day. All major news outlets carried the story of the outage in their morning bulletins, and I conducted 11 interviews, including three live radio interviews, appearances on all four major TV networks and four print journalist interviews.
Could we have done some things better? Of course we could have, but I'm also grateful for the way so many of our people mobilised under intense pressure and in unique circumstances to get us back online as quickly as we could. Optus also welcomes the opportunity to work with the government and industry to discuss any broader measures that may be suggested as a result of the outage, cognisant of the implications for managing all critical infrastructure, essential service and government service disruptions in the future.
In my time at Optus I have led our vision to be Australia's most loved everyday brand, with lasting customer relationships. We know that at the heart of that is being trusted and relied upon every day. Under my leadership we have shifted more investment to network resilience, and it has been a priority for the team. Beyond restoring our network, our focus now is on restoring trust. Thank you for time, and we're happy to take your questions.
CHAIR: Thank you very much. I know there are going to be lots of questions from colleagues in the room, so we will share the call around. We're here up until 11 am, so we've got lots of time. I would like to make sure that we keep this session as respectful as possible. We're here to get answers on behalf of the Australian people, and that's what they expect us to do.
I see in your submission the outline of the time frame. When did you first understand that there was a problem?
Mrs Bayer Rosmarin : When I got up in the morning I could see that the phone wasn't working, so I immediately decided to head into the office. At that time I thought I'd call the team on the way, because it's quite unusual for both the data and voice networks, which are run separately, to be down together. Once I was in the car and could not call the team I realised that it must be a more serious outage. Since I was already en route I continued driving—I did not stop and switch to an alternative SIM. I arrived in the office at around 7.35 am and headed straight to the network operations centre, where the team told me what was going on. I called a crisis, and we had our crisis meeting from 7.45 am until 8.30. My immediate next action, at 8.32, was to call the Minister for Communications and share what I knew.
CHAIR: At 8.30, when you spoke to the minister, were you aware that there had been an update in the early hours and that there had been a problem at that end?
Mrs Bayer Rosmarin : When I spoke to the minister I shared all the information that I knew from the crisis management team. It wasn't very much, because at that point we had no idea what had caused the issue, nor had a restoration time.
CHAIR: But you were aware that there had been an update that was meant to have happened at 4 am or earlier?
Mrs Bayer Rosmarin : I'm aware that the incident began at 4.05 am in the morning, yes.
CHAIR: When you spoke to the minister at 8.30, what did you tell her that you knew?
Mrs Bayer Rosmarin : I told her that I knew that we'd had this outage running and that the teams had been working on it from the early hours of the morning. I know that my team had contacted her office earlier, so they also had some information. Most importantly I assured her that we had everybody we needed on point to try and restore the network as quickly as we could and that I would keep her updated throughout the day.
CHAIR: Did you tell the minister that you were aware that people weren't able to call triple 0?
Mrs Bayer Rosmarin : When I spoke to the minister at that time we had every reason to believe that the triple 0 system would work as designed and that all calls would go through on alternative networks.
CHAIR: Was that just in relation to mobile devices?
Mrs Bayer Rosmarin : Correct.
CHAIR: So you would have known at 8.30 that landlines wouldn't have been able to call triple 0?
Mrs Bayer Rosmarin : It's always the case that if a landline is down you can't call triple 0. That's known information.
CHAIR: So at 8.35—did you understand that people were having problems calling triple 0 at that hour in the morning?
Mrs Bayer Rosmarin : At that hour we did not have any knowledge of that.
CHAIR: The outage was reported to have started 4.05 am. What time were the relevant technicians in Optus made aware of it?
Mrs Bayer Rosmarin : I might defer to Lambo to talk about the steps that were taken through the morning. But I believe it was around 4.05 that there first was an alert of an issue, and our teams who monitor 24/7 were the first ones to see that.
Mr Kanagaratnam : The incident started at 4.05 am. The initial line of investigation was to understand why connectivity had been lost, in terms of monitoring and management of the network. There were various incident bridges set up: one with the IT team at about 4.15, and then there was another ticket raised on the network side by 4.37. We started up a network bridge at 4.45. There were some challenges reaching some of our team because of the network being down; however, there were multiple attempts, and people were reached through Teams and also WhatsApp on alternative numbers. So our team was fully engaged and dealing with the problem from about 4.37.
CHAIR: What time was the first technician on site to deal with the outage?
Mr Kanagaratnam : The first technician on site—I'll have to come back to you in terms of that specific time line. But I'll just say that we had several people in the room at our NOC centre from 4.52.
Mrs Bayer Rosmarin : It might help if I explain, because 'a technician on site'—we're talking about lots of sites when it comes to the network.
CHAIR: Yes.
Mrs Bayer Rosmarin : We have a monitoring centre, which is staffed 24/7, and then we have a duty manager, who they can alert if there's an issue. Then they assemble what Lambo called a 'technical bridge', which is a virtual call that all the different technicians, engineers and field technicians, as required, can join. That bridge includes people from communications or those representing the frontline teams. So it's not that we were waiting for any technician to arrive somewhere; it's set up so people can join from home or from the monitoring centre or wherever they're meant to be.
CHAIR: So there was no need to send people physically to different sites?
Mrs Bayer Rosmarin : There was later.
Mr Kanagaratnam : There were actually people on call in certain locations. That's our standard operating procedure. So we had technicians available. The initial response, as Kelly said, is to understand what the incident is, what the scale of the outage is, and what exactly has failed, and then to look at multiple lines of investigations to restore. That was really the initial focus of the response to the incident.
CHAIR: Yes, but I'm trying to work out when you realised that you needed to physically send people to the centres. When did you realise that you needed a physical presence on the ground?
Mr Kanagaratnam : At about 6.50, we understood what had triggered the failure across all the routers that were impacted by the outage. As I said, there were staff in several of our exchanges—that's a 24/7 operation, in terms of people, that we have—and then we mobilised additional engineers. We had a total of 46 people on site, restoring, responding and resetting equipment to ensure that we restored service. So from 6.50 we started mobilising additional staff in locations where we didn't have people on site.
CHAIR: Just so we use language that people can understand, at 6.50 you sent people physically to these centres?
Mr Kanagaratnam : Yes.
CHAIR: Is that what you mean when you say 'mobilise'?
Mr Kanagaratnam : Yes, that's what I mean when I say 'mobilise'.
CHAIR: So at 6.50 you knew that there was a problem on the ground that would need to be physically dealt with. Is that correct?
Mrs Bayer Rosmarin : At that time we were still investigating multiple lines of inquiry. I can ask Lambo to go through a number of the different lines of inquiry we were pursuing. We had people there to try things physically. We were also trying things virtually as we were ascertaining the cause. Would you like to outline some of the hypotheses we tested?
Mr Kanagaratnam : The initial line of investigation we started with was not being able to authenticate on the network—all our staff could not reach the devices on the network remotely. We also had reports that some customers were able to access internet services by changing what is called a 'domain name server'. To explain simply, that's a system that translates a webpage into an IP address. Some customers, when they changed that, were able to access the internet. So we also looked at that as a line of investigation. We also ruled out what we call a 'distributed denial of service', which is a form of cyberattack where your network gets flooded with malicious data which is pushed through—
CHAIR: At what time did you decide that you could rule out a cyberattack?
Mr Kanagaratnam : We knew it was not a distributed denial of service attack quite early, but we wanted to make sure that there wasn't ongoing traffic being fed into our network. At about 10.20 we let Kelly and the team know that we were pretty confident it wasn't, because we had understood by then that the routers had reached this limit for another reason.
CHAIR: Was that a concern for you as soon as you realised this wasn't a cyberattack and that this was a problem of Optus's making?
Mrs Bayer Rosmarin : Regardless, it's a problem of Optus's making. But we did really want to understand if it was malicious or cyber, and the ongoing threat that could pose. There were some strange coincidences that made us quite worried about that. For example, we had the cyber incident the last time the Singtel board were in town, and they were in town again. So that did seem—
CHAIR: That is a weird coincidence, isn't it.
Mrs Bayer Rosmarin : very strange as a coincidence. Whilst they'd ruled out the denial of service attack as one technical type of cyberattack, there were other vectors of cyber and malicious activity and a threat intelligence that we were chasing down. It took the team until 10.20 to be able to confirm that. It was a serious concern for us in those hours up until 10.20.
CHAIR: I want to go back to the beginning of your morning. You woke up, your phone wasn't working, so you knew something was going on. Many Australians did the same thing. They woke up, went to look at their phone and their phone wasn't working. But you're the boss of Optus, so you knew something really wasn't working. You got in your car and you tried to access data. You made a voice call, and that wasn't working. What kind of backup does Optus have to make sure that when your own system goes down you can still communicate?
Mrs Bayer Rosmarin : We do have a process of key people having backup SIMs. One of the learnings we're taking from this incident is that while we have that process, the way that we allowed that to happen could have been improved. What I mean by that is, for example, I had a backup physical SIM which meant that to activate it I would have had to take action. Some of our people have eSIMs as their backup and so they switched more automatically. That's clearly a better practice. Rather than just insisting on a backup SIM, we're going to be much more emphatic about the manner in which those backup SIMs are implemented.
CHAIR: When you were in the middle of this crisis meeting—I'm trying to envisage this—you're the second largest communications company in the country and you couldn't communicate. But some of your people have the ability to switch SIMs. You've got a backup SIM, which is either an eSIM or a physical SIM. Most Australians don't have that.
Mrs Bayer Rosmarin : That's right.
CHAIR: You had insurance for yourselves, but your customers didn't. Do you think that's a problem?
Mrs Bayer Rosmarin : It is our intention to give our customers continuous connectivity and to give them a network they can rely on 24/7 for 365 days a year. We do everything we can to prevent these types of outages. We have multiple layers of redundancy, including geographical redundancy, physical redundancy and power redundancy. It is highly, highly unusual for all of the different networks, which are segregated, to be down at the same time.
CHAIR: You made some comments around the fact that you've been criticised for not speaking earlier. You mentioned that you wanted to make sure your team was okay dealing with all those urgent issues that you were confronted with. Did you not think that you could delegate the communication to your customers to somebody else?
Mrs Bayer Rosmarin : That's a very important question. Communications were delegated to the team. It's unusual for a CEO to appear at all during an outage because the public would expect that my focus would be on working with the teams to resolve the issue. Our communications team, as per the timeline we provided you, was giving updates to the media, fielding questions, and the team had the view at the time that this was being covered widely and all our customers knew what we knew, which is that the network was down, we were working on it and we were very sorry. Now, I appreciate that the team did what they could with the information they had and the channels available. And, yes, there's more we could have done.
CHAIR: At what point did you call your legal team?
Mrs Bayer Rosmarin : Our legal team is a standing part of our crisis team, so they were in the 7.45 meeting that was called.
CHAIR: At 7.45 the lawyers were in the room?
Mrs Bayer Rosmarin : It's one person, our chief legal counsel, who is part of our crisis response team.
CHAIR: Was part of that legal advice for Optus not to speak too quickly?
Mrs Bayer Rosmarin : I do not believe we got any legal advice in that meeting. That first crisis meeting was very much focused on the actions we could take to restore the network and reassure our customers during the day.
CHAIR: Were you given any advice throughout the day as to how to structure the apology and the concerns being raised by customers?
Mrs Bayer Rosmarin : I'm not sure what you mean by advice, but we work together very tightly as a team in a crisis.
CHAIR: Were you given advice from your legal team to not admit fault?
Mrs Bayer Rosmarin : I was not given such advice.
CHAIR: Were you given advice from your legal team to be careful with what you said in case of compensation?
Mrs Bayer Rosmarin : I was given no advice by my legal team to say or not say anything, and in all the interviews I did I was not on a script. I'm the CEO. I'm accountable. I took that accountability and I said what I believed was right.
Senator GROGAN: I'm interested in how you communicated both with the government and with your customers. You did just say that people wouldn't expect the CEO to appear, yet you talk about leadership and supporting your customers. I note the minister was out on radio at 7.30, broadly, with no information, just providing reassurance things were being done. Did you consider providing such reassurance earlier in the day?
Mrs Bayer Rosmarin : Our communications strategy is designed to empower our teams to be able to give that assurance on behalf of the company. We have a team, and any crisis management involves utilising that whole team. Whilst I personally hadn't gone out to do that, all our statements, which are in the appendix of our submission, contain an apology and reassure people saying we know there's an outage and we're doing everything we can to restore that outage.
Senator GROGAN: But from your full suite of staff, was there no other person you could send out to provide assurances on a broader series of networks through the radio, TV et cetera? I'm concerned that you put a statement out, but your customers would not be able to access that statement, right? The statement you provided was to everyone but your customers. The media may well have been able to reiterate that statement, but having the person from Optus go out—I'll it to you this way: do you have a series of protocols for such a thing for when a disaster like this happens and you have no comms with your customers? Do you have a protocol?
Mrs Bayer Rosmarin : Yes, we do.
Senator GROGAN: Can you step us through it?
Mrs Bayer Rosmarin : Thank you for the question, because it enables me to clarify that leveraging the media to get the message out is part of our protocol and our strategy. Issuing statements to the media from an Optus spokesperson that they can reiterate to reassure customers is part of our protocol and the way that we use the media to make sure our customers know what's going on. I thank all those media broadcasters that morning for covering the story. That was part of what gave us reassurance that our customers were getting the message. That's not incidental or accidental; that is actually part of our communications plan—to leverage the media.
Senator GROGAN: But—
CHAIR: Sorry, Senator Grogan—can I just jump in for a second? What people were saying was that they weren't hearing from you. They weren't hearing from you—that is what was being reported. That is not reassuring to the community at all.
Senator GROGAN: Chair, can I continue?
CHAIR: Yes.
Senator GROGAN: So step us through exactly—or maybe you have documents that you could table for us around your process, protocol and triggers for informing the government when you have a major issue, and for informing your customers when you have a major issue. If you could give us a headline walk through those and table them, that would be very helpful.
Mrs Bayer Rosmarin : Sure. Our protocol is to inform our customers as quickly as possible—
Senator GROGAN: Do you have details? Does your protocol step out detailed pathways for how you do this, or is it that high level, 'Yeah, let's make sure we let people know'?
Mrs Bayer Rosmarin : No, I assure you: it's not that high level. It revolves around this idea of us forming an incident management team. On that incident management team, there would be somebody allocated to customer communications. We have a media phone, which is available 24/7, and that's rotated through members of our comms team so that they're available at all times. It is up to that incident response team that forms to make the decision of what channels to use to get the communications out. I'd go as far as to say: we have pre-approved messages that they can put out without requiring any approvals or escalations, so that they can act in as swift and timely a manner as possible.
In this particular incident, because all of our customer communications were down, we understood that they weren't going to be able to log on and see a message or get an SMS, so having the media get the message out was determined to be the priority, and the person on the media phone between 4.30 am and 6.33 am, when we issued a media statement more broadly, fielded about 20 calls and was cognisant that our information was being broadcast on radio and TV—
Senator ROBERTS: Excuse me—that's 20 calls from the media?
Mrs Bayer Rosmarin : I believe so. And then, at 6.33, they issued an official statement, and we wanted to keep updating that statement so that it was fresh. Every hour and a half we wanted to issue another update, even if we didn't have more information, and so they followed that process.
In terms of notifying the government, one of the things that we've learnt through this incident is that the people who had backup SIMs and alternative communication were who we deemed to be the most essential personnel for resolving an incident, and not enough people in our government relations team—
Senator GROGAN: Can I just interrupt you for a moment, because I have to follow that bell—
CHAIR: We'll come back to you when you get back, Senator Grogan.
Senator GROGAN: It would be nice if I could have some more time. I'm finding a lot of what you are saying very, very fluffy. It would be really nice to have some specifics, because we could spend two hours sitting here not actually getting the details that we require.
CHAIR: Thanks, Senator Grogan. Senator Hughes.
Senator HUGHES: I'll just continue from where Senator Grogan was, because, as Senator Hanson-Young said, I certainly was listening to the radio that morning and heard that there was an outage, but it was overwhelmingly on 2GB, which is quite a large, listened-to station in the largest state. All were saying that they had absolutely no clue as to what was going on and making particular reference to the point that there had been no statement from you or from Optus that gave them any clarity. What I found, in having a look at your submission, was that one of the first things that went out was a social media post. Didn't that seem a little ironic to you—that you were letting people know there was an outage, on social media, when they didn't have access to it?
Mrs Bayer Rosmarin : As I said, our teams did the best they could with the channels and the information that were available to them. It is frustrating when you're in an outage of that magnitude and you're unable to provide clarity. So I fully appreciate how frustrating it was for all our customers not to know what the issue was or when it would be resolved, but that's not because we were withholding communicating that. We ourselves did not know what the issue was and when service would be restored.
CHAIR: But isn't that the problem? You provide a service to over 10 million people, and not just individuals but government agencies, emergency services and businesses, and all they got for hours was a couple of lines that said, 'Sorry, our service is out. We're working on it.' You've got to understand, surely, that that just is not good enough.
Mrs Bayer Rosmarin : Chair, as I said in my opening statement, our performance—to have an outage like this happen—was not good enough, and it's highly unusual for all of these different networks that are segregated, that have multiple layers of redundancy, to all be out simultaneously. That is a highly unusual event.
CHAIR: There clearly wasn't enough redundancy, was there?
Mrs Bayer Rosmarin : Can I ask my colleague to explain the layers of redundancy and what the root cause was?
CHAIR: I do want to go back to Senator Hughes, because she was in the middle of questions. But I also want to point out that your submission in relation to what the root cause was and naming SingTel has now been disputed by SingTel themselves. So we're not sure who to believe. It's time that someone got the story straight.
Mrs Bayer Rosmarin : If I may respond to that, our statement on the root cause that we put out was agreed and approved with SingTel. The statement that they put out is not a contradiction but rather a clarification. As we tried to explain—
CHAIR: They needed to clarify a statement that they'd already signed off on?
Mrs Bayer Rosmarin : They were trying to clarify the way that that statement was interpreted. As you can appreciate, we're dealing with a very complex technical fault here, and what we had tried to explain was that the root cause of the issue was that our Cisco routers hit a fail-safe mechanism which meant that each one of them independently shut down. The trigger event that led to that was the upgrade on the SingTel international peering network. We put out the statement, which then got interpreted by various commentators as being that the root cause was the SingTel upgrade, when the trigger was the SingTel upgrade but the root cause was the routers. So that is what has been attempted to be clarified. But I assure you they are consistent.
CHAIR: For a communications company, the communication is pretty lousy—both at the time of the crisis and in the aftermath. Senator Hughes.
Senator HUGHES: I come back to where we were. How big is Optus's communications team? How many people work in your communications team?
Mrs Bayer Rosmarin : I believe, in the total team, there are seven people, but I can take that on notice and clarify.
Senator HUGHES: That's just communications? That's not media? That's not the crisis management team?
Mrs Bayer Rosmarin : No, sorry. That's all of media, communications and PR.
Senator HUGHES: Okay. Have you got a media spokesperson? I'm assuming, if Optus is going out to make any random announcement, it's not you all the time. Is there a spokesman or spokeswoman, or spokesperson?
Mrs Bayer Rosmarin : Yes, there are a few people in that team who can be the Optus spokespeople.
Senator HUGHES: I know you've said you didn't know anything, but crisis management 101 is being seen to be on the front foot—to even say, 'You know what? We don't know what is actually going on at the moment, but we are doing our best to find out.' There just seemed to be a complete void. It is a very technical issue, and obviously your own teams were doing things before it was you. I still think, if I was your crisis communications manager, I would have had you on the phone to Ben Fordham and the other main radio stations so that they could at least share it with their networks and get grabs out there.
CHAIR: It might have been a better look to get out there before the minister was.
Senator HUGHES: Yes. But, if someone from your team had the phone overnight, knew at six o'clock in the morning and was fielding these media calls, why didn't they put out some form of comment or statement that 'we are working through what's happening and we will be in touch as we know more'?
Mrs Bayer Rosmarin : I really appreciate the question and I'd love for people to have the chance to look at the appendix to our submission.
Senator HUGHES: I have; I've gone through it.
Mrs Bayer Rosmarin : What you'll see is that the spokespeople in the communications team felt that they were doing that. Not only did they—
Senator HUGHES: Well, they put a tweet out, or a—
Mrs Bayer Rosmarin : Not only did they issue the statements on social media and our statement but they were also talking to many journalists and media influencers, representing exactly what you've just said to all of those stakeholders—
Senator HUGHES: But it wasn't being reported that way. People were actually saying that they hadn't spoken to Optus—they hadn't heard from Optus themselves.
Mrs Bayer Rosmarin : I wish I could explain why that's the case, and why people choose to report in a certain way. I can only tell you that our team was on the phones and talking to as many people as they could, reiterating that we were working on it and doing everything we could to restore services. That was all the information that they had at the time, as frustrating as that was.
Senator HUGHES: I just want to come to one point. The thing that I think a lot of people were upset by was—obviously, there was the triple 0 access and people's personal phones not being available—the businesses that were impacted, with their ability to take payments and to operate. We know it is a cost-of-living crisis at the moment and that businesses are doing it really tough. Some businesses lost, fundamentally, a day of trade, and if you think of some of those cafes, their busiest period is through the morning. You might have been back up by 2 pm, but they're done by then with everyone going to buy their morning coffee—very few people have cash anymore.
I noticed there was an article in The Australian which said that during a private briefing you said:
Obviously, it's devastating for some businesses that they weren't able to do what they normally do. Many businesses have contingency plans in place. And they were still able to move forward. That's a very case-by-case scenario.
When I read that, my first thought was: 'You're a big company. What was your contingency plan?' Obviously, we're here talking about that now. I'm not a technical person, I'm happy to say that I didn't even know there was a thing called an eSIM—and Senator Cadell will vouch for that! But if you are committed to customers—and this applies to all telcos—and if you're committed to ensuring businesses can have a contingency plan on their communications and on the ability to participate in commerce, is this part of the process? When you have account managers talking to business customers, do they say: 'Do you know what? Things do go wrong. They've gone wrong for other networks as well. What we would suggest to you is that your business have a backup eSIM so that you can still operate?' Has that even crossed anyone's mind, perhaps offering that layer of protection to business customers?
Mrs Bayer Rosmarin : Yes, we do have those conversations and many of our customers do have backups. Particularly, larger customers who we work with on structure. If they have other essential services, we do try and give them the levels of redundancy that they might need in a situation like this. It's clearly something that we want to work with all of our small-business customers on as well. That's why we're encouraging all small businesses who were affected, which we're very sorry for, to reach out to us individually, not only so that we can understand their specific circumstances but also set them up so that they are more resilient in the future—not that we're planning for anything like this to ever happen again.
Senator HUGHES: I was interested when you said—and I'm not going to be able to pronounce your surname correctly, so I might just go with Lambo as we were introduced, I apologise!
Mrs Bayer Rosmarin : Yes, go with Lambo.
Senator HUGHES: I noted you said that people were able to change their IP address or something in their phone, in their settings, so that they could then access data.
CHAIR: If you knew.
Senator HUGHES: If you knew! What was that? It seems to me that these are things that people might have found helpful to know. That would have been information—
Mrs Bayer Rosmarin : That wasn't true.
Senator HUGHES: Wasn't it? See that's how technical—
Mr Kanagaratnam : Maybe I'll just explain. For the average consumer, it's a very difficult thing to try and understand, so I wouldn't recommend it as a solution.
Senator HUGHES: Right, okay.
Mr Kanagaratnam : I was just explaining that there was a line of investigation, and it was one of the first things we looked at. That's because we got some feedback that it's the way they got around the connectivity issue. We thought that by restoring that service we would be able to then reconnect all our customers.
Mrs Bayer Rosmarin : But it turned out not to be the case—
Mr Kanagaratnam : Not to be the root cause.
Senator HUGHES: Right.
Mrs Bayer Rosmarin : So there was no way around the outage by changing settings.
Senator HUGHES: I thought you said that there was a way to do that.
Mr Kanagaratnam : There was a limited amount of customers who, if they'd done that, could have restored—very limited.
CHAIR: Senator Grogan?
Senator GROGAN: I'll put the other questions on the communications piece on notice to see if we can get a tighter response. I want to take you to the processes that you may have in place for network outages. Obviously, they were comprehensive. Are your plans more tailored to a particular section of the network going down?
Mrs Bayer Rosmarin : Our plans have to cater for a very wide range of possible network outages across different networks, geographies, localised, whether they're at the core or the periphery. I believe our plans are structured in a way that really empowers the right group of people to take the steps necessary to restore, communicate to customers and escalate through the chain of command, as required.
Senator GROGAN: Do you have a plan in place for a full outage like you had the other day?
Mr Kanagaratnam : I'll have to take that on notice. We did do a network outage exercise in October, but it wasn't for a full outage of the network.
Senator GROGAN: Do you do disaster scenario planning, and did you have a plan in place for something like the extent of the outage you had on 8 November?
Mr Kanagaratnam : We didn't have a plan in place for that specific scale of outage. It was unexpected. We have high levels of redundancy and it's not something that we expect to happen. It is certainly something we commit to learning from this outage, and we will take such exercises into consideration in the future.
Mrs Bayer Rosmarin : It might be helpful if Lambo could explain, for example, how much redundancy we have in each of those networks and why this was so unusual. I think that would help everybody's understanding.
Senator GROGAN: I think that's right, because in your submission you talk a lot about the inherent risk that there is for an outage in your line of work, which is fair enough. But you talk a lot about resilience and hardening measures, multiple layers of redundancy and multiple layers of protection. Why did none of those work?
Mr Kanagaratnam : I'll start, as Kelly suggested, by explaining what redundancy we have in place. We have multiple layers of redundancy in the network. If we look at how we connect the services to our customers, we have multiple exchanges or sites across the country and so we segregate different parts of the network according to that. At any time, if one exchange is isolated, it will only impact a certain amount of customers if the whole exchange is lost. Within that exchange we have multiple layers of redundancy in terms of connectivity. So we would have multiple routers—at least two routers—connecting each of the services that we connect. So, for example, if it's a fixed service for our consumer broadband customers or for the mobile voice network or the mobile data network, we'll have at least two routers providing connectivity to each of these services. Then what we do for mobile voice data and fixed voice is that we provide geographical redundancy so that traffic can switch seamlessly across the country. Again, to another exchange where we have multiple layers of redundancy.
For all our intercity fibre between our major cities, we have at least three different routes of connectivity, and we have multiple other layers of redundancy. The specific reason why we had this major outage was, as Kelly said and we've explained, there were almost 90 routers across the country that were impacted by the single issue that we had where a preset safety limit on these routers was reached or exceeded. What then happened was they all shut down, and effectively disconnected all these levels of redundancy, and effectively created the outage that we had.
I just want to come back to the initial question about Singtel, Optus and accountability. I want to make it very clear that accountability does lie with Optus. The outage was as a result of our defences for the change in routing information not working as they should have. That's something, obviously, we've addressed, and we've worked around the clock to ensure that we address all the issues that we've found. Again, I'll ask that you indulge me: I just want to say thank you to my team and all the partners that we worked with to make sure that we did that in the quickest time possible.
CHAIR: I have a question in relation to that particular point. Obviously, routine patch upgrades are common—common for Optus and common for many companies. Did you test the update before you rolled it out across the whole system?
Mr Kanagaratnam : Let me explain. The upgrade was done on what we call an international peering network, which is run by Singtel, or STiX. I wouldn't say that these upgrades are done frequently, or that they're routine, but they are done. Our network should have been able to deal with the—
CHAIR: When you allow these types of upgrades to happen, is there a test?
Mrs Bayer Rosmarin : If I may: this is an upgrade that's done by a partner, not by us.
Mr Kanagaratnam : Not by us.
Mrs Bayer Rosmarin : We get notified, but we have no ability to influence or test that. Our network has to be designed to cope with the redirection, or the diversion, away from where the upgrade is to an alternative link. It's actually one of those backup and redundancy options. When one of those links was being upgraded and was therefore not available, what was coming through that link needed to be diverted through another link, which happened to be configured differently. It then propagated through our network in a way that triggered these failsafes in each of the different routers.
CHAIR: So you don't necessarily have full control of your core routers at a time like this—is that what you're saying?
Mr Kanagaratnam : I just want to clarify that we connect to multiple international networks. That's—
CHAIR: Yes.
Mr Kanagaratnam : That's the way that our customers can reach all the different points on the internet, especially where they have to reach international destinations. All these networks work autonomously, but they connect in a standard and defined way, using what we call internet protocol. Those upgrades are done by all our partners. In this case, we should have had in place the defence mechanisms to ensure that any change in their networks don't impact our network.
CHAIR: But you didn't have that.
Mr Kanagaratnam : On that day, unfortunately, our defence mechanism did not work as it should have. The scale of the outage, obviously, was something that we didn't anticipate, because of this failsafe limit that was set.
CHAIR: I've got some other questions in relation to the routers and that system—we may need to put them on notice so we can get clear answers—but I want to come to the issue of customers not being unable to contact triple 0. We already heard before that when the minister was spoken to at 8.30 you were unaware that Optus customers were not able to contact triple 0. I just want to make sure we're correct about that, because that was in evidence that was given earlier this morning.
Mrs Bayer Rosmarin : Yes, we were not aware that any Optus customer was not able to call triple 0. We had full faith that the triple 0 system would be working at that time. I just want to make sure—there was something inherent in your question, where you said 'no Optus customers': at no time was it 'no Optus customers' who could. The experience varied depending on which device people were calling from, when we learnt of the issue later in the day.
CHAIR: Optus customers were not able to access triple 0 and you didn't—
Mrs Bayer Rosmarin : That was what I was trying to correct: most Optus customers were able to access triple 0.
CHAIR: How many Optus customers weren't able to access triple 0?
Mrs Bayer Rosmarin : There were 228 triple 0 calls that were unable to go through. We have done welfare checks on all of those 228 calls and, thankfully, everybody is okay.
CHAIR: When did you do the welfare checks?
Mrs Bayer Rosmarin : We started doing the welfare checks after our connectivity resumed.
CHAIR: Under the Telecommunications (Emergency Call Services) Determination ruling, you are required to do a welfare check on people who tried to call during the large network outage. You are saying you've done that for 228 people?
Mrs Bayer Rosmarin: I can triple-check the number and get back to you, but I believe it was 228.
CHAIR: Did you tell the emergency call services and other providers that you knew that customers were not able to get access to triple 0? Who did you inform?
Mrs Bayer Rosmarin: Thank you for the question, Chair, because we absolutely believe that the triple 0 system should have worked and it's critical for all Australians that that system can be relied upon. We don't manage the triple 0 system. It's a very complex system that involves all the carriers, it involves the device manufacturers and it involves the triple 0 operators.
CHAIR: Why couldn't people access triple 0?
Mrs Bayer Rosmarin: We are still investigating that, and we're really happy that the ACMA have called an investigation into why this did not work.
CHAIR: At this point you still don't know why Optus customers were not able to call triple 0.
Mrs Bayer Rosmarin: We've done some investigations into that, but we can't investigate that ourselves, because of the complex interrelationships, so we are really grateful that the ACMA has called an inquiry into this. We're going to work with them. We had a whole stream of work that was going on as soon as we become aware that some Optus customers weren't getting through. We were trying to solve that simultaneously to the outage. We weren't able to at the time. We do want that investigation to happen expeditiously and we're cooperating with that fully.
CHAIR: Do you think there should be a penalty because of that? It's a pretty serious breach of the emergency services protocol.
Mrs Bayer Rosmarin: I think it's too early to tell where the issue actually occurred. The triple 0 system—
CHAIR: So you think it's somebody else's fault?
Mrs Bayer Rosmarin: No. The triple 0 system is supposed to be able to pick up the traffic when we have an outage like this, so it is very important—
CHAIR: Piggyback.
Mrs Bayer Rosmarin : Correct, and if someone else has an outage, we should be picking up some of the calls. That's how the system should work. So we definitely—
CHAIR: But surely at Optus you're not suggesting that it's somebody else's fault?
Mrs Bayer Rosmarin: We're absolutely accountable for the outage.
CHAIR: So shouldn't there then be a penalty on Optus for failing to provide critical service to triple 0?
Mrs Bayer Rosmarin: As I've tried to explain, we don't run the triple 0 system. We participate in the triple 0 system, and the system—
CHAIR: But it's not anybody else's fault that your customers couldn't call triple 0. Surely it's Optus's fault, so isn't it Optus that should pay the fine or cough up the penalty?
Mrs Bayer Rosmarin: We, like you, are extremely passionate about the fact that our customers should have been able to call triple 0 even if our network was down.
CHAIR: So why don't you take responsibility today and say you will cop a penalty?
Mrs Bayer Rosmarin: As I've mentioned, we really want this issue to be thoroughly investigated, because multiple parties have to work together to make sure that, when there's an outage, the triple 0 calls still go through.
CHAIR: It sounds like you want to share the blame around.
Mrs Bayer Rosmarin: We absolutely take accountability for the outage. What I'm trying to explain is that the triple 0 system itself should have helped our customers during our outage. And we will take accountability for any role we've had in that, but there are also probably changes that need to be made in other parts of that system, potentially with different devices, because that—the device settings and, possibly, the way that works with the system—was the determinant of whether you could or could not complete the call. We are committed to investigating that thoroughly, and we really do want to get to the bottom of it, not just for us but for any time that there's an outage so that the system works.
CHAIR: I think this is going to wear thin. I think we're just going to have to cough up, accept responsibility, apologise and cop a penalty, surely. Anyway, I'm going to will give the call to Senator Hughes.
Senator HUGHES: I have just one question. I want to come back to something you just mentioned, Lambo, when we were talking about having backup plans and businesses being prepared for that. We know that there were a number of government departments that were hit: Home Affairs, the tax office, Workforce Australia. There were a number of government departments that lost services, and I would be very interested to talk about how Border Force was impacted. None of them appeared to have a backup plan, because they weren't up and running. They didn't go to an eSIM, or whatever they could have gone to as a contingency. How does that work? Why does the government not have in place a backup contingency plan should some pretty significant departments experience this sort of outage?
Mrs Bayer Rosmarin : If you don't mind, I'd like to take that question on notice, because for each customer that we deal with, we offer bespoke solutions that are designed for their needs. Some of our corporate networks were up and running and were not affected by this outage. I do not have information at hand for each specific customer as to what was and was not working, and what their backup plans were.
Senator HUGHES: Was there any communication between Optus and the Minister for Home Affairs or her office? She's responsible for Border Force. They have a pretty important role to play in keeping Australia safe, yet they did not have coverage on that Wednesday. I'm particularly interested in whether there was any communication between Optus and the Department of Home Affairs and the minister's office.
Mrs Bayer Rosmarin : Yes, I can confidently say there was communication with her office. Our account managers were in communication with all our customers that day—both corporate and government—to make sure they were being kept abreast of everything that we knew. I also personally reached out to Minister O'Neill, and our government affairs team was in contact with her office, too.
Senator CADELL: My understanding of the network is that you have Evolve internet as the corporate base, and then you have the cellular network and consumer mobile all linked to the network core. Is that the correct structure that you have?
Mr Kanagaratnam : Evolve is the enterprise or business network, and that's a separate network to the one that was impacted on Wednesday.
Senator CADELL: But it operates on the same network core.
Mr Kanagaratnam : No, they operate on two different sets of network cores. They are interconnected.
Senator CADELL: You talked about these third-party partners, and mentioned Singtel. Cisco is a great brand, and we're saying they were the routers that failed. Can you tell me what Akamai is? What role do they play?
Mr Kanagaratnam : Akamai is one of the largest content delivery networks. They aggregate content from multiple sources and deliver it into our network. They're one of the largest content delivery networks globally. They peer into our local network.
Senator CADELL: What is the role of Nokia?
Mr Kanagaratnam : Nokia provides multiple services for us. They are an equipment provider on our radio network and also our IP and transport networks. They also provide us with managed services for our network.
Senator CADELL: Did Nokia have any role in this software upgrade as part of the Singtel spike?
Mr Kanagaratnam : No, Nokia were not involved. As far as I know, Nokia were not involved, and I don't expect that they were involved.
Senator CADELL: Was the problem around a prefix filter that meant your Cisco router was overloaded?
Mr Kanagaratnam : The defence on our connectivity to the STiX network—yes, there was an issue with the prefix filter.
Senator CADELL: So the prefix filter was what overloaded the 90 Cisco routers? It was grabbing data from all over the world and grabbing traffic—
Mr Kanagaratnam : The defence mechanism that we should have had in place was exactly the filter that we should have had in place. But even if that failed, we wouldn't have expected such a large-scale outage. For us to lose 90 routers in one outage is not something that we contemplated.
Senator CADELL: I understand that. What role did Nokia play in trying to get these going? Did Nokia play any role in trying to get these going from a remote perspective?
Mr Kanagaratnam : Yes. Nokia is our managed services partner for our network, and they were involved from the beginning in managing and running the incident and recovering the network with us.
Senator CADELL: Where were their staff based?
Mr Kanagaratnam : Their staff are based in India in two locations.
Senator CADELL: When we went through the timetable of when we were showing up, even you couldn't access these things remotely because they were offline. None of these routers were able to be physically rebooted unless you were physically plugged in and connected to it.
Mr Kanagaratnam : That is correct, yes.
Mrs Bayer Rosmarin : To clarify, that was correct in this outage. We have a system that should enable remote—
Senator CADELL: But it did not.
Mrs Bayer Rosmarin : That was down as well.
Senator CADELL: It did not; that's what we're saying. So we were relying on overseas people in India. Your system relied on an Indian firm to outsource this problem to fix these things remotely when we couldn't do it across your entire network.
Mr Kanagaratnam : If I could explain, even if we had engineers in Australia, we would have had the same issue and the same challenges. For us to reset—maybe I can give you some context. We had 90 devices across 14 locations that were impacted. In addition to about half of those 90, we had to reboot 50 other network elements to restore connectivity.
Senator CADELL: I understand that.
Mr Kanagaratnam : So there was a total of 100 devices across 14 locations that we had to reboot and restore connectivity to.
Senator CADELL: When you saw that the first ticket was issued—I think at 4.20 or 4.05, you said—and a physical person showed up, is it true that they didn't have the right console cable to connect and reboot with and see if that was a problem at nine o'clock in the morning?
Mr Kanagaratnam : Sorry—you referred to a 'console cable?
Senator CADELL: A console cable to connect to the Cisco router .One of your people showed up at one of your locations to test if that was a problem and didn't have the right equipment to be able to connect to it.
Mr Kanagaratnam : They had some issues in terms of connecting to the console, but that was because there were various issues in terms of connectivity into the Cisco routers.
Senator CADELL: Telstra had an outage on 8 May over a software upgrade that caused unexpected errors. Did you game that with your own system to see what could happen to you if you were exposed, after Telstra's problem in early May?
Mr Kanagaratnam : We asked our partners who provide equipment to Telstra what happened, and they were obviously not willing to share details. But we have investigated that and put in place measures to ensure that we didn't have the same issue.
Senator CADELL: You said you had plans earlier for disaster recovery—not on this scale—but have you ever gamed them, or do you just rely on them being written and hope they'll work?
Mr Kanagaratnam : As I said, in October, we ran a scenario where we lost one of the states, WA. Also, in the same scenario, we did an assessment of a potential attack on one of our exchanges in South Australia.
Senator CADELL: How many senior network engineers does Optus employ?
Mr Kanagaratnam : I'll have to take that on notice. We have a team of 1,000 staff, and, on that day—
Senator CADELL: I'm not asking that question. I am asking about network engineers at the CIO sort of level—the technical side.
Mr Kanagaratnam : I'm the head of the network team, and I have a team of about 12 reporting to me.
Mrs Bayer Rosmarin : We'll take it on notice, because we do have a lot of engineers on staff, and we'll get back to you.
Senator CADELL: Seeing an external threat or software upgrade, do we still use any Huawei devices in the network?
Mr Kanagaratnam : We still do use Huawei equipment in the network, yes.
Senator CADELL: Is that what's used as termination devices to connect to the fibre?
Mrs Bayer Rosmarin : The Huawei devices are used on the periphery; they're not used in the core. So they were not involved in this incident.
Senator CADELL: I'm not asking that question. Are they used as network termination units to connect to the fibre?
Mr Kanagaratnam : In certain locations, they are being used. But, as I said, we are replacing them progressively across the network.
Senator CADELL: When was your network architecture last reviewed?
Mr Kanagaratnam : Our network architecture is reviewed on a consistent basis.
Senator CADELL: When was the last major review?
Mr Kanagaratnam : I'll have to come back to you on that one, but, as I said—
Mrs Bayer Rosmarin : Sorry; I can tell you that, in addition to reviewing it regularly ourselves, we also, from time to time, have third-party experts review and help us assess whether our resilience is as strong as it should be and whether the steps we're taking are adequate.
Senator CADELL: When was the last 'time to time' that happened?
Mrs Bayer Rosmarin : I'm aware that 18 months or two years ago we had Bell Labs assist us with ensuring that our network architecture and resilience were as strong as they could be.
Mr Kanagaratnam : We do regular testing of our failover. We do that when we do software upgrades or isolate nodes to do work on them. We've identified over 42 tests that we do on a regular basis to test the resilience and redundancy of the network.
Senator CADELL: Are Singtel, as an owner, overly demanding on dividends or capital returns to them?
Mrs Bayer Rosmarin : I think Singtel is a fantastic shareholder. They've invested more than $42 billion in the networks in Australia for very low returns.
Senator CADELL: A criticism from others in the industry is that Optus has taken to making an art form outsourcing what should be core competencies and that there is low telco experience in your CIO, or not a lot of telco experience. What confidence can we have? We're using Cisco systems, we're using external sources, and we've got a SingTel router. What systems do we have for security and control of your network in any way, shape or form?
Mrs Bayer Rosmarin : I really appreciate the question, because I think it's very important to say that we do outsource a number of components of our network management to global, leading companies and it is something that I do think we should look at as to whether we have the right level of outsourcing and insourcing. It is something that has been on the minds of the team—that we've already been thinking about. That is something we will look at further.
I can tell you that our Networks team is highly experienced. There are a number of engineers who've been network engineers for many years. They are a very hardworking team. They are absolutely devastated about what happened, and they go above and beyond to try and keep the network operating all the time. We do have a lot of experienced people. We take our responsibility to keep the network up and running incredibly seriously. It's in everybody's interest—our customers', our own, our shareholders', the community's. We do everything we can to avoid a situation like this, and we're very apologetic that it happened, but I don't think it should cause people to doubt just how much effort, attention, resourcing and financial commitment we have to resilience.
Senator CADELL: As the CEO, are you not aware of the low morale of the tech team because they feel it was dumbing down the internal abilities of the company in the IT sector, in the network sector? Do you think there was a general panic or disillusionment when this happened that they weren't able to respond to it on the ground?
Mr Kanagaratnam : I think maybe Kelly can I talk to this.
CHAIR: Just to be clear, Senator Cadell did ask the CEO the question about staff morale. I think it's important for her to answer that question.
Mrs Bayer Rosmarin : I'm very happy to answer that question, because we don't just wonder about staff morale but we also do staff engagement surveys. We actually got our results of the most recent staff engagement survey about three weeks ago, and the engagement of our teams was the highest it's ever been for Optus at 77 per cent.
Senator CADELL: In the technical space?
Mrs Bayer Rosmarin : Lambo, what was it in Networks?
Mr Kanagaratnam : My team's engagement score was 80 per cent. It was the third highest in the organisation.
Mrs Bayer Rosmarin : We do believe that the morale of the teams is strong and improving.
Senator CADELL: When you talk about the openness and the willingness to go forward, will you commit to releasing the Deloitte report on the cyberattack, or will you be appealing the court's decision on that?
Mrs Bayer Rosmarin : Thank you for the question. As you can appreciate, the Deloitte report contains a forensic investigation into our cyberdefences and is highly sensitive in its material. We are conscious that that attack is still the subject of an ongoing federal criminal investigation and that the contents could be so sensitive that they expose us to national security concerns, so the team will be looking at whether they appeal that decision.
Senator ROBERTS: Thank you, Mrs Bayer Rosmarin and Mr Kanagaratnum, for being here today. The first question I have has to do with your stakeholders. You have stakeholders in terms of your employees, your shareholders, your customers, the government and your service providers. Is there anyone else?
Mrs Bayer Rosmarin : Yes—and the community at large.
Senator ROBERTS: And you have a responsibility for communication. What I see is that your main priority was identifying the cause and being a leader for your people. Is that correct?
Mrs Bayer Rosmarin : My main priority was—
Senator ROBERTS: Early in the incident.
Mrs Bayer Rosmarin : My immediate priority early on in the incident was to make sure that all the resources of Optus were adequately focused on the jobs they needed to do to restore services as quickly as possible, take care of customers and keep everybody apprised of as much information as we had.
Senator ROBERTS: So you delegated the responsibility for initial communications, prior to finding out what the cause was and how long you'd be out, to your media people?
Mrs Bayer Rosmarin : That is correct.
Senator ROBERTS: And you got communications to the media at 6.33 am?
Mrs Bayer Rosmarin : Yes, that's correct. That was the official statement where we proactively sent it out to media.
Senator ROBERTS: You were focused on root causes and focused on the leadership of your team, and you left the media to someone else because that was the right priority?
Mrs Bayer Rosmarin : At that point in the morning, that is what we felt was best for the team—yes.
Senator ROBERTS: Thank you. That's what appeared to me to be the case. What responsibilities does Optus have to the minister and to ACMA?
Mrs Bayer Rosmarin : We obviously have a responsibility to work together with the minister and with our regulators to make sure that the Australian public has great connectivity, that we fully comply with all the expectations of the public, and that we have a strong, competitive sector where our customers benefit from choice, innovation and a range of services.
Senator ROBERTS: Once you identify the causes of this outage and you do a review of your response, will that review be given to the minister? Will it be communicated to her?
Mrs Bayer Rosmarin : The communications minister has already announced that they'll be doing a full and thorough review into the incident, and we are intending to fully cooperate with that and give them all the information that they request.
Senator ROBERTS: With so few providers in the telco space, I imagine that you all work fairly closely with ACMA and the minister. Is that correct?
Mrs Bayer Rosmarin : That is correct.
Senator ROBERTS: I'm also very concerned about the level of government intrusion across our country. I'm not talking about the Albanese Labor government specifically; I'm talking about governments in this country seeming to intrude a lot. Did the national governments affect you at all in terms of the cause, the recovery or the future? Do you have any concerns about the intrusion of government?
Mrs Bayer Rosmarin : On the contrary, I thought our conversations with government were very important. Government was there to help and support. As was mentioned earlier, Minister Rowland was on TV early, speaking to customers; that was very helpful. There was no area where I could say that I felt we were in any way impeded from doing the right thing and getting on with the job.
Senator ROBERTS: Speaking outside of this outage, is there any legislation that is needed for you to do a better job or any legislation that is currently impeding you and needs to be removed?
Mrs Bayer Rosmarin : If you wouldn't mind, I would like to take that on notice and chat to my broader team before responding.
Senator ROBERTS: Sure. Mr Kanagaratnam, I think it was you who mentioned the attacks. How often do you get cyberattacks, and what size are they?
Mr Kanagaratnam : Cyberattacks are an ongoing threat and challenge. We have to continue to defend against them. I don't have specific information in terms of the number that we have, but it's an ongoing threat and issue that we have to deal with and defend against all the time.
Mrs Bayer Rosmarin : Our cybersecurity reports to our chief information officer, Mark Potter, not to Lambo, but I know we have millions of cyberattacks every year that we defend against.
Senator ROBERTS: Thousands a day?
Mrs Bayer Rosmarin : I could get back to you with the exact numbers.
Senator ROBERTS: Yes, please. You mentioned, Mr Kanagaratnam, the service that Nokia provides from India. I've also recently been made aware—it's apparently a fact—that communication through the internet from the east coast of Australia to Western Australia is via China or via the United States, and it's the same with the communication from Western Australia back to the east coast. It's via those two countries—predominantly through China. Is that correct?
Mr Kanagaratnam : No, that's not correct. We have both terrestrial, as in fibre that traverses the land, and submarine cable connectivity, which we own and operate on our own, to connect the east coast and west coast of Australia.
Senator ROBERTS: So they're connected entirely on our continent?
Mr Kanagaratnam : That's right. To answer your question in terms of Nokia being offshore, they have a combination of resources offshore and locally, and we supplement that with our own team. One hundred per cent of our engineering network deployment or rollout and field teams are in-house or run by our team, and about 25 per cent of our operations assurance team is also onshore and run by Optus.
Senator ROBERTS: Apart from your service with Nokia, are there any connections that require overseas routes?
Mr Kanagaratnam : There would be to our offshore call centres and to some of the IT support teams globally. This is a practice that most companies—not just telcos—run, in terms of how they run their operations.
Senator ROBERTS: Mrs Bayer Rosmarin, are there any market-power or market-pressure concerns that you have, and do they impact on your company?
Mrs Bayer Rosmarin : Clearly we're a small industry with a dominant incumbent provider and only two other providers, who are much smaller. There are definitely some conversations we have on a continuous basis to make sure that the market is competitive, dynamic and fair.
Senator ROBERTS: So you have conversations with the minister?
Mrs Bayer Rosmarin : We do have conversations with the minister, of course.
Senator ROBERTS: On that topic?
Mrs Bayer Rosmarin : Yes.
Senator ROBERTS: Coming to compensation to customers who lost money, I understand you've offered customers a hundred dollars of data but it must be used by 30 December, is that really going to be significant to them?
Mrs Bayer Rosmarin : Thank you for asking for clarification on that. For our postpaid customers we've given them 200 gigabytes of data to use over the holiday period. I believe that's for a few months, but they have to have activated it by December. Depending on when they activate it they will get it for a couple of months—the idea being that they have access to that data over the holiday season, when people do tend to use and consume more data. For our prepaid customers, we've given them unlimited data on weekends through to the end of the year. For our fixed customers we've given them a free speed upgrade on their NBN to the end of the year. And we have invited anyone who has unique circumstances and wants us to look into those individually to contact us.
Senator ROBERTS: What about compensation for people who couldn't settle, for example, a house purchase, because I understand a lot of settlement is done electronically now. A lot of services in general are done electronically.
Mrs Bayer Rosmarin : On the issue of consequential losses, we felt that this was an issue that's much more broad and that, should government choose to look into this, we'd love to be part of that conversation. But there is no precedent for telcos or other essential providers covering consequential losses, and we are very conscious that this would have far-reaching implications not just for Optus and not just for all telcos, including the NBN, but also for other essential services, utilities and government services, so this needs to be a much broader conversation than us unilaterally determining how to go about that.
CHAIR: Sharing the blame around.
Senator DAVID POCOCK: I'm interested to learn about the financial impact on small businesses and whether you've modelled the impact that this had on small businesses across the country.
Mrs Bayer Rosmarin : As you would appreciate, every small business is different. The impacts are different on all of those operations. Some were able to take different actions. So it's not a task that we would be able to do.
Senator DAVID POCOCK: What have you tried to do to gauge the lost income or the impact it's had on small businesses?
Mrs Bayer Rosmarin : At the moment what we've done is ask for each of those businesses who have concerns to reach out to us and then we'll speak to them, understand what has occurred and also make sure that we can help them to be set up in the future.
Senator DAVID POCOCK: How do you contact those small businesses to do that?
Mrs Bayer Rosmarin : Some of our customers have account managers and we also have a dedicated business line.
Senator DAVID POCOCK: How many small businesses have contacted Optus seeking compensation for income lost during the outage?
Mrs Bayer Rosmarin : I do have that, but it's on my phone, if you can give me a second to look.
CHAIR: Let's hope it's working.
Mrs Bayer Rosmarin : I'm confident that it is working.
Senator HENDERSON: It seems a bit slow.
Mrs Bayer Rosmarin : I think that's me. We do have the fastest 5G.
CHAIR: When it's up.
Mrs Bayer Rosmarin : Which is most of the time. We've had 8,500 customers and small businesses reach out. So far, they've been discussing with us around $430,000 in compensation, and we've already applied $36,000.
Senator DAVID POCOCK: So $430,000 across the board, cumulatively, and you've paid out $36,000.
Mrs Bayer Rosmarin : So far. I don't have any details into the veracity of those claims and what's happening. That's the information that I have.
Senator DAVID POCOCK: What's the process to verify the veracity of claims and then pay out the remaining $430,000?
Mrs Bayer Rosmarin : I don't want to make any commitments on how much we are going to pay, Senator. What I am committed to, and what our teams are committed to, is to have a process to individually engage with each customer on their unique circumstances, to evaluate what's the right thing to do for that customer and to work with those customers to help ensure that they're set up well into the future.
Senator DAVID POCOCK: Have you established a dedicated small business call centre team to be able to deal with this, or are they just going through the usual avenues?
Mrs Bayer Rosmarin : We have a dedicated business centre for our customers to call into, and so that is the team who is managing this as well.
Senator DAVID POCOCK: Do you know how many small businesses were impacted by the outage in terms of payments?
Mrs Bayer Rosmarin : I'll take that on notice and get back to you, because some of the payments were working. It's only about 50 per cent, as I understand it, of payment terminals that weren't working or didn't have the right back up.
Mr Kanagaratnam : If the payment terminals were only reliant on the Optus mobile network, they would have been down. Otherwise, from what I understand, if they had the ability to have either alternative connectivity or to switch to another mobile provider, then I think they would have remained operational.
Senator DAVID POCOCK: Okay. On notice, can you give me the number? I'm interested, in terms of these sorts of outages, if you have any insurance or how it works? What do you see as your liability as a company when small businesses are relying on you to be able to do business and they're compromised?
Mrs Bayer Rosmarin : That liability is the same as any provider of essential services, utilities, government services. Our contracts are designed in a way that we are clear that we are providing our coverage on a best endeavours basis, and we recognise that outages can occur. We work very hard to make sure that they don't, but they are provided on a best endeavours basis.
Senator DAVID POCOCK: If 'best endeavours' is in the contract, why have you paid out compensation to small businesses then?
Mrs Bayer Rosmarin : I don't like to use the word 'compensation' because what we're doing is assessing the specific scenario of that customer and trying to do the right thing by that customer, who we want to have a long-term relationship with us.
Senator DAVID POCOCK: Finally, I'd like to confirm that you have paid out cash and not just in-kind services?
Mrs Bayer Rosmarin : I can double-check that for you.
CHAIR: I think that's an important point to clarify. You've given us the figure of $36,000 paid so far. We do need to know whether that is in services or cash terms. Do you have any sense that that's a combination of the two?
Mrs Bayer Rosmarin : I don't have any information right now, but I'm happy to take that on notice.
CHAIR: Thank you. Senator Grogan has a follow-up to that.
Senator GROGAN: With the conversations you're having with small business, if the offer you make to them is not acceptable to them, what recourse do they have?
Mrs Bayer Rosmarin : Our customers have recourse to the Telecommunication Industry Ombudsman. We have had a meeting with the ombudsman and we will work closely with them to make sure we have a process to deal with that in a timely and efficient manner with them.
Senator GROGAN: Have you had any that have been unhappy with the offer so far?
Mrs Bayer Rosmarin : I don't know.
Senator GROGAN: Okay.
CHAIR: Thanks, Senator Grogan. Just to clarify: you've said there have been claims or complaints or requests of up to 430,000 so far. I just want to clarify that figure.
Mrs Bayer Rosmarin : Yes, that's what I've been given by my team. Whether they're discussions, claims, complaints I can't be more specific, I'm afraid.
CHAIR: But you don't like the word 'compensation'?
Mrs Bayer Rosmarin : Chair, the only reason is because I do think that there's an important conversation to be had across government and industry about contingent loss because of the far-reaching implications that will have for all essential service providers, for all telcos, for all utilities, for all government services. I think that conversation is important because it will impact all of those organisations and the cost of those services moving forward. We would really welcome being a part of that conversation, but we don't see it as our place to unilaterally lead that.
CHAIR: I'm going to go to Senator Cadell, but I have one follow-up to a question I was asking earlier in relation to your own and your team's backup SIMs. Who's the provider for your backup SIMs?
Mrs Bayer Rosmarin : I now have all three. I used to have Vodafone, but I have Telstra now too.
Mr Kanagaratnam : I have a backup SIM from Vodafone.
Mrs Bayer Rosmarin : We do vary it. We don't specify it across the company, for obvious reasons.
CHAIR: If Optus goes down, you get to use Telstra or Vodafone?
Mrs Bayer Rosmarin : Yes.
CHAIR: It would be good if the public could easily swap, wouldn't it?
Senator CADELL: Domestic roaming. Hear, hear. Kelly, you earlier stated that you had Bell Labs do a review of architecture 18 months or two years ago. We've heard from Lambo that Nokia are a massive contractor in your network. Who owns Bell Labs?
Mr Kanagaratnam : Bell Labs are part of Nokia globally.
Senator CADELL: Correct, so they're marking their own homework. You're getting Bell Labs to mark their own services to you.
Mr Kanagaratnam : I can clarify: Nokia is one of our providers—
Senator CADELL: A significant one. You said what they do.
Mr Kanagaratnam : and so are Ericsson, Cisco and several others.
Senator CADELL: I understand all of that.
Mr Kanagaratnam : They are a very proficient, experienced and professional provider of this service.
Senator CADELL: They couldn't fix this though, could they?
Mrs Bayer Rosmarin : Senator, the fundamental premise of what you're saying is correct. Even though we'd done very detailed reviews into our architecture, processes and risks to make sure we could be as resilient as possible, we did not have an articulated, clear risk that each one of our 90 routers would independently shut itself down at the same time using a fail-safe on those Cisco routers that our team was not aware of. That specific risk that caused this outage was not identified in any of our reviews.
CHAIR: Senator Henderson, do you have any questions?
Senator HENDERSON: Mrs Bayer Rosmarin, this morning there was a media report that you intend to resign as CEO. Is it your intention to resign?
Mrs Bayer Rosmarin : I'm sure you can appreciate that my entire focus has been on restoring the outage issue and working with the team. It has not been a time to be thinking about myself.
Senator HENDERSON: Could you address the question. Are you intending to resign?
Mrs Bayer Rosmarin : I thought I answered the question. My focus is on the team, the customers and the community. My focus is not on myself.
Senator HENDERSON: So that report is not correct?
Mrs Bayer Rosmarin : I haven't seen any reports today. I've been preparing for being here.
CHAIR: Senator Henderson, do you want to ask another question?
Senator HENDERSON: Yes. Mrs Bayer Rosmarin, can I ask you to explain a bit more about the peer network and the issues with Singapore Internet Exchange. You've explained that the network was overwhelmed by updates from a peer network, Singapore Internet Exchange. The affected Optus autonomous system has been reported in the media as peering not only with Singtel but also with Akamai, FLAG Telecom and China Telecom's ChinaNet. Could one of those other peer networks, such as ChinaNet, automatically update and perhaps adversely affect the Optus network as Singtel's network has done?
Mr Kanagaratnam : After the outage, one of the recovery steps we did was to understand, and forensically work with our partners to understand, what the trigger and the root causes were. We have applied the necessary protection to ensure that none of our peering partners could create a situation where a repeat of the outage could happen again.
Senator HENDERSON: Why are you certain that that's the case?
Mr Kanagaratnam : Because we have applied the necessary protection, we've worked with our partners to verify and validate, we've worked through a series of workshops and our team has worked around the clock to make sure that all of the different scenarios and requirements have been worked through and put in place.
Senator HENDERSON: Mrs Bayer Rosmarin, the outage was caused by the actions of an overseas based network, as we know. You've referred to, in general terms, some communications with the Minister for Home Affairs. Can you detail those communications? Has the Minister for Home Affairs been in touch to understand exactly how this occurred? And prior to the outage occurring, had the government taken any steps to seek to ensure that Optus was not vulnerable to an international provider bringing down the network?
Mrs Bayer Rosmarin : Firstly, to be completely clear, we are accountable at Optus for the network that we operate. Lambo has full autonomy in making decisions about how the network operates, and I'm accountable as the CEO of the company. So, whilst the trigger event happened in the international pairing network—and all telecommunications companies pair with other networks—it was our responsibility to make sure that nothing that happens in one of those pairing networks has an impact like this.
Senator HENDERSON: I want you to address the question. The question is: have the government taken any steps to seek to ensure that Optus was not vulnerable to an international provider bringing down the network?
Mrs Bayer Rosmarin : We work very closely with the government and our regulators whenever we're introducing network elements or working with others, to make sure the government is comfortable with who we are dealing with and how we're dealing with them.
Senator HENDERSON: Early last Wednesday did you try to call the Minister for Communications or did the minister try to call you, and what time was that?
Mrs Bayer Rosmarin : I called the Minister for Communications at 8.32 am, although my team had been in touch with her office prior to that.
Senator HENDERSON: What time had your team been in touch?
Mrs Bayer Rosmarin : I believe it was just before 8 am.
Senator HENDERSON: Who from your team?
Mrs Bayer Rosmarin : I believe it was Andrew Sheridan but I'll take it on notice to confirm that that's who it was.
Senator HENDERSON: What was the nature of your discussions with the minister?
Mrs Bayer Rosmarin : We had a reasonably brief conversation. It was very constructive. I explained what we knew at the time, which was unfortunately not much, and expressed our commitment to resolving the issue. The minister offered her full support, as needed, and we agreed that I would keep her updated during the day.
Senator HENDERSON: The minister went on radio I think at 7.30 and gave assurances. On what basis did she do that given that no communication with Optus had occurred at that time?
Mrs Bayer Rosmarin : At that time there was communication from Optus that confirmed that we knew about the problem and we were working on it.
Senator HENDERSON: You just said the first communication was just before eight o'clock with Andrew Sheridan.
Mrs Bayer Rosmarin : That was a direct communication between our offices, but the communication, more generally, the statement that we'd issued that we were working on it, was out there, so I believe she was fully justified to say that.
Senator HENDERSON: So the minister was relying only on the public statement issued by Optus?
Mrs Bayer Rosmarin : I think that would be a question for the minister.
Senator HENDERSON: Did the minister explain what action, if any, the government was taking to help bring this crisis to an end?
Mrs Bayer Rosmarin : I believe that she just offered her full support for us to focus on the issue.
Senator HENDERSON: Did the minister ask you about what government services or facilities were impacted by the Optus outage?
Mrs Bayer Rosmarin : Yes, she did.
Senator HENDERSON: What was your response?
Mrs Bayer Rosmarin : I explained that all of our mobile data, voice and fixed networks were down, but not all of our corporate networks. That's the information I had at the time.
Senator HENDERSON: To what extent were government services impacted, given your corporate networks were not impacted?
Mrs Bayer Rosmarin : It varied on a customer-by-customer basis.
Senator HENDERSON: I'd be grateful if you could provide on notice the details of the government services impacted. Did Minister Rowland ask for assistance in helping to deal with outages of government services to the extent they were impacted?
Mrs Bayer Rosmarin : She did not ask that of me, no.
Senator HENDERSON: Did she ask anyone else for assistance?
Mrs Bayer Rosmarin : I don't know.
Senator HENDERSON: Could you investigate that and take that on notice and come back to the committee?
Mrs Bayer Rosmarin : Sure.
Senator HENDERSON: Did Minister Rowland say to you whether the government had any contingency plans in place to deal with the loss of essential services such as the triple 0 service?
Mrs Bayer Rosmarin : Obviously, we have a lot of communications with government and it's probably not appropriate for us to keep going through what those communications—
Senator HENDERSON: It's very appropriate, because I'm asking you a specific question. Did Minister Rowland say to you whether the government had any contingency plans in place to assist with the loss of access to essential services. It's really important you answer that question.
Mrs Bayer Rosmarin : I don't recall that.
Senator HENDERSON: Did the minister advise whether she was talking to any other telcos, trying to arrange for other telcos to step in and provide services on an urgent basis?
Mrs Bayer Rosmarin : No, that was not mentioned. But that was something we also discussed in our first crisis meeting—whether there's even the ability to have customers go on another network on a scale like that—and it's not a capability that exists.
Senator HENDERSON: So what did the government do to actually help restore services, if anything?
Mrs Bayer Rosmarin : Keeping the Optus network up and running is Optus's full accountability.
Senator HENDERSON: Are you aware of any substantive action taken by the Minister for Communications with respect to how the government assisted Optus with the outage, to solve the outage?
Mrs Bayer Rosmarin : The government did announce an immediate inquiry into what had happened. That inquiry is broad and will cover other telecommunications networks as well. We think that's very constructive and helpful and we intend to cooperate with that fully.
Senator HENDERSON: Thank you very much. Thanks, Chair—no further questions.
CHAIR: Thank you. All of us at this table, and our parliamentary colleagues, have I imagine been inundated by members of the public who have been impacted by this—over 10 million customers. We've covered the serious elements of people not being able to access triple 0. That is very serious. We know that there are individuals who lost income because they weren't able to do their job throughout the day or have their own customers pay for services. I've had members of the public reach out and say that they weren't able to be in communication with family in the middle of brain cancer surgery, and people who weren't able to access childcare services because childcare centres wouldn't take people's children if there was no way of contacting them. Trains went out. My own daughter's school wasn't able to be contacted by the telephone because they use the Optus network. Do you really think that $1.60 is enough compensation for all of this?
Mrs Bayer Rosmarin : We completely agree with you. We understand how essential our services are to our customers. We are very sorry for the outage, and we know that there is nothing we can do or say that will give people back the time that they lost and some of the things that occurred on that day. That's why we tried to do something meaningful for our customers. We agree with you that just refunding people for the day with between $1 and $2 was not going to cut it, and that's why we went further, to offer those data offers to our customers.
CHAIR: Have you been surprised at the frustration that your customers have expressed at that data offer? I haven't heard anyone who said: 'Oh, great. Thanks, Optus.'
Mrs Bayer Rosmarin : We have a lot of customers, and I have heard a wide range of responses. There are customers who have written to me to say, 'Thank you for the extra data.' So there are a wide range of responses. But we do completely understand the frustration. It's in everybody's interest that we keep the networks up and running, and we are doing what we can to show our customers that we care, that we apologise, and that we appreciate their patience and their loyalty. The thing they want most from us is for this to not happen again and for us to keep the networks up and running.
CHAIR: What guarantees will you give today that this won't happen again? What have you done to make sure it won't happen again?
Mrs Bayer Rosmarin : I'm going to hand to my colleague Lambo to talk about the steps that we've taken. What I can guarantee you is that we will do everything we possibly can to try and prevent another incident like this from occurring and that we've taken steps to make sure that this same specific outage could not occur. Do you want to outline the steps, Lambo?
Mr Kanagaratnam : Yes. Since the outage on Wednesday morning we've been working forensically around the clock with our partners to firstly understand what the root cause was, what the trigger was and how we could increase the resilience of the network to the outage.
We have implemented four key changes on the network. The first one we've done is to make some changes on our international gateway routers that connect to our peering partners. That's to ensure that the routes don't propagate in the same way they did on that morning. The second change we've made is to increase the connectivity and resilience between our enterprise and business network and the consumer network that went down on that morning. We have also improved the availability of what we call the 'management network', which is redundant but we've increased the protection on it so it doesn't get impacted in the same way that it did when the outage occurred. And, finally, we implemented a networkwide change freeze to ensure that we don't have any other impact on the network during this period. That's so that we have stability.
CHAIR: What about an overhaul of your PR team and your crisis response management? That was one of the biggest problems which members of the public have raised with us. What all of us at this table have put to you today is that the communication with Australians in the midst of this was appalling. What have you done to fix that?
Mrs Bayer Rosmarin : Thanks for the question, Chair. There are always opportunities for us to do better. As I said in my opening statement, I wholeheartedly believe that the team did the best that they could with the information they had at the time and with the channels available. Having said that, there have been suggestions to us that we do press conferences versus one-on-one interviews and that our messages be of a different tone et cetera. We will take all that feedback on board to try to make sure that we do better in the future whilst, at the same time, trying to make sure there is no future where that's needed.
Senator GROGAN: I just want to follow through on that line: what it sounds like for us—I think I'm probably speaking for all of us—is that you're not hearing what we're saying. My understanding, or my experience—and I know it's the same for some of my colleagues—is that if you work for a large business then there's a lot of crisis planning and risk management work. On a risk management matrix, the situation that occurred on 8 November would sit down there as 'highly unlikely' but also 'catastrophic'. You would understand a risk matrix?
Mrs Bayer Rosmarin : Yes, absolutely.
Senator GROGAN: That would lead you, surely, as an organisation of your size, to have a plan for something catastrophic that would include a significant public relations piece? You have to assure your customers, who are concerned, confused and not knowing what's going on; they need to see a face or hear a voice so that they know, 'This is actually the company telling us what's going on,' or, 'They're telling us that they don't know but they're finding out.'
CHAIR: Showing some compassion and empathy!
Senator GROGAN: I haven't heard anything where it sounds like you've taken that piece and said, 'Oh, yes, that's one of the things we're looking at.' There has been plenty of commentary since 8 November about your public relations approach to this, but there's nothing in what you're telling us about how you're looking at improving, and the lessons you're taking from this—how you're going to improve into the future and how you're going to provide more surety for your customers. I'm not hearing anything about that—about the comms and also about risk management.
Mrs Bayer Rosmarin : I apologise, Senator. I have been trying to say that we're taking lessons on board about how we do that. We did follow the plans we had in place. One of the things you said has really resonated with me; you said that maybe people now want a face and a voice, and that a statement and our teams talking in the background via the media was less ideal than if we had had a real face and voice. That was what I had tried to bring to this from 10:30. I think that's a great suggestion and we will of course take that on board. I do want to just make sure that people understand there was no intention from Optus not to communicate. We felt that our messages were out there for the customers and that customers didn't know there was an outage and they didn't know we were working on it. As frustrated as everybody was that we didn't have more information to give, which is what people really wanted, about what the problem was and when it would be fixed, I can't tell you how frustrating that was on our side as well. We wish we'd had something more to communicate in those early hours. We are absolutely open to taking all the lessons and suggestions, and I appreciate your constructive suggestions which will feed into what we've learned moving forward.
Senator GROGAN: This takes me back to the management of risk. Obviously, at your company, there are a lot of customers who would be weighing up what they do next. That always happens. That surely is part of your risk management piece, in terms of the hard business end and not losing customers. And that's all the way down to the kinds of services that you provide to government, train networks, hospitals et cetera. Your risk management framework—your risk management plans and approach—is probably the piece that I'm most concerned about. How are you gaming this? How are you brainstorming this? How are you plotting out, 'How do we make sure that we're the most robust we can be in the face of an incident occurring that we have not predicted?'
Mrs Bayer Rosmarin : Thank you so much for that question, because we do have a very robust risk management framework. That framework does allow us to look at different scenarios, their impacts and how we plan to avoid those. That's quite extensive across a range of risks, as you would imagine, and something that we practice and evaluate regularly.
Senator GROGAN: When you say 'regularly', how regularly? I know when I was asking earlier, you said that you'd had one trial on part of the network earlier in the year. What is the sort of frequency?
Mrs Bayer Rosmarin : Each area does their own testing frequently, at their own frequency. As a company, we update that risk process at least annually. We have an executive risk committee that meets at least quarterly to go through everything that we need to do and all the risks that are arising. We have a very robust process there.
But I also want to go to the heart of your question, which is that we also have a very real lived experience as a company as to the implications when an unexpected risk occurs. That's because of the cyberattack, and the team has worked incredibly hard to regain trust by our customers since then. In the last 14 months, we've put ourselves back into a position of gaining market share in our core mobile product line. We have lifted our customer satisfaction back to the levels before that occurred. We actually achieved our lowest Telecommunications Industry Ombudsman complaints ever in the company's history last month. As a company, we completely understand the implications that come from one of those risks eventuating, and we've put in the hard work to recover from that once before. There's nobody in the company who would have wanted something like this to happen again—not just because we have risk management processes and strategies to go through but because we have a lived experience of it as well.
CHAIR: Senator Henderson.
Senator HENDERSON: I want to ask you about Minister O'Neil. Did Minister O'Neil, or anyone from her office, contact you or anyone in Optus regarding fears that this could have been a cyberattack or cyberthreat?
Mrs Bayer Rosmarin : I contacted the minister during the morning and made sure that she knew our teams were in contact. Her office and ours were in contact during the day.
Senator HENDERSON: At what time was that?
Mrs Bayer Rosmarin : I'll have to check the exact time. I believe it was around 9:30 that I spoke to her personally. But again, we can get back to you on exactly what time our office was in contact.
Senator HENDERSON: Were you able to advise the nature of the outage at that point in time?
Mrs Bayer Rosmarin : No. As I've mentioned, it took us until after the network was restored to be able to describe the cause of the outage.
Senator HENDERSON: I know. I appreciate that but, in terms of ruling out a cyberattack, what advice did you give the minister?
Mrs Bayer Rosmarin : We were only able to rule out a cyberattack at 10.20, and at that point, about 15 minutes later, I went on the radio and made that announcement publicly. We were able to communicate that to everybody after that time.
Senator HENDERSON: Did the Minister for Home Affairs provide assistance in any way up until the point of time when a cyberattack was ruled out?
Mrs Bayer Rosmarin : As I mentioned, we were in contact with the Australian Cyber Security Centre, which we work with closely and regularly. They were providing us with assistance.
Senator HENDERSON: Sorry—just to go back to my question: did the minister make any offer of assistance?
Mrs Bayer Rosmarin : I don't think specifically, but I'm sure she would've been aware that we would be working with the appropriate teams on that.
Senator HENDERSON: Can I ask whether you've received any notice of any legal claims, or are there any proceedings which have been instituted?
Mrs Bayer Rosmarin : Not to my knowledge.
Senator HENDERSON: Any threats of legal claims?
Mrs Bayer Rosmarin : Not to my knowledge.
CHAIR: Could you take those on notice.
Mrs Bayer Rosmarin : Yes, I could take those on notice and ask our legal team.
Senator HENDERSON: Thank you. Could I ask you about the PR and strategy advice you've received in relation to managing the fallout of the outage? What has been the cost of that PR and strategy advice?
Mrs Bayer Rosmarin : I'll have to take that on notice.
Senator HENDERSON: Are you also able to provide on notice to the committee a copy of all of the advice and recommendations you've received from your PR and strategy team and your government relations team in relation to today's hearing?
Mrs Bayer Rosmarin : I'm not sure I'd be able to do that, because many of them were in conversations and brainstorming sessions. In a crisis situation that is moving so fast, it's not as formal as tabling—
Senator HENDERSON: But, to the extent that you've got talking points or other briefing notes or background documents, could you provide a copy of those to the committee?
Mrs Bayer Rosmarin : I will certainly have a look as to whether there's anything like that, but I know, for example, when I did all my media, I did not have briefing documents. I was there to represent the company and—
Senator HENDERSON: Sure, but just leading up to today's appearance by you—if you could provide all of that material. My last question relates to mobile roaming. Did you investigate mobile roaming as an option to provide connectivity to Optus's 10 million Australians? When I was the shadow minister for communications, I passionately spoke about the importance of mobile roaming, particularly in rural, regional and remote areas. What steps did you take to investigate whether mobile roaming was an option for your customers?
Mrs Bayer Rosmarin : As I mentioned, in our first crisis meeting we did ask whether that was a possibility, and we understand that that's not technically feasible at the moment. For that to occur, there are a range of technical things that would need to happen between the different providers, and that is not currently in place.
Senator HENDERSON: Mobile roaming is used, in fact, at the moment, in relation to emergency phone calls where there's only one Optus tower or one Telstra tower in a rural area. So you currently have that technology. Can you just explain why that wasn't explored?
Mrs Bayer Rosmarin : Can you explain, Lambo?
Mr Kanagaratnam : Yes. As Kelly has said, we don't have a mobile roaming agreement with Telstra or TPG. TPG roams on our network in regional areas on 3G. However, we don't have a facility for our subscribers to roam on another network.
Senator HENDERSON: Did you investigate the possibility of putting that in place on an emergency basis with the likes of Telstra?
Mr Kanagaratnam : It is not possible unless you have all the pre-technical configuration set up, interconnectivity—
CHAIR: Is this something that you will now investigate, given you've told us you don't ever want to see this happen again?
Mr Kanagaratnam : That's something that we will need to discuss with the other mobile providers, and I'm not—
CHAIR: Is this something you will discuss?
Mrs Bayer Rosmarin : It's something we will discuss with the others, but I also want to just mention that there are some practical implications of that—
CHAIR: How about practically delivering the service to your customers that the customers need?
Mrs Bayer Rosmarin : For example, in this instance, if we had automatically roamed our entire customer base onto another network, that could have overwhelmed and congested and brought down that network if it hadn't adequately planned for that.
Senator HENDERSON: In other words, it is technically possible, but—
Mrs Bayer Rosmarin : No, it was not technically possible, but, in the—
Senator HENDERSON: Not at the time, but it is—
Mrs Bayer Rosmarin : theoretical—
Senator HENDERSON: Yes, that's right. It is technically feasible.
Mrs Bayer Rosmarin : What I'm saying is: it would require a lot of planning, coordination and work to be implemented on a practical basis.
Senator HENDERSON: Are you prepared to put that work in and those plans into place to guard against another possible outage, so that Australians will not lose their connectivity and suffer as much as they have?
Mrs Bayer Rosmarin : What I can say is: we will definitely take away the suggestion and work through it with the other providers.
CHAIR: I must say: I'm flabbergasted! We asked you previously: 'What will you do to make sure this doesn't happen again?' and this wasn't even one of the options you listed. Is it because you don't want your customers being able to roam on a competitor's network? Is this about protecting your own profits ahead of the interests of the customer? I don't understand why this wouldn't already be on your list of things to investigate. I think Senator Henderson is absolutely correct.
Mrs Bayer Rosmarin : Senator, there are two things. Firstly, we don't see profits and customers as opposed in any way. We only make a profit if we have happy customers and if more customers choose us and more customers choose to stay with us. So, for us to be profitable, we want to do the right thing for customers. Secondly—I may say this another way. Given that Optus has networks that are set up—and we, at the moment, have about 30 per cent market share—hypothetically, if that there was a fault on Telstra's network, where they have 50 per cent market share, and suddenly all of those subscribers were to be on our network, we would have to have already invested in the capacity to be able to cater for that many customers simultaneously. So there are a lot of considerations, if we go down this path, about investment capacity and how that all works to make sure that you don't inadvertently, if one goes down, bring down another network by overwhelming it with scale.
CHAIR: Well, I suggest you get on with talking to your competitors and making sure that the Australian people are actually looked after and can access a service that is essential. We are going to have to wind up. I know Senator Pocock has a last burning question, and so does Senator Cadell. Can we keep them short.
Senator DAVID POCOCK: Thanks again for your time. One of my team members just called the Optus business line to inquire about compensation, and they were told that they couldn't lodge a claim and that Optus would get back to them at some stage when they had more information. What's going on there? You said earlier that you have had small businesses lodge complaints. What do I tell small businesses?
Mrs Bayer Rosmarin : If you could pass me the details of who called, I'll look into why the process was not followed.
Senator DAVID POCOCK: That was literally half an hour ago, and they said: 'You can't lodge a claim. We don't have the information about what's going to happen, and so we'll get back to you at some point once we have more information.' Why don't they have the information?
Mrs Bayer Rosmarin : I would have to look into that. So, if you could pass me more details, I'll make sure we do.
Senator DAVID POCOCK: Does your Optus business line have the information necessarily to take claims when people call up?
Mrs Bayer Rosmarin : I believe that they do, and, as I mentioned, we've already had a number of customers who have been able to do that. So, as I said, I'll look into it.
CHAIR: Senator Pocock, we will have to wind up. Senator Cadell, quickly.
Senator CADELL: Very quickly—you answered about the engagement levels of your staff when I asked about that. Since then, I've been emailed by a number of your staff. One of them said: 'I just note that the engagement surveys we did in the last couple of years had nothing about attitude or outsourcing and keeping intellectual property in house. They are pretty much about Optus values. There are some things about staff development but nothing about workload or how skilled a person you are. So, in telling you about the high engagement score, it's very much delivering it to you.' Given you didn't know the weakness in the network, you haven't responded to the customers well and you haven't reflected your staff's attitude, isn't it time for new leadership at Optus?
Mrs Bayer Rosmarin : Thank you, Senator. I will take that on board.
CHAIR: We are going to have to end today's hearing. I do thank you both for making yourselves available today at such short notice. I know this happened only a week ago, but it was important to make sure that the Australian people and the millions impacted were able to see you and hear directly from you about what is going on and what you're going to do about it. That concludes today's proceedings. We obviously will be continuing our inquiries with other stakeholders. There are questions we've asked you to take on notice, and we look forward to receiving the answers to them. We've set down Friday 24 November as the response date for those questions. Thank you, both. Thanks to the secretariat, Hansard and my fellow senators on the committee.
Committee adjourned at 11:04