Note: Where available, the PDF/Word icon below is provided to view the complete and fully formatted document
Auditor-General—Audit report No. 5 for 2021-22—Performance audit—Improving immunisation coverage: Department of Health


Download PDF Download PDF

The Auditor-General Auditor-General Report No. 5 2021-22 Performance Audit

Improving Immunisation Coverage

Department of Health

Australian National Audit Office

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

2

© Commonwealth of Australia 2021

ISSN 1036-7632 (Print) ISSN 2203-0352 (Online) ISBN 978-1-76033-679-0 (Print) ISBN 978-1-76033-680-6 (Online)

Except for the content in this document supplied by third parties, the Australian National Audit Office logo, the Commonwealth Coat of Arms, and any material protected by a trade mark, this document is licensed by the Australian National Audit Office for use under the terms of a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 Australia licence. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/3.0/au/.

You are free to copy and communicate the document in its current form for non-commercial purposes, as long as you attribute the document to the Australian National Audit Office and abide by the other licence terms. You may not alter or adapt the work in any way.

Permission to use material for which the copyright is owned by a third party must be sought from the relevant copyright owner. As far as practicable, such material will be clearly labelled.

For terms of use of the Commonwealth Coat of Arms, visit the It’s an Honour website at https://www.pmc.gov.au/government/its-honour.

Requests and inquiries concerning reproduction and rights should be addressed to:

Senior Executive Director Corporate Management Group Australian National Audit Office GPO Box 707 Canberra ACT 2601

Or via email: communication@anao.gov.au.

Audi

tor-General Report No. 5 2021-22 Improving Immunisation Coverage

3

Canber

ra ACT

22 September 2021

Dear Mr President Dear Mr Speaker

In accordance with the authority contained in the Auditor-General Act 1997, I have undertaken an independent performance audit in the Department of Health. The report is titled Improving Immunisation Coverage. Pursuant to Senate Standing Order 166 relating to the presentation of documents when the Senate is not sitting, I present the report of this audit to the Parliament.

Following its presentation and receipt, the report will be placed on the Australian National Audit Office’s website — http://www.anao.gov.au.

Yours sincerely

Grant Hehir Auditor-General

The Honourable the President of the Senate The Honourable the Speaker of the House of Representatives Parliament House Canberra ACT

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

4

AUDITING FOR AUSTRALIA

The Auditor-General is head of the Australian National Audit Office (ANAO). The ANAO assists the Auditor-General to carry out his duties under the Auditor-General Act 1997 to undertake performance audits, financial statement audits and assurance reviews of Commonwealth public sector bodies and to provide independent reports and advice for the Parliament, the Australian Government and the community. The aim is to improve Commonwealth public sector administration and accountability.

For further information contact: Australian National Audit Office GPO Box 707 Canberra ACT 2601

Phone: (02) 6203 7300 Email: ag1@anao.gov.au

Auditor-General reports and information about the ANAO are available on our website: http://www.anao.gov.au

Audit team

Jennifer Eddie Elizabeth Robinson Hugh Balgarnie Deborah Jackson

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

5

Contents Summary and recommendations .................................................................................................................... 7

Background ............................................................................................................................................... 7

Conclusion ................................................................................................................................................. 8

Supporting findings .................................................................................................................................... 8

Recommendations ..................................................................................................................................... 9

Summary of Department of Health’s response ......................................................................................... 9

Key messages from this audit for all Australian Government entities ..................................................... 10

Audit findings .............................................................................................................................................. 11

1. Background ............................................................................................................................................. 12

National immunisation strategies ............................................................................................................ 12

National Immunisation Program .............................................................................................................. 13

Governance arrangements ...................................................................................................................... 13

National Partnership on Essential Vaccines............................................................................................ 16

Rationale for undertaking the audit ......................................................................................................... 17

Audit approach ........................................................................................................................................ 17

2. Improving immunisation coverage ........................................................................................................... 18

Has Health implemented the key actions for 2013-18, and is it implementing the key actions for 2019-24, as identified in the national immunisation strategy for improving immunisation coverage? ........................................................................................................................................... 18

Has immunisation coverage in Australia improved? ............................................................................... 33

3. Monitoring immunisation coverage .......................................................................................................... 37

Does Health gain assurance about the quality of the data it uses for monitoring and reporting immunisation coverage? .................................................................................................................... 37

Does Health identify geographic areas or cohorts of low coverage and take action to address low coverage? ........................................................................................................................................... 47

Has Health assessed its approach to improving immunisation coverage? ............................................. 53

Appendices ................................................................................................................................................. 57

Appendix 1 Entity responses ................................................................................................................. 58

Appendix 2 Improvements observed by the ANAO ............................................................................... 60

Appendix 3 National Immunisation Program Schedule, 1 July 2020 ..................................................... 61

Auditor-General Report No.5 2021-22 Improving Immunisation Coverage

 Funding has increased to more than

$488.7 million for National Immunisation Program (NIP) vaccines in 2021-22.

 The audit provides assurance to the Australian

Parliament and the public on the effectiveness of the Department of Health’s (Health’s) approach to improving and monitoring immunisation coverage.

 Health’s approach to improving and

monitoring immunisation coverage has been largely effective.

 Health has implemented or is implementing

most of the key actions in 2013-18 and 2019-24 national immunisation strategies.

 Reported immunisation coverage rates have

improved between 2012 and 2020 for all children at ages one and five years.

 Health does not gain assurance about the

quality of the data it uses for monitoring and reporting immunisation coverage.

 Methodologies and definitions used for

‘fully immunised’ rates, and the way in which these rates are reported, could lead readers to misinterpret immunisation coverage data.

 There were three recommendations related

to Health’s oversight of immunisation coverage.

 Health agreed to all recommendations.

 Reported immunisation coverage rates for all

children in 2020:

 94.1% for one year olds

 92.5% for two year olds

 95.1% for five year olds

 Reported immunisation coverage rates for

Aboriginal and Torres Strait Islander children in 2020:

 93.8% for one year olds

 91.4% for two year olds

 97.2% for five year olds

 Health does not regularly report on adult

immunisation coverage.

$488.7m Commonwealth funding for NIP vaccines in 2021-22.

95.1% Number of Australian five year olds fully immunised in 2020, as

calculated by Health.

39

Number of vaccine doses on the NIP Schedule up to age five years.

4

Number of vaccine doses in Health’s calculation of fully immunised for five year olds.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

7

Summary and recommendations

Background 1. Immunisation is an effective way to protect individuals from harmful infections and to prevent the spread of disease in the community.1 The Australian and state and territory health ministers agreed in 2008 that a new national immunisation strategy be developed. The Department of Health (Health) and its National Immunisation Committee oversaw the development of the strategy, which was endorsed by the state and territory governments and published in 2013. An updated version was published in 2019. Both versions of the strategy focus on the governance and implementation of the National Immunisation Program (NIP) and include the strategic priority ‘improve immunisation coverage’.

2. Through the NIP, the Australian Government provides vaccines against 17 diseases to eligible people.2 National, state and territory arrangements for vaccination are articulated in the National Partnership on Essential Vaccines (NPEV) agreement. The NPEV agreement outlines funding and delivery arrangements, roles and responsibilities of the Australian Government and states and territories and a performance framework.

Rationale for undertaking the audit

3. For 2019-20 Health reported that immunisation coverage for all five year olds was 94.77 per cent and coverage for Aboriginal and Torres Strait Islander one-year-old children was 93.4 per cent, against a target rate of 95 per cent. Funding for vaccine purchasing and services to support immunisation uptake has increased from $10 million a year in the mid-1970s to more than $488.7 million for NIP vaccines in 2021-22. The audit provides assurance to the Australian Parliament and the public on the effectiveness of Health’s approach to improving and monitoring immunisation coverage.

Audit objective and criteria

4. The audit objective was to assess the effectiveness of Health’s approach to improving and monitoring immunisation coverage.

5. To form a conclusion against this audit objective, the following high-level criteria were adopted:

• Has Health’s approach to improving immunisation coverage been effective?

• Has Health effectively monitored immunisation coverage?

1 Australian Institute of Health and Welfare, Vaccine preventable disease in Australia, Canberra, 2018, p. 1. Although the terms immunisation and vaccination are sometimes used interchangeably, they are not the same. Vaccination is the process of receiving a vaccine. Immunisation is the process of receiving a vaccine and becoming immune to the disease as a result. 2 The 17 diseases included in the NIP are: Hepatitis B, diphtheria, tetanus, pertussis (whooping cough),

Haemophilus influenzae type b (Hib) disease, poliomyelitis, pneumococcal, rotavirus, measles, mumps, rubella, meningococcal, varicella (chickenpox), hepatitis A, human papillomavirus (HPV), influenza and herpes zoster (shingles).

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

8

Conclusion 6. The Department of Health’s approach to improving and monitoring immunisation coverage has been largely effective, as reported immunisation rates have been improving.

7. While Health has not established an implementation plan for the national immunisation strategies, it has implemented or is implementing most of the key actions identified in the 2013- 18 National Immunisation Strategy and is progressing the key actions in the 2019-24 strategy. Health’s approach to improving immunisation coverage, which includes distributing vaccines under the National Partnership on Essential Vaccines and implementing vaccination awareness campaigns, has been largely effective. Immunisation coverage has improved for children and adolescents, but Health does not have sufficient data to determine if immunisation coverage has improved for other cohorts.

8. Health’s approach to monitoring immunisation coverage has been partially effective. Health does not gain assurance about the quality of the data it uses for monitoring and reporting immunisation coverage. In addition, its calculation of some coverage rates and the way they are publicly reported could lead readers to misinterpret coverage rates. While Health identifies and addresses areas of low immunisation coverage areas for children and the adolescent HPV vaccine, it does not identify or monitor areas of low coverage for other cohorts due to insufficient data. The performance measures for immunisation coverage are not fully adequate and, while Health has taken some action to assess performance, it does not systematically address issues raised in evaluations and program assessments.

Supporting findings

Improving immunisation coverage

9. Health has implemented or is implementing most of the key actions identified in the 2013- 18 National Immunisation Strategy for improving immunisation coverage and is implementing the key actions under the 2019-24 National Immunisation Strategy. Health has not established an implementation plan for either version of the strategy or clearly reported on performance targets for vaccination awareness campaigns.

10. Reported immunisation coverage rates have improved for: all children, including Aboriginal and Torres Strait Islander children, at ages one and five years from 2012 to 2020; adolescent girls receiving two doses of the HPV vaccine from 2012 to 2017; and adolescent boys receiving two doses of the HPV vaccine from 2014 to 2017. Immunisation rates have not improved for children at age two years. Health does not report immunisation coverage rates for other vaccines or cohorts.

Monitoring immunisation coverage

11. Health does not gain assurance about the quality of the data it uses for monitoring and reporting immunisation coverage and has not clarified responsibility for data quality. Definitions and methodologies used to define ‘fully immunised’ at ages one, two and five years, and the way in which these rates are reported, could lead readers to misinterpret immunisation coverage rates.

Summary and recommendations

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

9

12. Health identifies and addresses geographic areas and cohorts of low coverage for children and adolescents by encouraging states and territories to improve rates in low coverages areas. Health does not identify areas of low immunisation coverage for other cohorts due to insufficient data.

13. Health has conducted two maturity assessments on the NIP and contracts the National Centre for Immunisation Research and Surveillance to evaluate individual vaccination programs. However, it has not assessed its overall approach to improving immunisation coverage and does not systematically consider or address key learnings from the assessments undertaken. Health has established and reports on performance measures for immunisation, but these measures are not fully adequate for determining if Health’s objective for immunisation is being achieved.

Recommendations

Recommendation no. 1 Paragraph 2.37 The Department of Health report against performance targets for vaccination awareness campaigns.

Department of Health response: Agreed.

Recommendation no. 2 Paragraph 2.67 The Department of Health implement a plan to operationalise the National Immunisation Strategy, which includes assigning clear

responsibility for actions and outcomes and setting timeframes for delivery.

Department of Health response: Agreed.

Recommendation no. 3

Paragraph 3.36

The Department of Health ensure that reporting about immunisation coverage is accurate, and that data definitions and the methodology used to calculate immunisation coverage rates are clear.

Department of Health response: Agreed.

Summary of Department of Health’s response The Department welcomes the report's findings and accepts its recommendations. The Department is committed to implementing the Australian National Audit Office's recommendations and is taking steps to address the issues identified in this audit.

It is pleasing to note the report recognises the Department's progress in implementing the key actions identified in the 2013-18 and 2019-24 National Immunisation Strategies (NIS). The Department is particularly proud of achieving high immunisation coverage rates for children, including Aboriginal and Torres Strait Islander children, and adolescents receiving HPV vaccination. Increasing coverage of these at-risk population groups has been enhanced through effective communication strategies and facilitating broader access to immunisation through funded vaccination catch-up arrangements.

The audit highlighted opportunities for improvement regarding data quality assurance and governance arrangements supporting the National Immunisation Program (NIP). The Department has initiatives currently underway that will address these findings including the introduction of mandatory reporting of COVID-19 and NIP vaccines to the Australian Immunisation Register (AIR).

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

10

This will enable the AIR to become a complete and reliable dataset of vaccines administered in Australia. This expanded dataset will inform strategies to increase uptake of vaccines nationally across an expanded range of cohorts, improve the management of vaccine-preventable disease outbreaks and support clinicians to offer evidence-b

ased care.

In addition, the Department has already commenced the design of a monitoring framework for the NIS Strategy 2019-2024 and will implement all the relevant recommendations.

Key messages from this audit for all Australian Government entities 14. Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Program implementation • Successful program implementation requires a fit-for-purpose implementation plan that sets clear responsibilities and timeframes and establishes a process for senior management oversight and accountability.

• When responsibility for program delivery is shared across entities, roles and responsibilities should be clearly assigned.

• If entities obtain data from other entities or sources, they should have appropriate processes in place to gain sufficient assurance about the quality of that data.

Performance and impact measurement • To assess the effectiveness of government awareness campaigns, it is necessary to establish clear and measurable performance targets and report against the set targets

• Performance measures should be adequate for assessing progress against a program’s objective.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

11

Audit findings

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

12

1. Background

1.1 Immunisation is an effective way to protect individuals from harmful infections and to prevent the spread of disease in the community.3 Since the introduction of routine immunisation of infants in Australia in the 1950s, death or disability from many once-common infectious diseases is now rare.

National immunisation strategies 1.2 The Australian and state and territory health ministers agreed in 2008 that a new national immunisation strategy be developed. The Department of Health (Health) and its National Immunisation Committee oversaw the development of the strategy, which was endorsed by Australian, state and territory government representatives in June 2013 and noted by Australian health ministers in November 2013.4 Health published the National Immunisation Strategy for Australia 2013-18 at the end of 2013. An updated version, the National Immunisation Strategy for Australia 2019-2024, was endorsed by the National Immunisation Committee and agreed by Australian health ministers in 2019.

1.3 Both versions of the strategy focus on the governance and implementation of the National Immunisation Program (NIP) and include the following eight strategic priorities:

• improve immunisation coverage;

• ensure effective governance of the NIP;

• ensure secure vaccine supply and efficient use of vaccines for the NIP;

• continue to enhance vaccine safety monitoring systems;

• maintain and ensure community confidence in the NIP through effective communication strategies;

• strengthen monitoring and evaluation of the NIP through assessment and analysis of immunisation register data and vaccine preventable disease surveillance;

• ensure an adequately skilled immunisation workforce through promoting effective training for immunisation providers; and

• maintain Australia’s strong contribution to the region.

1.4 The strategies outline key actions for achieving each strategic priority. This audit focuses on the first strategic priority ‘improve immunisation coverage’, for which the following key actions were outlined in the 2019-24 strategy:

• maintain or improve immunisation coverage in accordance with the NIP Schedule;

3 Australian Institute of Health and Welfare, Vaccine preventable disease in Australia, Canberra, 2018, p. 1. Although the terms immunisation and vaccination are sometimes used interchangeably, they are not the same. Vaccination is the process of receiving a vaccine. Immunisation is the process of receiving a vaccine and becoming immune to the disease as a result. 4 The National Immunisation Committee is chaired by a senior executive from Health and its membership

includes health professionals, peak bodies, consumers and researchers, as well as other representatives from the Australian Government, and state and territory governments.

Background

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

13

• work towards achieving immunisation coverage rates of 95 per cent for children aged one, two and five years;

• work towards achieving and maintaining immunisation coverage rates of 95 per cent for Aboriginal and Torres Strait Islander children aged one, two and five years;

• improve immunisation coverage for population groups at higher risk;

• continue implementing strategies to improve immunisation coverage in areas where coverage is low;

• facilitate access to immunisation services for all Australians, regardless of financial or geographical barriers;

• implement strategies to improve and better understand adolescent immunisation coverage; and

• improve monitoring and uptake of influenza, pneumococcal and herpes zoster vaccination.

National Immunisation Program 1.5 The NIP was established in 1997 with the aim to increase national immunisation coverage to reduce the incidence of vaccine preventable diseases in Australia.5 Through the NIP, the Australian Government provides vaccines against 17 diseases to eligible people.6 The NIP is an initiative involving the Australian Government and state, territory and local governments, along with healthcare providers, administrators, researchers, advisory groups and committees.

Governance arrangements 1.6 The governance arrangements for immunisation in Australia involve a range of responsibilities for the design, delivery and uptake of immunisation, which are shared among different levels of government and other stakeholders, as depicted in Figure 1.1.

5 The NIP was also known as the ‘Immunise Australia Program’ when it was introduced. 6 The 17 diseases included in the NIP are: Hepatitis B, diphtheria, tetanus, pertussis (whooping cough), Haemophilus influenzae type b (Hib) disease, poliomyelitis, pneumococcal, rotavirus, measles, mumps, rubella, meningococcal, varicella (chickenpox), hepatitis A, human papillomavirus (HPV), influenza and herpes

zoster (shingles).

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

14

Figure 1.1: Australian immunisation landscape

Source: Department of Health, 2019-24 National Immunisation Strategy, p. 38.

1.7 Health administers and coordinates the NIP and other immunisation programs in conjunction with the Australian Health Protection Principal Committee (AHPPC), the National Immunisation Committee and the Australian Technical Advisory Group on Immunisation (ATAGI). Health’s responsibilities include:

• developing immunisation policy and legislation; • overseeing systems and processes to monitor the safety and efficacy of vaccines;

Background

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

15

• providing secretariat support to ATAGI, Jurisdictional Immunisation Coordinators and the National Immunisation Committee; • managing the National Partnership on Essential Vaccines (NPEV); • coordinating delivery of the national immunisation strategy; • providing communication materials to vaccination providers and the public; • maintaining the Australian Immunisation Register (AIR), which includes managing the AIR

program agreement with Services Australia7; • supporting the implementation of the NPEV through coordinating annual assessments of state and territory performance against defined benchmarks; • providing policy advice on the government’s ‘No Jab, No Pay’ initiative; and • managing tenders and contracts for the supply of vaccines to state and territory

governments.

Australian Health Protection Principal Committee 1.8 The AHPPC is the key decision-making committee for health emergencies. It is chaired by the Australian Chief Medical Officer and its membership includes all state and territory chief health officers. The AHPPC delegates some responsibilities for immunisation to the National Immunisation Committee and the Communicable Diseases Network Australia (CDNA).

Communicable Diseases Network Australia 1.9 The CDNA is a sub-committee of the AHPPC. It offers strategic advice to governments on public health actions to minimise the impact of communicable diseases in Australia. Membership includes representatives from the Australian and state and territory governments.

National Immunisation Committee 1.10 The National Immunisation Committee’s role is to provide advice on the NIP. It is chaired by a senior executive from Health and its membership includes health professionals, peak bodies, consumers and researchers, as well as other representatives from the Australian Government, and state and territory governments. The National Immunisation Committee reports to the AHPPC through the CDNA. It also consults and collaborates with ATAGI and Jurisdictional Immunisation Coordinators.8

Australian Technical Advisory Group on Immunisation 1.11 ATAGI advises the Minister for Health (the Minister) on the NIP and other immunisation policies, programs and issues. This includes identifying and prioritising gaps in the immunisation landscape to improve impact, confidence and equity. ATAGI also provides technical advice on the medical administration of vaccines, including those available through the NIP, and produces the Australian Immunisation Handbook.9 ATAGI’s membership includes technical experts, medical practitioners and a consumer representative.

7 The AIR is the national register to record all vaccines given to all people in Australia. 8 Each state and territory has a Jurisdictional Immunisation Coordinator, whose role is to discuss and resolve issues related to the NIP and the National Partnership Agreement on Essential Vaccines. 9 Department of Health, The Australian Immunisation Handbook [Internet], available from

https://immunisationhandbook.health.gov.au/ [accessed 27 April 2021]. The handbook provides online clinical advice for health professionals about using vaccines safely and effectively.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

16

National Partnership on Essential Vaccines 1.12 National and state and territory arrangements for immunisation are articulated in the National Partnership on Essential Vaccines (NPEV) agreement. The Australian Government and the state and territory governments signed the first NPEV agreement in 2009 and the second in 2017.10 The NPEV agreement describes the arrangements for the funding and delivery of a national, coordinated approach to maintaining and improving immunisation coverage through the NIP. It outlines the roles and responsibilities of the Australian Government and state and territory governments and provides the framework for reward payments to states and territories for achieving performance benchmarks.

1.13 The objectives and outcomes of the first and second NPEV agreements are outlined at Table 1.1.

Table 1.1: Objectives and outcomes of the National Partnership on Essential Vaccines

2009 2017

Objectives

• Improved health and well-being of Australians through the cost-effective delivery of immunisation programs under the NIP

• Maintaining and, where possible, improving immunisation coverage rates through immunisation initiatives to reduce the incidence of vaccine preventable diseases in the Australian population

• Protect the Australian public from the spread of vaccine preventable diseases through the cost-effective and efficient delivery of immunisation programs under the NIP

Outcomes

• Minimise the incidence of major vaccine preventable diseases in Australia

• Maintain and where possible increase immunisation coverage rates for vulnerable groups and, in particular, minimise disparities between Indigenous and non-Indigenous Australians

• All eligible Australians are able to access high quality and free essential vaccines through the NIP in a timely manner

• Increase community understanding and support for the public health benefits of immunisation

• Minimise the incidence of vaccine preventable diseases: in the eligible Australian population for diseases with vaccines listed under the NIP; in Aboriginal and Torres Strait Islander people for diseases with vaccines listed under the NIP; and in the eligible Australian population in geographic areas of low coverage

• Minimise the incidence of Human Papillomavirus (HPV) in the eligible Australian population

• Ensure that Australian HPV immunisation data is provided to the Commonwealth annually

• Ensure that vaccines listed under the NIP are managed in a way that minimises wastage and leakage, with a target rate of wastage and leakage of 5 per cent or lower

Source: ANAO analysis of the 2009 and 2017 National Partnership on Essential Vaccines agreements.

10 The NPEV agreements were preceded by Australian immunisation agreements between the Australian Government and state and territory governments.

Background

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

17

Rationale for undertaking the audit 1.14 For 2019-20 Health reported that immunisation coverage for all five year olds was 94.77 per cent and coverage for Aboriginal and Torres Strait Islander one-year-old children was 93.4 per cent against a target rate of 95 per cent. Funding for vaccine purchasing and services to support immunisation uptake has increased from $10 million a year in the mid-1970s to more than $488.7 million for NIP vaccines in 2021-22.11 The audit provides assurance to the Australian Parliament and the public on the effectiveness of Health’s approach to improving and monitoring immunisation coverage.

Audit approach Audit objective, criteria and scope 1.15 The audit objective was to assess the effectiveness of Health’s approach to improving and monitoring immunisation coverage.

1.16 To form a conclusion against this audit objective, the following high-level criteria were adopted:

• Has Health’s approach to improving immunisation coverage been effective?

• Has Health effectively monitored immunisation coverage?

1.17 The audit scope includes Health’s approach to improving immunisation coverage, which is the first strategic priority of Health’s 2013-18 and 2019-24 national immunisation strategies. This includes immunisation coverage for: all Australian children at age one, two and five years; Aboriginal and Torres Strait Islander children at age one, two and five years; adolescents; and Australians who are 65 years and older. The audit scope does not include: the administration of the National Partnership Agreement on Essential Vaccines (NPEV); the COVID-19 vaccination program; or Services Australia’s administration of the Australian Immunisation Register (which was the subject of an ANAO performance audit in 2015).12

Audit methodology 1.18 To address the audit criteria and achieve the audit objective, the audit team:

• examined documentation and data held by Health relating to improving and monitoring immunisation coverage;

• interviewed relevant Health officers; and

• interviewed relevant Services Australia officers.

1.19 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $272,000.

1.20 Team members for this audit were Jennifer Eddie, Elizabeth Robinson, Hugh Balgarnie and Deborah Jackson.

11 This does not include funding for COVID-19 vaccines. 12 Auditor-General Report No. 46 2014-15, Administration of the Australian Childhood Immunisation Register, available from https://www.anao.gov.au/work/performance-audit/administration-australian-childhood-immunisation-register [accessed 5 January 2021].

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

18

2. Improving immunisation coverage

Areas examined This chapter examines whether the Department of Health’s (Health’s) approach to improving immunisation coverage has been effective.

Conclusion While Health has not established an implementation plan for the national immunisation strategies, it has implemented or is implementing most of the key actions identified in the 2013- 18 National Immunisation Strategy and is progressing the key actions in the 2019-24 strategy. Health’s approach to improving immunisation coverage, which includes distributing vaccines under the National Partnership on Essential Vaccines and implementing vaccination awareness campaigns, has been largely effective. Immunisation coverage has improved for children and adolescents, but Health does not have sufficient data to determine if immunisation coverage has improved for other cohorts.

Areas for improvement The ANAO made two recommendations aimed at establishing a plan to operationalise the National Immunisation Strategy and reporting against performance targets for vaccination awareness campaigns.

The ANAO also suggested that Health could consider using its disease surveillance data to more clearly identify risks of vaccine preventable diseases in the Australian population.

2.1 The national immunisation strategy outlines Health’s approach to improving immunisation coverage. The strategy focuses on improving rates for vaccines administered under the National Immunisation Program (the NIP), which aims to reduce the incidence of vaccine preventable diseases in Australia through increased national immunisation coverage.

2.2 To determine if Health’s approach to improving immunisation coverage has been effective, the ANAO examined whether:

• Health has implemented the key actions for 2013-18, and is implementing the key actions for 2019-24, as identified in the national immunisation strategy for improving immunisation coverage; and

• immunisation coverage in Australia has improved.

Has Health implemented the key actions for 2013-18, and is it implementing the key actions for 2019-24, as identified in the national immunisation strategy for improving immunisation coverage?

Health has implemented or is implementing most of the key actions identified in the 2013-18 National Immunisation Strategy for improving immunisation coverage and is implementing the key actions under the 2019-24 National Immunisation Strategy. Health has not established an implementation plan for either version of the strategy or clearly reported on performance targets for vaccination awareness campaigns.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

19

2.3 Under the 2013-18 National Immunisation Strategy, Health outlined nine key actions for improving immunisation coverage. The key actions were updated for the 2019-24 strategy: six remained the same or had slight wording changes; three were removed; and two were added.

2.4 The ANAO’s assessment of the status of Health’s key actions for improving immunisation coverage are outlined at Table 2.1, followed by a discussion of progress against these actions.

Table 2.1: Status of key actions identified in the 2013-18 and 2019-24 national immunisation strategies for improving immunisation coverage

Key actions under the 2013-18 strategy

Status as at 2018 Key actions under the 2019-24 strategy

Status as at 2021

Maintain or improve immunisation coverage in accordance with the National Immunisation Program (NIP) Schedulea



Maintain or improve immunisation coverage in accordance with the NIP Schedulea 

Work towards achieving immunisation coverage rates of 95% for children aged 1, 2 and 5 years 

Work towards achieving and maintaining immunisation coverage rates of 95% for Aboriginal and Torres Strait Islander children aged 1, 2 and 5 years



Improve immunisation coverage for high risk population groups 

Improve immunisation coverage for population groups at higher risk 

Identify geographic areas or cohorts of low coverage and implement strategies to improve immunisation coverage where coverage is low



Continue implementing strategies to improve immunisation coverage in areas where coverage is low 

Ensure equity of access to immunisation services for all Australians regardless of financial or geographical barriers



Facilitate access to immunisation services for all Australians, regardless of financial or geographical barriers



Maintain and monitor the effectiveness of childhood vaccination awareness and promotion campaigns and incentivesb

 - -

Implement strategies to improve and better understand adolescent immunisation coverage 

Implement strategies to improve and better understand adolescent immunisation coverage. 

Improve influenza and pneumococcal vaccination rates  Improve monitoring and uptake of influenza, pneumococcal and herpes

zoster vaccination



Through disease surveillance, identify the risks posed by unvaccinated cohorts in the populationb

 - -

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

20

Key actions under the 2013-18 strategy

Status as at 2018 Key actions under the 2019-24 strategy

Status as at 2021

Develop an agreed position on the provision of free catch-up immunisation schedulesb  - -

Key:  Strong progress or complete ▲ In progress  Not completed

Note a: For all children, including Aboriginal and Torres Strait Islander children up to five years old. Note b: Removed as a key action for improving immunisation coverage in the 2019-24 National Immunisation Strategy. Source: ANAO analysis of the 2013-18 and 2019-24 national immunisation strategies.

Maintain or improve immunisation coverage in accordance with the NIP Schedule

2.5 The first key action for improving immunisation coverage in both versions of the national immunisation strategy is to ‘maintain or improve immunisation coverage in accordance with the NIP Schedule’. There were an additional two related ‘key actions’ in the 2019-24 strategy, which identified target rates for children:

• work towards achieving immunisation coverage rates of 95 per cent for all children aged one, two and five years; and

• work towards achieving and maintaining immunisation coverage rates of 95 per cent for Aboriginal and Torres Strait Islander children aged one, two and five years.

2.6 Strong progress has been made against these three key actions, with reported immunisation coverage rates for children at age five years and Aboriginal and Torres Strait Islander children at age five years surpassing the 95 per cent target in 2020.

2.7 Health does not have an implementation plan for the strategy (discussed at paragraphs 2.65-2.66) and does not regularly report on actions taken. A progress review of the strategy in 2015 and 2017 included only outcomes at the state level and immunisation coverage rates. It did not outline what Health had done to ‘maintain or improve immunisation coverage’.

2.8 The ANAO’s examination found that the main actions Health had undertaken to ‘maintain and improve immunisation coverage in accordance with the NIP Schedule’ were to:

• distribute vaccines to the states and territories under the National Partnership on Essential Vaccines (NPEV);

• encourage states and territories to improve immunisation coverage for children and Aboriginal and Torres Strait Islander children, including in low coverage areas, through performance-based incentive payments under the NPEV agreements (discussed at paragraphs 2.22-2.25 and 3.39-3.46);

• implement vaccination awareness campaigns (discussed at paragraphs 2.31-2.42); and

• develop a free catch-up immunisation program (discussed at paragraphs 2.62-2.64).

2.9 At July 2013 the NIP Schedule outlined vaccines against 16 diseases provided under the NIP for children, adolescents and at-risk groups.13 The most recent NIP Schedule (in effect from 1 July

13 The 16 diseases were: Hepatitis B, diphtheria, tetanus, pertussis (whooping cough), Haemophilus influenzae type b (Hib) disease, poliomyelitis, pneumococcal, rotavirus, measles, mumps, rubella, meningococcal, varicella (chickenpox), hepatitis A, human papillomavirus (HPV) and influenza.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

21

2020) includes vaccines against 17 diseases, with the addition of the herpes zoster (shingles) vaccine for people 70 years and over (Appendix 3 lists the vaccines included in the 2020 NIP Schedule).14 The majority of NIP vaccines are provided to children, with the 2013 and 2020 NIP Schedules including vaccines for children against 13 diseases.15 These vaccines are given to children from birth to age four years, with additional vaccines available to Aboriginal and Torres Strait Islander children.

2.10 Under the 2020 NIP Schedule, adolescents are eligible to receive vaccines against five diseases: human papillomavirus (HPV), diphtheria, tetanus, pertussis (whooping cough) and meningococcal ACWY. Adults 70 years and over are eligible for the pneumococcal and shingles vaccines and pregnant women are offered the whooping cough vaccine.16

2.11 In addition people with specified medical risk conditions17 and high risk groups are offered additional vaccine doses and there is an annual influenza vaccination program for eligible cohorts.18

2.12 National immunisation coverage rates for all children were above 90 per cent before the first national immunisation strategy started in 2013 — as calculated by Health’s definition of ‘fully immunised’ for each cohort (the definition of ‘fully immunised’ and how it is calculated is discussed in paragraphs 3.20-3.29). By the end of the first strategy in 2018 strong progress had been made against the key action to ‘maintain or improve immunisation coverage in accordance with the NIP Schedule’. As outlined in Table 2.2, between 2012 and 2018 reported national immunisation rates

increased at the one-year and five-year marks, but decreased at the two-year mark (the decrease for two year olds is discussed at paragraph 2.72).19

Table 2.2: Comparison of reported national immunisation coverage for children up to age 5 years, 2012 and 2018

Age group 2012 2018

Change from 2012 to 2018

1 year — all children 91.69% 94.04% + 2.35

2 years — all children 92.59% 90.75% - 1.84

5 years — all children 90.82% 94.67% + 3.85

1 year — Aboriginal and Torres Strait Islander children 85.40% 92.62% + 7.22

2 years — Aboriginal and Torres Strait Islander children 92.19% 88.20% - 3.99

5 years — Aboriginal and Torres Strait Islander children 89.81% 96.66% + 6.85

Note: These rates are based on the Department of Health’s definition for ‘fully immunised’ for each cohort. Source: ANAO analysis of Department of Health quarterly immunisation reports.

14 The herpes zoster (shingles) vaccine was added to the NIP Schedule in November 2016 for people 70 years and over. 15 Children aged six months to less than five years are also eligible for the annual influenza vaccination program. 16 Pregnant women are also eligible for the annual influenza vaccination program. 17 Specified medical risk conditions include asplenia, hyposplenia, complement deficiency, those undergoing

treatment with eculizumab and conditions that increase the risk of pneumococcal disease. 18 People eligible for the annual influenza vaccination include: children six months to less than five years of age; people six months and over with specified medical risk conditions; people 65 years and over; pregnant women; and Aboriginal and Torres Strait Islander people six months and over. 19 For the analysis of childhood immunisation rates, the ANAO selected 2012 as the baseline year as it is the year

before the start of the 2013-18 National Immunisation Strategy.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

22

2.13 The second national immunisation strategy, which commenced in 2019, outlined a target of achieving immunisation coverage rates of 95 per cent for all children and all Aboriginal and Torres Strait Islander children aged one, two and five years. Health reported that it met this target for five year olds and was making progress towards the target for one and two year olds, as outlined in Table 2.3.

Table 2.3: Comparison of reported national immunisation coverage rates for children up to 5 years, 2018 and 2020.

Age group 2018 2020

Change from 2018 to 2020

1 year — all children 94.04% 94.85% + 0.81

2 years — all children 90.75% 92.55% + 1.80

5 years — all children 94.67% 95.09% + 0.42

1 year — Aboriginal and Torres Strait Islander children 92.62% 93.79% + 1.17

2 years — Aboriginal and Torres Strait Islander children 88.20% 91.43% + 3.23

5 years — Aboriginal and Torres Strait Islander children 96.66% 97.25% + 0.59

Note: These rates are based on the Department of Health’s definition for ‘fully immunised’ for each cohort. Source: ANAO analysis of Department of Health quarterly immunisation reports.

2.14 Health did not have sufficient data for determining if immunisation coverage had increased for other cohorts, as reporting to the Australian Immunisation Register (AIR) was not mandatory until 2021 and reporting levels have been low for cohorts other than children.

Improve immunisation coverage for high risk population groups

2.15 As noted in Table 2.1, the key action to ‘improve immunisation coverage for high risk population groups’ is still in progress. High risk population groups are those at higher risk from vaccine preventable diseases. The groups identified in the strategy include Aboriginal and Torres Strait Islander people and pregnant women.

2.16 Reported immunisation coverage rates for Aboriginal and Torres Strait Islander children at one year of age have increased from 85.4 per cent in 2012 to 93.8 per cent in 2020, as noted in Table 2.2 and Table 2.3. Health does not have sufficient data to monitor coverage for Aboriginal and Torres Strait Islander people older than five years as reporting to the AIR was not mandatory.

2.17 One of Health’s performance measures, outlined in its Portfolio Budget Statements, is ‘immunisation rates for Aboriginal and Torres Strait Islander children 12 to 15 months of age are increased to close the gap’. Health has reported against this measure in its annual reports since 2015-16. In 2019-20, Health reported a result of 93.4 per cent for this measure against a target of 93 per cent.20

2.18 The 2009 and 2017 NPEV agreements also include a performance benchmark for improving immunisation coverage for Aboriginal and Torres Strait Islander people, which is linked to a reward

20 The difference between the rates noted in paragraphs 2.16 and 2.17 are due to different reporting periods.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

23

payment.21 All states and territories met this benchmark for 2017-18,

2018-19 and 2019-20.

2.19 While pregnant women are eligible to receive vaccines against whooping cough and influenza under the NIP, Health does not collect data on immunisation coverage rates for this group. Health advised the ANAO that it does not flag in the AIR whether a woman is pregnant when a vaccine is administered. In 2018 Health commissioned research on immunisation during pregnancy. This research was used to inform a communications campaign in 2019-20 that promoted the uptake of whooping cough and influenza vaccines for pregnant women.

2.20 In some cases, people can be at higher risk because they work in an area with a high risk of disease transmission, such as health care, aged care or child care. The national immunisation strategy states that in these cases, employers are responsible for workplace vaccination through

workplace health and safety programs.

2.21 The 2021-22 Health Portfolio Budget Statements introduced a new performance measure focused on immunisation coverage rates for adults at increased risk of vaccine preventable diseases due to age or underlying medical conditions. Changes were also made to vaccine reporting legislation in 2021 with a view to improve the ability to monitor coverage rates for older Australians. This is discussed at paragraph 3.6.

Improve immunisation coverage in areas where coverage is low

2.22 Some progress is being made towards meeting the key action to ‘Improve immunisation coverage in areas where coverage is low’.

2.23 The 2009 and 2017 NPEV agreements included performance benchmarks for improving childhood immunisation rates in areas of low coverage.22 To encourage states and territories to improve coverage rates in areas and cohorts of low coverage, Health linked the performance benchmarks to annual reward payments.

2.24 Coverage is measured across Statistical Area Level 3 (SA3) geographic regions.23 Rates increased in the lowest coverage areas for children at five years old, between 2015 (when Health started publishing data by SA3) and 2020. However, the three SA3 regions with the lowest coverage

in 2015 were still among the bottom five SA3 regions in 2020, as shown in Figure 2.1.24

21 The benchmark in the 2017 agreement is: ‘An increase in the vaccination coverage rates for Aboriginal and Torres Strait Islander people in at least two of the following three cohorts: [one year olds; two year olds; and five year olds], relative to the baseline’. States and territories were eligible for a separate reward payment (worth 0.75 per cent of the total cost of the state’s or territory’s vaccine purchases) for each of the five benchmarks they met under the 2017 agreement (up to a total of 3.75 per cent of the cost of the state’s or territory’s vaccine purchases). 22 The benchmark in the 2009 agreement is ‘Maintaining or increasing coverage in agreed areas of low

immunisation coverage’ and applies to children [at one year] and [and five years]. The benchmark in the 2017 agreement is ‘An increase in vaccination coverage rates for [five year olds] in four of the ten lowest vaccination coverage SA3 geographical areas, relative to the baseline’. 23 SA3 are geographical areas designed by the Australian Bureau of Statistics to provide a regional breakdown of

Australia. They generally have a population of between 30,000 and 130,000 people. As of 2016, there are 358 SA3 regions for Australia. 24 The national average for children fully immunised at age five years, as calculated by Health, increased by 2.4 per cent from 92.6 per cent in 2015 to 95 per cent in 2020.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

24

Figure 2.1: SA3 regions with the lowest immunisation coverage for children at age five years, comparison between 2015 and 2020

SA3 regions with the lowest coverage in 2015 2015

SA3 regions with the lowest coverage in 2020 2020

Change from 2015

Eastern Suburbs - North, NSW 83.2%

Richmond Valley - Coastal, NSW 83.9% + 0.4%

Richmond Valley - Coastal, NSW 83.5% Adelaide City, SA 87.0% + 1.3%

Gold Coast Hinterland, QLD 84.3% Noosa, QLD 87.6% - 2.8%

Gold Coast Hinterland, QLD 88.1% + 3.8%

Eastern Suburbs - North, NSW 88.2% + 5.0%

Source: ANAO analysis of Department of Health quarterly immunisation reports.

2.25 Low coverage is discussed further at paragraphs 3.39-3.51.

Facilitate access and ensure equity of access to immunisation services for all Australians

2.26 The key action to ‘facilitate access to immunisation services for all Australians, regardless of financial or geographical barriers’ is in progress and the key action ‘ensure equity of access’ was discontinued at the end of the first strategy.

2.27 The wording in the key action was changed from ‘ensure equity of access’ in the first strategy to ‘facilitate access’ in the second strategy. Health does not define ‘equity of access’ or ‘facilitate access’ in the strategies. The Australian Technical Advisory Group on Immunisation’s (ATAGI’s) strategic priorities, as at September 2020, included: improve equity in outcomes by improving access to vaccination for persons of all ages; identify vulnerable populations and difficult to reach groups and advise on strategies to target those at greatest risk.

2.28 Through the NPEV agreement, states and territories are responsible for supporting individuals’ access to immunisation services under the NIP. A review of the 2017 NPEV found the agreement is partially meeting the principle of equity by allocating states and territories with greater vaccine requirements more funding. The review noted that when awarding benchmark payments, the agreement: does not consider state and territory-specific immunisation factors; milestone payments are not equitable; and other equity factors such as remote populations are not accounted for.

2.29 Health contracts the National Centre for Immunisation Research and Surveillance (NCIRS) to evaluate individual vaccine programs.25 Two of the six evaluations conducted between 2015 and 2021 considered access issues, specifically those encountered by Aboriginal and Torres Strait Islander people. In the 2015 process evaluation on the Hepatitis A vaccination program for Aboriginal and Torres Strait Islander Children, NCIRS noted cultural barriers to the uptake of the

25 NCIRS is a research organisation whose activities include the surveillance of vaccine preventable diseases, monitoring of vaccine coverage, vaccination program evaluations and vaccine safety monitoring.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

25

hepatitis A vaccine, such as the reluctance of parents of Aboriginal and Torres Strait Islander children to use mainstream health services and the limited availability of alternative culturally appropriate services.26 NCIRS also cited geographical challenges such as transport to vaccine services and the follow-up of transient families.

2.30 The 2018 NCIRS process evaluation of the influenza vaccination program for Aboriginal and Torres Strait Islander children noted access factors mentioned by interviewed stakeholders, such as issues with opening hours and location of vaccine services and the lack of bulk billing practices.27 The report also mentioned eligibility as a barrier because Aboriginal and Torres Strait Islander children between the ages of five and 15 years of age were not eligible for the free influenza vaccine. 28 Health does not systematically address or action items raised in the NCIRS reports, but Health made one change following this report — increasing influenza vaccine eligibility for Aboriginal and Torres Strait Islander people to all those above six months of age.29

Maintain and monitor the effectiveness of childhood vaccination awareness and promotion campaigns and incentives

2.31 The key action to ‘maintain and monitor the effectiveness of childhood vaccination awareness and promotion campaigns and incentives’ was in progress in 2018. The key action was not included in the 2019-24 strategy, but Health is continuing to undertake childhood vaccination awareness and promotion activities.

2.32 In 2017 Health launched the childhood vaccination campaign, ‘Get the Facts’, which was the first national vaccination campaign in 20 years. It was the first of six campaigns that ran from 2017 to 2020.30 The campaigns covered:

• Childhood vaccination ‘Get the Facts’ (four phases from 2017 to 2020);

• Pregnancy and influenza (one phase in 2017-18);

• Seasonal influenza vaccination (two phases from 2018 to 2020);

• HPV vaccinations for adolescents (three phases from 2018 to 2020);

• Meningococcal ACWY vaccinations for adolescents (one phase in 2019); and

• Maternal vaccinations (two phases from 2019 to 2020).

2.33 Health established objectives for the six campaigns. Most objectives focused on increasing awareness and understanding of the benefits of vaccination and increasing the uptake of vaccines.

26 NCIRS, Process evaluation: Hepatitis A immunisation program for Aboriginal and Torres Strait Islander Children in the Northern Territory, Queensland, South Australia and Western Australia, 30 November 2015. 27 NCIRS, Evaluation of the influenza immunisation program for Aboriginal and Torres Strait Islander children aged 6 months to <5 years: Process Evaluation & Coverage Report, 6 February 2018. 28 Under the July 2018 NIP Schedule, Aboriginal and Torres Strait Islander children aged six months to five years

and 15 years and over were eligible for the funded annual influenza vaccination program. Other children were not eligible for the funded influenza vaccine, except for those 6 months and over with medical risk factors. 29 Under the July 2020 NIP Schedule, those eligible for the funded annual influenza program include: all Aboriginal and Torres Strait Islander people over the age of six months; all children between the ages of six

months and five years; people over six months with medical risk conditions; people aged 65 years and over and pregnant women. 30 Excludes campaigns about the COVID-19 vaccines.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

26

2.34 To assess the effectiveness of government campaigns, it is necessary to establish clear and measurable performance targets and report progress against the targets. Health established targets for the final phase of four campaigns, as shown in Table 2.4.

Table 2.4: Vaccination awareness campaign performance targets

Campaign Phase Performance targets

Childhood vaccination ‘Get the Facts’

1-3 None

4 Reach: Prompted campaign recognition >40%

Response: Key campaign message recall >40%

Impact: Significant increases in campaign related actions from benchmark. Most common actions taken >40% among campaign recognisers.

Pregnancy and influenza

1 None

Seasonal influenza vaccination

1-2 None

HPV vaccinations for adolescents

1-2 None

3 Increase the average number of daily visitors to relevant pages on health.gov.au by 10% from pre to post campaign period

Increase the average number of daily video views for hero video and motion graphic by 10% from pre to post campaign period

Meningococcal ACWY vaccinations for adolescents

1 Maintain or increase the number of visitors to relevant pages on health.gov.au from pre to post campaign period.

Maintain or increase the number of video views for hero video and motion graphic from pre to post campaign period.

Maternal vaccinations

1 None

2 Increase the average number of daily visitors to relevant pages on health.gov.au by 10% from pre to post campaign period

Increase the average number of daily video views for hero video and motion graphic by 10% from pre to post campaign period

Source: ANAO analysis of Department of Health documentation.

2.35 In Auditor-General Report No.7 2019-20 Government Advertising: June 2015 to April 2019, the ANAO made a recommendation to Health to ‘set clear and measurable performance targets for short and long-term advertising campaigns and report against the targets’. Health agreed to the recommendation and stated that it:

will update the campaign management guidelines to include the requirement for all advertising campaigns implemented by the department to have clear and measurable performance targets for short and long term advertising campaigns in line with […] the whole of government campaign evaluation framework and will report against the targets.31

31 Auditor-General Report No.7 2019-20 Government Advertising: June 2015 to April 2019, paragraph 4.43-44.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

27

2.36 The ANAO found that although Health established performance targets for vaccination awareness campaigns commenced after agreeing to the recommendation, it had not reported against any of these targets.

Recommendation no. 1 2.37 The Department of Health report against performance targets for vaccination awareness campaigns.

Department of Health response: Agreed.

2.38 The Department will set and report against clear performance targets in line with the behaviour change model for communication campaigns and the whole of government campaign evaluation framework.

2.39 Health monitored the media performance of each campaign through:

• media performance reports, which contain the results of media performance against benchmarks, such as, the number of clicks a digital advertisement receives;

• social media moderation reports, which contain data collected from social media forums to ensure campaign content was appropriate; and

• campaign summary reports, which included an overview of campaign activities and key learnings.

2.40 Health also commissioned evaluations of the largest of the six campaigns, the ‘Get the Facts’ childhood vaccination campaign. The objectives of the ‘Get the Facts’ campaign, which targeted parents with children up to five years old and pregnant women, were to:

• inform parents of the benefits of childhood vaccinations both to the individual and the community;

• inform parents of the facts about childhood vaccinations and address any perceived misconceptions;

• increase awareness and confidence in the National Immunisation Program with health professionals, as a trusted source of information;

• increase confidence in the program’s role in protecting Australians from vaccine preventable diseases; and

• generate an increased intention to participate and complete the childhood vaccination schedule.

2.41 The phase one evaluation assessed four of the five campaign objectives and found that overall, the campaign was performing well against the objectives.32 The evaluation of phase two found that awareness had remained constant. Parents’ self-reported action after seeing the campaign was similar to phase one, but the results were weaker for pregnant women. While the evaluation contained relevant information, it did not conclude on whether the campaign was effective in achieving its objectives.

32 The evaluation did not include the objective related to increasing awareness with health professionals.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

28

2.42 Health included performance targets in its communication strategy for phase four of the ‘Get the Facts’ campaign in November 2019 in accordance with its updated guidelines. Health advised the ANAO that it did not report against these targets, because a formal evaluation was already a requirement under the whole-of-government campaign evaluation framework. Although relevant information was included in the phase four evaluation report, the performance targets were not identified in the evaluation report and were not clearly reported against.

‘No Jab, No Pay’ initiative

2.43 The ‘No Jab, No Pay’ initiative was introduced in January 2016. It requires that children meet the immunisation schedule requirements for their family to receive family assistance payments.33 In 2020, Health commissioned a study by NCIRS to assess the catch-up vaccination of older children and adolescents during the first two years of the initiative. The study stated:

The effects of ‘No jab, no pay’ have not been formally assessed. As vaccination coverage increased both before and after its introduction, its contribution to these changes cannot be assessed on the basis of standard childhood milestone reporting alone.34

2.44 NCIRS found that ‘nearly one in five young people not fully vaccinated against measles were vaccinated during the first two years of the policy’.35

Implement strategies to improve and better understand adolescent immunisation coverage

2.45 Progress has been made towards the key action to ‘implement strategies to improve and better understand adolescent immunisation coverage’ as it relates to the HPV vaccine. Health has not implemented specific strategies to improve or better understand other adolescent vaccines such as diphtheria, tetanus and whooping cough.

2.46 In 2016 Health commissioned market research into the attitudes of parents and adolescents towards the HPV vaccine. In 2019 Health commissioned further market research into the knowledge base of adolescents and their parents around school-based vaccinations and catch up programs.

2.47 The NPEV agreement between the Australian and state and territory governments is the primary strategy used by Health to improve HPV immunisation rates in adolescents. The 2017 NPEV agreement includes the performance benchmark, ‘An increase in the vaccination coverage rate for both adolescent boys and adolescent girls for HPV’. States and territories received a payment for each cohort that had increased coverage rates. In 2019-20 all states and territories, except New South Wales, were deemed to have met this benchmark.36

2.48 Between 2012 and 2020 the percentage of girls who had received two doses of the HPV vaccine increased from 79.7 per cent to 82.4 per cent. From February 2013 the HPV program was

33 The ‘No Jab, No Pay’ initiative extended existing requirements for receiving family assistance payments by expanding the immunisation assessment age, removing non-medical objection and tightening guidelines for medical exemptions. The ‘No Jab, No Pay’ policy is administered by the Department of Social Services. 34 Hull B et al., ‘”No jab, no pay”: catch-up vaccination activity during its first two years’, Medical Journal of

Australia, 213 (8), 2020, pp. 364-365. 35 Hull B et al., ‘”No jab, no pay”: catch-up vaccination activity during its first two years’, Medical Journal of Australia, 213 (8), 2020, p. 364. 36 New South Wales was not assessed against the HPV benchmark or deemed to have met it because it

implemented the two-dose HPV program in advance of amendments to the NIP.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

29

extended to boys aged 12-13. The rate of boys who had received two doses of the HPV vaccine increased from 70.4 per cent in 2014 to 79.5 per cent in 2020. Health did not assess the impact of the campaign on the uptake of the HPV vaccine.

2.49 In addition to the NPEV benchmark, Health ran a campaign from 2018 to 2020 to increase awareness of the HPV vaccine among the parents of adolescents and young adults eligible for catch-up vaccinations. The campaign ran on social media and included digital online videos, factsheets for vaccination providers and posters and brochures available on Health’s website.

2.50 In 2021 Health funded NCIRS to produce an impact evaluation of the Australian HPV vaccination program. The evaluation found the HPV vaccine had been successfully incorporated into Australia’s NIP and achieved relatively high coverage. The evaluation also made recommendations on how to overcome barriers to achieving higher coverage, such as the development of electronic consent forms for school-based immunisation programs.

2.51 In 2018 the government announced an investment of $52 million over four years to deliver the meningococcal ACWY vaccine through the NIP to adolescents as part of school-based vaccination programs. In 2019 Health ran a three-week social media and digital online video campaign to increase awareness of the meningococcal vaccine among adolescents and their parents. The campaign video was viewed around 300,000 times. Health did not assess the impact of the campaign on the uptake of the meningococcal vaccine.

Improve monitoring and uptake of influenza, pneumococcal and herpes zoster vaccination

2.52 Progress has been made for the key action to ‘improve the monitoring and uptake of influenza, pneumococcal and herpes zoster vaccinations’.

2.53 Health has not regularly monitored influenza, pneumococcal or herpes zoster (shingles) immunisation coverage rates for adults. Some data on immunisation coverage for older Australians is recorded on the AIR, which was established as a whole-of-life register in October 2016. Until 2021 reporting of vaccinations to the AIR was not mandatory, and reporting for older Australians was low. As a result of changes to AIR reporting legislation, it has been mandatory for vaccination providers to report administered influenza vaccines since March 2021 and to report administered NIP vaccines since July 2021.

2.54 NCIRS conducted an exploratory analysis of the first two years of adult vaccination data recorded on the AIR, which was published in November 2019. For influenza vaccinations, NCIRS found that the recorded uptake was 31.5 per cent for people aged 65 years and over in 2017 and 46.3 per cent in 2018. NCIRS noted ‘these influenza vaccination uptake figures likely substantially underestimate true uptake. Uptake in adults aged >65 years from previous national surveys has been around 70% or greater.’37 For shingles vaccinations, NCIRS found that between October 2016 and September 2018, the vaccination uptake for people aged 70 to 80 years was 31.2 per cent. NCIRS noted that the uptake of the shingles vaccine is ‘likely to be considerably higher’ given that

37 NCIRS, Exploratory analysis of the first 2 years of adult vaccination data recorded on the AIR, November 2019, p. 7. The Australian Institute of Health and Welfare and a survey commissioned by Health estimated influenza vaccination rates for 2014-15 were greater than 70 per cent in adults aged 65 and over.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

30

the number of vaccine doses recorded on the AIR ‘was approximately half the number of doses distributed under the NIP over the study period’.38

2.55 NCIRS did not estimate immunisation rates for the pneumococcal vaccine because of ‘the complexity of recommendations by age, Indigenous status and presence of at-risk medical conditions, and the short duration of data available on the register’. NCIRS reported the doses recorded on the AIR for adults as a percentage of vaccine distributed, which was 60.5 per cent between January 2017 and September 2018.

Through disease surveillance, identify the risks posed by unvaccinated cohorts in the population 2.56 The key action ‘through disease surveillance, identify the risks posed by unvaccinated cohorts in the population’ had not been completed by 2018, as Health had not established a clear process for identifying risks posed by unvaccinated cohorts. The key action was not included in the 2019-24 strategy.

2.57 Disease surveillance can be used to identify disease risks in the population and to monitor the effectiveness of disease prevention strategies, such as national immunisation programs. In Australia, responsibility for disease surveillance and disease risk management is shared among a number of key players at national, state and local levels and involves: coordination by the Australian Government, service delivery by the states and territories; and decision-making by committees with federal, state and territory representatives. The flow of disease surveillance information between key players in Australia is outlined at Figure 2.2.

Figure 2.2: Flow of disease surveillance information between key players in Australia

Australian Technical Advisory Group on Immunisation (ATAGI)

Commonwealth Department of Health

Australian Health Protection Principal Committee (AHPPC)

States and Territories

Communicable Diseases Network Australia (CDNA)

National Centre for Immunisation Research and Surveillance (NCIRS)

National Notifiable Diseases Surveillance System (NNDSS)

Source: ANAO analysis of Department of Health and NCIRS information.

2.58 Australia’s disease surveillance system is based on notifications of instances of disease collected by the state and territory health departments under their public health legislation and forwarded to Health through the National Notifiable Disease Surveillance System (NNDSS). The NNDSS was established in 1990 and is the primary system for disease surveillance in Australia. It is a passive surveillance system, receiving daily de-identified notifications of more than 50

38 NCIRS, Exploratory analysis of the first 2 years of adult vaccination data recorded on the AIR, November 2019, p. 7.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

31

communicable diseases, including 15 vaccine preventable diseases. The Communicable Diseases Network Australia (CDNA) meets fortnightly to share and evaluate the latest developments in disease surveillance, which includes a discussion on the data from the NNDSS. Aggregated NNDSS data is publicly available on Health’s website and published in the quarterly journal Communicable Diseases Intelligence.

2.59 Health funds NCIRS to undertake vaccine preventable disease surveillance activities. Since 2015 NCIRS has: produced two disease surveillance reports; conducted four national serosurveys and three epidemiological reviews; and continued to lead the Paediatric Active Enhanced Disease Surveillance (PAEDS) network, which is a hospital-based active surveillance system.39 NCIRS disease surveillance activities are used to provide support to Health and ATAGI for the development and implementation of national immunisation policies. NCIRS also supports ATAGI to develop content

for the Australian Immunisation Handbook.

2.60 Health does not have a risk management strategy in place for identifying and mitigating risks posed by unvaccinated cohorts in the population. The system for disease surveillance in Australia, however, does contribute to identifying risks relating to vaccine preventable diseases and to addressing those risks through policy changes, such as the addition of new vaccines to the NIP Schedule. An example of how disease surveillance led to a vaccine being included in the NIP is outlined in Case study 1.

Case study 1. Disease surveillance and the inclusion of the Meningococcal AWCY vaccine in the National Immunisation Program

From 2013 to 2017 the rate of invasive meningococcal disease notification increased in Australia, predominantly driven by increases in serogroup W (MenW). The CDNA convened a working group on MenW in 2015. From 2016 to 2017 the burden of disease was highest in children under two years old, with a secondary peak in adolescents. ATAGI discussed the increasing incidence caused by MenW at its regular meetings. In 2016 Health, representatives from the CDNA, CDNA MenW Working Group and NCIRS contributed to research on the rise of MenW in Australia from 2013 to 2015, which was based on NNDSS data.

In February 2017 ATAGI noted that vaccination programs with meningococcal ACWY vaccines were being implemented in several Australian states, but these programs differed by target age group, mode of delivery and vaccine brand. In June 2017 ATAGI reviewed the efficacy and safety of three meningococcal vaccines for their potential use in Australia. ATAGI updated the meningococcal chapter of the Australian Immunisation Handbook in 2018. In January 2018 the Pharmaceutical Benefits Advisory Committee accepted ATAGI advice and recommended that the meningococcal ACWY vaccine replace the meningococcal C vaccine on the NIP Schedule for children at 12 months of age. Meningococcal ACWY vaccines replaced Meningococcal C vaccines on the NIP Schedule for children at 12 months in July 2018.

Source: ANAO analysis of Department of Health documents and ATAGI meeting bulletins.

39 The two disease surveillance reports are: Vaccine Preventable Diseases and Vaccination Coverage in Aboriginal and Torres Strait Islander People, Australia, 2011-2015; and Summary of national surveillance data on vaccine preventable diseases in Australia, 2012-2015. Serosurveillance provides estimates of antibody levels against infectious diseases and is used for measuring population immunity due to past infection or vaccination. Epidemiological reviews look at the distribution and burden of disease in populations.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

32

2.61 The ANAO suggests that Health consider using its disease surveillance data to more clearly identify risks of vaccine preventable diseases in the Australian population to support early responses to emerging risks.

Develop an agreed position on the provision of free catch-up immunisation schedules

2.62 The key action to ‘develop an agreed position on the provision of free catch-up immunisation schedules’ was competed in 2017, when the government agreed with states and territories to provide free catch-up vaccines to all individuals up to the age of 19 years.

2.63 In the 2017-18 Health Portfolio Budget Statements the government committed $14.1 million over four years through the National Immunisation Program to fund ongoing catch-up vaccines for almost 375,000 Australians aged 10 to 19 years and more than 8000 adult refugees and humanitarian entrants. Health supplied resources including worksheets, factsheets and an online calculator to help vaccination providers plan a catch up schedule for individuals up to age 19 years, who had not received all vaccinations appropriate for their age. In 2019 Health also commissioned market research into awareness of the catch-up program among adolescents and their parents. The research found many respondents were unaware free catch-up vaccines were available up to the age of 19 and would appreciate more proactive communication about the program.

2.64 Health advised the ANAO that AIR data does not include an identifier for vaccines given as part of the NIP catch-up program. Health does not monitor how many people access the catch-up program and does not have oversight on whether the vaccines paid for under this program have been administered to the people eligible for the program.

Implementation plan for national immunisation strategies

2.65 When Health presented the national immunisation strategy to the state and territory governments and to Australian health ministers in 2013, it stated that it would develop an implementation plan to support the strategy in consultation with the states and territories. An implementation plan was not developed for the first or second versions of the strategy. The ANAO found during fieldwork that there was a lack of records and knowledge among departmental staff

about who was responsible for specific key actions or what had been done to achieve them.

2.66 A lack of clear responsibility, set timeframes and accountability can decrease the effectiveness of a program’s implementation. An implementation plan would facilitate effective oversight of the strategy, enabling Health to assign clear responsibility for each key action under the strategy, set timeframes for deliverables and report to senior management on progress.

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

33

Recommendation no. 2 2.67 The Department of Health implement a plan to operationalise the National Immunisation Strategy, which includes assigning clear responsibility for actions and outcomes and setting timeframes for delivery

Department of Health response: Agreed.

2.68 The Department will operationalise the National Immunisation Strategy through assigning responsibilities, and tracking timeframes for delivery, actions, and outcomes to improve management and governance of the National Immunisation Program.

Has immunisation coverage in Australia improved?

Reported immunisation coverage rates have improved for: all children, including Aboriginal and Torres Strait Islander children, at ages one and five years from 2012 to 2020; adolescent girls receiving two doses of the HPV vaccine from 2012 to 2017; and adolescent boys receiving two doses of the HPV vaccine from 2014 to 2017. Immunisation rates have not improved for children at age two years. Health does not report immunisation coverage rates for other vaccines or cohorts.

2.69 Health publishes immunisation coverage on its website for: all children at age one, two and five years; and Aboriginal and Torres Strait Islander children at age one, two and five years. It also published the rates of HPV vaccination for adolescent girls and boys up to 2017, but not for 2018, 2019 or 2020. It does not report on immunisation coverage for other vaccines or cohorts, such as adults 65 years and older.

All children at age one, two and five years

2.70 Reported immunisation coverage rates have improved for children at age one year from 91.7 per cent in 2012 to 94.9 per cent in 2020.40 Reported rates have also increased for children at age five years from 90.8 per cent in 2012 to 95.1 per cent in 2020. See Figure 2.3.

40 For the analysis of childhood immunisation rates, the ANAO selected 2012 as the baseline year as it is the year before the start of the 2013-18 National Immunisation Strategy.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

34

Figure 2.3: Immunisation coverage for all children at age 1, 2 and 5 years, 2012-20

Source: ANAO analysis of department of Health quarterly immunisation coverage data.

2.71 While immunisation coverage rates for children at age one and five years increased, rates for children at age two years have not increased since 2012, as shown in Table 2.6.

Table 2.5: Immunisation coverage for children at age 2 years, 2012-20

Age 2012 2013 2014 2015 2016 2017 2018 2019 2020

2 years 92.6% 92.2% 91.2% 89.3% 91.4% 90.5% 90.8% 91.6% 92.5%

Source: ANAO analysis of department of Health quarterly immunisation coverage data.

2.72 The reported coverage rate at age two years had already decreased slightly between 2012 and 2013, but it decreased further at the end of 2014 and into 2015 because of a change to Health’s definition of ‘fully immunised’ at age two years. In July 2014, the definition, as well as the AIR algorithm for determining coverage rates at age two years, was expanded to include doses of vaccines containing meningococcal C (MenC) and varicella (chickenpox) and a second dose of a vaccine containing measles, mumps and rubella (MMR2). This followed an update to the NIP Schedule in July 2013, where the second dose of an MMR-containing vaccine moved on the schedule from four years to 18 months. The coverage rate at age two years decreased again in late 2016 into 2017 following the inclusion of the fourth dose of the diphtheria, tetanus and pertussis vaccine (DTP4) in the definition for fully immunised at age two years.

Aboriginal and Torres Strait Islander children at age one, two and five years

2.73 Immunisation coverage rates reported for Aboriginal and Torres Strait Islander children have improved: at age one year, with rates increasing from 85.4 per cent in 2012 to 93.8 per cent in 2020; and at age five years, with rates increasing from 89.8 per cent in 2012 to 97.2 per cent in 2020. The reported rates for Aboriginal and Torres Strait Islander children at age two years

70%

75%

80%

85%

90%

95%

100%

2012 2013 2014 2015 2016 2017 2018 2019 2020

1 Year 2 Years 5 Years

Definition for ‘fully immunised’ at age 2 years changed to include MMR2, MenC

and varicella

Definition for ‘fully immunised’ at age 2 years changed to include DTP4

Improving immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

35

decreased from 92.2 per cent in 2012 to 91.4 per cent in 2020 because of a change to the definition of ‘fully immunised’ for two year olds, as discussed in paragraph 2.72. See Figure 2.4.

Figure 2.4: Immunisation coverage for Aboriginal and Torres Strait Islander children at 1, 2 and 5 years, 2012-20

Source: ANAO analysis of Department of Health quarterly immunisation coverage data.

Adolescents

2.74 Under the 2020 NIP Schedule, adolescents are eligible for vaccines against five diseases: HPV; diphtheria; tetanus; pertussis (whooping cough); and meningococcal ACWY. Health publishes immunisation coverage data for HPV vaccines, but not for the other four vaccines. Health has not established a target rate for HPV vaccine uptake and has not published HPV coverage data for 2018, 2019 or 2020.

2.75 The HPV vaccine is delivered as part of school vaccination programs to adolescents at age 12-13 years. Vaccines administered under this program were reported to the HPV Register from 2012 to 2017 and have been reported directly to the AIR since 2018. Data held in the HPV Register was migrated to the AIR in 2018.

2.76 Between 2012 and 2017 the percentage of girls vaccinated with three doses of the HPV vaccine increased from 71.9 per cent to 80.2 per cent. In 2013 the HPV program was extended to include boys aged 12-13. The rate of boys vaccinated with three doses of the HPV increased from 62.4 per cent in 2014 to 75.9 per cent in 2017. See Table 2.6.

70%

75%

80%

85%

90%

95%

100%

2012 2013 2014 2015 2016 2017 2018 2019 2020

1 Year 2 Years 5 Years

Definition for ‘fully immunised’ at age 2 years changed to include MMR2, MenC and

varicella

Definition for ‘fully immunised’ at age 2 years changed to include DTP4

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

36

Table 2.6: Reported HPV coverage for adolescents at age 15 years, 2012-17a

Cohort 2012 2013 2014 2015 2016 2017

Girls — 3 doses 71.9% 72.1% 74.8% 78.7% 79.7% 80.2%

Boys — 3 doses - - 62.4% 67.8% 73.8% 75.9%

Note a: The ANAO has not included rates for 2018, 2019 and 2020 because the data was not comparable to prior year data and Health could not provide assurance about the reliability of the data. Source: ANAO analysis of Department of Health documentation.

2.77 There were three key changes to the HPV vaccine program in 2018:

• a two-dose (9-valent) HPV vaccine replaced the three-dose (4-valent) vaccine on the NIP Schedule;

• HPV coverage data transferred from the HPV Register to the AIR and from 2018 onwards administered HPV vaccines were reported directly to the AIR by vaccination providers; and

• Health changed the methodology for determining HPV rates.

2.78 These changes have contributed to data challenges for HPV rates, as discussed in paragraphs 3.16-3.19, and to the delay in publishing HPV rates for 2018, 2019 and 2020.

Adults 65 years and older

2.79 Health does not have sufficient data to monitor immunisation coverage rates for adults, as reporting to the AIR had been voluntary, for NIP Schedule vaccines, until 1 July 2021. As discussed at paragraph 2.54, NCIRS found that, for 2018, 46.3 per cent of people over 65 years were vaccinated for influenza and 31.2 per cent of people between 70-80 years were vaccinated for shingles but noted that the figures likely substantially underestimated true vaccination uptake.41

41 The Australian Institute of Health and Welfare and a survey commissioned by Health estimated influenza vaccination rates for 2014-15 were greater than 70 per cent in adults aged 65 and over.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

37

3. Monitoring immunisation coverage

Areas examined This chapter examines whether the Department of Health (Health) has effectively monitored immunisation coverage.

Conclusion Health’s approach to monitoring immunisation coverage has been partially effective. Health does not gain assurance about the quality of the data it uses for monitoring and reporting immunisation coverage. In addition, its calculation of some coverage rates and the way they are publicly reported could lead readers to misinterpret coverage rates. While Health identifies and addresses areas of low immunisation coverage areas for children and the adolescent HPV vaccine, it does not identify or monitor areas of low coverage for other cohorts due to insufficient data. The performance measures for immunisation coverage are not fully adequate and, while Health has taken some action to assess performance, it does not systematically address issues raised in evaluations and program assessments.

Areas for improvement The ANAO made one recommendation aimed at ensuring that reporting about immunisation coverage is accurate and data definitions and calculation methodologies are clear.

The ANAO also suggested that Health: clarify responsibility for data quality and ensure that it has sufficient assurance about immunisation coverage data; consider whether the use of the ‘higher dose’ assumption is still appropriate; and ensure performance measures for the immunisation program are fully adequate.

3.1 Effectively monitoring immunisation coverage requires accurate and complete data. Health derives its immunisation data from the Australian Immunisation Register (AIR).

3.2 To determine if Health effectively monitors immunisation coverage, the ANAO examined whether Health:

• gains assurance about the data it uses for monitoring and reporting immunisation coverage;

• identifies geographic areas or cohorts of low coverage and takes action to address low coverage; and

• has assessed its approach to improving immunisation coverage.

Does Health gain assurance about the quality of the data it uses for monitoring and reporting immunisation coverage?

Health does not gain assurance about the quality of the data it uses for monitoring and reporting immunisation coverage and has not clarified responsibility for data quality. Definitions and methodologies used to define ‘fully immunised’ at ages one, two and five years, and the way in which these rates are reported, could lead readers to misinterpret immunisation coverage rates.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

38

3.3 To determine if Health gains assurance about the data it used for monitoring and reporting immunisation coverage, the ANAO examined how Health:

• derives immunisation coverage data;

• gains assurance about data quality;

• defines ‘fully immunised’;

• calculates ‘fully immunised’ rates; and

• reports on childhood immunisation rates.

Immunisation coverage data

3.4 Health derives its data on immunisation coverage from the AIR. The AIR is a national register for recording vaccines given to people of all ages in Australia. It was established as the Australian Childhood Immunisation Register (ACIR) in 1996 and initially only included vaccines that had been administered to children. In 2016 the ACIR was expanded into a whole-of-life national register to allow for the reporting of all vaccines given to people throughout their lifetime and the name was changed to the AIR. The AIR and the use of AIR data is governed by the Australian Immunisation Register Act 2015.

3.5 Only vaccination providers, such as general practitioners or community health centres, can report to the AIR.42 Until 2021 reporting to the AIR was not mandatory. Reporting levels have been high for childhood immunisation and low for other cohorts.

3.6 In February 2021 the Australian Immunisation Register Act 2015 was amended to require that vaccination providers report to the AIR on relevant vaccinations they have administered, with ‘relevant vaccinations’ to be prescribed by rules set by the Minister. The Australian Immunisation Register Rule 2015 (the AIR Rule) was also amended in February 2021 to prescribe that all COVID-19 and influenza vaccines must be reported to the AIR from 20 February 2021 and 1 March 2021 respectively. A subsequent amendment to the AIR Rule in June 2021 made the reporting of NIP Schedule vaccines mandatory from 1 July 2021. Civil penalties apply to vaccination providers that do not report relevant vaccinations as prescribed by the AIR Rule.43

3.7 Services Australia administers the AIR on behalf of Health according to a program agreement, which is supplemented by additional instructions when required.44 The 2018 program agreement outlines: roles and responsibilities for Health and Services Australia; deliverables; data release management policies and guidelines; funding arrangements; and the AIR governance structure. Health engages with Services Australia through a number of forums to discuss management of the AIR and to provide instructions on changes or enhancements, for example, when Health adds a new vaccine to the NIP Schedule.45

42 Section 4 of the Australian Immunisation Register Act 2015. 43 The civil penalty for failing to comply with mandatory reporting to the AIR was set at 30 penalty units. As at 1 July 2020, the value of a penalty unit is $222. 44 When the program agreement was signed in 2018, Services Australia was called the Department of Human

Services.

45 Forums include the AIR Committee, the AIR Program Agreement Managers Committee and the AIR Operations Group.

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

39

Assurance about data quality

3.8 The quality of data in the AIR is partly dependent on vaccination providers inputting accurate vaccination data. Services Australia has a number of controls and assurance activities in place to manage the quality of data in the AIR.46 These controls and activities are not outlined in the program agreement between Health and Services Australia. Further, the program agreement does not outline responsibility for data quality.

3.9 Under the program agreement, Services Australia is to provide Health with quarterly immunisation coverage reports within 10 working days of the end of each quarter. These include two sets of reports — one set for all records in the AIR and another set for Aboriginal and Torres Strait Islander records. Each set of reports is to include data at the following levels:

• state and territory;

• local government area;

• statistical local area;

• Primary Health Network;

• public health unit;

• postcode;

• Statistical Area Level 2 (SA2); and

• Statistical Area Level 3 (SA3).

3.10 Services Australia is also to provide additional specified reports to Health, state and territory health departments, the National Centre for Immunisation Research and Surveillance (NCIRS) and vaccination providers.47

3.11 The ANAO suggests that Health clarify responsibility for data quality and ensure that it has sufficient assurance about immunisation coverage data, including for childhood immunisation and HPV vaccinations.

Childhood immunisation data

3.12 For each quarter since 2015, Health has published childhood immunisation coverage data sets at the national, state and territory, Primary Health Network and SA3 levels.

3.13 For the national and state and territory data sets, ANAO analysis found that Health’s published quarterly data for all children up to age five years and for Aboriginal and Torres Strait Islander children up to age five years matched the AIR quarterly data that was provided to Health

46 The ANAO did not audit Services Australia’s data quality controls and assurance activities. Services Australia provided the ANAO with information about its data assurance activities, which include: checking data for noticeable anomalies; comparing quarterly data with previous quarters to identify significant increases or decreases; and reviewing ‘pending’ vaccination records where the AIR was unable to uniquely match an individual to an existing record. 47 Additional reports include information such as how many people are registered on the AIR, the number of

vaccinations given by states and territories and how many people on the AIR are not fully immunised. The additional reports have different reporting frequencies, including as required, monthly, quarterly, six-monthly and annually.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

40

by Services Australia. Health also applies a clear methodology for converting quarterly immunisation data into annual immunisation rates.

3.14 Health started publishing quarterly immunisation coverage reports for children at ages one, two and five years by SA3 in 2015. At the time, there were 351 SA3 areas in Australia (based on the 2011 SA3 codes). The Australian Bureau of Statistics (ABS) updated the SA3 areas in 2016, with the updates affecting 11 SA3 codes and increasing the number of SA3 regions by seven. The program agreement does not specify that the current SA3 codes be used. The SA3 codes in force at the time the program agreement was signed in 2018 were the 2016 SA3 codes. The ANAO found that, as at July 2021 Health was publishing SA3 data using the 2011 SA3 codes. Health should clarify with Services Australia its expectations regarding the parameters of the data provided from the AIR.

3.15 As the result of a geo-coding issue introduced during an upgrade to Services Australia’s systems in May 2017, 800,000 childhood immunisation records from May 2017 to early 2019 (when the issue was discovered) were not included in the SA3 and Primary Health Network reports. The SA3 coverage reports are published on Health’s website and are used for measuring the performance of states and territories under the National Partnership on Essential Vaccines (NPEV) agreement. As the issue had not been discovered prior to the preparation of the 2017-18 performance benchmark report, the assessment of state and territory performance was based on inaccurate data. For 2018-19, Health decided that because the SA3 coverage reports for the period contained geocoding errors, all states and territories would be deemed to have met the benchmark for improving coverage in low coverage areas for the purpose of receiving the linked reward payment.48

HPV vaccination data

3.16 As discussed in paragraphs 2.75-2.78, historical HPV vaccination data was transferred from the HPV Register to the AIR in 2018.49 A 2021 NCIRS report on the HPV immunisation program found a number of challenges related to the transition of HPV vaccination reporting to the AIR.50 These included ‘differences in the method of calculation and reporting of coverage rates, and the need for ongoing efforts to raise awareness among [vaccination] providers of the importance of HPV vaccination reporting’. The NCIRS reported that the challenges associated with the transition of existing records from the HPV Register to the AIR included:

• records without Medicare numbers;

• incorrect vaccination provider numbers;

• no consistent method of recording school details; and

• changes in names and addresses over time.

48 AIHW, National Partnership on Essential Vaccines: performance report 2018-19, 27 August 2020, p. iv. Available at www.aihw.gov.au [accessed 6 August 2021]. 49 Health advised the ANAO that HPV vaccination data was transferred from the HPV Register to the AIR in 2018 to ensure: HPV data was complete, as HPV vaccines administered by general practitioners and health clinics

were already reported directly to the AIR; HPV data is collected using the same standards for the AIR and Medicare records; and an individual’s AIR record contains all vaccinations received. 50 NCIRS, Impact evaluation of Australian national human papillomavirus vaccination program, Final Report, February 2021, pp. 84-85. Available at www.ncirs.org.au [accessed 24 May 2021].

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

41

3.17 According to the NCIRS report, these challenges ‘prevented the automatic matching and required developing new matching systems in the software or laborious manual matching of individual records’. Approximately 60,000 individual HPV vaccination records were able to be manually matched, but 30,000 could not be matched.51

3.18 There were also issues with duplicate records, with the NCIRS report stating:

The last published cohort for coverage estimates from the HPV Register was for adolescents aged 12-19 years in 2017, with data as notified to mid-2018. Notably by this time WA had ceased reporting to the HPV Register directly and the HPV Register was receiving large numbers of notifications from AIR, many of which duplicated state-based notifications. Best efforts were made to identify and remove such duplicates but because of missing data or data quality issues, this was not always possible.

3.19 Health reported immunisation rates from the HPV Register for adolescents from 2012 to 2017. As a result of the issues with the completeness of HPV data and the changes to the methodology, Health has not yet published HPV rates for 2018, 2019 and 2020.

Definitions for ‘fully immunised’ coverage rates

3.20 The ABS Data Quality Framework outlines seven dimensions of data quality, including interpretability, which enables information to be understood and used appropriately.52 The definitions for key terms should be clear so that readers are not confused or easily misled.

3.21 According to the 2020 National Immunisation Program (NIP) schedule, a child without any risk factors should receive at least 39 doses of vaccines against 13 diseases by age five — as outlined in Table 3.1.53 Several of these doses are provided in combination vaccines. For example, children receive a six-in-one vaccine (at two months, four months and six months) against diphtheria, tetanus, pertussis (whooping cough), hepatitis B, polio and Haemophilus influenzae type b (Hib).

51 Services Australia and the NCIRS report noted that as state and territory immunisation databases are integrated into the AIR, more records are expected to be matched; and that records can also be matched if an individual notifies Services Australia that a vaccine has been administered and the vaccination can be confirmed by the vaccination provider. 52 The seven dimensions in the ABS data quality framework are institutional environment, relevance, timeliness,

accuracy, coherence, interpretability and accessibility. 53 Children aged six months to less than five years are also eligible for the annual influenza vaccination program.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

42

Table 3.1: 2020 NIP Schedule vaccines and number of doses, all children up to 5 years

Vaccine 2

months

4

months

6

months

12

months

18

months

4

years

Total doses

Rotavirus 1 1 - - - 2

Diphtheria 1 1 1 - 1 1 5

Tetanus 1 1 1 - 1 1 5

Pertussis 1 1 1 - 1 1 5

Polio 1 1 1 - - 1 4

Haemophilus influenzae type b (Hib) 1 1 1 - 1 - 4

Hepatitis Ba 1 1 1 - - 3

Measles - - - 1 1 - 2

Mumps - - - 1 1 - 2

Rubella - - - 1 1 - 2

Pneumococcal 1 1 - 1 - - 3

Meningococcal ACWY - - - 1 - - 1

Varicella - - - 1 1

Total doses 8 8 6 5 8 4 39

Note a: An additional dose of Hepatitis B is sometimes given at birth. Source: ANAO analysis of the 1 July 2020 NIP Schedule.

3.22 Health reports quarterly on childhood immunisation coverage at age one, two and five years. However, these rates do not include all NIP Schedule vaccines. For example, the definition Health uses for ‘fully immunised’ at age five is different to the definition of ‘fully immunised’ as outlined on the NIP Schedule. The NIP Schedule includes 39 doses against 13 diseases in a child’s first five years (27 doses in the first year, eight doses in the second year and four doses in the fourth year), but only the last four of these doses are included in Health’s calculation of ‘fully immunised’ at five years old. Table 3.2 shows the doses on the NIP Schedule and the doses Health includes in its calculation for ‘fully immunised’ for each age cohort.

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

43

Table 3.2: NIP Schedule vaccine doses compared to vaccine doses included in Health’s ‘fully immunised’ calculations — at age 1, 2 and 5 years, 2019

Age Doses on 2020 NIP Schedule (including those given in previous age groups) Doses in department’s calculation of ‘fully immunised’a

1 year 27 doses against 12 diseases:

• 2 doses rotavirus

• 3 doses diphtheria, tetanus and pertussis

• 3 doses polio • 3 doses Hibb

• 3 doses hepatitis B

• 1 dose measles, mumps and rubella

• 3 doses pneumococcal • 1 does meningococcal

7 doses against 7 diseases:

• 3rd dose diphtheria, tetanus and pertussis

• 3rd dose polio

• 3rd dose Hibb

• 3rd dose hepatitis B

• 2nd dose pneumococcal

2 years 35 doses against 13 diseases:

• 2 doses rotavirus

• 4 doses diphtheria, tetanus and pertussis

• 3 doses polio

• 4 doses Hibb

• 3 doses hepatitis B

• 2 doses measles, mumps, rubella

• 3 doses pneumococcal

• 1 dose meningococcal

• 1 dose varicella

12 doses against 12 diseases:

• 4th dose diphtheria, tetanus and pertussis

• 3rd dose polio

• 4th dose Hibb

• 3rd dose hepatitis B

• 2nd dose measles, mumps, rubella

• 3rd dose pneumococcal

• 1 dose meningococcal

• 1 dose varicella

5 years 39 doses against 13 diseases:

• 2 doses rotavirus

• 5 doses diphtheria, tetanus and pertussis

• 4 doses polio

• 4 doses Hibb

• 3 doses hepatitis B

• 2 doses measles, mumps, rubella

• 3 doses pneumococcal

• 1 dose meningococcal

• 1 dose varicella

4 doses against 4 diseases:

• 5th dose diphtheria, tetanus and pertussis

• 4th dose polio

Note a: Health assumes that prior doses were given, but prior doses are not included in the calculation. Note b: Haemophilus influenzae type b (Hib). Source: ANAO analysis of Department of Health documentation.

3.23 The method for reporting immunisation coverage at five years also does not align with current requirements under the ‘No Jab, No Pay’ initiative. Most vaccinations received in a child’s first five years (for example, measles-mumps-rubella (MMR), Hib, pneumococcal, hepatitis B,

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

44

meningococcal and varicella) are not included in the calculation for ‘fully immunised’ at five years old.

3.24 In October 2019 Health recommended to the Minister that it develop a new ‘school ready’ indicator for reporting childhood immunisation coverage at age five years. The proposed indicator was to be based on more, but not all, of the NIP Schedule vaccinations received between birth and age five years. A proposed indicator considered by Health included the highest dose given for vaccines against 12 diseases (all those on the NIP Schedule except for the rotavirus vaccine). The

AIR contains data for the 12 vaccinations included in the proposed school ready indicator. Health ran a simulation looking at the proposed indicator for the 2017-18 financial year. Immunisation coverage rates using the proposed indicator would have been 3.8 per cent lower than the rates published using the current definition for ‘fully immunised’ at age five for the same period. Health advised the ANAO that it put the project on hold in 2020 to focus on COVID-19 vaccination priorities.

Methodology for calculating ‘fully immunised’ rates

3.25 The methodology for calculating and reporting childhood immunisation rates has not been updated since reporting began in 1998. There are three elements of the methodology that lead to differences between vaccine doses on the NIP Schedule for each age group and the doses included in the fully immunised assessment for each cohort.

• Health relies on a ‘higher dose assumption’.

• The number of diseases children are to be vaccinated against is different from the number of diseases included in the calculation of fully immunised for each cohort.

• Health does not have a consistent approach to including or excluding vaccinations from a previous age cohort (for example, the third dose of the polio vaccine is included for fully immunised at age two years, even though this vaccine is given at age six months. By contrast, at age five years, the only vaccines included in the count are those a child would have had after age four years).

3.26 Under Health’s 1998 higher dose assumption methodology, if a child’s records indicate they have received a vaccine at the age that the last vaccine in a sequence is due (for example, the third dose), it is assumed that the earlier vaccine doses in the sequence (for example, doses one and two) have been given. This assumption is not tested or verified.54 With the inclusion of the ‘higher dose assumption’ in the methodology, the current definitions of ‘fully immunised’ for one, two and five years of age do not capture all vaccine doses given to a child — even for the vaccines that are included in the calculation. For example, by age one year, children should have received at least 27 doses against 12 diseases, but the methodology for determining whether a child at age one year is fully immunised includes only seven doses against seven diseases.

3.27 The most recent study on the methodology was in 2001, which NCIRS undertook to estimate the impact of the higher dose assumption on immunisation rates at age one year. The results of the study were published in 2003 in Health’s journal, Communicable Diseases Intelligence. The article stated:

54 National Centre for Disease Control and Department of Health, Methodology for measuring Australia’s childhood immunisation coverage, p. 36, CDI Vol 22, No. 3, 19 March 1998.

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

45

The assumption was considered appropriate when reporting of immunisation coverage … first commenced in 1998 because the ACIR is based on the Medicare database and a delay in Medicare registration was likely to affect recording of the first and second vaccine doses due at 2 and 4 months of age. Without the ‘third dose assumption’, the underestimation of coverage by the ACIR, related to incomplete reporting by providers and delayed Medicare registration, would be substantially greater. Since 1998, incentives to general practitioners to notify vaccinations to the ACIR have been introduced and there has been a significant reduction in delays in Medicare registration. As the ACIR has matured and coverage has increased substantially, continued use of the ‘third dose assumption’ in calculating and reporting immunisation coverage has been questioned.55

3.28 NCIRS found that the difference between the rates at age one year with and without the higher dose assumption was 7.02 per cent. An assessment that included all required doses resulted in a rate of 84.16 per cent at age one year, while Health had reported a rate of 91.18 per cent. NCIRS conducted a survey of a sample of children from that 7.02 per cent who were counted as fully immunised at age one year even though one or more required doses was not on their record.56 NCIRS found 86.3 per cent of the sampled children were fully immunised at one year. NCIRS determined that the adjusted overall rate for fully immunised at age one year would have been 90.22 per cent — about one per cent lower than the published rate of 91.18 per cent.

3.29 Health has not reviewed the use of the higher dose assumption since the 2001 NCIRS review. Given that vaccination providers are now reporting to the AIR more consistently, Health could consider whether the use of the higher dose assumption is still appropriate.

Reporting on childhood immunisation coverage 3.30 Health reports information about immunisation coverage through a range of channels, including in its annual report, on its website and in its vaccination awareness campaign materials. However, as discussed in the examples below, Health does not clearly explain or outline its definitions for immunisation rates in its reporting.

3.31 From 2015-16 to 2019-20 Health’s annual reports included results against the performance measure ‘Immunisation coverage rates in children at 5 years of age are increased and maintained at the protective rate of 95%’. None of these reports defined how the immunisation coverage rates were calculated.

3.32 Health has a series of webpages devoted to childhood immunisation coverage. The main ‘Childhood immunisation coverage’ page introduces the topic by stating:

Immunisation coverage is the percentage of children who have had all vaccines recommended for their age. For the second quarter in a row we met the target of 95% coverage for five year olds, with 95.22% coverage. Coverage for one year olds has increased to 94.91%, and for two year olds is 92.53%.57

55 ‘Estimating immunisation coverage: is the ‘third dose assumption’ still valid?’, Communicable Diseases Intelligence Volume 27, No 3, September 2003, p. 357. The study was conducted by NCIRS and funded by the department. 56 Approximately 4500 children (seven per cent) of the approximately 63,000 children in the cohort were eligible

for the study. Of those eligible, 225 (five per cent) were surveyed. 57 Department of Health, Childhood immunisation coverage [Internet], available from https://www.health.gov.au/health-topics/immunisation/childhood-immunisation-coverage#what-is-childhood-immunisation-coverage [accessed 18 June 2021].

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

46

3.33 Health’s website also includes a webpage on immunisation coverage rates for all children, which shows rates at the national, state and territory and local level.58 The pages do not include an explanation or definition of what is included in the calculations for the rates. Further, Health’s statement that ‘immunisation coverage is the percentage of children who have had all vaccines recommended for their age’ is not accurate. Health does not include all recommended vaccines in its calculations.

3.34 The ‘Get the Facts’ vaccination awareness campaign website states ‘Nearly 95% of five-year-old children in Australia are vaccinated’.59 An embedded video, ‘Immunisation facts in 90 seconds’ states ‘Nearly 95 per cent of Aussie five year olds are now fully vaccinated’. A linked Questions about vaccination booklet includes the statement ‘about 94 per cent of Australian children aged under 5 are fully immunised against 15 diseases’.60 These statements are not accurate. The ‘fully immunised’ statistic for five year olds is based on the assessment of whether children at age five have received four vaccine doses against four diseases. Further, many Australian children would be vaccinated against the 13 diseases on the NIP childhood immunisation schedule as well as against influenza, for a total of 14, not 15 diseases, as stated.

3.35 Health uses data definitions that do not align with the NIP Schedule and it does not include data definitions in its reporting. Consequently, reporting about immunisation coverage is not always accurate and could lead readers to believe that rates for children are marginally higher than those calculated based on all NIP Schedule vaccines and doses.

Recommendation no. 3 3.36 The Department of Health ensure that reporting about immunisation coverage is accurate, and that data definitions and the methodology used to calculate immunisation coverage rates are clear.

Department of Health response: Agreed.

3.37 While the calculation of immunisation coverage is undertaken in accordance with agreed methodology, the Department will improve the accuracy of reporting and transparency on the methodology applied to reduce the risk that reporting could be misinterpreted.

3.38 The Department will also strengthen the program agreement with Services Australia and document internal quality assurance processes to enhance the quality of the Department's immunisation coverage data under the National Immunisation Program.

58 Department of Health, Immunisation coverage rates for all children [Internet], available from https://www.health.gov.au/health-topics/immunisation/childhood-immunisation-coverage/immunisation-coverage-rates-for-all-children [accessed 18 June 2021].

59 Department of Health, Get the Facts: The top facts about immunisation [Internet], available from https://campaigns.health.gov.au/immunisationfacts/top-facts-about-immunisation [accessed 15 June 2021]. 60 Department of Health, Questions about vaccination [Internet], 2018, p. 5, available from https://campaigns.health.gov.au/immunisationfacts [accessed 25 May 2021].

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

47

Does Health identify geographic areas or cohorts of low coverage and take action to address low coverage?

Health identifies and addresses geographic areas and cohorts of low coverage for children and adolescents by encouraging states and territories to improve rates in low coverages areas. Health does not identify areas of low immunisation coverage for other cohorts due to insufficient data.

3.39 The ANAO examined how Health identifies and addresses areas of low coverage for children, adolescents and adults.

Low coverage for children

3.40 In 2020 reported immunisation rates were above 90 per cent for children at ages one, two and five years in every state and territory, as shown in Table 3.3. For ages one year and five years, most states and territories had reached Health’s target of 95 per cent by 2020.

Table 3.3: Fully immunised at ages 1, 2 and 5 years, by state and territory, 2020a

State or territory 1 year 2 years 5 years

Australian Capital Territory 96.1% 93.3% 95.5%

New South Wales 94.8% 92.3% 94.9%

Victoria 95.1% 93.2% 96.1%

Queensland 94.6% 92.8% 94.7%

South Australia 95.0% 92.8% 95.1%

Western Australia 94.5% 91.4% 94.1%

Tasmania 95.1% 90.4% 94.8%

Northern Territory 95.5% 91.9% 94.9%

Australia 94.8% 92.5% 95.1%

Key: Reported rates above Health’s target of 95 per cent are shaded in green. Note a: ‘Fully immunised’ as defined by the Department of Health. Source: ANAO analysis of Department of Health documentation.

3.41 As mentioned in paragraphs 3.8-3.15, Health also publishes childhood immunisation rates at the local level. In 2020 childhood immunisation coverage by local SA3 ranged from 80 per cent to 99 per cent across Australia:

• for children at one year, reported immunisation rates ranged from 82 per cent in ‘Richmond Valley-Coastal’ in New South Wales to 99 per cent in ‘Grampians’ in Victoria (see Figure 3.1);

• for children at two years, reported immunisation rates ranged from 80 per cent in ‘Richmond Valley-Coastal’ in New South Wales to 98 per cent in ‘Broken Hill and Far West’ in New South Wales (see Figure 3.2); and

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

48

• for children at five years, reported immunisation rates ranged from 84 per cent in ‘Richmond Valley-Coastal’ in New South Wales to 99 per cent in Daly-Tiwi-West Arnhem’ in the Northern Territory (see Figure 3.3).61

Figure 3.1: Australian immunisation coverage at age 1 year, by SA3, 2020

Note: The Department of Health did not publish data for eight of the 358 SA3 areas, which are white in the figure. Source: ANAO analysis of Department of Health and Australian Bureau of Statistics data. Prepared using ‘National Map’, a Geoscience Australia tool.

61 As discussed in the section above, the rates for ‘fully immunised’ at each cohort do not include all vaccines administered up to that age.

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

49

Figure 3.2: Australian immunisation coverage at age 2 years, by SA3, 2020

Note: The Department of Health did not publish data for eight of the 358 SA3 areas, which are white in the figure. Source: ANAO analysis of Department of Health and Australian Bureau of Statistics data. Prepared using ‘National Map’, a Geoscience Australia tool.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

50

Figure 3.3: Australian immunisation coverage at age 5 years, by SA3, 2020

Note: The Department of Health did not publish data for eight of the 358 SA3 areas, which are white in the figure. Source: ANAO analysis of Department of Health and Australian Bureau of Statistics data. Prepared using ‘National Map’, a Geoscience Australia tool.

NPEV performance benchmarks for low coverage

3.42 Health has used the 2009 and 2017 NPEV agreements with state and territory governments to identify and monitor areas of low coverage. Each version of the agreement outlines performance benchmarks for the states and territories that are linked to annual incentive payments from the Australian Government. States and territories were eligible for a single reward payment if they met at least two of the four performance benchmarks in the 2009 NPEV agreement and were eligible for a separate reward payment for each of the five benchmarks they met in the 2017 agreement.62

62 Under the 2017 NPEV agreement, 0.75 per cent of the cost of each state’s and territory’s total vaccine purchases was linked to each of the five performance benchmarks, up to a total of 3.75 per cent. Under the 2009 NPEV agreement, each state and territory would receive a single reward payment if it had met at least two of the four benchmarks, with the reward payment ranging from 20 per cent (2009-10) to 60 per cent (2011-12 to 2016-17) of the state’s or territory’s incentive payment, which was equal to the state’s or territory’s share of four per cent of the total essential vaccine funding expenditure for the year.

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

51

The NPEV performance benchmarks include improving immunisation coverage in areas and cohorts of low coverage, as outlined in Table 3.4.

Table 3.4: NPEV performance benchmarks, 2009 and 2017 NPEV agreements

2009 agreement 2017 agreement

Maintaining or increasing vaccine coverage for [Aboriginal and Torres Strait Islander people]

An increase in the vaccination coverage rates for Aboriginal and Torres Strait Islander people in at least two of the following three cohorts: [one year olds; two year olds; and five year olds], relative to the baselinea

Maintaining or increasing coverage in agreed areas of low immunisation coverage

An increase in vaccination coverage rates for [five year olds] in four of the ten lowest vaccination coverage SA3 geographical areas, relative to the baseline. (States to notify the Commonwealth by August the four areas to be targeted that year)

Maintaining or increasing vaccination coverage for four year olds

An increase in vaccination coverage rates for [five year olds] relative to the baselinea

Maintaining or decreasing wastage and leakage Annual decrease in the wastage and leakage rate for agreed vaccines, relative to the baseline (where a State achieves a

wastage and leakage rate of 5 per cent or lower, it will be deemed to have met the benchmark)

- An increase in the vaccination coverage rate for both adolescent

boys and adolescent girls for HPV, relative to the baseline

Note a: According to the NPEV agreement, where a state or territory achieves a coverage rate for the year of 95 per cent or higher for a particular cohort, it will be deemed to have met the target for that cohort. Source: 2009 and 2017 national partnership agreements on essential vaccines.

3.43 For the 2009 low coverage performance benchmark, ‘agreed areas’ of low immunisation coverage were defined as areas where rates were ‘more than 5% below national levels for immunisation coverage rates for 12 - < 15 months, and 60 - < 63 months of age’. Each state and territory was assessed annually against this benchmark. From 2009-10 to 2016-17 all states and territories met the benchmark on low coverage areas in seven of the eight years.63 However, in most instances a state or territory was ‘deemed to have met’ the benchmark as there were no areas in its jurisdiction that met the criteria of a low coverage area. In the 2016-17 NPEV performance report, for example, all states and territories were assessed as having met the benchmark, with the caveat that no jurisdictions had ‘agreed low coverage areas’.

3.44 In 2009-10 rates in low coverage areas ranged from:

• 86.7 per cent to 88.3 per cent for one year olds; and

• 80.9 per cent to 82.5 per cent for five year olds.

3.45 By 2015-16, the rates in low coverage areas had increased, with rates ranging from:

• 88.8 per cent to 89.2 per cent for one year olds; and

63 2012-13 was the only year where states did not meet one of their targets. New South Wales and Tasmania each met one of their two targets; Western Australia met both of their targets; and the remaining states and territories did not nominate areas and were deemed to have met the benchmark.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

52

• 86.7 per cent to 90.2 per cent for five year olds.64

3.46 The 2017 NPEV agreement defined areas of low coverage as areas with rates below the target rate of 95 per cent. Each state and territory is assessed annually and receives payments for increasing immunisation rates in each of their four nominated areas. In 2017-18 all states and territories met the benchmark, except for one state, which partially met the benchmark. The performance of the states and territories could not be assessed in 2018-19 and all were ‘deemed to have met’ the benchmark for those years (see paragraph 3.15). For 2019-20, three states and territories met the low coverage benchmark and five partially met the benchmark (see Table 3.5).

Table 3.5: Assessment against NPEV benchmark: increasing vaccination coverage for five year olds in low coverage areas, 2019-20

State or territory Benchmark meta Number of geographical areas

with increase (of four)

Australian Capital Territory Partially 3

New South Wales Partially 3

Victoria Partially 2

Queensland Yes 4

South Australia Yes 4

Western Australia Yes 4

Tasmania Partially 2

Northern Territory Partially 2

Note a: To meet the benchmark, the state or territory must increase vaccination coverage rates for 60-<63 month olds (that is, five year olds) in four of the ten lowest vaccination coverage SA3 geographical areas, relative to the baseline. Source: ANAO analysis of National Partnership on Essential Vaccines: performance report 2019-20, p. 28.

3.47 To address geographic areas of low immunisation coverage, Health used geo-targeting in its childhood immunisation education campaign ‘Get the Facts’. A 2019 evaluation of phase three of the campaign found geo-targeted areas were more likely than the rest of Australia to take relevant action following exposure to the campaign (91 per cent compared to 80 per cent).

Low coverage for adolescents

3.48 The 2017 NPEV agreement identified adolescents as a target cohort and incentivises states and territories to increase the immunisation coverage rate for adolescent boys and girls relative to the baseline, with states receiving payments for increasing vaccination rates in one or two cohorts. The benchmark was not assessed in 2017-18 due to the ongoing transition of data from the HPV register to the AIR. In 2018-19 all states and territories were assessed as having met the

64 In 2016-17 no states identified an area of low coverage and all states were deemed to have met the low coverage benchmark, so 2015-16 rates have been used to show the increase over the period of the 2009 NPEV agreement.

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

53

performance benchmark. According to the 2019-20 NPEV performance report all states and territories except New South Wales were deemed to have met the benchmark.65

3.49 Adolescents are also eligible to receive the meningococcal ACWY vaccine and booster doses of diphtheria, tetanus and pertussis (whooping cough). Health does not have sufficient data to identify areas of low coverage for these vaccines.

3.50 Health also monitors vaccination coverage rates for children and adolescents through annual immunisation coverage reports produced by NCIRS. These reports identify trends in immunisation data and their relationship to relevant policy and program changes. The 2017 NCIRS annual coverage report outlined cohorts of lower coverage, including Aboriginal and Torres Strait Islander adolescents, noting that 79 per cent of 15-year-old Aboriginal and Torres Strait Islander girls who received the first dose of HPV vaccine had completed the course of three doses, compared to 91 per cent among non-Indigenous girls. Adolescent HPV coverage rates have not been included in the annual coverage reports since 2017.

Low coverage for adults

3.51 Health does not have sufficient data to monitor or identify geographic areas of low coverage for adult or high risk cohorts due to low levels of reporting of vaccine uptake. The NPEV does not identify any performance benchmarks in relation to these groups.

Has Health assessed its approach to improving immunisation coverage?

Health has conducted two maturity assessments on the NIP and contracts NCIRS to evaluate individual vaccination programs. However, it has not assessed its overall approach to improving immunisation coverage and does not systematically consider or address key learnings from the assessments undertaken. Health has established and reports on performance measures for immunisation, but these measures are not fully adequate for determining if Health’s objective for immunisation is being achieved.

3.52 Establishing adequate performance measures and assessing a program against the measures enables an entity to measure a program’s progress or success, identify areas for improvement and be held accountable for its outcomes.

3.53 To review how Health assesses its approach to improving immunisation coverage, the ANAO examined Health’s relevant performance measures, assessments and evaluations.

Performance measures

3.54 The 2021-22 Health Portfolio Budget Statements outlined the following objective for Program 1.9 Immunisation: Reduce the incidence of vaccine preventable diseases to protect individuals and increase national immunisation coverage rates to protect the Australian community. Three performance measures support this objective:

65 New South Wales was not assessed against the HPV benchmark or deemed to have met it because it implemented the two-dose HPV program in advance of amendments to the NIP.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

54

• Immunisation coverage rates for children at five years of age are increased and maintained at the protective rate of 95% (2021-22 target: ≥95%);

• Immunisation coverage rates for Aboriginal and Torres Strait Islander children 12-15 months of age are increased to close the gap (2021-22 target: ≥94%); and

• Immunisation coverage rates for adults at increased risk of vaccine preventable diseases due to age or underlying medical conditions (no targets).

3.55 The first two performance measures are measurable and have targets.66 The third measure, which is new in 2021-22, is unclear and not measurable. The measure describes a cohort (adults at increased risk of vaccine preventable diseases due to age or underlying medical condition) but not what it is intending to achieve for this cohort.

3.56 The three measures are relevant, but it is not clear why these cohorts were chosen over other cohorts who receive vaccines under the NIP (such as all children at age one or two years, or Aboriginal and Torres Strait Islander children at age two or five years. To enable assessment against its objectives, Health should ensure that its performance measures for the immunisation program are fully adequate.

Assessments and evaluations

3.57 While Health has not developed a program evaluation framework for the NIP, it has conducted two maturity assessments of the NIP and contracted NCIRS to evaluate the introduction of vaccines to the NIP.

Maturity assessments of the National Immunisation Program

3.58 In 2018 Health conducted a maturity assessment of the NIP, which found it was generally operating as intended, with documented policies and processes, strong stakeholder communication and increasing digitisation of processes. The assessment identified areas for improvement to the program’s governance and change management processes and noted AIR data transfer issues.

3.59 In 2020 Health commissioned a maturity assessment of the NIP. The objective was to assess the existence, quality and alignment of NIP processes, procedures and governance arrangements. The assessment found that: NIP activities were closely aligned to its strategic priorities; key business activities were supported by formalised processes; and processes were being followed. The review identified shortfalls in relation to management reports, ad-hoc performance evaluation and a reliance on manual reporting. The review recommended actions to increase the program’s maturity level, including the use of robust reporting applications to reduce reliance on manual processing. Health has made some improvements to address the issues identified in the review and is working to improve its data management, but it is still reliant on manual processing and spreadsheets in some cases.

66 The Department of Finance’s Resource Management Guide No.131 (RMG 131) — Developing good performance information — provides guidance to entities on the criteria and assessment characteristics of good performance information. Based on this guidance, the ANAO has assessed whether the performance measures for the immunisation program are measurable, relevant and adequate. RMG 131 is available from www.finance.gov.au [accessed 7 June 2021].

Monitoring immunisation coverage

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

55

NCIRS vaccination program evaluations

3.60 Health has contracted NCIRS to evaluate the introduction of vaccines on the NIP since 2006. Between 2015 and 2021 NCIRS conducted six evaluations of vaccine programs:

• Process Evaluation: Hepatitis A immunisation program for Aboriginal and Torres Strait Islander Children in the Northern Territory, Queensland, South Australia and Western Australia (2015);

• Introduction of measles-mumps-rubella-varicella (MMRV) vaccine on the National Immunisation Program (2016);

• Introduction of the Haemophilus influenzae type b-meningococcal C (Hib-MenC) vaccine on the National Immunisation Program (2016);

• Process Evaluation of the influenza immunisation program for Aboriginal and Torres Strait Islander children aged 6 months to <5 years (2018);

• Evaluation of the National Shingles Vaccination Program Process and Early Impact Evaluation (2019); and

• Impact Evaluation of the National Human Papillomavirus Vaccination Program (2021).

3.61 While the scope varied for each evaluation, the aims included: to describe the implementation of the vaccine program; and to identify the strengths and challenges of the program. Some also looked at vaccine safety and access issues. NCIRS used tools such as interviews with key stakeholders, online surveys of vaccination providers, computer-assisted telephone interviews of consumers and a review of policy, operational and communication documents. Most reports included recommendations, many of them suggestions from interviewed stakeholders.

3.62 Health does not have a process for ensuring that learnings from program assessments and evaluations are considered and does not systematically address or action items raised in the NCIRS evaluation reports or track progress against the recommendations. Health advised the ANAO that ‘each report is requested with a view to understanding what is happening in the landscape and whether or not the changes can/should be addressed under the National Immunisation Program’. Health provided an example where a recommendation in an NCIRS report led to a policy change. The 2018 process evaluation of the influenza program for Aboriginal and Torres Strait Islander children included the stakeholder recommendation: ‘Free influenza vaccine for all Aboriginal and Torres Strait Islander children and adults’. From July 2020, the NIP Schedule was updated to include the influenza vaccine for all Aboriginal and Torres Strait Islander people aged 6 months of age and older.

3.63 Establishing a process for considering and applying key learnings and reporting to senior management on how Health is responding to key findings and recommendations would enable Health to achieve greater value from these evaluations and assessments and allow for better oversight on emerging issues.

Grant Hehir Auditor-General

Canberra ACT 22 September 2021

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

57

Appendices

Audi

tor-General Report No. 5 2021-22 Improving Immunisation Coverage

58

Appendix 1 Entity responses

Appendix 1

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

59

Audi

tor-General Report No. 5 2021-22 Improving Immunisation Coverage

60

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021-22 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;

• introducing or revising policies, strategies, guidelines or administrative processes; and

• initiating reviews or investigations.

4. During the course of the audit, the ANAO did not observe changes in Heath’s approach to improving immunisation coverage.

Auditor-General Report No. 5 2021-22 Improving Immunisation Coverage

61

Appendix 3 National Immunisation Program Schedule, 1 July 2020

Table A.1: NIP Schedule for children from birth to age 4 years, 1 July 2020

Age Disease

Birth Hepatitis B

2 months

Diphtheria, tetanus, pertussis (whooping cough), hepatitis B, polio, Haemophilus influenzae type b (Hib)

Rotavirus

Pneumococcal

Meningococcal Ba

4 months

Diphtheria, tetanus, pertussis (whooping cough), hepatitis B, polio, Haemophilus influenzae type b (Hib)

Rotavirus

Pneumococcal

Meningococcal Ba

6 months Diphtheria, tetanus, pertussis (whooping cough), hepatitis B, polio, Haemophilus influenzae type b (Hib)

Pneumococcalb, c

Meningococcal Bd

12 months

Meningococcal ACWY

Measles, mumps, rubella

Pneumococcal

Meningococcal Ba

18 months Haemophilus influenzae type b (Hib)

Measles, mumps, rubella, varicella (chickenpox)

Diphtheria, tetanus, pertussis (whooping cough)

Hepatitis Ac

4 years Diphtheria, tetanus, pertussis (whooping cough), polio

Pneumococcalb, c

Hepatitis Ac

Key: Shaded boxes indicate additional vaccines available to Aboriginal and Torres Strait Islander children or those with specified risk conditions. Note a: Available to Aboriginal and Torres Strait Islander children. Note b: Available to children with specified risk conditions. Note c: Available to Aboriginal and Torres Strait Islander children in Western Australian, Northern Territory, South

Australia and Queensland. Note d: Available to Aboriginal and Torres Strait Islander children with specified medical risk conditions. Source: ANAO representation of information from the National Immunisation Program Schedule, 1 July 2020.

Audi

tor-General Report No. 5 2021-22 Improving Immunisation Coverage

62

Table A.2: NIP Schedule for adolescents and adults, 1 July 2020

Age Disease

12-13 years Human papillomavirus (HPV)

Diphtheria, tetanus, pertussis (whooping cough)

14-16 years

Meningococcal ACWY

50 years and over

Pneumococcal (Aboriginal and Torres Strait Islander people)

70 years and over

Pneumococcal

70-79 years Herpes zoster (shingles)a

Pregnant women

Pertussis (whooping cough)

Key: Shaded boxes indicate additional vaccines available to Aboriginal and Torres Strait Islander people). Note a: The herpes zoster (shingles) vaccine was added to the NIP Schedule in November 2016. Source: ANAO representation of information from the National Immunisation Program Schedule, 1 July 2020.