Note: Where available, the PDF/Word icon below is provided to view the complete and fully formatted document
Auditor-General—Audit report No. 42 of 2021-22—Performance audit—Procurement of delivery partners for the Entrepreneurs’ Programme: Department of Industry, Science, Energy and Resources


Download PDF Download PDF

The Auditor-General Auditor-General Report No. 42 2021-22 Performance Audit

Procurement of Delivery Partners for the Entrepreneurs’ Programme

Department of Industry, Science, Energy and Resources

Australian National Audit Office

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

2

© Commonwealth of Australia 2022

ISSN 1036-7632 (Print) ISSN 2203-0352 (Online) ISBN 978-1-76033-753-7 (Print) ISBN 978-1-76033-754-4 (Online)

Except for the content in this document supplied by third parties, the Australian National Audit Office logo, the Commonwealth Coat of Arms, and any material protected by a trade mark, this document is licensed by the Australian National Audit Office for use under the terms of a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 Australia licence. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/3.0/au/.

You are free to copy and communicate the document in its current form for non-commercial purposes, as long as you attribute the document to the Australian National Audit Office and abide by the other licence terms. You may not alter or adapt the work in any way.

Permission to use material for which the copyright is owned by a third party must be sought from the relevant copyright owner. As far as practicable, such material will be clearly labelled.

For terms of use of the Commonwealth Coat of Arms, visit the It’s an Honour website at https://www.pmc.gov.au/government/its-honour.

Requests and inquiries concerning reproduction and rights should be addressed to:

Senior Executive Director Corporate Management Group Australian National Audit Office GPO Box 707 Canberra ACT 2601

Or via email: communication@anao.gov.au.

Audi

tor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

3

Canber

ra ACT

24 June 2022

Dear Mr President Dear Mr Speaker

In accordance with the authority contained in the Auditor-General Act 1997, I have undertaken an independent performance audit in the Department of Industry, Science, Energy and Resources. The report is titled Procurement of Delivery Partners for the Entrepreneurs’ Programme. Pursuant to Senate Standing Order 166 relating to the presentation of documents when the Senate is not sitting, I present the report of this audit to the Parliament.

Following its presentation and receipt, the report will be placed on the Australian National Audit Office’s website — http://www.anao.gov.au.

Yours sincerely

Grant Hehir Auditor-General

The Honourable the President of the Senate The Honourable the Speaker of the House of Representatives Parliament House Canberra ACT

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

4

AUDITING FOR AUSTRALIA

The Auditor-General is head of the Australian National Audit Office (ANAO). The ANAO assists the Auditor-General to carry out his duties under the Auditor-General Act 1997 to undertake performance audits, financial statement audits and assurance reviews of Commonwealth public sector bodies and to provide independent reports and advice for the Parliament, the Australian Government and the community. The aim is to improve Commonwealth public sector administration and accountability.

For further information contact: Australian National Audit Office GPO Box 707 Canberra ACT 2601

Phone: (02) 6203 7300 Email: ag1@anao.gov.au

Auditor-General reports and information about the ANAO are available on our website: http://www.anao.gov.au

Audit team

Sean Neubeck Jocelyn Watts Joshua Carruthers Brian Boyd

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

5

Contents Summary and recommendations .................................................................................................................... 7

Background ............................................................................................................................................... 7

Conclusion ................................................................................................................................................. 8

Supporting findings .................................................................................................................................... 8

Recommendations ................................................................................................................................... 11

Summary of entity response .................................................................................................................... 13

Key messages from this audit for all Australian Government entities ..................................................... 14

Audit findings .............................................................................................................................................. 15

1. Background ............................................................................................................................................... 16

Introduction .............................................................................................................................................. 16

Rationale for undertaking the audit ......................................................................................................... 17

Audit approach ........................................................................................................................................ 17

2. Did the procurement process demonstrate achievement of value for money? ....................................... 19 Were relevant evaluation criteria included in request documentation to enable the proper identification, assessment and comparison of submissions? ............................................................ 19

Were the successful candidates those assessed as providing the best value for money? .................... 25 Were procurements conducted ethically, including identifying and managing any conflicts of interest and transparently managing incumbency advantage?.......................................................... 38

Were appropriate procurement records maintained? .............................................................................. 46

Was the procurement approach open, fair and non-discriminatory? ...................................................... 47

3. Are the contracts being appropriately managed to achieve the objectives of the procurement? ........... 53 Have relevant contracts and amendments been accurately reported on AusTender within 42 days of a contract being entered into or amended? ........................................................................... 53

Is there an appropriate contract management framework in place? ....................................................... 56

Do the contracts set clear performance requirements, and a means to effectively measure performance against the requirements? ............................................................................................ 64

Appendices ................................................................................................................................................. 69

Appendix 1 Entity response ................................................................................................................... 70

Appendix 2 Improvements observed by the ANAO ............................................................................... 81

Appendix 3 Evaluation criteria in the RFT ............................................................................................. 82

Appendix 4 Errors identified by the ANAO in the department’s tender evaluation report ...................... 85 Appendix 5 Probity plan requirements not fully implemented ................................................................ 89

Appendix 6 Tenderer complaints about the procurement ...................................................................... 92

Appendix 7 Non-compliances with the Commonwealth Procurement Rules......................................... 95

Appendix 8 Non-compliance with ethical requirements ....................................................................... 100

Appendix 9 Portfolio management tool dashboard extract .................................................................. 104

Auditor-General Report No.42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

 The Entrepreneurs’ Programme was

announced as part of the 2014-15 Budget with funding of $484.2 million and forms part of the Australian Government’s industry policy.

 The procurement of delivery partners is a key

element for the delivery of a services and grants program that is part of the National Innovation and Science Agenda.

 The design and conduct of the

procurement did not comply with the Commonwealth Procurement Rules, and the signed contracts are not being appropriately managed.

 In its conduct of the procurement, the

department did not demonstrate achievement of value for money. There was not open and effective competition for the delivery partner roles as competing tenders were not treated fairly or equitably.

 The delivery partner contracts are not

being appropriately managed. The contract management framework is inadequate and the contracts do not include an effective performance management framework.

 The Auditor-General made ten

recommendations to the Department of Industry, Science, Energy and Resources. The department agreed to all ten of the recommendations.

 Following a redesign process, a Request for

Tender was issued in September 2019 for the engagement of delivery partners to deliver expert business advisory and facilitation services for the Entrepreneurs’ Programme.  The estimated value of the procurement was

$182 million for the maximum contract term of five years.

 Fifty-five responses were received to the

Request for Tender, of which 10 were from incumbent providers.  Seven contracts were entered into.

55

Tender submissions received in response to the Request for Tender.

14

Tender submissions shortlisted after evaluation against the first two of six criteria.

$144m

Value of the seven delivery partner contracts at time they were signed.

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

7

Summary and recommendations

Background 1. The Entrepreneurs’ Programme was announced as part of the 2014-15 Budget with funding of $484.2 million. It was launched on 1 July 2014. The program forms part of the Australian Government’s industry policy outlined in the Industry Innovation and Science Agenda, released on 14 October 2014, and the National Innovation and Science Agenda, announced on 7 December 2015. The Department of Industry, Science, Energy and Resources (DISER or the department) is responsible for administering the program.1

2. On 27 September 2019, the department issued a Request for Tender (RFT) for the engagement of delivery partners to deliver expert business advisory and facilitation services for the Entrepreneurs’ Programme.2 The estimated value of the procurement was $182 million for the maximum contract term of five years.

3. Fifty-five responses were received to the RFT, of which 10 were from incumbent providers (18 per cent).

4. Seven tenderers (one to deliver commercialisation services nationally; one to deliver incubation support and innovation connection services nationally; and five to deliver advisory and facilitation services in nominated geographic areas) were awarded contracts for a term of three years from 1 July 2020 to 30 June 2023, with two one-year extension options. Five of the seven tenderers (71 per cent) awarded contracts were incumbent providers.

Rationale for undertaking the audit

5. This procurement is a key element for the delivery of a services and grants program that is part of the National Innovation and Science Agenda, the design of which was examined in Auditor-General Report No.10 of 2017-18. The procurement was DISER’s largest value procurement in 2019-20 and represented 37 per cent of the total value of contracts entered into by DISER and reported on AusTender for that year. The audit will provide assurance to the Parliament about whether value for money was demonstrably achieved and whether services have been provided under the contracts to date.

Audit objective and criteria

6. The audit objective was to examine whether the design and conduct of the procurement process for delivery partners for the Entrepreneurs’ Programme complied with the Commonwealth Procurement Rules, and whether the signed contracts are being appropriately managed.

7. To form a conclusion against this objective, the following high-level criteria were applied.

• Did the procurement process demonstrate achievement of value for money?

1 With effect from 1 July 2022, the department will become the Department of Industry, Science and Resources. 2 The development of the RFT was informed by a ‘redesign project’ with five of the ten incumbent delivery partners (formerly named ‘industry partners’) engaged to assist with the redesign.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

8

• Are the contracts being managed appropriately to achieve the objectives of the procurement?

Conclusion 8. The design and conduct of the procurement did not comply with the Commonwealth Procurement Rules (CPRs), and the signed contracts are not being appropriately managed.

9. In its conduct of the procurement the department did not demonstrate achievement of value for money, the core rule of the CPRs. Although the Request for Tender (RFT) resulted in 53 compliant tender responses being received suggesting a competitive selection process, the department’s approach was deficient in significant respects such that there was not open and effective competition for the delivery partner roles. In particular, the significant majority of tenders received were not fully evaluated against each of the published criteria. The department’s conduct of the procurement process also fell short of the ethical requirements set out in the CPRs, with key aspects of the approach employed either not outlined in the RFT or inconsistent with the RFT, competing tenders not being treated fairly or equitably, and probity risks not being appropriately managed.

10. The delivery partner contracts are not being appropriately managed. The contract management framework is inadequate, and the department’s approach has not resulted in contract deliverables being provided on time or required that service provision is to an appropriate standard before payments are made. The contracts do not include an effective performance management framework.

Supporting findings

Procurement process

11. Six relevant evaluation criteria were included in the RFT. The RFT did not disclose to tender participants that the department would employ a staged procurement approach involving the first two criteria being employed as the basis for shortlisting, such that most tenders were not evaluated against the remaining four criteria (including the price criterion). In addition, the application of the evaluation criteria was not supported by the timely development and approval of an appropriate tender evaluation plan. The version of the evaluation plan that existed at the time the RFT was issued did not include the same criteria as the RFT. The tender evaluation team was not established at the time the RFT was released or prior to evaluations commencing. (See paragraphs 2.2 to 2.20)

12. DISER’s approach to the evaluation of tender responses did not allow for value for money to be demonstrably achieved. The department’s evaluation approach:

• shortlisted out 39 of the 53 compliant tender responses (74 per cent) on the basis of an assessment against the first two of the six criteria, although for six respondents an evaluation was conducted against only one of those two criteria. Shortlisting was also not consistently undertaken given, of the 30 tenders rated as ‘acceptable’ or better against the first two criteria, 14 (47 per cent) were shortlisted whereas 16 (53 per cent) were not;

Summary and recommendations

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

9

• involved only 14 of the compliant tender responses (26 per cent) being evaluated against the price criterion, with a significant error made in the price evaluation for one of those tender responses (leading to it being ranked more highly than it should have been);

• evaluated a reduced shortlist of 10 tenders against the ‘corporate and financial viability’ criterion; and

• did not evaluate any of the tenders against either the risk criterion or the Commonwealth policies criterion.

13. In addition, the selection of successful providers in two of the three outcomes areas did not align with the results of the evaluation work. The evaluation work did not support the selection of candidates in the commercialisation and growth outcomes. The successful candidate for the commercialisation outcome was not identified as the first ranked tenderer. For the growth outcome, DISER tailored arrangements to select the successful candidate to provide growth services in Queensland and errors were made in the pricing analysis which supported the selection of the successful candidate to provide growth services in the Northern Territory. (See paragraphs 2.21 to 2.67)

14. The procurements were not conducted in accordance with the guidance provided by the CPRs about how to conduct procurements ethically.

• The department involved the probity adviser it engaged in the drafting of the RFT and the conduct of the evaluation activities, an approach that adversely affects the independence of the probity advice.

• The procurement of the probity adviser was deficient, including an absence of open and effective competition for the role.

• While a probity plan was documented, key elements were not implemented, for example:

− a conflict of interest register was not maintained;

− the probity register was incomplete;

− a list of key persons with access to controlled information was not maintained by the procurement manager and conflict of interest declarations were not obtained from all persons that had access to controlled information. Those that did not provide a declaration included the procurement delegate and internal legal and procurement advisers who formed part of the specialist advice and support team; and

− where declarations were provided not all conflicts were identified and, where conflicts were identified, management actions were not put in place to avoid or mitigate them.

• Incumbency advantages were not transparently managed, in particular:

− there were no specific contractual or payment arrangements in place to govern the department’s engagement of five of the ten existing industry partners to contribute to the redesign and payment for this work;

− the department did not implement in full the probity risk management measures recommended by its probity adviser (for example, a probity plan and protocols for the redesign work was not in place at the commencement of work and the

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

10

department did not prohibit the industry partner personnel involved in the redesign from assisting in the preparation of tender responses); and

− departmental records indicate that information about the redesign of the program was revealed to incumbent tenderers involved in the redesign work providing them with a competitive advantage (this disclosure of information to the incumbents involved with the redesign work, and the seeking of urgent advice from the probity adviser about it, was not recorded in the probity register). (See paragraphs 2.70 to 2.97)

15. Appropriate procurement records were partly maintained. While available records addressed the requirement for the procurement as well as the process that was followed and relevant approvals, evidence to support key decisions was not maintained. In addition, the tender evaluation report did not accurately reflect the evaluation process that was employed or satisfactorily demonstrate that value for money had been achieved. (See paragraphs 2.100 to 2.102)

16. While DISER selected an open tender procurement method and issued an RFT, the procurement did not demonstrate open and effective competition in accordance with the CPRs. The department’s procurement approach resulted in incumbent providers receiving greater consideration than non-incumbents and this was reflected in the tender outcomes with 83 per cent of contracts being awarded to an incumbent provider whereas they represented 20 per cent of tenders received. The procurement approach did not give appropriate consideration to Commonwealth policies relating to Small and Medium Enterprises (SMEs). While 81 per cent of tenders submitted were from SMEs, only 20 per cent (9 of 44) of those were shortlisted (15 SMEs rated as ‘acceptable’ against the two shortlisting criteria nevertheless did not proceed any further in the evaluation). (See paragraphs 2.105 to 2.117)

Contract management

17. The department has with one exception met its obligations to accurately report the contracts and amendments on AusTender within the required 42 days. On one occasion the reported value did not align with the signed contract variation with Deloitte Consulting Pty Ltd (Deloitte), resulting in the reported value being $165,517 higher than the value of the contract variation. (See paragraphs 3.2 to 3.3)

18. DISER’s contract management framework for the program is inadequate. In particular, contract management plans have not been developed for any of the contracts and the department’s approach has not secured:

• provision of contract deliverables on time;

• provision of services to an appropriate standard before payments are made; and

• appropriate management of all identified conflicts of interest. (See paragraphs 3.4 to 3.43)

19. The delivery partner contracts do not include an effective performance management framework. The contracts do not contain:

• specific service levels that each delivery partner must achieve or exceed;

• any performance measures and related targets to assess delivery partner performance; and

Summary and recommendations

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

11

• a means to adjust payment based on the performance of delivery partners. (See paragraphs 3.46 to 3.62)

Recommendations

Recommendation no. 1 Paragraph 2.13 When planning to employ a staged process to evaluating tenders, the Department of Industry, Science, Energy and Resources clearly

identify this in its approach to market along with the criteria that will be used to shortlist potential suppliers, and if applicable, any expected limits on the number of potential suppliers that will be shortlisted at each stage.

Department of Industry, Science, Energy and Resources response: Agreed.

Recommendation no. 2 Paragraph 2.68 When evaluating tender responses, the Department of Industry, Science, Energy and Resources fully evaluate responses received

consistent with the approach set out in the approach to the market, with the results of this work relied upon to select the successful candidate(s).

Department of Industry, Science, Energy and Resources response: Agreed.

Recommendation no. 3 Paragraph 2.77 The Department of Industry, Science, Energy and Resources improve its procurement framework to specifically address the

engagement of probity advisers, including ensuring that advisers are independent and objective by not engaging the same probity advisers on an ongoing or serial basis.

Department of Industry, Science, Energy and Resources response: Agreed.

Recommendation no. 4

Paragraph 2.98

The Department of Industry, Science, Energy and Resources improve its procurement framework to specifically address how it will manage the risk of any incumbency advantages when conducting procurement processes.

Department of Industry, Science, Energy and Resources response: Agreed.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

12

Recommendation no. 5

Paragraph 2.103

The Department of Industry, Science, Energy and Resources improve its procurement record keeping so that accurate and concise information exists on:

• the process that was followed;

• how value for money was considered and achieved;

• relevant approvals; and

• relevant decisions and basis of those decisions.

Department of Industry, Science, Energy and Resources response: Agreed.

Recommendation no. 6 Paragraph 2.118 The Department of Industry, Science, Energy and Resources: • not exercise the extension options in the awarded contracts;

• commences work to conduct a new procurement process that will be completed before the existing contracts expire on 30 June 2023; and

• conducts the next procurement process in a manner that fully complies with the Commonwealth Procurement Rules.

Department of Industry, Science, Energy and Resources response: Agreed in principle.

Recommendation no. 7 Paragraph 3.7 The Department of Industry, Science, Energy and Resources develop contract management plans for each of the delivery partner

contracts.

Department of Industry, Science, Energy and Resources response: Agreed.

Recommendation no. 8 Paragraph 3.17 The Department of Industry, Science, Energy and Resources adopt a proactive approach to managing delivery partner conflicts of

interest by:

• ensuring conflict of interest declarations are completed, updated regularly or their ongoing currency otherwise confirmed; and

• regular review of delivery partner reporting of conflict of interest in order to monitor compliance with contractual obligations.

Department of Industry, Science, Energy and Resources response: Agreed.

Summary and recommendations

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

13

Recommendation no. 9

Paragraph 3.44

The Department of Industry, Science, Energy and Resources strengthen its management of the Entrepreneur’s Programme delivery contracts, including by taking prompt action in circumstances where delivery deadlines are not met and verifying that services have been provided before payments are made.

Department of Industry, Science, Energy and Resources response: Agreed.

Recommendation no. 10 Paragraph 3.63 When contracting for the delivery of services, the Department of Industry, Science, Energy and Resources apply competitive pressure

when establishing performance expectations, and how these will relate to contractual payments. This can be achieved by the department setting out its expected performance management system and standards in the approach to the market

documentation, or by asking respondents to set out their proposals to inform a comparative evaluation.

Department of Industry, Science, Energy and Resources response: Agreed.

Summary of entity response 20. The proposed audit report was provided to DISER. Extracts of the proposed report were also provided to: Mills Oakley, DMV Consulting, Bevington Group, Deloitte, Australian Industry Group (Ai Group), Business Australia (formerly NSW Business Chamber Ltd (NSWBC)), Business SA, Commonwealth Scientific and Industrial Research Organisation (CSIRO), Darwin Innovation Hub, and i4 Connect Pty Ltd (i4 Connect). The letters of response that were received for inclusion in the audit report are at Appendix 1. DISER’s summary response is provided below.

The Department of Industry, Science, Energy and Resources (the department) agrees to all of the ANAO's recommendations and takes seriously the important issues raised.

While the department's administration of the procurement fell short of the appropriate standards of transparency, consistency and fairness, particularly in late 2019 and early 2020, the Entrepreneurs' Programme has successfully supported over 22,000 Australian companies to strengthen and grow since its launch in 2014. Businesses participating in the growth element of the program have experienced average increases of 3.5 new jobs and an additional $1.47 million in turnover.

The department is actively working to address all recommendations, and any cultural or systemic improvements necessary, as a matter of priority, including adopting a formal change program approach.

The department has already put contract management plans in place for the Entrepreneurs' Programme. We are immediately developing performance metrics and standards for delivery partners, as well as a probity plan specific to the programme, which will comprehensively address conflict of interest.

We are also establishing a robust department-wide contract management framework, and a centralised and strengthened approach to the management of probity and conflict of interest.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

14

Having successfully implemented a probity framework for grants management, we are now focused on strengthening our Probity Framework to cover procurements and contracts, with a clear line of sight by the departmental executive and our internal Audit and Assurance Committee.

Key messages from this audit for all Australian Government entities 21. Below is a summary of key messages, including instances of good practice, which have been identified in this audit and may be relevant for the operations of other Australian Government entities.

Procurement • Competition is a key element of the Australian Government’s procurement framework set out in the Commonwealth Procurement Rules (CPRs). Generally, the more competitive the procurement process, the better placed an entity is to demonstrate that it has achieved value

for money. Competition encourages respondents to submit more efficient, effective and economical proposals. It also ensures that the purchasing entity has access to comparative services and rates, placing it in an informed position when evaluating the responses.

• Acting ethically when conducting procurements requires more than documenting plans and guidance, obtaining and filing conflict of interest declarations and obtaining sign offs at key points from probity advisers. It requires ethical principles to be applied to all actions throughout the entire procurement process and for management action to be taken when ethical issues are identified. It also requires good recordkeeping so that it can be transparently demonstrated that the procurement was conducted ethically.

• While probity advisers can be engaged to assist entities conduct procurements ethically as required by the CPRs, responsibility and accountability for the procurement cannot be outsourced by entities to their probity (or any other) advisers. When engaging an external probity adviser, it is important that entities conduct the procurement process in a way that is ethical. It is also important that entities not engage the same probity adviser on an ongoing basis and ensure that the tasks allocated to the probity adviser do not threaten the independence and objectivity of the probity adviser.

• Entities should treat all tenderers, and potential tenderers, in a fair and non-discriminatory manner. To assist, entities should ensure that their procurement frameworks specifically address how risks of incumbency advantage are managed so that the procurement process is conducted with no bias or favouritism, and to maximise the likelihood of maximising value for money to the Australian Government through competitive selection processes. This requires that attention be given to whether the incumbent has access to information that would provide an unfair advantage and how to manage this and, when setting procurement timeframes, making sure that deadlines are not made so short that the incumbent has an unfair advantage.

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

15

Audit findings

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

16

1. Background

Introduction 1.1 The Entrepreneurs’ Programme was announced as part of the 2014-15 Budget with funding of $484.2 million. It was launched on 1 July 2014. The program forms part of the Australian Government’s industry policy outlined in the Industry Innovation and Science Agenda, released on 14 October 2014, and the National Innovation and Science Agenda, announced on 7 December 2015. The Department of Industry, Science, Energy and Resources (DISER or the department) is responsible for administering the program.3

1.2 On 27 September 2019, the department issued a Request for Tender (RFT) for the engagement of delivery partners to deliver expert business advisory and facilitation services for the Entrepreneurs’ Programme. The estimated value of the procurement was $182 million for the maximum contract term of five years. The department expected it would engage approximately 10 delivery partners able to employ a minimum of five specified personnel. At the time, the program was being delivered by providers appointed from a procurement process conducted in 2014-15. Most of those contracts expired on 30 June 2020.4 The new delivery partners engaged through this RFT process were to commence services from 1 July 2020.

1.3 The development of the RFT was informed by a ‘redesign project’. Five of the ten incumbent delivery partners (formerly named ‘industry partners’) were engaged to assist with the redesign to draw on their ‘deep knowledge of customer needs/program experience.’ Three of the incumbents engaged to contribute to the redesign project were subsequently awarded contracts as delivery partners.5 As a result of the project, the department decided on an operating model focused on the outcomes of ‘growth, innovation, and commercialisation’. The outcomes are supported by four program elements: business management, innovation connections, incubator support and accelerating commercialisation.6 Grants are available to businesses under the business management, innovation connections, and accelerating commercialisation services of the Entrepreneurs’ Programme.

1.4 In the RFT, the department sought:

• a single delivery partner to provide commercialisation services to customers nationally;

• a single delivery partner to provide innovation and incubator support services nationally; and

3 With effect from 1 July 2022, the department will become the Department of Industry, Science and Resources. 4 In June 2020, the department approved extensions for four of the five unsuccessful incumbent providers’ contracts to 31 December 2020. The contract extensions were for ‘complementary services’ to ‘promote a

successful transition of existing services, operating in parallel to the new delivery partners to avoid duplication’. The total cost of the contract extensions was $620,100. 5 The three successful incumbents each contributed multiple personnel to the redesign project: Deloitte Consulting Pty Ltd (Deloitte) (six personnel, including two as members of the Design Team); Australian

Industry Group (Ai Group) (four personnel, including one as a member of the Design Team); and NSW Business Chamber Ltd (NSWBC) (three personnel). All other participating companies supplied one person. 6 Under the previous arrangements, accelerating commercialisation services were not provided by a delivery partner.

Background

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

17

• one or more delivery partners to provide expert business advisory and facilitation services in nominated geographic areas on a state and/or regional basis.

1.5 Fifty-five responses were received to the RFT, of which 10 were from incumbent providers (18 per cent). Two responses did not satisfy the conditions for participation and/or the minimum content and format requirements. The remaining 53 progressed for evaluation.

1.6 Seven tenderers (one to deliver commercialisation services nationally; one to deliver incubation support and innovation connection services nationally; and five to deliver advisory and facilitation services in nominated geographic areas) were awarded contracts for a term of three years from 1 July 2020 to 30 June 2023, with two one-year extension options. Five of the seven tenderers (71 per cent) awarded contracts were incumbent providers.

1.7 Under the contracts, performance is to be assessed and managed by the department using a combination of quality, volume and outcome measures to evaluate each delivery partner’s contribution to the overall performance of the program against its objectives. Performance

measurement is a combination of the number of customers, the outcomes achieved and the impact for the customer.

Rationale for undertaking the audit 1.8 This procurement is a key element for the delivery of a services and grants program that is part of the National Innovation and Science Agenda, the design of which was examined in Auditor-General Report No.10 of 2017-18. The procurement was DISER’s largest value procurement in 2019-20 and represented 37 per cent of the total value of contracts entered into by DISER and reported on AusTender for that year. The audit will provide assurance to the Parliament about whether value for money was demonstrably achieved and whether services have been provided under the contracts to date.

Audit approach

Audit objective, criteria and scope

1.9 The audit objective was to examine whether the design and conduct of the procurement process for delivery partners for the Entrepreneurs’ Programme complied with the Commonwealth Procurement Rules, and whether the signed contracts are being appropriately managed.

1.10 To form a conclusion against this objective, the following high-level criteria were applied.

• Did the procurement process demonstrate achievement of value for money?

• Are the contracts being managed appropriately to achieve the objectives of the procurement?

1.11 The audit examined the design and conduct of the 2019-20 procurement process for delivery partners for the Entrepreneurs’ Programme, and the management of the contracts for all delivery partners since services commenced in July 2020. The audit scope did not include:

• the 2014-15 procurement process for industry partners and the awarded contracts; and

• the development of program guidelines and the evaluation and award of grant funding under the Entrepreneurs’ Programme.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

18

Audit methodology

1.12 The audit methodology included examination and analysis of DISER records; and meetings with relevant DISER staff.

1.13 The audit was conducted in accordance with ANAO Auditing Standards at a cost to the ANAO of approximately $507,000.

1.14 The team members for this audit were Sean Neubeck, Jocelyn Watts, Joshua Carruthers and Brian Boyd.

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

19

2. Did the procurement process demonstrate achievement of value for money?

Areas examined The ANAO examined whether the procurement process demonstrated achievement of value for money.

Conclusion In its conduct of the procurement the department did not demonstrate achievement of value for money, the core rule of the Commonwealth Procurement Rules (CPRs). Although the Request for Tender (RFT) resulted in 53 compliant tender responses being received suggesting a competitive selection process, the department’s approach was deficient in significant respects such that there was not open and effective competition for the delivery partner roles. In particular, the significant majority of tenders received were not fully evaluated against each of the published criteria. The department’s conduct of the procurement process also fell short of the ethical requirements set out in the CPRs, with key aspects of the approach employed either not outlined in the RFT or inconsistent with the RFT, competing tenders not being treated fairly or equitably, and probity risks not being appropriately managed.

Areas for improvement The ANAO made six recommendations related to shortlisting of responses to an open approach to the market, the conduct of tender evaluation processes, the engagement of probity advisers, record keeping, managing the risks of incumbency advantages that may exist in procurement processes, and commencing a new procurement process that is more transparent and adheres to probity principles.

2.1 To assess whether the procurement demonstrated achievement of value for money, the ANAO examined whether an open and competitive procurement process was conducted that complied with the CPRs.7 This reflects that competition is a key element of the Australian Government’s procurement framework set out in the CPRs.8 Generally, the more competitive the procurement process, the better placed an entity is to demonstrate that it has achieved value for money. Competition encourages respondents to submit more efficient, effective and economical proposals. It also ensures that the purchasing entity has access to comparative services and rates, placing it in an informed position when evaluating the responses. Openness in procurement involves giving suppliers fair and equitable access to opportunities to compete for work while maintaining transparency and integrity of process.

Were relevant evaluation criteria included in request documentation to enable the proper identification, assessment and comparison of submissions?

Six relevant evaluation criteria were included in the RFT. The RFT did not disclose to tender participants that the department would employ a staged procurement approach involving the

7 Appendix 7 summarises the ANAO’s analysis of where the conduct of the procurement did not comply with the CPRs. 8 Department of Finance (Finance), Commonwealth Procurement Rules, paragraph 5.1.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

20

first two criteria being employed as the basis for shortlisting, such that most tenders were not evaluated against the remaining four criteria (including the price criterion). In addition, the application of the evaluation criteria was not supported by the timely development and approval of an appropriate tender evaluation plan. The version of the evaluation plan that existed at the time the RFT was issued did not include the same criteria as the RFT. The tender evaluation team was not established at the time the RFT was released or prior to evaluations commencing.

2.2 The CPRs require that evaluation criteria be included in request documentation to enable proper identification, assessment and comparison of submissions on a fair, common and appropriately transparent basis.9 Request documentation must include a complete description of evaluation criteria to be considered in assessing submissions and, if applicable to the evaluation, the relative importance of those criteria.10

2.3 The RFT documentation outlined that tenders would be assessed on the basis of value for money consistent with Commonwealth procurement policies, through the application of evaluation criteria. The RFT included six evaluation criteria to be applied for the purposes of evaluation, two of which were weighted and the remaining were not weighted. The RFT stated that the criteria were

not in order of importance.11 The criteria were:

• organisational capability (70 per cent weighting);

• promotion and marketing and industry knowledge (30 per cent weighting);

• price (not weighted);

• risk (not weighted);

• corporate and financial viability (not weighted); and

• Commonwealth policies (not weighted).12

Tender evaluation plan

2.4 An evaluation plan should be established before market responses are sought.13 Drafting the evaluation plan and RFT together helps to avoid any inconsistencies between the evaluation plan and requirements specified in RFT documentation, and to ensure that the evaluation methodology proposed in the plan is consistent with the RFT.14 An appropriately drafted and

9 Finance, Commonwealth Procurement Rules, paragraph 7.12. 10 Finance, Commonwealth Procurement Rules, paragraph 10.6. 11 In March 2022, the Department of Industry, Science, Energy and Resources (DISER or the department) advised the ANAO in relation to the statement in the RFT that the criteria were not in order of importance that it

‘recognises the unintentional error but does not agree that it negatively affected the conduct or outcome of the procurement.’ 12 The evaluation criteria published in the RFT, including the detailed sub criteria and weightings, are set out in Appendix 3. 13 Auditor-General Report No.23 2017-18 Delivery of Moorebank Intermodal Terminal, p.13; Australian

Government Solicitor, Commercial notes No.1 [Internet], AGS, available from https://www.ags.gov.au/publications/commercial-notes/comnote01.htm [accessed December 2021]; Australian Government Solicitor, Commercial notes No.15 [Internet], AGS, available from https://www.ags.gov.au/publications/commercial-notes/ComNote15.htm [accessed December 2021]. 14 Australian Government Solicitor, Commercial notes No.15 [Internet], AGS, available from

https://www.ags.gov.au/publications/commercial-notes/ComNote15.htm [accessed December 2021].

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

21

implemented evaluation plan assists to demonstrate that tender submissions have been objectively and fairly evaluated in accordance with the RFT.

2.5 In March 2022, the Department of Science, Industry and Resources (DISER or the department) advised the ANAO that:

The Tender Evaluation Plan was largely completed as a working document when the RFT was published, and was finalised and approved prior to the evaluation process commencing.

2.6 The ANAO’s analysis of the contemporaneous records of the conduct of the procurement did not support the department’s advice that the evaluation plan had been largely completed at the time the RFT was published. The draft plan in circulation at the time the RFT was published was based on the plan used for the previous tender for delivery partners and included various comments from the probity adviser and departmental officers. It also could not be considered a largely complete document, given the draft at the time the RFT was published did not reflect key aspects of the approach later employed for the evaluation. In particular, the following aspects were missing.

• The draft plan did not identify who would conduct the evaluation:

− the draft plan stated that a six member Tender Assessment Panel (TAP) would be employed whereas the evaluation of tenders was undertaken by a three member Panel, notwithstanding that a three person TAP had been established by the department prior to the RFT being published (see paragraph 2.16);

− this draft version of the evaluation plan also did not reflect that four ‘internal experts’ would play a role in the conduct of the evaluations (while the use of ‘internal experts’ was identified in the final version of the evaluation plan that was attached to the tender evaluation report their role was not defined in that document).

• The draft plan did not reflect the evaluation criteria set out in the RFT:

− the draft evaluation plan at the time the RFT was published set out five weighted evaluation criteria being service delivery (25 per cent), key personnel (35 per cent), project management (10 per cent), industry knowledge (15 per cent) and promotion and marketing (15 per cent); whereas

− six criteria had been published in the RFT, of which two had been weighted — organisational capability (70 per cent weighting), promotion and marketing and industry knowledge (30 per cent weighting), price, risk, corporate and financial viability, and Commonwealth policies.

2.7 The evaluation plan15 for the procurement process for delivery partners for the Entrepreneurs’ Programme was:

• not signed off and approved before the RFT was issued on 27 September 2019 or before tender submissions were opened on 28 November 2019;

• not signed off and agreed by all TAP members;

• not signed off and approved by the probity adviser; and

15 The stages set out in the tender evaluation plan were as follows: stage 1: registration and screening; stage 2: technical evaluation; stage 3: pricing evaluation; stage 4: contract compliance evaluation; and stage 5: value for money including overall risk evaluation.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

22

• not at any time signed off and approved by the delegate for the procurement.

2.8 The draft plan was provided to the TAP to be signed off after stage 2 technical evaluations were completed on 19 December 2019. As evidence that the plan had been signed, DISER provided the ANAO in October 2021 with a scanned signature page including the TAP Chair’s signature from 23 December 2019. One member of the TAP had provided agreement to the terms and conditions of the plan by email on 19 December 2019. No evidence that the remaining member signed the plan was recorded.

2.9 The draft plan was also intended to be approved by the procurement delegate and by the probity adviser. Signatures by the delegate and probity adviser were not recorded.

2.10 While the RFT stated that the evaluation criteria were not listed in order of importance, the department’s tender evaluation plan set out a multi-staged process where tender submissions were required to meet the first two weighted evaluation criteria to be shortlisted for further evaluation against the remaining criteria.

2.11 The ‘multi-stage procurement method’ is one of the procurement methods identified in the CPRs.16 DISER did not inform tenderers that the procurement involved a staged approach. Specifically, the RFT did not outline to potential tenderers that a threshold would be applied and that there would be a limit17 on the number of tender submissions that would proceed for further

evaluation against the remaining criteria. This was important information for tenderers to know to inform their decision about whether to participate in the tender process.18 The contributions received by the ANAO to the Citizens Contribution facility for this audit included two from tenderers that clearly evidenced the tenderers were unaware that there had been a shortlisting process employed based solely on evaluation against the first two criteria listed in the RFT.

2.12 The ANAO compared the RFT to other approaches to market for four procurements19 the ANAO has audited where shortlisting was employed (going back to 2004). Compared with the approach to market documentation for those procurements, the Entrepreneurs’ Programme RFT was significantly deficient. All the other procurements had:

• clearly stepped out how shortlisting would be employed, including clearly identifying the criteria for shortlisting as well as those for full evaluation; and

• where relevant, also clearly outlined to the market the Commonwealth’s expectations in terms of how many respondents were likely to be shortlisted.

16 Finance, Commonwealth Procurement Rules, paragraph 9.8. 17 Prior to commencing evaluations, the department envisaged a shortlist of around 15 tender submissions would proceed to the stage 3 pricing evaluation. 18 The CPRs state that participation in procurement imposes costs on relevant entities and potential suppliers.

Those costs should be considered when designing a process that is commensurate with the scale, scope and risk of the proposed procurement: Finance, Commonwealth Procurement Rules, paragraph 5.2. 19 These procurements were undertaken by a range of different Commonwealth entities. Refer to Auditor-General Report No.43 2008-09 Construction of the Christmas Island Detention Centre; Auditor-

General Report No.45 2016-17 Replacement Antarctic Vessel; Auditor-General Report No.46 2016-17 Conduct of the OneSKY Tender; Auditor-General Report No.16 2019-20 Western Sydney Airport Procurement Activities.

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

23

Recommendation no. 1 2.13 When planning to employ a staged process to evaluating tenders, the Department of Industry, Science, Energy and Resources clearly identify this in its approach to market along with the criteria that will be used to shortlist potential suppliers, and if applicable, any expected limits on the number of potential suppliers that will be shortlisted at each stage.

Department of Industry, Science, Energy and Resources response: Agreed.

2.14 The changes we are making to our procurement-related frameworks, processes and procedures will provide for the clear identification of selection criteria and evaluation process in its approach to the market, and these changes will be included in our procurement checklist against which we will monitor compliance. Our department-wide Probity Framework will address transparency, consistency and fairness requirements.

Tender evaluation team

2.15 Guidance to entities is that the procurement team should be selected carefully and early, preferably before the RFT is released, and probity training should be provided to the team as soon as they are identified, not when the evaluation is about to start.20

2.16 DISER established the TAP comprising a chairperson and two members prior to issuing the RFT. A probity adviser had also been engaged in advance of the RFT’s release (see paragraph 2.71 below). Neither the panel of internal experts nor the pricing assessment team had been established prior to commencement of evaluation.21 The role of the internal experts was not defined in the tender evaluation plan. The pricing assessment team was to:

• actively participate in briefings on the RFT and the evaluation process;

• analyse the whole-of-life contract cost for each tender submission at stage 3 of the evaluation process; and

• provide a report to the TAP containing the price ranking and analysis and risk assessment of pricing elements for consideration as part of the stage 5 value for money assessment.

2.17 A probity briefing was provided to the Entrepreneurs’ Programme branch on 13 September 2019 in advance of an Entrepreneurs’ Programme Annual Forum. The forum was attended by the incumbent delivery partners and facilitators. DISER has not retained a record of the

20 Australian Government Solicitor, Commercial notes No.1 [Internet], AGS, available from https://www.ags.gov.au/publications/commercial-notes/comnote01.htm [accessed December 2021]. 21 Drafts of the tender evaluation plan noted that the department had sent a request to the Entrepreneurs’ Programme Committee for assistance with an independent assessment of the shortlisted responses from

potential tenderers for the commercialisation and innovation outcomes. This note was later removed from the version of the plan attached to the tender evaluation report. In March 2022, DISER advised the ANAO that the ‘EPC was not requested to provide assistance with an independent assessment of shortlisted tenderers’, which departmental records show was not clear to the TAP as late as December 2019. The EPC operates under delegation from Industry Innovation and Science Australia. The role of the EPC is to provide merit assessments and merit rankings of referred applications in relation to the accelerating commercialisation and incubator support initiative components of the Entrepreneurs’ Programme.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

24

briefing, including an attendance list. The ANAO’s analysis of records indicates that key personnel, including the TAP Chair, were not present at the probity briefing provided in advance of the Forum.

2.18 A probity briefing to the TAP and other members of the procurement team was provided by the probity adviser on 28 November 2019, the day after the RFT closing date. Three of the four internal panel experts did not attend the briefing. The pricing assessment team had not been established at the time of the briefing.

2.19 DMV Consulting was engaged to perform the stage 3 pricing evaluation. The procurement process for the engagement, which was not consistent with the CPRs, was as follows:

• on 31 October 2019, the probity adviser provided the procurement team with recommendations for financial and pricing analysts;

• on 11 November 2019, delegate approval was provided to engage a financial analyst to do a cost comparison and financial health check of shortlisted tender responses;

• the request for approval outlined that, subject to delegate approval, the department would approach four businesses to begin discussions about the estimated costs (the CPRs state that ‘the expected value of a procurement must be estimated before a decision on the procurement method is made’)22;

• the department requested a quote from each of the four businesses on

15 November 2019;

• the department received two quotes for comparison. The comparison was made on the basis that the shortlist would be of 15 tenderers;

• on 9 December 2019, the department sought a revised quote from DMV Consulting based on the department’s own estimates of time required to conduct the financial viability reviews and cost comparison of pricing schedules. The department advised DMV Consulting that it would be engaged under the department’s panel arrangement for the supply of Accounting, Audit, and Related Professional Services. Neither the cost estimate nor the procurement method was identified in the original procurement process approval; and

• on 16 December 2019, the delegate provided stage 2 approval to engage DMV Consulting to provide financial analysis services for the delivery partner tender as well as another tender process being undertaken for the Entrepreneurs’ Programme. The total approved contract value was $120,428 with the award of this contract reported by the department on AusTender on 24 December 2019.

2.20 The total amount paid to DMV Consulting was $56,823.23

22 Finance, Commonwealth Procurement Rules, paragraph 9.2. 23 The other tender process that the quote included did not proceed. The amount paid to DMV Consulting was for the pricing analysis on the delivery partner procurement.

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

25

Were the successful candidates those assessed as providing the best value for money?

DISER’s approach to the evaluation of tender responses did not allow for value for money to be demonstrably achieved. The department’s evaluation approach:

• shortlisted out 39 of the 53 compliant tender responses (74 per cent) on the basis of an assessment against the first two of the six criteria, although for six respondents an evaluation was conducted against only one of those two criteria. Shortlisting was also not consistently undertaken given, of the 30 tenders rated as ‘acceptable’ or better against the first two criteria, 14 (47 per cent) were shortlisted whereas 16 (53 per cent) were not;

• involved only 14 of the compliant tender responses (26 per cent) being evaluated against the price criterion, with a significant error made in the price evaluation for one of those tender responses (leading to it being ranked more highly than it should have been);

• evaluated a reduced shortlist of 10 tenders against the ‘corporate and financial viability’ criterion; and

• did not evaluate any of the tenders against either the risk criterion or the

Commonwealth policies criterion.

In addition, the selection of successful providers in two of the three outcomes areas did not align with the results of the evaluation work. The evaluation work did not support the selection of candidates in the commercialisation and growth outcomes. The successful candidate for the commercialisation outcome was not identified as the first ranked tenderer. For the growth outcome, DISER tailored arrangements to select the successful candidate to provide growth services in Queensland and errors were made in the pricing analysis which supported the selection of the successful candidate to provide growth services in the Northern Territory.

Registration and Screening

2.21 The CPRs require request documentation to include any conditions for participation and any minimum content and format requirements.24 Further consideration must be given only to submissions that meet minimum content and format requirements.25

2.22 Fifty-five tender responses were received. The initial screening concluded that two tender responses did not satisfy the conditions for participation and/or the minimum content and format requirements. There were 13 tender responses for which additional information was sought to satisfy the conditions for participation and minimum content and format requirements. Of those 13 tender responses, 11 were progressed to stage 2 evaluations before the additional information was received. This meant that DISER did not comply with the CPR requirement outlined in paragraph 2.21.

2.23 As part of the screening process, the tender evaluation plan allowed for tender submissions to be excluded, with supporting legal advice, if the tender response was ‘clearly uncompetitive’

24 Finance, Commonwealth Procurement Rules, paragraph 10.6. 25 Finance, Commonwealth Procurement Rules, paragraph 10.34.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

26

when compared with others received. The TAP received probity advice that Darwin Innovation Hub’s tender submission should be excluded on the basis of being non-competitive and that ‘bid repair’ was required to address non-compliance with the minimum content and format requirements.

2.24 The TAP later decided to shortlist Darwin Innovation Hub’s tender submission after a panel member reassessed the original rating and the remaining members agreed to shortlist. The records of the TAP’s decision to shortlist were inadequate to explain the decision that was taken.

Application of evaluation criteria

2.25 The TAP did not evaluate tender submissions in accordance with the published evaluation criteria and methodology outlined in the RFT. Figure 2.1 below shows the extent to which tender submissions were evaluated against each of the criteria.

Figure 2.1: Application of evaluation criteria

Note: This graph includes 53 tenderers progressed to the stage 2 technical evaluations. Two tenderers were excluded from further consideration because they did not meet minimum content and format requirements (refer to paragraph 2.22). Source: ANAO analysis of DISER records.

Criteria 1 and 2 — ‘organisational capability’ and ‘promotion and marketing and industry knowledge’

2.26 The first two weighted criteria were applied as part of the stage 2 technical evaluations. These were conducted by the TAP from 3 December to 18 December 2019 (refer to Figure 2.2 below). The TAP was provided with an individual ‘Stage 2 Technical Worth Assessment Form’ by the

0

10

20

30

40

50

60

Organisational capability Promotion, marketing and

industry knowledge

Price Risk Corporate and

financial viability

Commonwealth policies

Number of tenderers

Evaluation criteria applied Evaluation criteria not applied

Number of tenderers

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

27

probity adviser26 on 1 December 2019, two days before technical evaluations commenced. At stage 2, the TAP was to perform a comparative assessment of the ability and relative merit of each tender response against the objective comparative assessment rating scale in the tender evaluation plan.

2.27 While all 53 tender submissions that progressed to stage 2 were evaluated against the ‘organisational capability’ criterion, six tender submissions did not receive an evaluation against the ‘promotion and marketing and industry knowledge’ criterion.

2.28 In contrast to the RFT, which stated that the evaluation criteria were not in order of importance, DISER applied the first two weighted criteria as a threshold and shortlisted tender submissions following the stage 2 technical evaluation against the first two criteria.

2.29 The 14 shortlisted tender responses were provided to the internal panel experts for further evaluation. The tender responses were divided among the internal panel experts according to each assessor’s knowledge of the outcomes and/or their experience within relevant and applicable policy. The TAP provided the internal panel experts with a set of specific questions as well as questions relevant to each shortlisted tender response. Internal panel experts were not required to allocate a score or comment against the full evaluation criteria contained in the RFT, only those relevant to the internal expert’s expertise.

Criterion 3 — Price

2.30 The shortlisting meant that of the 53 tender submissions that were progressed for evaluation, 14 (26 per cent) received evaluation against the price criterion. As set out in Appendix 6, a number of tenderers have complained about the conduct of the tender process, including the extent to which the successful candidate was more expensive than the price they offered.

2.31 As outlined at paragraph 2.19, DMV Consulting was engaged to conduct the pricing evaluation. The initial stage 3 pricing assessment involved a comparison of the prices tendered by each of the 14 shortlisted tenderers. The tender submissions were separated according to the three outcomes (growth, innovation, and commercialisation) and ranked from ‘best’ to ‘least best’ based on the ‘tendered price per facilitator’.

2.32 A significant error was made in the conduct of the pricing evaluation, which resulted in Darwin Innovation Hub’s tender response being ranked as the best tenderer for the growth outcome.27 Had Darwin Innovation Hub’s price per facilitator been calculated correctly it would have ranked seventh of the ten tenderers assessed for price in the growth outcome. In March 2022, DISER advised the ANAO that it:

was not aware of the error with DMV’s assessment of the Darwin Innovation Hub (DiH) pricing until it was raised by the ANAO. This error did not have a material impact on the overall pricing, or value for money assessment.28

26 See further at paragraph 2.75 in relation to the probity adviser having involvement in procurement work. 27 Darwin Innovation Hub’s price per facilitator was calculated in the pricing evaluation report as $132,873 (GST exclusive) based on its actual number of proposed facilitators (11). Based on its proposed 5 full time equivalent (FTE) facilitators, the price per facilitator was $292,320 (GST exclusive). For all other tender responses

evaluated, FTE was used as the basis to calculate the price per facilitator. 28 As set out above, the department’s view was not supported by the ANAO’s analysis which demonstrated the impact this error had on the tender outcome.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

28

Criterion 4 — Risk

2.33 According to the tender evaluation plan, the evaluation of risk was to form part of the overall value for money assessment at stage 5. Risks were to be identified throughout the evaluation process, including during the technical and financial evaluations, and were to be formally assessed and taken into account in the rating during the overall best value for money considerations in stage 5.

2.34 The TAP did not perform a formal risk assessment which encompassed risks identified throughout the process for any of the tender responses. While aspects of the risk criterion were considered throughout the process, evaluation of risk was limited to:

• compliance checks at stage 2 that tender submissions included the information requested in the RFT. While the template used at stage 2 included a section for TAP members to record evaluation criterion risks identified through the evaluation process, this section was not completed for any of the assessments;

• limited analysis of risks as part of the value for money assessment of the 14 shortlisted tenderers, which was largely based on the tenderers’ interview responses;

• consideration of price risk in relation to three tender responses for the commercialisation outcome; and

• consideration of the level of compliance with the draft contract of the seven tenderers identified as the recommended tenderers in the evaluation report.29

Criterion 5 — Corporate and financial viability

2.35 DISER reduced the shortlist to 10 tender submissions following interviews with the 14 shortlisted tenderers and an initial ‘value for money assessment’ based on the initial stage 3 pricing and the results of the internal panel expert evaluations.

2.36 Evaluation against the corporate and financial viability criterion was conducted by DMV Consulting and was limited to the reduced shortlist of 10 tender submissions. DMV Consulting provided individual reports for each of the three outcomes (growth, innovation, and commercialisation), and assigned a risk rating for each of the tender submissions against the corporate and financial viability criterion.

2.37 DMV Consulting proposed mitigation strategies for three tenderers:

• Darwin Innovation Hub — tenderer to obtain a bank guarantee from Paspalis family;

• Respondent 53 — tenderer to provide evidence of liquidity prior to contract signing; and

• i4 Connect Pty Ltd (i4 Connect) — tenderer to provide evidence of liquidity prior to contract signing.

29 Stage 4 of the tender evaluation plan was to assess the level of compliance of tender responses with the draft form of the contract provided in the request documentation. Compliance with the RFT, including the draft form of contract, was a subcriterion of the ‘risk’ criterion (refer to Appendix 3 for the evaluation criteria in full).

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

29

2.38 In March 2022, in response to the ANAO’s request for information regarding the mitigation strategies, DISER advised the ANAO that:

A bank guarantee was not obtained from the Paspalis Family Trust as the TAP determined they were sufficiently assured of DiH’s financial position.

Evidence of i4 Connect’s liquidity was obtained prior to contract signing. DISER emailed i4 Connect on 11 February 2020 with a series of questions relating to pricing. This included questions on their current financial position, ability to pay debts and whether there were bank guarantees or alternative approaches to deal with the issues of there being few substantial assets.

i4 Connect’s responses were provided to the Pricing Analyst and the Probity Advisor. Both indicated the responses were satisfactory and the TAP was satisfied with this additional information.

2.39 The DMV Consulting report that proposed the mitigation strategies was prepared after receiving i4 Connect’s responses and was provided to the department on 17 February 2020. DMV Consulting’s report stated that ‘whilst the Tenderer has indicated that they have negotiated loans to assist with short-term liquidity, these would need to be sighted prior to contract signing.’

Criterion 6 — Commonwealth policies

2.40 The CPRs require officials to assess the economic benefit of a procurement to the Australian economy as part of determining the value for money. This assessment requires officials to gather appropriate information on economic benefits as part of the decision-making process and document how economic benefit has been considered as part of the overall value for money assessment.30

2.41 DISER did not comply with this requirement. Evaluation against the Commonwealth policies criterion allowed for this requirement to be met, as well as consideration of other relevant Commonwealth procurement connected policies. Consideration of the criterion was restricted to a compliance check at stage 2 that tender submissions included the information requested in the RFT. The TAP did not document an assessment against the evaluation criteria, including how economic benefit was considered, as part of the overall value for money assessment for any of the tender responses.

Technical scoring and application of weightings

2.42 As part of stage 2, each TAP member was asked to record their assessment of each tender response and rate them (on a scale of 0 to 5) against the weighted criteria (‘organisational capability’ and ‘promotion and marketing and industry knowledge’). After completing individual assessments, the TAP was to meet to reach a consensus on the proper word description based on the rating scale and the consequent supporting score for each evaluation criterion. Weightings were then to be applied to each score for each of the evaluation criteria in accordance with the weightings specified in the RFT.

30 Finance, Commonwealth Procurement Rules, paragraph 4.7. See also Finance, Consideration of broader domestic benefits in procurement [Internet], Finance, available from https://www.finance.gov.au/sites/default/files/2020-08/consideration-of-broader-economic-benefits-in-procurement.pdf [accessed December 2021].

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

30

2.43 The TAP did not employ a consistent and transparent process in evaluating each of the tender submissions. The process was adversely impacted by the following.

• The number of evaluations, such that:

− only four (eight per cent) tender responses received an evaluation from all three TAP members;

− 26 (49 per cent) tender responses received an evaluation from two TAP members; and

− 23 (42 per cent) tender responses received an evaluation from only one TAP member.

• The technical scoring of tender submissions only being applied incidentally to producing the shortlist of tender responses, such that the TAP did not apply a transparent process. In particular:

− the individual TAP members scored responses inconsistently (for example, one assessor gave an overall score while the other two did not);

− there is no record of the TAP meeting to moderate and agree a final score for each submission;

− while overall scores were recorded in a ‘tracking’ document, how those scores were determined was not transparent and the scores recorded in the tender evaluation report are inconsistent with the tracking document;

− there is no evidence of weightings having been applied; and

− the TAP did not produce a ranking of each tender response as an outcome of the technical evaluation, or apply a consistent threshold for shortlisting such that 16 of the 30 tender submissions rated as ‘acceptable’ or better were not shortlisted (see paragraph 2.49).

2.44 An important probity step when adopting an ‘individual scoring’ followed by a ‘consensus scoring’ method of assessment is to ensure all members have had sufficient time to conduct their individual scoring prior to convening to undertake the consensus scoring process.31 The ANAO analysed the conduct of the assessment process in the context of the extent of information tenderers were required to submit in response to the RFT (for example, all six criteria were required to be addressed), the number of tenders the department expected to receive, and the planned approach of individual TAP members conducting separate assessments prior to undertaking consensus scoring.

2.45 Figure 2.2 outlines the timing of evaluation assessments over the two week evaluation period. A significant proportion of assessments were undertaken late in the two week period and there was an uneven distribution of assessments conducted by individual TAP members, with the assessor independent to the program undertaking the fewest assessments. To expedite the evaluation process, part way through the technical evaluation period, the TAP Chair (in consultation with the probity adviser) decided that a division of the tender responses among the TAP was

31 New South Wales Independent Commission Against Corruption (ICAC), Probity in procurement: tips from a professional [Internet], ICAC, available from https://www.icac.nsw.gov.au/prevention/corruption-prevention-publications/latest-corruption-prevention-publications/probity-in-procurement-tips-from-a-professional [accessed November 2021].

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

31

required. The reasoning provided in the tender evaluation report was that this was a result of the large number of responses received. This rationale was at odds with:

• the RFT indicating that approximately 10 delivery partners would be engaged;

• the market analysis recorded in DISER’s procurement plan that had indicated that the number of responses received was similar to that expected; and

• the 2015 Entrepreneurs’ Programme Industry Partner tender where more than 50 responses had been received.

Figure 2.2: Timing of evaluation assessments

Source: ANAO analysis of DISER records.

0

2

4

6

8

10

12

Tues 3 Dec 19 Weds 4 Dec 19

Thurs 5 Dec 19 Fri 6 Dec 19

Sat 7 Dec 19 Sun 8 Dec 19

Mon 9 Dec 19 Tues 10 Dec 19

Weds 11 Dec 19 Thurs 12 Dec 19

Fri 13 Dec 19 Sat 14 Dec 19

Sun 15 Dec 19 Mon 16 Dec 19

Tue 17 Dec 19 Weds 18 Dec 19

Number of technical assessments completed

Assessor 1 Assessor 2 Assessor 3 (independent of the program)

TAP division of responses

Key moderation meeting

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

33

Shortlisting

2.46 Following the stage 2 evaluations against the first two criteria, DISER shortlisted 14 tender submissions for further evaluation against the remaining criteria. This meant that of the 53 tender submissions that were identified as having met the conditions for participation and the minimum content and format requirements, 39 tender submissions (74 per cent) received no further evaluation against the remaining criteria.

2.47 As outlined at paragraphs 2.3 and 2.10, the RFT stated that the criteria were not in any order of importance. The department relied on a standard clause included in the RFT that ‘[t]he Department may use its discretion to shortlist a sub-set of tenderers at any time during the RFT process’. This clause did not provide potential tenderers with a transparent indication of the extent to which the department would shortlist tender submissions such that a significant majority (74 per cent) would not be evaluated against any criteria other than the first two. Further, DISER

did not transparently advise unsuccessful tenderers that shortlisting had occurred based on the first two criteria, with two unsuccessful tenderers (which were not shortlisted) told by the department in April 2020 that:

All tender submissions were assessed against each of the evaluation criteria contained in the RFT through a comparative rating assessment process overseen by an independent probity adviser.

2.48 In April 2022, DISER advised the ANAO that ‘the email sent to unsuccessful tenderers should not have stated that the tenders had been evaluated against all evaluation criteria’, and further advised the ANAO in May 2022 that it would provide the correct advice to those unsuccessful tenderers.

2.49 DISER advised the ANAO in March 2022 that ‘it shortlisted tenders that demonstrated the capability to deliver the requested services’ and that ‘if the tenderer could not provide the required level of technically capability, [it] did not progress’. At odds with this advice to the ANAO, is that while the tender evaluation report outlined that there were a total of 30 tenders rated as ‘acceptable’ or better against the first two criteria:

• 14 tenderers (47 per cent) of those rated as ‘acceptable’ or better32 were shortlisted; and

• 16 tenderers (53 per cent) rated as ‘acceptable’ against those criteria were not shortlisted.

2.50 Further, in respect to the shortlisting approach:

• only three of eleven tenders (27 per cent) from non-incumbent tenderers33 (each for the growth outcome with none for the innovation outcome) were shortlisted; and

• of the three non-incumbents shortlisted for growth, two were for Northern Territory and one was for Western Australia, meaning that in the remaining five states/territories there was no competition from non-incumbents.

2.51 Following an initial ‘value for money’ assessment and interview responses, the TAP reduced the shortlist of tender submissions further to ten. The remaining tender submissions proceeded to

32 Comprising all five tenders rated as ‘Exceptional’ (5), all seven tenders rated as ‘Good’ (4) and two of the 18 tenders rated as ‘Acceptable’ (3). 33 While there were three non-incumbent tenderers shortlisted for the commercialisation outcome, these services were not provided by delivery partners previously, and no incumbent providers submitted tenders to

provide those services for the Entrepreneurs’ Programme.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

34

the final value for money assessment stage, which included an evaluation against the corporate and financial viability criterion and further comparative assessments of price. Of the eight included in the final shortlist for the growth and innovation outcomes34, only one was a non-incumbent (Darwin Innovation Hub), which was successful and awarded a contract.

2.52 As noted at paragraphs 2.34 and 2.41 above, the TAP did not properly conduct an evaluation against either of the risk or Commonwealth policies criteria.

2.53 In April 2022, DISER advised the ANAO that:

The RFT could have been clearer in the TAP’s intention to shortlist based on the capacity to deliver the required services. The RFT should not have contained the words “criteria not in order of importance” This was an unintentional error noting the RFT was approved by the Probity Adviser.

Value for money assessment and recommendations

2.54 After reducing the shortlist to 10 tender submissions, the TAP conducted the overall value for money assessment. As stated in paragraphs 1.2 and 1.4, the RFT advised potential tenderers that the department was expecting to engage up to 10 delivery partners, including:

• a single delivery partner to provide commercialisation services to customers nationally (in relation to the commercialisation outcome);

• a single delivery partner to provide innovation and incubator support services nationally (in relation to the innovation outcome); and

• one or more delivery partners to provide expert business advisory and facilitation services in nominated geographic areas on a state and/or regional basis (in relation to the growth outcome).

2.55 The tender evaluation plan required the TAP to rank the tender submissions on the basis of the technical merit of each response, the associated risks and the cost of the solution for each response. The TAP did not do this. Rather, the TAP ranked the remaining ten tender submissions in ‘bands’ as ‘competitive and recommended’, ‘competitive and not recommended’ or ‘uncompetitive’ against each of the outcomes (commercialisation, innovation, and growth). This meant that DISER was not able to transparently demonstrate that the successful candidates were recommended based on their merits as determined by the evaluation against each of the six specified criteria.

Commercialisation

2.56 Two tender responses offering commercialisation services were shortlisted for comparative assessment: Respondent 53 and i4 Connect. Table 2.1 shows DISER’s ranking of each tenderer in respect of price and risk ratings for ‘corporate’ and ‘price’.

34 While there were two non-incumbent tenderers shortlisted for the commercialisation outcome at the ‘value for money stage’, these services were not provided by delivery partners previously, and no incumbent providers submitted tenders to provide those services for the Entrepreneurs’ Programme.

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

35

Table 2.1: DISER ranking of commercialisation tender responses

Rank Tenderer name Total

Tender Price per facilitator

Total tender price

Corporate risk Price risk Preferred tenderer

(Yes/No)

1 Respondent 53 $1,113,868 $21,163,500 High Medium-

High

No

2 i4 Connect $1,455,208 $34,925,000 High Low Yes

Source: ANAO analysis of DISER records.

2.57 While Respondent 53 was identified as the first ranking tenderer, the TAP identified i4 Connect as the recommended tenderer.

2.58 The analysis presented in the tender evaluation report in favour of selecting i4 Connect did not satisfactorily address the significant price difference and there was bias evident in the analysis to support the decision to recommend i4 Connect over Respondent 53.

• The TAP gave considerable weight to ‘inherent risks’ in Respondent 53’s bid regarding the financial and corporate viability of a consortium with limited financial history, and governance structure. While risks were acknowledged in the analysis of i4 Connect, these were outweighed by the TAP’s confidence in the management team.

• Notwithstanding that ‘price risk’ was not evaluated for any of the other outcomes, the TAP gave considerable weight to the price risk associated with Respondent 53’s bid.

• The strengths of Respondent 53’s bid identified by the TAP in its value for money assessment were not presented in the analysis in the tender evaluation report.

• The tender evaluation report included statements recycled from the

2014-15 procurement’s tender evaluation report in regard to a different tenderer, which indicates it was not a complete and genuine attempt to support the recommendation.

Innovation

2.59 Two tender responses, both from incumbent providers, for innovation services were shortlisted for a comparative assessment: Respondent 1 and Commonwealth Scientific and Industrial Research Organisation (CSIRO). The tender evaluation report concluded that there was ‘minimal difference between the two tender submissions on the basis of both price and corporate and financial viability’. On a price per facilitator basis, CSIRO’s price was marginally lower.

2.60 CSIRO was rated as both ‘competitive’ and ‘recommended’ while Respondent 1's was rated as ‘competitive’ and ‘not recommended’.

Growth

2.61 In relation to the growth outcome, the RFT stated that the department was ‘seeking to engage one or more delivery partners to provide expert business advisory and facilitation services in nominated geographic areas on a state and/or regional basis’. According to the tender evaluation report, the TAP identified that ‘placed-based knowledge, relationships and local networks for delivery services was pivotal’. As a result, the TAP decided that a single provider for growth services would not be recommended as the preferred tenderer.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

36

2.62 Two of the shortlisted tenderers (Australian Industry Group (Ai Group) and Deloitte Consulting Pty Ltd (Deloitte)) both tendered for national service delivery, and one other (NSW Business Chamber Ltd (NSWBC)) tendered for services across the designated states and regions of the eastern seaboard, as well as the Northern Territory.35 The TAP approached each of the three tenderers regarding their tendered delivery model to confirm that they were prepared to revise their tender responses to deliver services on a state and/or regional basis. To accommodate this approach, the TAP also requested additional pricing information.

2.63 DMV Consulting was engaged to conduct a comparative price assessment of the seven shortlisted tender responses for growth in relation to the following regions:

• Queensland;

• New South Wales;

• Victoria and Tasmania;

• South Australia;

• Northern Territory; and

• Western Australia.

2.64 The assessment compared the total prices of three or more tender responses in each of these regions and ranked them accordingly. The tenderers that submitted bids for national services — Ai Group and Deloitte — featured in the comparative assessments for all six regions, while NSWBC featured in three of the six regions (Queensland, New South Wales, and Victoria and Tasmania). NSWBC’s tender submission also included the Northern Territory in its service coverage but was not included in the comparative assessment for the Northern Territory.

2.65 The pricing information used for comparative assessment in DMV Consulting’s report was unreliable. The report did not include a transparent methodology for how it calculated the adjusted prices of the tender submissions of Ai Group, Deloitte and NSWBC in accordance with the baseline FTE used in each region. The report also contained errors in relation to the pricing for Darwin Innovation Hub’s tender submission, including the error in the initial pricing analysis report, discussed in paragraph 2.32, and stated the total adjusted tender price as being $4,384,799 GST

inclusive which was actually the GST exclusive amount, as shown in Table 2.2. This meant that the total price was represented as being $0.44 million less than the actual tendered price.

2.66 Table 2.2 highlights the TAP’s first ranked tenderers for each region based on the comparative assessment of price.

35 NSWBC’s proposed service coverage included four designated state/territory areas (New South Wales/Australian Capital Territory, Victoria, Tasmania, Northern Territory), and four designated regions (Far North Queensland, South East Queensland, Geelong and surrounds, and Hunter region).

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

37

Table 2.2: Outcome of comparative analysis for ‘growth’

Region Tenderer

Name

FTE used for evaluation purposes

Total

adjusted tender price (3 years)

Total Price per facilitator (FTE)

Corporate risk

Queensland Ai Group 21 $19,284,192 $918,295 Low

New South Wales NSWBC 31 $28,186,431 $909,240 Low

Victoria/Tasmania Ai Group 28 $25,678,578 $917,092 Low

South Australia Business SA 11 $8,504,987 $773,181 Low

Northern Territory Darwin Innovation Hub

5 $4,384,799a $876,960b Medium

Western Australia Respondent 15 10 $8,411,052 $841,105 Low

Note a: Darwin Innovation Hub’s total adjusted tender price was reported in the table incorrectly. The correct amount was $4,823,280. Note b: Darwin Innovation Hub’s total price per facilitator (FTE) based on the correct total adjusted tender price is $964,656. Source: ANAO analysis of DISER records.

2.67 Table 2.3 shows the TAP’s final selection of recommended candidates for growth services. The ANAO observed the following inconsistencies with the selections.

• Opportunities were provided to Ai Group, Deloitte and NSWBC to revise or improve tender submissions with alternative delivery models to what they had initially proposed, along with additional pricing information in support of those alternative delivery models.

• While Deloitte — an incumbent provider — was not identified as the best candidate in any particular region, DISER tailored arrangements to select Deloitte to provide growth services in Queensland on the basis that it was able to provide additional ‘national specialist roles’. The RFT did not include a requirement for ‘national’ specialist roles.36

• Darwin Innovation Hub’s tender submission was originally regarded as both non-compliant and non-competitive before the TAP decided it would shortlist the submission (see paragraphs 2.23 and 2.24). An error was made in the initial pricing assessments which lead to it initially being ranked as the best price for growth and subsequently being shortlisted for the final value for money assessment. A further error in the comparative assessment of price in the Northern Territory meant that the difference in price with the next ranked tender submission was overstated.

36 In March 2022, the department advised the ANAO that: Following shortlisting against technical criteria and a value for money assessment, Deloitte demonstrated they had exceptional capability and capacity to collaborate and work flexibly to deliver the requirements outlined in the evaluation criteria of the RFT. Therefore, Deloitte demonstrated the capability and capacity to deliver growth and/or specialist services to all and any States and territories.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

38

Table 2.3: Final selection for ‘growth’

State/Region Tenderer Incumbent provider?

Victoria, Tasmania and Western Australia Ai Group Yes

New South Wales/Australian Capital Territory NSWBC Yes

Queensland and ‘specialist roles’ Deloitte Yes

Northern Territory Darwin Innovation Hub No

South Australia Business SA Yes

Source: ANAO analysis of DISER records.

Recommendation no. 2 2.68 When evaluating tender responses, the Department of Industry, Science, Energy and Resources fully evaluate responses received consistent with the approach set out in the approach to the market, with the results of this work relied upon to select the successful candidate(s).

Department of Industry, Science, Energy and Resources response: Agreed.

2.69 The changes we are making to our procurement-related frameworks, processes and procedures will provide for the clear identification of selection criteria and evaluation process in its approach to the market, and these changes will be included in our procurement checklist against which we will monitor compliance. Our department-wide Probity Framework will address transparency, consistency and fairness requirements.

Were procurements conducted ethically, including identifying and managing any conflicts of interest and transparently managing incumbency advantage?

The procurements were not conducted in accordance with the guidance provided by the CPRs about how to conduct procurements ethically.

• The department involved the probity adviser it engaged in the drafting of the RFT and the conduct of the evaluation activities, an approach that adversely affects the independence of the probity advice.

• The procurement of the probity adviser was deficient, including an absence of open and effective competition for the role.

• While a probity plan was documented, key elements were not implemented, for example:

− a conflict of interest register was not maintained;

− the probity register was incomplete;

− a list of key persons with access to controlled information was not maintained by the procurement manager and conflict of interest declarations were not obtained from all persons that had access to controlled information. Those that did not provide a declaration included the procurement delegate and internal

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

39

legal and procurement advisers who formed part of the specialist advice and support team; and

− where declarations were provided not all conflicts were identified and, where conflicts were identified, management actions were not put in place to avoid or mitigate them.

• Incumbency advantages were not transparently managed, in particular:

− there were no specific contractual or payment arrangements in place to govern the department’s engagement of five of the ten existing industry partners to contribute to the redesign and payment for this work;

− the department did not implement in full the probity risk management measures recommended by its probity adviser (for example, a probity plan and protocols for the redesign work was not in place at the commencement of work and the department did not prohibit the industry partner personnel involved in the redesign from assisting in the preparation of tender responses); and

− departmental records indicate that information about the redesign of the program was revealed to incumbent tenderers involved in the redesign work providing them with a competitive advantage (this disclosure of information to the incumbents involved with the redesign work, and the seeking of urgent advice from the probity adviser about it, was not recorded in the probity register).

2.70 The CPRs require that procuring entities act ethically throughout the procurement, including by recognising and dealing with actual, potential and perceived conflicts of interest and by dealing equitably with potential suppliers, tenderers and suppliers.37

Probity adviser

2.71 In August 2019, DISER appointed Mills Oakley as the probity adviser for the RFT to advise on key documentation for the procurement process, conduct probity briefings for procurement personnel and advise on ad hoc probity and process issues.38 While the estimated cost of the engagement was $76,316, the contract notice published on AusTender has a contract value of $116,316.39 The ANAO’s analysis of DISER records indicates that the total expenditure on probity services provided by Mills Oakley for the tender was $151,019.40

37 Finance, Commonwealth Procurement Rules, paragraph 6.6. Appendix 8 summarises the ANAO’s analysis of those areas where DISER did not comply with the ethical standards set out in the CPRs. 38 The Department of Finance guidance on ethics and probity in procurement notes that while external probity advice can be a valuable asset in a tender process, the appointment of such specialists and the sign-offs

provided by those specialists, do not remove the agency’s accountability for the process: Finance, Ethics and Probity in Procurement [Internet], Finance, available from https://www.finance.gov.au/government/procurement/buying-australian-government/ethics-and-probity-procurement [accessed December 2021]. 39 The contract notice reflects the varied price of the contract for probity services. The variation was agreed

between the department and probity adviser in November 2019. 40 The amount expended on probity advice is $74,704 higher than the original approved contract value and $40,000 higher than the approved value of the contract.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

40

Engagement of the probity adviser

2.72 DISER sourced the probity services through a departmental legal services panel arrangement. Three quotes were obtained from different law firms on the panel and evaluated based on each provider’s capacity and capability, the price offered and any identified risks. The engagement of Mills Oakley as probity adviser did not transparently demonstrate value for money and lacked probity due to the evaluation:

• criteria applied by DISER not being included in the request documentation41;

• of each candidate’s capability unfairly advantaged Mills Oakley because the deciding factor was its previous experience with the Entrepreneurs’ Programme (Mills Oakley was the probity adviser on the previous procurement of industry partners for the Entrepreneurs’ Programme and was directly approached in June 2019 to provide probity advice on the redesign of the program — refer to paragraph 2.92 below); and

• of each provider’s price unfairly advantaged Mills Oakley because, while another candidate offered a lower estimated price, ‘the Mills Oakley quote showed their greater relevant experience which indicated that they had a better grasp of what work was required of them.’

2.73 Engaging the same probity adviser on an ongoing or serial basis over several related or unrelated issues increases self-interest and familiarity risks that may threaten the actual or perceived independence of the practitioner.42 DISER’s focus on the benefits of Mills Oakley’s previous experience in its evaluation also failed to consider whether this experience could adversely affect the independence of its probity advice.

Scope of probity services

2.74 Probity experts should be independent and free from all conflicts of interest and have sound knowledge of all relevant government policies and procedures.43 DISER engaged Mills Oakley to provide advice on probity and process issues.44 As noted in paragraph 2.71, the cost of the probity services was double that estimated by the department when it undertook the procurement. DISER correspondence with the probity adviser indicates that the increased cost reflects:

a number of additional iterations of the RFT (including detailed review and drafting and amendment of the evaluation criteria, schedules, and response attachments) and other work additional to [the probity adviser’s] original scope such as the provision of a draft tender evaluation plan and detailed drafting work on the RFT and response schedules between them and connections with them and evaluation criteria.

41 The CPRs require evaluation criteria to be included in request documentation to enable the proper identification, assessment and comparison of submissions on a fair, common and appropriately transparent basis: Finance, Commonwealth Procurement Rules, paragraph 7.14. 42 New South Wales (NSW) Auditor-General, Engagement of probity advisers and probity auditors, NSW Auditor-

General, available from https://www.audit.nsw.gov.au/our-work/reports/engagement-of-probity-advisers-and-probity-auditors [accessed November 2021]. 43 Finance, Ethics and Probity in Procurement [Internet], Finance, available from https://www.finance.gov.au/government/procurement/buying-australian-government/ethics-and-probity-

procurement [accessed December 2021]. 44 Finance guidance distinguishes between ‘process’ and ‘probity’ advisers. A process adviser is an expert who provides advice on tender process structure. A probity adviser is an expert who advises on probity issues as they arise during a tender process: Finance, Ethics and Probity in Procurement [Internet].

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

41

2.75 At the request of the department, Mills Oakley had involvement in procurement work including:

• drafting planning documents including the probity plan and tender evaluation plan;

• drafting the RFT documentation, including redrafting the evaluation criteria and response schedules;

• drafting the draft form of contract attached to the RFT;

• drafting one of the evaluation templates used by the TAP;

• recommending financial analysts to undertake pricing and value for money assessments; and

• participating as a procurement team member in meetings with a tenderer during the value for money evaluation phase of the tender process.

2.76 DISER’s decision to source probity and procurement process advice from the same adviser adversely affected the independence of the probity advice. For example, in September 2019, Mills Oakley advised the delegate that ‘on the basis of the review criteria and investigations contained in this letter, we are satisfied that the RFT and draft form contract meet the probity and procurement requirements.’ This was not appropriate because the same team from Mills Oakley was responsible for developing the RFT and providing the interim probity report.

Recommendation no. 3 2.77 The Department of Industry, Science, Energy and Resources improve its procurement framework to specifically address the engagement of probity advisers, including ensuring that advisers are independent and objective by not engaging the same probity advisers on an ongoing or serial basis.

Department of Industry, Science, Energy and Resources response: Agreed.

2.78 The engagement of probity advisers will be addressed in our department-wide Probity Framework, including how to ensure independence and objectivity and manage risk.

Probity plan and protocols

2.79 The probity plan for the procurement was approved by the delegate in September 2019.

2.80 The probity plan formed ‘a key part of assurance to delegates, suppliers and the Commonwealth that the procurement process for the program is conducted in a manner that is fair, equitable and defensible.’ The probity plan applied to ‘key persons’ involved in the procurement, including members of the assessment panel and other persons nominated by the procurement manager. Probity protocols were also issued to members of the procurement team, providing detailed instruction on reporting probity issues, assessment and record keeping requirements, information management, declaration of conflicts of interest and communications with potential tenderers.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

42

2.81 The probity protocols established by the probity plan were not fully implemented45, and had limited effect on the conduct of the procurement. Key deficiencies in implementation related to the management of conflicts of interest (refer to paragraph 2.82 to 2.86 below) and maintenance of a complete probity register. In addition, while probity briefings were undertaken for the procurement team, key personnel involved in the procurement did not attend them (refer to paragraphs 2.17 and 2.18 above).

Conflicts of interest management

2.82 DISER’s probity plan for the procurement stated that a conflict of interest arises in any circumstance in which an official’s public duty is (or is seen to be) in conflict with their private interest.

2.83 Department of Finance guidance to entities on ethics and probity in procurement states that ‘persons involved in the tender process, including contractors such as legal, commercial or probity experts, should make a written declaration of any actual, potential or perceived conflicts of interests prior to taking part in the process’.46 Consistent with this guidance, the probity protocols required members of the procurement team to declare actual or potential conflicts of interest in relation to the procurement. The department has not retained on file all conflict of interest declarations, including that of the external financial analyst responsible for preparing pricing evaluations and corporate and financial viability checks of shortlisted tenderers, and not all members of the procurement team signed conflict of interest declarations, including:

• the procurement delegate47; and

• internal legal and procurement advisers who formed part of the specialist advice and support team.

2.84 In March 2022, DISER advised the ANAO that in its view:

Requiring a specific conflict of interest declaration from any person, even those with a tenuous and superficial link to the procurement process is overly bureaucratic, administratively burdensome and does not appropriately manage risk.48

2.85 Not all conflicts of interest were identified. For example, one internal expert did not declare his (known) employment relationship of nine years with one of the tenderers — a breach of the probity protocols — and, as a result, no steps were taken to manage this conflict.

2.86 DISER’s approach to management of conflicts of interest for the procurement and compliance with probity protocols was largely passive. No management actions were put in place

45 Refer to Appendix 5 for a detailed examination of the implementation of probity controls contained within the probity plan. 46 Finance, Ethics and Probity in Procurement [Internet], Finance, available from https://www.finance.gov.au/government/procurement/buying-australian-government/ethics-and-probity-

procurement [accessed December 2021]. 47 The department’s conflict of interest and insider trading policy requires senior executive service (SES) staff to complete annual conflict of interest declarations. The procurement delegate completed the SES conflict of interest declaration for 2019-20, with no declared interests relating to the procurement. 48 The department’s probity plan required the procurement manager to nominate 'key persons' to allow access

to controlled information. Those persons were also required to prepare conflict of interest declarations in accordance with the department's probity plan. No such list was maintained within the department's probity plan or records management system. The ANAO's analysis was based on who had access to controlled information and who the procurement manager requested to complete conflict of interest declarations.

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

43

to avoid or mitigate identified conflicts of interest. This was reflected in a number of missed opportunities to avoid or effectively manage potential conflict of interest concerns for procurement team members who held contract management positions in the department.

Incumbency advantage

2.87 The CPRs state that all potential suppliers to government must be treated equitably based on their commercial, legal, technical and financial abilities.49 When an incumbent provider competes for new work, it may have (or be perceived to have) certain advantages, such as understanding of an agency’s needs, established relationships with agency staff, and knowledge that is not available to other potential suppliers.50

Composition of the TAP

2.88 DISER advised the ANAO in October 2021 that a key probity control to manage incumbency risks was the composition of the TAP. One member of the TAP was external to the Entrepreneurs’ Programme (from within the AusIndustry division of the department) and the other two members were employed in the Entrepreneurs’ Programme branch. Further, Figure 2.2 above demonstrates that the number of technical evaluations was not divided equally between panel members.

• Assessor 1 (internal to the Entrepreneurs’ Programme) completed 34 stage 2 technical assessments (39 per cent of assessments completed).

• Assessor 2 (internal to the Entrepreneurs’ Programme) completed 44 stage 2 technical assessments (51 per cent of assessments completed).

• Assessor 3 (external to the Entrepreneurs’ Programme) completed 9 stage 2 technical assessments (10 per cent of assessments completed).

2.89 While the design of the TAP was intended to mitigate incumbency risks, the unequal participation of assessors in stage 2 technical evaluations demonstrates that the effectiveness of this control was minimal.

Probity controls for the redesign project

2.90 Incumbent providers may be consulted on technical, operational or other information to be used in a procurement process. There are significant risks with this approach that require management, as an incumbent may (or be perceived to) gain insight from the information collected, giving it an unfair advantage.51

2.91 Prior to commencing the RFT, DISER undertook a redesign project with the stated aim to improve the operation of the program, including the delivery mechanism and the structure of the program. DISER engaged five of the ten existing industry partners to assist with the redesign to draw

49 Finance, Commonwealth Procurement Rules, paragraph 5.4. The CPRs note that ethical behaviour includes dealing with potential suppliers, tenderers and suppliers equitably: Finance, Commonwealth Procurement Rules, paragraph 6.6b. 50 NSW ICAC, Dealing with incumbent providers in procurement [Internet], NSW ICAC, available from

https://www.icac.nsw.gov.au/prevention/corruption-prevention-publications/latest-corruption-prevention-publications/dealing-with-incumbent-providers-in-procurement [accessed December 2021]. 51 ibid.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

44

on their ‘deep knowledge of customer needs/program experience.’52 In response to a request from the ANAO for advice on the contractual arrangements for engaging five of the incumbent providers to assist with the redesign, and how much they were paid for this assistance, in March 2022 DISER advised the ANAO that:

DISER did not enter into new contracts with providers for the redesign project. DISER accessed its existing arrangements with the supplier and paid for their services under those arrangements.53

2.92 DISER engaged Mills Oakley to provide probity advice on actual and perceived conflicts of interest involved in industry partners participating in the redesign project, who may later tender for services in a resultant procurement process.54 While Mills Oakley noted that engagement of industry partners was acceptable from a probity and legal perspective, strong probity protocols should be implemented to provide ‘sufficient probity and practical protection for the department, the process, the individuals and their employers in any future EP procurement process’. Table 2.4 sets out the probity adviser’s recommended probity measures for the redesign project and the extent to which these probity controls were implemented.

Table 2.4: Implementation of probity controls for the redesign project

Recommendation Was this implemented?

Implement an appropriate probity plan and protocols. A probity plan and protocols were not implemented until after the redesign process was paused in

September 2019.

Providing additional information in the RFT and fully disclosing the nature and extent of industry partner involvement in the development of the new program.

Information regarding the redesign and the industry partners’ involvement in the process was included in the RFT.

Industry partners and their employers execute deed polls setting out confidentiality obligations in favour of the department.

Industry partners completed deed polls, but key provisions were removed from the deed polls at the request of industry partners. Refer to paragraph 2.93 for more information.

Conducting a comprehensive industry briefing. The department held two industry briefings where they addressed probity matters

Giving tenderers sufficient time to respond to the tender. Tender submissions were open for two months between 27 September 2019 and

27 November 2019.a

52 Six personnel were engaged from Deloitte (including two personnel engaged as part of the ‘design team’), four personnel were engaged from Ai Group (including one person engaged as part of the ‘design team’), and three personnel engaged from NSWBC to contribute to the redesign. All other participating companies supplied one person. 53 Similarly, DISER records show that in April 2020 it was attempting to obtain ‘pro bono’ assistance from one of

the successful Entrepreneurs’ Programme tenderers with other work on the basis that the firm should be prepared to make a ‘corporate contribution’ to the department in recognition of the amount of money it was being paid under its Entrepreneurs’ Programme contract. 54 As noted at paragraph 2.72 above, DISER approached Mills Oakley for a quote for probity services for the

redesign project in June 2019. No other suppliers on the panel were approached for quotes and no records were maintained as to why this approach was considered to provide value for money. According to ANAO analysis of DISER records, DISER spent $10,058 on this probity advice.

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

45

Recommendation Was this implemented?

The department provides industry partners with a probity briefing. The department did not provide probity briefings for industry partners involved in the redesign

process.

Note a: While the RFT was originally due to close on 28 October 2019, the closing date was extended three times during the period tender submissions were open. Complaints were received by the Minister’s Office about the procurement process, which were provided to the procurement delegate. For example, one complaint stated that the tender application is ‘extremely paper work heavy’ and includes ‘requests for a lot of superfluous and theoretical requirements.’ ANAO analysis of DISER records indicates that the procurement delegate agreed to the second extension of the timeframe because of ‘the degree of agitation by the [Minister’s] Office and their view of the possibility of complaints’. The procurement delegate approved the third extension due to the ‘strong market interest’, number of requests for clarification (approximately 200) and complaints received by the Minister’s Office. Source: ANAO analysis of DISER records.

2.93 Table 2.4 illustrates that some key probity controls for the redesign process were not implemented or eroded during the redesign process. For example, Mills Oakley recommended including provisions in the confidentiality deed polls prohibiting industry partner personnel involved in the redesign project from assisting in the preparation of the tender response. DISER removed these provisions, despite the heightened risk. Removal of these provisions occurred at the request of industry partners, who raised concerns that these provisions would unfairly disadvantage their tender preparation.

Breach of probity for the redesign project

2.94 The redesign project was put on hold before the release of the RFT in September 2019. While DISER had planned for the redesign project to continue through the RFT, Mills Oakley advised the department that to mitigate potential probity risks the redesign project should be paused.

2.95 In a briefing to the Deputy Secretary, DISER officials stated that ‘following legal advice regarding probity issues, the redesign project has been put on hold until the RFT process is complete’. In contrast, the ANAO’s examination of DISER email records indicates that the redesign project was put on hold because of a breach of the information barriers put in place. The breach was that advice was provided to those incumbent providers involved in the redesign that effectively meant that the work they had performed had been completed and the delivery model decided, rather than being simply concept development work to present options for the department to consider and make a decision later on. This breach provided those incumbent providers with information about the tender process before the RFT was released.55

2.96 The breach occurred on 30 August 2019 and a meeting with the probity adviser to obtain ‘urgent’ advice was held the next Monday, 2 September 2019. No record of the nature of the probity issue and how it was handled was entered in the probity register.

2.97 The probity advice recommended putting the project on hold until after completion of tender evaluation and delivery partners were selected. Nevertheless, the redesign project recommenced in December 2019, including consultation with one of the incumbent providers — Deloitte — three months before the delivery partners were selected. While an email to Deloitte stated that resumption of the process was based on ‘probity advice’, written advice was not evident from DISER’s records with the department advising the ANAO in March 2022 that this

55 The Request for Tender was released on 27 September 2019.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

46

occurred via a phone call with a record of the call not made. The re-engagement of Deloitte was also not recorded in the probity register.

Recommendation no. 4 2.98 The Department of Industry, Science, Energy and Resources improve its procurement framework to specifically address how it will manage the risk of any incumbency advantages when conducting procurement processes.

Department of Industry, Science, Energy and Resources response: Agreed.

2.99 Our department-wide Probity Framework will address how to manage incumbency advantages, including through appropriate controls and monitoring compliance.

Were appropriate procurement records maintained?

Appropriate procurement records were partly maintained. While available records addressed the requirement for the procurement as well as the process that was followed and relevant approvals, evidence to support key decisions was not maintained. In addition, the tender evaluation report did not accurately reflect the evaluation process that was employed or satisfactorily demonstrate that value for money had been achieved.

2.100 The CPRs state that officials must maintain a level of documentation commensurate with the scale, scope and risk of each procurement. The documentation should provide accurate and concise information on the requirement for the procurement; the process that was followed; how value for money was considered and achieved; relevant approvals; and relevant decisions and the basis of those decisions.56 As described in paragraph 1.8, the Entrepreneurs’ Programme delivery partner procurement was the largest value procurement for DISER in 2019-20, representing 37 per cent of contracts entered into by DISER.

2.101 The requirement for the procurement was recorded in the stage 1 approval endorsed by the delegate in September 2019 and the tender evaluation report. Relevant approvals were endorsed by the delegate through the department’s financial management system, and the process that was followed was set out in the tender evaluation report to the delegate.

2.102 The ANAO observed the following deficiencies in the department’s record management practices for the procurement.

• Key documentation relating to the procurement was not signed or approved by the TAP or delegate (refer to paragraphs 2.8 and 2.9).

• While the tender evaluation report was signed by the TAP members, dates were not recorded. The procurement delegate signed and dated the tender evaluation report but did not indicate whether the report was ‘approved’ or ‘not approved’.

56 Finance, Commonwealth Procurement Rules, paragraph 7.2 to 7.3.

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

47

• Incorrect or misleading statements were included in the tender evaluation report to the delegate (refer to Appendix 4 for further details).57

• There are no records (such as minutes) of the TAP’s meetings. DISER officials provided notes in emails as evidence of TAP deliberations.

• Records of individual evaluations by TAP members are incomplete (refer to paragraph 2.25 to 2.41). Two records of individual evaluations are blank.

• There are insufficient records regarding the probity of the procurement process. For example, the probity register is not a complete record of all occurrences concerned with the probity of the procurement process (refer to Appendix 5 for more information).

• Key procurement records were not stored in the secure procurement library. During the course of the audit, DISER added approximately 200 records to the procurement library including conflict of interest declarations, correspondence with the probity adviser and TAP deliberations.

Recommendation no. 5 2.103 The Department of Industry, Science, Energy and Resources improve its procurement record keeping so that accurate and concise information exists on:

• the process that was followed;

• how value for money was considered and achieved;

• relevant approvals; and

• relevant decisions and basis of those decisions.

Department of Industry, Science, Energy and Resources response: Agreed.

2.104 Our department-wide contract management framework and Probity Framework will include assurance processes to ensure the right documentation is in place across the life cycle of a procurement and contract. We have already made changes to our standard operating procedures for the Entrepreneurs’ Programme to improve record-keeping.

Was the procurement approach open, fair and non-discriminatory?

While DISER selected an open tender procurement method and issued a request for tender, the procurement did not demonstrate open and effective competition in accordance with the CPRs. The department’s procurement approach resulted in incumbent providers receiving greater consideration than non-incumbents. This was reflected in the tender outcomes with 83 per cent of contracts being awarded to an incumbent provider whereas they represented 20 per cent of tenders received. The procurement approach did not give appropriate consideration to Commonwealth policies relating to Small and Medium Enterprises (SMEs).

57 Tender evaluation reports should be objective, impartial and written to give the decision-maker all the information they need. This includes all relevant information, good and bad, about preferred tenderers and not just the information that supports the preferred tenderer: NSW ICAC, Dealing with incumbent providers in procurement [Internet], NSW ICAC, available from https://www.icac.nsw.gov.au/prevention/corruption-prevention-publications/latest-corruption-prevention-publications/dealing-with-incumbent-providers-in-procurement [accessed December 2021].

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

48

While 81 per cent of tenders submitted were from SMEs, 20 per cent (9 of 44) of those were shortlisted (15 SMEs rated as ‘acceptable’ against the two shortlisting criteria nevertheless did not proceed any further in the evaluation).

2.105 The CPRs state that procurements should encourage competition and outline that effective competition requires non-discrimination and the use of competitive procurement processes.58

Open and effective competition

2.106 As described in paragraphs 2.28 and 2.46 above, DISER shortlisted tender submissions following the stage 2 technical evaluation against the first two criteria59 with 14 of the 30 tenders assessed as ‘acceptable’ or better against those two criteria shortlisted and 16 of the tenders assessed as ‘acceptable’ or better against those two criteria not shortlisted. Shortlisting did not provide for open and effective competition to be maintained.

2.107 Figure 2.3 illustrates the extent to which shortlisting limited competition at key stages of the procurement for each of the three outcomes (growth, commercialisation, and innovation). As noted in paragraph 2.47, the shortlisting process had a significant bearing on the likelihood of tenderers being successful such that the department should have communicated in the RFT the extent to which shortlisting would be applied.

Figure 2.3: Number of tenderers at key stages of the procurement

Source: ANAO analysis of DISER records.

58 Finance, Commonwealth Procurement Rules, paragraph 4.4. 59 Although shortlisting was on the basis of the first two criteria, for six tender respondents an evaluation was conducted against only one of those two criteria.

0

5

10

15

20

25

30

35

40

45

50

Growth Commercialisation Innovation

Number of tenderers

Tenders submitted per outcome Shortlisted tenderers

Shortlisted for value for money

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

49

2.108 Figure 2.4 illustrates that the department’s approach to the procurement resulted in incumbent providers receiving greater consideration. While 20 per cent of tenders submitted for the growth and innovation outcomes were from incumbent providers, tender submissions from incumbents represented 73 per cent of shortlisted submissions.60 In contrast, three submissions (27 per cent) from non-incumbent tenderers for growth and innovation were shortlisted, and one (Darwin Innovation Hub) was awarded a contract. The other non-incumbent that was successful — i4 Connect — was for the commercialisation outcome, for which there were no incumbent delivery partners.

Figure 2.4: Proportion of incumbent providers at key stages of the procurement

Note: There were 51 tenders submitted for the growth and innovation outcomes; 11 tenders shortlisted for the growth and innovation outcomes; and six contracts awarded for the growth and innovation outcomes. Note: This figure does not include tenders submitted exclusively for the commercialisation outcome. Commercialisation services were not provided by a delivery partner under the previous Entrepreneurs’

Programme. No incumbent providers submitted a tender for commercialisation. Source: ANAO analysis of DISER records.

2.109 The consequences of DISER’s shortlisting were that:

• three submissions (27 per cent) from non-incumbent providers for growth (three) and innovation (zero) were shortlisted;

• of the three non-incumbents shortlisted for growth, two were for services in Northern Territory and one was for services in Western Australia;

• in New South Wales/Australian Capital Territory, South Australia, Tasmania, Queensland, and Victoria there was no competition from non-incumbents; and

60 Commercialisation services were not provided by a delivery partner under the previous Entrepreneurs’ Programme, and no incumbent providers submitted a tender for commercialisation.

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Tenders submitted Tenders shortlisted Contracts awarded

Percentage of tenderers

Not incumbent Incumbent

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

50

• after further shortlisting there was one non-incumbent — Darwin Innovation Hub — remaining for the evaluation against the ‘corporate and financial viability’ criterion and for the final value for money assessment.

2.110 The CPRs state that to:

ensure that SMEs can engage in fair competition for Australian Government business, officials should apply procurement practices that do not unfairly discriminate against SMEs and provide appropriate opportunities for SMEs to compete.61

2.111 ANAO analysis indicates that the department’s approach to evaluation did not include appropriate consideration of Commonwealth policies relating to SMEs (refer to paragraphs 2.40 to 2.41). The result of the evaluation was that:

• while 81 per cent of tenders submitted were from SMEs, 20 per cent (9 of 44) of those were shortlisted with 15 of the 16 tenderers (94 per cent) assessed as ‘acceptable’ or better against the two criteria used for shortlisting that were not shortlisted being from SMEs;

• 19 per cent of tenders submitted were from non-SMEs, and 50 per cent of those (5 of 10) were shortlisted; and

• three of the seven contracts awarded were to tenderers identified as SMEs.62

Fair and equitable treatment

2.112 Conducting a procurement process without due regard to probity and fair dealing may leave it open to potential challenges or complaints.63

2.113 Consistent with the ANAO’s analysis that the procurement did not demonstrate open and effective competition, and that tender submissions were not evaluated transparently in accordance with the published criteria and methodology, the ANAO identified a number of complaints from tenderers. Seven complaints from five tenderers were identified by the ANAO, either made to DISER or to the ANAO through the citizens contribution facility on the ANAO website (see Appendix 6 for details of the complaints and the extent to which each tenderer’s submission was evaluated against the evaluation criteria). The concerns raised in the complaints were that:

• tender prices had not been properly considered in the evaluation and that value for money was not demonstrated;

• tender submissions were not evaluated fairly and equitably such that SMEs were able to compete;

61 Finance, Commonwealth Procurement Rules, paragraph 5.5. 62 Darwin Innovation Hub is part of the Paspalis Group of Companies and a trustee for the Paspalis Enterprise Trust; it is also part of a consortium that includes the Centre for Appropriate Technology Limited, Desert Knowledge Australia, and Charles Darwin University, and was awarded funding through the Entrepreneurs’

Programme (Incubator Support and the Expert in Residence program). i4 Connect was formed as a Special Purpose Vehicle for the sole purpose of responding to this RFT. Business SA is a trading name for the South Australia Employers’ Chamber of Commerce and Industry Incorporated. 63 Australian Government Solicitor, Commercial notes No.15 [Internet], AGS, available from

https://www.ags.gov.au/publications/commercial-notes/ComNote15.htm [accessed December 2021].

Did the procurement process demonstrate achievement of value for money?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

51

• the information requirements for the tender submission were substantial, requiring significant time and costs to prepare, and the feedback provided by the department did not provide confidence that the evaluation was an impartial assessment of the merits of the submission as a whole; and

• the process of awarding the contract lacked probity.

2.114 Following the rejection of a submission or the award of a contract, officials must promptly inform affected tenderers of the decision.64 All unsuccessful tenderers were advised on 30 March 2020, including the two tenderers which were assessed as not meeting the minimum content and format requirements on 11 December 201965 and the 39 tenderers which were not shortlisted on 17 December 2019.

2.115 After receiving the department’s notification of the outcome, two unsuccessful tenderers (Respondents 2 and 44) raised concerns with the department regarding its handling of ‘competitive neutrality’ in the procurement given that the successful tenderer, CSIRO, is a Commonwealth entity. In its response to the tenderers, the department advised that:

All tender submissions were assessed against each of the evaluation criteria contained in the RFT through a comparative rating assessment process overseen by an independent probity adviser. The assessment process focused on capability and capacity to deliver the services first, before a pricing analysis was conducted. The pricing analysis and the subsequent value for money determination, which included bringing together the price assessment, assessment of risk and all other unweighted evaluation criteria determined that the CSIRO response complied with competitive neutrality requirements including those relating to pricing.

2.116 DISER’s advice to the tenderers was misleading and did not reflect the evaluation process that was conducted. Inconsistent with the advice that all tenderers were assessed against each of the evaluation criteria:

• a complete evaluation of Respondent 2’s tender submission was not conducted. Only two of the three TAP members conducted any assessment and those assessments only addressed two of the six criteria. The submission was rated 'technically suitable' against the first two criteria but after further consideration by the TAP the submission was not shortlisted as it was deemed ‘uncompetitive’ when compared to other submissions for the innovation outcome;

• a complete evaluation of Respondent 44’s tender submission was not conducted. Only one of the three TAP members conducted any assessment. The assessment only addressed two of the six criteria. The submission was rated 'technically unsuitable' and 'uncompetitive' against the first two criteria and was not shortlisted for further consideration; and

• neither submission was evaluated against the price, risk, corporate and financial viability and Commonwealth policies criteria.

64 Finance, Commonwealth Procurement Rules, paragraph 7.17. 65 DISER did not record the date the two tenderers were formally excluded (in accordance with their tender evaluation plan). This date is based on when the department received probity advice to exclude the tenders on the basis of not having met the minimum content and format requirements.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

52

2.117 The ANAO also observed inconsistent treatment of tenderers throughout the procurement process, such as:

• providing only 14 tenderers of the 30 tender submissions rated as ‘acceptable’ or above to the ‘internal panel experts’ for further technical evaluation (see paragraph 2.29);

• providing opportunities for select tenderers to improve or revise tender submissions (see paragraph 2.67);

• meeting with one tenderer (i4 Connect) to discuss its response to the department’s request for additional information (see paragraph 2.75) when this opportunity was not provided to another tenderer that had been asked for additional information (and which was not awarded a contract); and

• tailoring its approach to a particular tenderer (see paragraph 2.67).

Recommendation no. 6 2.118 The Department of Industry, Science, Energy and Resources:

• not exercise the extension options in the awarded contracts;

• commences work to conduct a new procurement process that will be completed before the existing contracts expire on 30 June 2023; and

• conducts the next procurement process in a manner that fully complies with the Commonwealth Procurement Rules.

Department of Industry, Science, Energy and Resources response: Agreed in principle.

2.119 While the department supports this recommendation in principle, the programme is being evaluated and decisions about its future, including programme design, will be subject to government consideration.

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

53

3. Are the contracts being appropriately managed to achieve the objectives of the procurement?

Areas examined The ANAO examined whether the contracts are being managed appropriately by the Department of Industry, Science, Energy and Resources (DISER) to achieve the objectives of the procurement.

Conclusion The delivery partner contracts are not being appropriately managed. The contract management framework is inadequate, and the department’s approach has not resulted in contract deliverables being provided on time or required that service provision is to an appropriate standard before payments are made. The contracts do not include an effective performance management framework.

Areas for improvement The ANAO made four recommendations. The first three relate to the existing delivery partner contracts (the development of contract management plans, adopting a proactive approach to managing delivery partner conflicts of interest, and the department taking prompt action in circumstances where delivery deadlines are not met and verifying that services have been provided before payments are made). The fourth recommendation is that, in future procurements, the department establish performance expectations and how these relate to contractual payments. This will place the department in a position to apply competitive pressure so as to secure the best possible outcome from the procurement process.

3.1 To assess whether the contracts are being appropriately managed, the ANAO examined the department’s reporting of the contracts and variations, the appropriateness of the department’s contract management framework and approach to securing delivery of contract deliverables, and whether the contracts include clear performance requirements and a means to effectively measure performance against the requirements.

Have relevant contracts and amendments been accurately reported on AusTender within 42 days of a contract being entered into or amended?

The department has with one exception met its obligations to accurately report the contracts and amendments on AusTender within the required 42 days. On one occasion the reported value did not align with the signed contract variation with Deloitte Consulting Pty Ltd (Deloitte), resulting in the reported value being $165,517 higher than the value of the contract variation.

3.2 The Commonwealth Procurement Rules (CPRs) establish reporting requirements for all relevant entities that enter or amend a contract value at or above the reporting threshold.66 DISER

66 Relevant entities must report contracts and amendments on AusTender within 42 days of entering into (or amending) a contract if they are valued at or above the reporting threshold: Finance, Commonwealth Procurement Rules, paragraph 7.18.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

54

(a non-corporate Commonwealth entity) has a reporting threshold of $10,000 with a reporting deadline within 42 days of entering or amending a contract.

3.3 Contracts with the successful tenderers were signed in March 2020. The department reported each of the contracts accurately within 42 days of being entered into. Since contract signature, there has been a total of 19 contract variations across the seven delivery partner contracts (see paragraphs 3.31 to 3.42). Three of the variations were below the $10,000 reporting threshold. Table 3.1 below outlines the department’s compliance with the reporting requirements.

Table 3.1: Delivery partner procurement AusTender reporting compliance

Delivery partner Contract signature date Original contract value

Current contract value

Variance

between original and current contract (%)

Total compliant AusTender reporting obligationsa

AusTender

reported variance

Australian Industry Group (Ai Group) 27 Mar 2020 $30,757,238

$34,059,327

11 4 of 4 $0

Business SA 18 Mar 2020 $8,553,300

$9,912,634

16 2 of 2 $0

Commonwealth Scientific and Industrial Research Organisation (CSIRO)

27 Mar 2020 $16,952,719

$16,208,844

(4) 4 of 4 $0

Deloitte 27 Mar 2020 $31,852,659 $37,031,346

16 2 of 3 $165,517

Darwin Innovation Hub 18 Mar 2020 $4,861,319 $5,838,400

20 3 of 3b $0

i4 Connect Pty Ltd (i4 Connect) 27 Mar 2020 $30,309,465

$31,388,800

4 2 of 2 $0

NSW Business Chamber Ltd (NSWBC)

27 Mar 2020 $20,845,745

$29,078,914

39 5 of 5 $0

Total N/A $144,132,444 $163,518,265 13 22 of 23 $165,517

Note a: This column refers to the number of times DISER met the reporting requirements on AusTender, including the original contract and variations that exceeded the $10,000 threshold. Note b: On 28 January 2022, DISER reported a contract amendment on AusTender to update the details of the contract (with no change in contract value) but there was no corresponding approved contract variation in the departmental records available to the ANAO for examination. Source: ANAO analysis of departmental records.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

56

Is there an appropriate contract management framework in place?

DISER’s contract management framework for the program is inadequate. In particular, contract management plans have not been developed for any of the contracts and the department’s approach has not secured:

• provision of contract deliverables on time;

• provision of services to an appropriate standard before payments are made; and

• appropriate management of all identified conflicts of interest.

3.4 The delivery and payment of services and the ongoing management of the contract are important elements in achieving the objectives of the procurement.67 An appropriate contract management framework supports the effective management of procurement risks and the achievement of procurement objectives.68 Its purpose is to provide a clear and standardised approach to managing and administering contracts.

Contract administration

3.5 DISER’s contract management framework for the Entrepreneurs’ Programme comprises the delivery partner contracts, policy documents which relate to delivery partner performance, promotion and marketing, and business intelligence, and standard operating procedure documents.

3.6 The Australian Government Contract Management Guide requires contract management plans for complex and strategic contracts.69 DISER’s procurement toolkit contains a contract management template for complex and/or high value contracts. Notwithstanding that the delivery partner contracts were DISER’s largest value procurement in 2019-20 and represented 37 per cent of the total value of contracts entered into by DISER and reported on AusTender for that year, the department has not developed contract management plans for the delivery partner contracts. Contract management plans are useful tools for managing risks to the success of the contract, and for ensuring that what is negotiated as value for money is achieved.70

67 Finance, Commonwealth Procurement Rules, paragraph 2.10. 68 Finance, Australian Government Contract Management Guide [Internet], Finance, available from https://www.finance.gov.au/sites/default/files/2020-12/Contract%20Management%20Guide%20December%202020%20-%20Master.pdf [accessed January 2022].

69 ibid.

70 ibid.

Are the contracts being appropriately managed to achieve the objectives of the procurement?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

57

Recommendation no. 7 3.7 The Department of Industry, Science, Energy and Resources develop contract management plans for each of the delivery partner contracts.

Department of Industry, Science, Energy and Resources response: Agreed.

3.8 Contract management plans for each of the delivery partner contracts came into effect in March 2022.

3.9 The department’s internal guidance also states that contract managers are responsible for assessing the risks identified during ‘the planning stages of [the] procurement process, including those that were captured in [the] risk management plan or contract management plan.’ The department has not maintained risk management or contract management plans for the delivery partner contracts.

3.10 The Australian Government Contract Management Guide states that ‘proper systems for identifying, gathering and recording relevant information will help you to manage your contract effectively and provide a clear audit and accountability trail.’71 The department has not had these proper systems to support appropriate management, including:

• the department does not maintain consolidated contracts reflecting all contract variations made;

• there is no clearly articulated process for acceptance of delivery partner quarterly reporting (refer to paragraphs 3.28 and 3.29); and

• an appropriate performance framework with clearly defined performance requirements, and a means to effectively measure performance against the expectations is not in place (refer to paragraphs 3.46 to 3.62).

Monitoring of delivery partner conflicts of interest

3.11 The delivery partner contracts require delivery partners to notify DISER ‘immediately’ in writing when a conflict of interest arises. The notification must make full disclosure of the conflict and the delivery partner must take such steps DISER requires to resolve or deal with the conflict. Delivery partners are required to notify DISER of all actual, potential or perceived conflicts of interest of their personnel including their employees, facilitators or contractors.

3.12 DISER has not developed systematic processes or guidance for delivery partners to comply with their contractual obligations.72 DISER advised the ANAO in February 2022 that:

delivery partners…must identify to the department any actual, perceived or potential conflicts of interest they believe will or may arise during the delivery of the program and, where a conflict of

71 ibid.

72 Guidance has been developed for commercialisation facilitators relating to declaration of conflicts of interest. In February 2022, DISER provided the ANAO with the Accelerating Commercialisation Facilitator Procedures Manual. This document contains procedures detailing how and when commercialisation facilitators should disclose conflicts of interest in relation to requests for commercialisation guidance and how declarations of interest will be managed during the grants application process. Similar manuals were not provided for other program outcomes.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

58

interest is identified, specify how that conflict will be addressed and monitored to ensure that it does not compromise program outcomes.

3.13 In contrast, the ANAO’s analysis of DISER records indicates that delivery partners have not consistently complied with their contractual obligation to ‘immediately’ notify DISER of conflicts of interest, and that DISER has failed to take appropriate action when delivery partners do not meet their contractual obligations, including when actual conflicts occur. For example, a facilitator for the strengthening business program (employed by NSWBC) submitted an expression of interest for a grant under the incubator support initiative. NSWBC did not notify the department of this potential conflict of interest in accordance with its contractual obligations, but rather this potential conflict was brought to the attention of the department by a facilitator for the incubator support program. DISER took appropriate action to eliminate the conflict of interest, with the outcome being that the facilitator withdrew the application for the grant.

3.14 On another occasion, DISER did not take appropriate action when dealing with an actual conflict of interest. A member of i4 Connect’s management team placed an offer for investment in a company receiving facilitation services from i4 Connect and applying for an accelerating commercialisation grant under the Entrepreneurs’ Programme.73 This conflict of interest was disclosed when the grant application was assessed by the Entrepreneurs’ Programme Committee in June 2021, nine months after the company commenced its engagement with i4 Connect.

3.15 The department sought legal advice which confirmed that this constituted an ‘actual conflict of interest’. Although the conflict of interest was not resolved, the program delegate endorsed the declaration of interest without requiring any management actions to mitigate the conflict of interest other than that i4 Connect be reminded of its contractual obligations in using information they may be privy to in their jobs for personal benefit. The company was awarded a $398,350 accelerating commercialisation grant (the full grant amount sought).

3.16 The department’s internal legal area has identified that conflicts of interest in the Entrepreneurs’ Programme are dealt with inconsistently and on an ad-hoc basis and that ‘proactive management of probity risk leads to better outcomes for both the department and for the people to whom the risk relates.’

73 i4 Connect is the delivery partner responsible for delivery of accelerating commercialisation services.

Are the contracts being appropriately managed to achieve the objectives of the procurement?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

59

Recommendation no. 8 3.17 The Department of Industry, Science, Energy and Resources adopt a proactive approach to managing delivery partner conflicts of interest by:

• ensuring conflict of interest declarations are completed, updated regularly or their ongoing currency otherwise confirmed; and

• regular review of delivery partner reporting of conflict of interest in order to monitor compliance with contractual obligations.

Department of Industry, Science, Energy and Resources response: Agreed.

3.18 Conflict of interest requirements are included in the delivery partner contract management plans and the department will regularly review reporting and compliance, as per contract conditions. Probity and conflict of interest training will be rolled out for all Programme staff. More broadly, our department-wide Probity Framework will include policies, processes and tools to manage conflict of interest across departmental activities.

Payment of delivery partners

3.19 The delivery partner contracts are ‘fee for services’ arrangements where DISER pays six of the seven delivery partners quarterly for services delivered. One delivery partner is paid monthly for services delivered. The ANAO’s analysis of DISER records indicates that total cumulative expenditure for the delivery partner contracts between 1 July 2020 and 31 October 2021 is $70.8 million, comprising of:

• $62.5 million of regular service charge payments;

• $8 million of service establishment payments paid to six of seven delivery partners upon engagement of 50 per cent of the required specified personnel prior to services commencement74; and

• $278,216 of pre-service payments paid to all delivery partners on completion of the induction for specified personnel prior to services commencement.

3.20 DISER has not established robust service verification procedures for the delivery partner contracts. Internal departmental guidance on making payments states that on receipt of an invoice, the receiving official must ‘obtain assurance that goods and services have been received.’ Service verifications were not recorded consistently, with 15 per cent of invoices analysed by the ANAO not having evidence of services verification, resulting in $11.4 million being paid without evidence that services had been delivered. While on the remaining occasions verifications have been recorded, it is also not clear how services are verified prior to payment of invoices (refer to paragraphs 3.28 and 3.29 for more information).

74 One delivery partner —i4 connect — received the service establishment payment in two instalments. The delivery partner received $500,000 upon contract signature and $2,726,325 upon engagement of 50 per cent of its specified personnel. The five other delivery partners received their total service establishment payment upon engagement of 50 per cent of their specified personnel.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

60

Contract deliverables

3.21 Table 3.2 sets out the contractual deliverables and the milestone dates that the delivery partners are required to meet.

Table 3.2: Deliverables

Deliverable Milestone date

Engagement of all required specified personnel to deliver program services 1 July 2020.

Quarterly management report 30 days after the conclusion of each quarter.

Draft work and communications plan for the 2020-21 financial year Within four weeks from the services commencement date of 1 July 2020.

Draft work and communications plan The draft must be delivered to the department at least six weeks prior to the start of the new financial year to which the annual report refers.

Source: ANAO analysis of delivery partner contracts.

Specified Personnel

3.22 DISER required engagement of all specified personnel by the services commencement date of 1 July 2020.75 The ANAO’s analysis of DISER records indicates that five of seven delivery partners did not engage the required number of specified personnel by this date. While this requirement was not met, there were no deductions made from the first quarterly payments. In March 2022, DISER advised the ANAO that it ‘exercised discretion when processing first quarterly payments. The number of facilitators were considered sufficient to support service delivery’. In its advice, the department noted that the only option available in the contract to vary payments was ‘to suspend payment due to the contractor not complying with its obligations under the contract’.

Work and communications plan

3.23 The draft work and communications plan is developed by the delivery partner and details how the delivery partner will deliver the services, including activities, targets and qualitative measures. Once the department accepts the work and communications plan the delivery partner must perform the services in accordance with the plan. Table 3.3 outlines delivery partner compliance with work and communications plan contractual deadlines.

Table 3.3: Delivery partner compliance with work and communications plan deadline

Delivery partner 2020-21 2021-22

Ai Group 94 days late 163 days late

Business SA 99 days late 132 days late

CSIRO 113 days late 247 days late

Darwin Innovation Hub 104 days late 184 days late

Deloitte 94 days late 124 days late

75 Specified personnel include the delivery partners’ subcontractors and facilitators employed responsible for delivery of program services.

Are the contracts being appropriately managed to achieve the objectives of the procurement?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

61

Delivery partner 2020-21 2021-22

i4 Connect 122 days late 205 days late

NSWBC 31 days late 86 days late

Source: ANAO analysis of delivery partner work and communications plans.

3.24 Table 3.3 illustrates that no delivery partner has complied with the contractual deadline for this deliverable, and that performance deteriorated further in 2021-22 compared with 2020-21. For example, in financial year 2020-21 the average delay for submission across all delivery partners was 94 days (more than three months) and up to four months late, and in financial year 2021-22 the average delay was 163 days (more than five months) and up to eight months late. This situation reinforces the risks to service outcomes where contract managers do not, through their management of the contract, reinforce the importance of meeting requirements from the commencement of the provision of services, as identified in a recent ANAO audit report.76

3.25 As stated in paragraph 3.23, once the work plan is accepted by the department, the delivery partner must perform the services in accordance with the plan. The delay means that the services for the financial year have not been agreed at the beginning of the financial year. This is indicative of the lack of an appropriate contract management framework being employed by the department.

Quarterly management report

3.26 The quarterly management report contains updates on the delivery partner’s year-to-date performance against the work and communications plan. The report must also provide information on current and emerging trends in the industry sectors and jurisdictions covered by the delivery partner. Table 3.4 sets out the timeliness of delivery partner quarterly reporting and the relationship between quarterly invoice payment and receipt of the quarterly report.

Table 3.4: Timeliness of delivery partner reporting

Delivery partner Proportion of

quarterly reports submitted late (per cent)

Average number of days late Proportion of invoices verified before receipt

of quarterly report (per cent)

Ai Group 0 N/A 60

Business SA 40 11 60

CSIRO 80 11 100

Darwin Innovation Hub 100 18 100

Deloitte 0 N/A 20

i4 Connect 80 29 N/A

NSWBC 60 2 80

Total 51 15 60

Note a: DISER pays i4 Connect on a monthly basis (refer to paragraph 3.19), which is not dependent on the submission of the quarterly management report or any other deliverable. Source: ANAO analysis of DISER quarterly reports and invoices.

76 Auditor-General Report No.19 2020-21 Management of the Search and Rescue Aircraft Contract, p.10.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

62

3.27 Table 3.4 indicates that there are varying levels of compliance with quarterly report submission timeliness. Fifty-one per cent of the quarterly reports submitted were outside of the contractually required deadline with an average delay of 15 days. Two delivery partners — Ai Group and Deloitte — have submitted all quarterly reports within the contractual deadline. In contrast, Darwin Innovation Hub has not submitted any quarterly reports within the contractual deadline, with an average delay of 18 days.

3.28 The delivery partner contracts state that payment of invoices is subject to the department’s acceptance of the quarterly management report.77 Table 3.4 highlights that the department has not applied the terms of the contract, with 60 per cent of invoices recorded as having been verified and processed for payment without submission of the quarterly reports by the delivery partners.

3.29 The absence of a clearly documented acceptance process of this key deliverable is another significant flaw in the department’s administration of the delivery partner contracts. DISER has not retained appropriate records to demonstrate that this acceptance occurs on a quarterly basis for all delivery partners. In February 2022, DISER advised the ANAO that ‘acceptance of the reports will be provided via email and/or discussed as part of regular…meetings with delivery partners. In response to an ANAO request for a copy of emails to delivery partners confirming formal acceptance of quarterly reports in accordance with the contract, the department provided in May 2022 two emails sent internally within the department but not to the delivery partners.

Contract variations

3.30 When contract variations amount to a significant change to the contract or significantly vary the scope of the contract, consideration should be given to whether78:

• other potential suppliers may have responded differently to the amended contract scope in the tendering process which may have produced a different value for money outcome; or

• the variation may compromise the original procurement’s value for money assessment.

3.31 As at 18 February 2021, there has been a total of 19 variations to the delivery partner contracts since the original contracts were signed in March 2020. All delivery partner contracts have increased in value, except for the CSIRO contract due to the removal of the incubator support services (see paragraphs 3.41 to 3.42 below). The overall cost of the contracts has increased by $19.4 million (13 per cent).

3.32 The contract variations have varied the scope of the services to be provided, including:

• the introduction of two new program elements, ‘strengthening business’ and ‘strengthening northern Australia business’; and

• the removal of a program element, ‘incubator support’, aligned to the ‘innovation’ outcome.

77 This includes the i4 connect contract, even though the contracted payment schedule is on a monthly basis. 78 Finance, Australian Government Contract Management Guide [Internet], Finance, available from https://www.finance.gov.au/sites/default/files/2020-12/Contract%20Management%20Guide%20December%202020%20-%20Master.pdf [accessed January 2022].

Are the contracts being appropriately managed to achieve the objectives of the procurement?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

63

New program elements

3.33 Since the contracts were signed, DISER has introduced two new program elements to the Entrepreneurs’ Programme, ‘strengthening business’ and ‘strengthening northern Australia business’, with facilitation services to be provided by existing delivery partners.

3.34 Strengthening business was announced as part of the Australian Government’s bushfire recovery package for small business on 11 March 2020 and included 21 new facilitators to be engaged by existing delivery partners at an expected cost of $12.8 million.79

3.35 Eight of the 19 (42 per cent) contract variations relate to the introduction of the strengthening business program element.

3.36 On 6 May 2021, the Australian Government announced the Northern Australia Development Program (NADP).80 ‘Strengthening northern Australia business’ is implementing advisory services in support of the NADP. In January and February 2022, DISER varied the contracts

with Deloitte, Ai Group and Darwin Innovation Hub to include an additional 11 new facilitators at an expected cost of $5.5 million for the strengthening northern Australia program element.

3.37 The new program elements were not part of the original procurement and did not form part of the original value for money considerations that lead to the decision to engage the delivery partners.

3.38 The delivery partners, Ai Group, NSWBC and Business SA, were selected to engage the facilitators for strengthening business on the basis that they provided ‘growth’ services in the bushfire affected regions of New South Wales, South Australia and Victoria. The contract with Deloitte was varied in September 2020, to provide three ‘digital solutions specialists’ at a cost of $1.8 million for the strengthening business program element.

3.39 Deloitte, Ai Group and Darwin Innovation Hub were selected to deliver strengthening northern Australia business services due to their geographic coverage in providing growth services, track record on business advisory services and expertise. The request for delegate approval also recorded ‘urgency’ as the justification for the decision to approach the existing delivery partners as a separate standalone procurement would have added months to the timeframe, which would reduce the effectiveness of this approach. This is notwithstanding that Government approval for the program was provided in May 2021 and the contract variations were not signed until late January 2022 and early February 2022 (eight months after Government approval).

3.40 Reflecting the general nature of the contracts, and their lack of performance measures, service levels and targets (see paragraphs 3.51 to 3.56), the contract variations did not include any specific requirements for the strengthening business or strengthening northern Australia business services. The variations to the contracts were to increase the number of specified personnel (facilitators) and the contract value.

79 Michaelia Cash (Minister for Employment, Skills, Small and Family Business) ‘Red tape cut, access easier for bushfire affected small business’, media release, 11 March 2020. Available from https://www.minister.industry.gov.au/ministers/michaeliacash/media-releases/red-tape-cut-access-easier-bushfire-affected-small-businesses. 80 Keith Pitt (Minister for Resources and Water) ‘Investing in new opportunities for northern Australia |

Ministers for the Department of Industry, Science, Energy and Resources’, media release, 6 May 2021. Available from https://www.minister.industry.gov.au/ministers/pitt/media-releases/investing-new-opportunities-northern-australia.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

64

Incubator support services

3.41 As discussed in paragraph 2.60, CSIRO was awarded the contract to deliver innovation and incubator support services. Incubator support services was included as a program element in the RFT but discontinued in May 2021 as part of a savings measure announced in the 2021-22 federal budget. The savings achieved by early closure of the program was approximately $9.8 million over the forward estimates.

3.42 To partially achieve this savings measure, the CSIRO contract was varied in June 2021 to remove the two facilitators associated with delivery of the incubator support services. Removal of these two facilitator positions reduced the total contract value from $17.3m to $16.2m.

Approvals

3.43 Section 23 of the Public Governance, Performance and Accountability Act 2013 states that accountable authorities of a department may enter into, vary or administer a contract. DISER’s accountable authority instructions (AAIs) state that the delegate must not vary an arrangement unless it is within the scope of their delegation. DISER has not retained a copy of the necessary section 23 approvals for 10 of the 19 contract variations signed to date.

Recommendation no. 9 3.44 The Department of Industry, Science, Energy and Resources strengthen its management of the Entrepreneur’s Programme delivery contracts, including by taking prompt action in circumstances where delivery deadlines are not met and verifying that services have been provided before payments are made.

Department of Industry, Science, Energy and Resources response: Agreed.

3.45 The contract management plans set out clear expectations for contract deliverables, and we have updated our standard operating procedure to mandate approval and verification of services prior to payment.

Do the contracts set clear performance requirements, and a means to effectively measure performance against the requirements?

The delivery partner contracts do not include an effective performance management framework. The contracts do not contain:

• specific service levels that each delivery partner must achieve or exceed;

• any performance measures and related targets to assess delivery partner performance; and

• a means to adjust payment based on the performance of delivery partners.

3.46 Performance management is key to delivering value for money and involves ensuring that goods or services are delivered as required under the contract.81 To assess whether the contracts

81 Finance, Australian Government Contract Management Guide [Internet], Finance, available from https://www.finance.gov.au/sites/default/files/2020-12/Contract%20Management%20Guide%20December%202020%20-%20Master.pdf [accessed January 2022].

Are the contracts being appropriately managed to achieve the objectives of the procurement?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

65

set clear performance requirements, and a means to effectively measure performance against the requirements, the ANAO examined:

• the request documentation, including the draft contract;

• the signed delivery partner contracts; and

• the relevant policy documents referred to in the contracts.

3.47 The draft contract developed and released as part of the Request for Tender (RFT) on 27 September 2019, advised that the performance management regime had not been fully settled, and contained an indicative performance management regime which stated that:

quantitative and qualitative measures and targets to assess performance will be developed by the department in consultation with prospective delivery partners during the contract negotiation stage.

Procurement of consultant to develop a performance management framework

3.48 DISER engaged consultants, Bevington Group, to produce a performance framework to support contract negotiations with successful tenderers at a cost of $56,760. The procurement did not comply with the CPRs and did not transparently demonstrate value for money.

• DISER identified the consultant through a ‘Consultancy and Business Services’ panel, but that panel had expired, and instead employed a limited tender procurement method.

• In justifying the limited tender procurement approach, the department cited a ‘short timeframe’ as its justification. While ‘reasons of extreme urgency brought about by events unforeseen82’ is a condition for limited tender, the ANAO’s analysis is that the short timeframes were not brought about by unforeseeable events given the RFT had foreshadowed that the performance framework would need to be developed.

• DISER did not estimate the expected value of the procurement before deciding on the procurement method.

• Evidence of the approval to approach the market or the original request for quote was not retained.

• Quotes were not requested from multiple providers.

• DISER did not consider the costs in its value for money justification, and only focused on the short timeframe and Bevington’s prior program knowledge.

3.49 In December 2019, the department accepted delivery of the new performance framework from Bevington. This was despite departmental officers recording that they were not satisfied with the work completed by Bevington. The framework was changed by the department before contract negotiations commenced to align it more closely with the outcomes of the redesign work. In March 2022, DISER advised the ANAO that:

Bevington was engaged as an expert adviser to assist the department to consider performance frameworks and outcome assessment. The work Bevington completed informed the final performance framework and a range of ways the program could consider measuring activities, efficiency and outcomes.

82 Finance, Commonwealth Procurement Rules, paragraph 10.3b.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

66

DISER proactively contract managed the engagement with Bevington. There were some delays with delivery of key activities under the contract due to Bevington’s availability. However, these deliverables were rectified and managed to the department’s satisfaction.

Contracted service requirements

3.50 The delivery partner contracts require three types of services to be provided:

• business facilitation and advisory services by specified personal (facilitators);

• promotion and marketing to promote the Entrepreneurs’ Programme; and

• provision of market and business intelligence to the department.

3.51 The contracts set out the general requirements in relation to the provision of those services for all delivery partners. While ‘service level’ is a defined term in the contract83, the contracts do not contain specific service levels that each delivery partner must achieve or exceed.

3.52 The contracts refer to external supporting documents that provide further guidance on performance expectations for each of the three services, including:

• performance and capability policy;

• communications and marketing guidelines; and

• market and business intelligence framework.

Performance management

3.53 The Australian Government Contract Management Guide states that performance management involves:

• performance measurement: collecting accurate and timely data of supplier performance;

• performance assessment: deciding on whether the supplier’s performance meets the standards contained in the contract; and

• performance adjustment: understanding what constitutes good performance, correcting areas of underperformance, amending the contract requirements to meet changing needs.84

3.54 The delivery partner contracts state that

performance will be assessed and managed by the department using a combination of quality, volume and outcome measures to evaluate each delivery partner’s contribution to the overall performance of the program and its objectives.

3.55 The contracts do not contain any performance measures to assess delivery partner performance.

3.56 The contracts state that the process for performance management will be detailed in the performance and capability policy. The policy contains a mix of qualitative and quantitative

83 Service level is defined as meaning ‘the level of performance which the Contractor must achieve or exceed when providing the Services as specified in item 5 of Schedule 2’. 84 Finance, Australian Government Contract Management Guide [Internet], Finance, available from https://www.finance.gov.au/sites/default/files/2020-

12/Contract%20Management%20Guide%20December%202020%20-%20Master.pdf [accessed January 2022].

Are the contracts being appropriately managed to achieve the objectives of the procurement?

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

67

measures and ‘expected behaviours’ the delivery partners will be assessed against. No targets are set in the policy against the performance measures and no process is set out to assess performance.

3.57 As outlined in paragraph 3.23, delivery partners must perform services in accordance with the work and communications plan, which should include ‘targets and qualitative measures.’ The ANAO’s analysis of the 2020-21 work and communications plans identified a clear disconnect between the performance and capability policy and the work and communications plans. None of the delivery partner work and communications plans include targets for all measures contained in the performance and capability policy.85

3.58 While there is a lack of an effective performance management framework in place, DISER does maintain oversight of delivery partner and facilitator performance through portfolio management tool dashboards.86 The portfolio management tool is intended to provide an overview of:

• services and how they have progressed;

• collaboration activity such as referrals and uptake of specialist services

• client demographics such as ‘full-time equivalent’, ‘revenue’, ‘export’ and ‘research and development’ activity;

• outcomes being achieved in actions implemented, their impact and business performance; and

• implementation activity — where client capability is developed.

3.59 The department holds quarterly reviews of dashboards with delivery partners.

Performance adjustment

3.60 There are benefits from linking contract payments to the achievement of key performance indicators.87 The draft contract included a provision by which DISER could reduce quarterly payments for failure to meet contracted service levels. The signed contracts do not contain this provision or a similar provision which provides for adjustment of payment in relation to service levels. None of the successful tenderers had requested that this clause be removed. Rather, its removal was initiated by DISER. In February 2022, DISER advised the ANAO that the clause

was removed from the draft contract (released as part of the RFT). This was replaced with the Performance and Capability Policy approach that was referenced in the final contracts issued to all Delivery Partners.

3.61 The performance and capability policy does not contain a provision that allows for adjustment of payments. The only contractual rights available to the department to manage underperformance are rectification notices, withholding of payment until underperformance is addressed, reducing the scope of services or rescinding the contract.

3.62 Without targets or any basis for comparison, the department is unable to assess whether delivery partner performance is satisfactory and is also unable to demonstrate that the contracts

85 The only example of where targets were agreed was Deloitte’s first annual work and communications plan which contained two ‘parameters for success’ that relate to the measures in the policy. These parameters were not reported against in the quarterly reports. 86 An extract of these dashboards is contained in Appendix 9. 87 Auditor-General Report No.19 2020-21 Management of the Search and Rescue Aircraft Contract, p.10.

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

68

are delivering against the expectations that informed the decision to select the successful tenderers.

Recommendation no. 10 3.63 When contracting for the delivery of services, the Department of Industry, Science, Energy and Resources apply competitive pressure when establishing performance expectations, and how these will relate to contractual payments. This can be achieved by the department setting out its expected performance management system and standards in the approach to the market documentation, or by asking respondents to set out their proposals to inform a comparative evaluation.

Department of Industry, Science, Energy and Resources response: Agreed.

Gran

t Hehir

Auditor-General

Canberra ACT 24 June 2022

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

69

Appendices

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

70

Appendix 1 Entity response

Appendix 1

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

71

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

72

Appendix 1

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

73

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

74

ANAO comments on Mills Oakley response

(a) Paragraphs 2.74 and 2.75 sets out the work undertaken by Mills Oakley in addition to probity advice. (b) The ANAO's finding, set out in paragraph 2.76, was that DISER’s decision to source probity and procurement process advice from the same adviser adversely affected the

independence of the probity advice. (c) In finalising the report, while the ANAO considered Mills Oakley's suggested editorial amendments to the proposed report, no amendments were made to paragraphs 2.75 and

2.76 as the ANAO concluded that the wording accurately reflects the contemporaneous evidence of the conduct of the procurement of delivery partners for the Entrepreneurs' Programme.

Appendix 1

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

75

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

76

ANAO comments on DMV Consulting response

(d) The DMV Consulting report stated in its 'Pricing Evaluation Approach' that 'The Pricing Assessment process involved an analysis of all information provided by each Tenderer in Attachment 10 of the submitted Tender documents. A review was also completed of their overall tender submission to gain an understanding of the relationship between the prices quoted and their overall approach to the delivery of services, particularly in regard to programme management and travel assumptions.' The Darwin Innovation Hub tender response set out that the tenderer would be providing 5 Full Time Equivalent (FTE) personnel through 11 individuals with a table in the tender response identifying those personnel and the contribution each person would make to the overall 5 FTE resources being tendered. (e) Refer to paragraph 2.32 in the audit report in relation to the impact of the pricing

evaluation error. (f) The audit approach reflects that the audit was of the Department of Industry, Science, Energy and Resources pursuant to the Auditor-General Act 1997 (the Act) and that, while section 18B of the Act enables the Auditor-General to audit the performance of

Commonwealth partners such as DMV Consulting, this audit did not employ that power. Consistent with the ANAO’s usual practice as well as principles of transparency and accountability, the ANAO report has named the consultants employed by the department in conducting this procurement. As required by the Commonwealth Procurement Rules (CPRs), the department has also publicly reported the names of the consultants it engaged on AusTender.

Appendix 1

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

77

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

78

Appendix 1

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

79

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

80

Audi

tor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

81

Appendix 2 Improvements observed by the ANAO

1. The existence of independent external audit, and the accompanying potential for scrutiny improves performance. Improvements in administrative and management practices usually occur: in anticipation of ANAO audit activity; during an audit engagement; as interim findings are made; and/or after the audit has been completed and formal findings are communicated.

2. The Joint Committee of Public Accounts and Audit (JCPAA) has encouraged the ANAO to consider ways in which the ANAO could capture and describe some of these impacts. The ANAO’s 2021-22 Corporate Plan states that the ANAO’ s annual performance statements will provide a narrative that will consider, amongst other matters, analysis of key improvements made by entities during a performance audit process based on information included in tabled performance audit reports.

3. Performance audits involve close engagement between the ANAO and the audited entity as well as other stakeholders involved in the program or activity being audited. Throughout the audit engagement, the ANAO outlines to the entity the preliminary audit findings, conclusions and potential audit recommendations. This ensures that final recommendations are appropriately targeted and encourages entities to take early remedial action on any identified matters during the course of an audit. Remedial actions entities may take during the audit include:

• strengthening governance arrangements;

• introducing or revising policies, strategies, guidelines or administrative processes; and

• initiating reviews or investigations.

4. During the course of the audit, the ANAO did not observe any change in the Department of Industry, Science, Energy and Resources’ approach to the procurement of delivery partners for the Entrepreneurs’ Programme.

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

82

Appendix 3 Evaluation criteria in the RFT

Evaluation criteria and description Sub-criteria Weighting

Organisational capability: the overall organisational ability of the tenderer to deliver and support the delivery of requirements including meeting contractual obligations and having the financial capacity to deliver the requirements.

General:

• organisational experience delivering expert business advisory and facilitation services;

• ability to provide a minimum of five full-time equivalent specified personnel of suitable quality and experience to deliver program services;

• understanding of and ability to deliver expert advisory and facilitation services (as relevant);

• ability to deliver program services appropriate to the individual needs of businesses;

• ability to implement quality assurance systems and processes for consistent delivery of high-quality services;

• ability to deliver program services in a way that is flexible, adaptable, and scalable;

• ability to appropriately manage service delivery timeframes as specified by the department;

• ability to respond to significant increases or decreases in the level of services or changes to the services resulting from changes to Commonwealth policy or the number of eligible businesses entering the program;

• ability to work with other delivery partners to form a state and/or national network;

• ability to connect with and facilitate referrals of eligible businesses to other relevant Government programs including but not limited to Industry Growth Centres;

• have a service delivery model to comply with the delivery principle to ensure continuity of service to a business as outlined in item 2 of Schedule 1- Statement of Requirement;

• ability to offer other ad hoc project services to further support the achievement of the Programme’s objectives; and

• ability to instil confidence for businesses entering the Programme, some of whom may be seeking expert business advisory assistance for the first time, including those from culturally and linguistically diverse, or Indigenous backgrounds.

70%

Project management:

• ability to ensure quality assurance and performance management plans are in place and achievable;

• ability to manage workflow including effective resource allocation, workload balancing and ensuring appropriate specialist skills are deployed in response to customer needs within required timeframes;

• ability to ensure ongoing professional development of tenderer’s specified personnel;

Appendix 3

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

83

Evaluation criteria and description Sub-criteria Weighting

• ability to manage resourcing (equipment and facilities including ICT);

• ability to have adequate ICT security protocols in place to ensure customer information is stored securely and only accessed or used for the purpose of delivering program services;

• ability to manage any delays in providing the services;

• ability to comply with the program’s administrative requirements;

• ability to transition in and transition out of the program; and

• ability to appropriately manage work health and safety issues.

Specified personnel:

• extent of specified personnel’s firsthand business experience and knowledge of current issues affecting eligible businesses;

• suitability of specified personnel’s skills to fulfil the roles of facilitators as outlined in the RFT;

• tenderer’s ability to replace specified personnel if positions become vacant during the course of the contract or additional specified personnel are required due to increased demand for program services; and

• diversity of specified personnel.

Note: If the tenderer does not currently have all specified personnel, please outline the methodology that will be applied to attract and appoint suitably skilled and qualified specified personnel.

Promotion and marketing and industry knowledge: the overall ability of the tenderer to:

• promote the program to ensure adequate take up and demand for program services; and

• provide the department with high level market and business intelligence to support continuous improvement.

Promotion and Marketing:

• ability to promote program services to eligible businesses through marketing, communications and business development activities; and

• ability to promote program services to eligible businesses to ensure adequate demand for and take-up of program services is maintained.

30%

Industry Knowledge:

• industry knowledge, interaction and linkages with eligible businesses;

• connection with and understanding of what is occurring across the business landscape for SMEs, including start-ups; and

• ability to provide the department with market and business intelligence relating to the program’s eligible businesses, including any value-added services to support the department’s achievement of program objectives.

Price: pricing proposed by the tenderer to deliver

• pricing offered including all costs, fees, allowances and charges associated with the implementation and completion of obligations under the Draft Form Contract;

Not

weighted

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

84

Evaluation criteria and description Sub-criteria Weighting

the program services and deliverables. • pricing structure; and • proposed payment schedules and life cycle costs.

Risk: any risks inherent in the tender.

Any risks inherent in the tender which may include (but is not limited to):

• management of any actual, perceived or potential conflicts of interest;

• level of compliance with this RFT (including the Draft Form of Contract);

• adequacy of the insurance proposed by the tenderer;

• past performance of contractual obligations by the tenderer. For the purposes of this sub-criterion, tenderer also encompasses any related body corporate, proposed subcontractor or their related body corporate, or special purpose vehicle (in which any of these entities have been involved); and

• the nature and health of the tenderer’s or proposed subcontractors’ previous contractual relationships with the Commonwealth and behaviour.

Not

weighted

Corporate and financial viability. • proposed corporate structure and management structure, including senior management;

• tenderer’s business size; locations and duration of operation;

• details of any litigation or any other relevant issues that may affect the tenderer’s performance;

• financial reference checks;

• the proposed corporate structure of the tenderer and proposed subcontractors, including for any related bodies corporate; and

• the financial and corporate viability of the tenderer and proposed subcontractors to fulfil the obligations set out in the Draft Form of Contract.

Not

weighted

Commonwealth policies: the tenderer’s ability to sufficiently conform and provide the services in accordance with relevant Commonwealth procurement connected policies.

Indigenous Procurement Policy:

In evaluating tenders, the department will take into consideration the tenderer’s proposed approach to:

• using Indigenous enterprises in its supply chain; and

• the employment of Indigenous Australians.

Not

weighted

Requirement of economic benefit for Australia:

• In evaluating tenders, the department will take into consideration the tenderer’s proposed approach to providing benefits to the Australian economy including in the areas of employment, environmental sustainability and innovation.

Source: DISER Request for Tender documentation.

Appendix 4 Errors identified by the ANAO in the department’s tender evaluation report

Page no.

Statement Description of/reason for error

Part 4.5 Step 2 — Technical Evaluation

21 A ‘Stage 2 Technical Assessment Form (TAF)’ template was developed by the tender support team in conjunction with the probity adviser for use by the TAP members to uniformly document evaluation comments and scores.

TAP members did not uniformly document evaluation comments and scores. The template was used inconsistently. It was adapted by one member and used in a different manner to the other two members. For example, scoring was recorded differently.

22 Initial individual technical evaluations were conducted by the tender assessment panel (TAP) in accordance with the procedures outlined in Part B: Evaluation Process, Step 2: Technical Evaluation (part 14) of the tender evaluation plan (TEP).

The tender evaluation plan required each team member to identify and record any risk identified through the evaluation process. Risks were not identified and recorded.

22 After commencement of the technical evaluation and as a result of the large number of responses received, the TAP chair determined (in consultation with the probity adviser) that a division of the remaining responses for the technical evaluation was required to expedite the evaluation process.

As a result of its market analysis, the department expected strong interest which would provide a good range of competitive responses. The 2015 Entrepreneurs’ Programme industry partners tender received more than 50 responses.

22 The TAP met with the probity adviser on 12 December 2019 and remaining responses were randomly divided amongst the three panel members.

The meeting was not documented.

22 The TAP agreed that at least two ‘Stage 2’ Technical Assessments would be completed across each response. Accordingly, each response was reviewed by two of the three technical assessors.

Not all responses were assessed by at least two TAP members. Of the 53 stage 2 technical assessments:

• 4 had three assessments;

• 26 had two assessments; and

• 23 had one assessment.

22 TAP individual technical assessments were then moderated by the tender support team. For those tender responses where two or more assessments were conducted, the scores were averaged to create a ‘combined score’ out of 5. The process

conducted is not properly called ‘moderation’.

The moderation section on each of the individual technical assessment forms was left blank for all assessments.

22 After completion of the technical evaluation, a final technical score was agreed and the following tenderers were evaluated as technically unsuitable and not progressed for further evaluation…

There is no evidence of an ‘agreed’ score having occurred by a process of moderation. Rather, where multiple assessments were conducted, the scores were averaged to create a score out of 5.

The list of tenders excluded at this point included:

• 6 scored as 3 (Acceptable);

• 8 scored as 2 (Poor); and

• 15 scored as 1 (“Unsuitable”).

The appropriate description in the tender evaluation plan’s rating scale for a score of 1 was ‘unsatisfactory’.

23 The TAP agreed the above list of tenderers were not competitive in accordance with the evaluation criteria. The stage 2 technical assessments were focused only on the two weighted criteria and scores were applied to each of the two criteria. The process for

applying the weightings to each of the two criteria and determining the overall score was not recorded.

There was not a reasonable basis to exclude the listed tenderers as not all criteria had been evaluated against. For example, as there was no consideration of the tenderers’ price, it was not appropriate to determine that the tenderers were ‘not competitive’ in accordance with the evaluation criteria.

The RFT did not represent to the market that tender prices would not be considered prior to shortlisting and that the two weighted criteria would be applied as a shortlisting threshold.

Part 4.6 Outcomes from Stage 2 Technical Assessments — Shortlisting

24 The TAP met on 17 December 2019 to review the shortlisted Tenderers. Individual scores and comments from the stage 2 technical assessments were discussed. The TAP agreed that a combined score of 5 or below (a rating of 2 and 3 from individual TAP members) was regarded as an uncompetitive response and accordingly these were determined as not suitable.

While the initial shortlist was compiled using the ‘moderated’ (that is, averaged) scores to determine technical suitability, the TAP subsequently decided to combine the individual ratings and apply a new threshold of 5, below which a tender response was regarded as ‘uncompetitive’.

As above, this threshold was not represented to the market as a basis for shortlisting before all criteria would be applied to the evaluation of tenders.

25 Tenderers 2 and 29 both scored 6 (a rating of 3 and 3 from individual TAP members). When compared to other submissions received for the commercialisation and innovation outcome, the responses demonstrated a lack of cohesiveness and understanding of the requirements. Therefore, they were also deemed uncompetitive and did not proceed.

The TAP ignored the previously applied threshold for these particular tender responses and excluded them from further evaluation on the basis of other reasons not considered during the evaluation against the technical criteria.

25 Tenderers 38 and 45 each received a score of 6 (a rating of 3 and 3 from individual TAP members) for their response to the growth outcome. After further discussion, the TAP agreed the responses did not indicate they fully understood the types of businesses the program works with and the TAP were not convinced they would be able to sufficiently meet reporting requirements, or provide the level of business intelligence the program will expect under new arrangements from 1 July 2020.

The TAP ignored the previously applied threshold for these particular tender responses and excluded them from further evaluation on the basis of other reasons not considered during the evaluation against the technical criteria.

Tenderer 38 also submitted a response for the commercialisation outcome. The report does not include the reasons it was not shortlisted for this outcome.

25 The process applied throughout this stage was conducted with support from the independent probity adviser (Mills Oakley). The approach ensures the department can have a high level of confidence that the stage 2 technical assessment process, ranking, shortlisting and internal expert assessment of tenderer responses for the RFT was conducted fairly and equitably in accordance with all probity requirements.

The probity adviser was not sufficiently independent from the process (refer to paragraphs 2.71 to 2.76).

Part 4.8 Stage 3 Pricing and Financial Capability Evaluation

26 DMV Consulting completed a review of the pricing, corporate and financial information of the 14 shortlisted Tenderers. DMV Consulting completed an initial review of pricing for the stage 3 assessment of the 14 shortlisted tender responses. The review of corporate

and financial viability occurred later as part of the value for money assessment and only included 10 of the shortlisted tenderers.

26 This was the most effective, efficient and economical use of accountable agency resources in the circumstances and the approach was approved by the probity adviser.

In its assessment of quotes received for the engagement of the financial analyst services, the department had already assumed it would be shortlisting approximately 15 tenderers for pricing analysis. The department had not advised the market it would be shortlisting tender responses before an evaluation against the price criterion.

27 Tenderers were separated according to the outcome/s and ranked by price (best to least best). The table listed Darwin Innovation Hub as the first ranked tenderer for the growth outcome. This was based on an error made in the financial analysis

which incorrectly calculated Darwin Innovation Hub’s average price per facilitator as $132,873 instead of $292,320. Darwin Innovation Hub should have been the seventh ranked tender.

Part 4.9 Stage 5 — value for money assessment — initial assessment and confirmation of the shortlisting

27 The TAP met on 3 February 2020 to conduct the first stage of the value for money assessment. There was no record of this meeting that documented how the first stage of the assessment was conducted.

27 Based on the value for money assessments and interview response, the shortlist of 14 tenderers was reduced to 11. Tenderers 24, 30 and 54 were excluded from the process and regarded as both Uncompetitive and Not recommended.

Only 10 tenderers received an evaluation against the corporate and financial viability criterion.

It is unclear which criteria were used to determine that the tender responses were uncompetitive.

Part 4.12 Contacting Referees

30 Referee checks were conducted for all of the shortlisted tenderers. Referee checks were conducted for a reduced shortlist of 11 tenderers.

Part 4.13 Step 4 — Contract Compliance

30 Identified instances of draft contract non-compliance and requests for confidentiality for shortlisted tenderers were recorded and actioned in accordance with the procedures outlined in part 19 of the tender evaluation plan.

Assessment of the level of compliance with the draft form of contract issued with the RFT was conducted only on the seven preferred (and eventually, successful) tenderers, not on all shortlisted tenderers.

Part 4.14 Step 5 — Completion of the value for money assessment

31 The value for money assessment was conducted in accordance with the procedures outlined in part 21 stage 5: value for money assessment of the tender evaluation plan.

The value for money assessment was not conducted in accordance with the procedures outlined in the tender evaluation plan because:

• the TAP did not ‘rank the responses on the basis of its assessment of the technical merit of each tender response, the associated risks and the costs of the solution for each response’ as stated in the tender evaluation plan;

• instead, the TAP prepared a list of shortlisted tenderers which it had assessed as either:

− not recommended and uncompetitive;

− not recommended and competitive; and

− recommended and competitive; and

• the TAP did not assess risks identified and recorded throughout the evaluation process and did not complete an overall risk assessment for each tender response in accordance with the risk matrices attached to the tender evaluation plan.

Source: ANAO analysis of departmental records.

Audi

tor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

89

Appendix 5 Probity plan requirements not fully implemented

Paragraph Probity plan requirement Were the requirements implemented?

5.1.4 The procurement manager must maintain a conflict of interest register that records all conflicts of interests in relation to the procurement process.

A conflict-of-interest register was not maintained.

7.1.2 The procurement manager must ensure that, as soon as practicable, a person who is a key person, throughout the procurement process receives a copy of the probity protocols and completes a conflict-of-interest declaration or deed poll.

No list of key persons was maintained by the procurement manager and therefore key persons were not identified.

Fourteen key personnel signed the required conflict of interest declarations (including 11 Australian Public Service (APS) personnel and three external contractors). Other key members of the procurement team did not sign conflict of interest declarations.

7.1.3 Key persons and other persons authorised pursuant to clause 7.1.1 to access controlled information must cease access to controlled information unless, within 10 business days of receiving that access, the person either:

• has satisfied the requirements of clause 7.1.2; or

• has been authorised under, and has complied with, any applicable conditions place on that person pursuant to clause 9.1.2

Of the 11 APS personnel who signed conflict of interest declarations six were signed within 10 business days of receiving access to controlled information. Other personnel who did not sign conflict of interest declarations did not have their access to the controlled information removed.

Other personnel were granted access to controlled information without ever signing conflict of interest declarations.

7.1.4 The procurement manager must maintain a list of all key persons and all persons authorised, pursuant to clause 7.1, to access controlled information.

A list of key persons with access to controlled information was not maintained by the procurement manager.

7.2.1 The procurement manager must cause the maintenance of a probity register that contains:

• the original of each executed conflict of interest declaration or deed poll;

• a log of occurrences concerned with the probity of the procurement process;

• a log of records relating to the management of conflicts of interest during the procurement, including any complaints notified to the probity adviser in accordance with this plan; and

• a record of each communication with a potential tenderer that is required to be recorded under the probity protocols.

The procurement manager maintained a probity register, which contained some information about the probity of the procurement process.

The probity register:

• did not contain the original of each executed conflict of interest declaration or deed poll. Two out of fourteen signed declarations were noted in the probity register;

• was not a complete log of occurrences concerned with the probity of the procurement process;

• did not contain a log with all records relating to the management of conflicts of interest; and

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

90

Paragraph Probity plan requirement Were the requirements implemented?

• did not record all communications with potential tenderers in accordance with the probity protocols.

8.1.1 The procurement manager must take reasonable steps to ensure, in respect of the procurement process, that:

• key persons maintain proper records or deliberations and decisions relating to the procurement process, including reasons for any decisions on substantial issues;

• a filing system is maintained consistent with relevant departmental instructions, and that the file contains all controlled information, as well as any information about the procurement process made generally available to the public, media or potential tenderers;

• all documents relating to the procurement process are appropriately classified in accordance with departmental practices and directions, including marking documents ‘commercial in confidence’ where necessary; and

• departmental requirements for the secure storage of information are met.

This requirement was partly met.

• Proper records were partly kept and decisions relating to the procurement process were partly documented. A tender evaluation report was prepared for the delegate with a number of attachments. No formal records of tender assessment panel (TAP) meetings exist, with TAP deliberations largely contained in email records.

• An electronic filing system was maintained in accordance with departmental instructions.

• All documents relating to the procurement process were appropriately classified.

9.1.2 The procurement manager must take reasonable steps to ensure that:

• key persons are aware of, and comply with, their obligations under this plan;

• persons authorised to access controlled information comply with any conditions on that authorisation;

• the members of the assessment panel receive a probity briefing related to their obligations in the assessment process;

• key persons comply with requirements of the department’s accountable authority instructions and other policies that are relevant to the procurement process or procurement processes generally.

Reasonable steps were partly taken by the procurement manager in relation to these requirements.

• No list of key persons was maintained. The procurement manager provided the probity protocols to some members of the procurement team.

• No conditions were placed on access to controlled information.

• Most members of the TAP received a probity briefing relating to their obligations in the assessment process. Three ‘internal experts’ did not receive the briefing.

• Key persons who returned conflict of interest declarations acknowledged their awareness of controlled information and confidentiality obligations. There is no evidence to suggest that reasonable steps were taken to ensure that key persons complied with the department’s accountable authority instructions.

Appendix 5

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

91

Paragraph Probity plan requirement Were the requirements implemented?

10.1.2 Any queries from potential Tenderers about the procurement process must be referred to the procurement manager, who will consult with the probity adviser.

Not all queries from potential tenderers about the procurement process were referred to the procurement manager.

Source: ANAO analysis of DISER records.

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

92

Appendix 6 Tenderer complaints about the procurement

Tenderer Nature of complaint Extent to which the submission was

evaluated

Respondent 39 The respondent was concerned that the procurement process favoured larger companies and did not treat Small and Medium Enterprises (SMEs) fairly and equitably as required by the CPRs.

• A complete evaluation was not conducted by the TAP. Only two of the three TAP members conducted any assessment. Those assessments only addressed two of the six criteria.

• The submission was rated ‘technically unsuitable’ and ‘uncompetitive’ against the first two criteria and was not shortlisted for further consideration.

• The submission was not evaluated against the price, risk, corporate and financial viability and Commonwealth policies criteria.

Respondent 41 The respondent was concerned that:

• tender prices were not properly considered in the evaluation given the price offered by the respondent was substantially less than the value of the contract awarded to the successful tenderer;

• the information requirements for the tender submission were substantial, requiring significant time and costs to prepare, and the feedback provided by the department did not provide confidence that the evaluation was an impartial assessment of the merits of the submission as a whole; and

• the process of awarding the contract lacked probity, was not fair or impartial and did not deliver value for money for taxpayers.

• A complete evaluation was not conducted by the TAP. Only two of the three TAP members conducted any assessment. Those assessments only addressed two of the six criteria.

• The submission was rated ‘as technically suitable’ against the first two criteria.

• After further consideration by the TAP, the submission was not shortlisted for the growth outcome.

• No further consideration in relation to the response for the commercialisation outcome was recorded by the TAP.

• The submission was not shortlisted for further consideration.

• the submission was not evaluated against the price, risk, corporate and financial viability and Commonwealth policies criteria.

Respondent 51 The respondent was concerned that the procurement did not meet the stated objectives.

• A complete evaluation was not conducted by the TAP. Only two of the three TAP members conducted any assessment. Those assessments only addressed two of the six criteria.

• The submission was rated ‘technically unsuitable’ and ‘uncompetitive’ against the first two criteria and was not shortlisted for further consideration.

• The submission was not evaluated against the price, risk, corporate and financial

Appendix 6

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

93

Tenderer Nature of complaint Extent to which the submission was

evaluated

viability and Commonwealth policies criteria.

Respondent 53 Three individual members of the consortium submitted separate complaints. The concerns raised were that:

• the assessment process lacked rigour and transparency; and

• value for money was not demonstrated given the price offered by the respondent was substantially less than the value of the contract awarded to the successful tenderer.

• Assessments against the first two weighted criteria were conducted by two TAP members.

• The submission was rated ‘competitive’ against the first two criteria and was shortlisted for further consideration.

• The submission was ranked first for the commercialisation outcome but was not recommended as the preferred tenderer. The respondent's tender price was:

− $341,340 per facilitator less than that of the preferred tenderer; and

− $13,761,500 in total less than that of the preferred tenderer.

• The department's assessment of the price risk as ‘medium-high’ compared to the successful tenderer’s ‘low’ rating was a significant factor in it not being recommended as a preferred tenderer.

Respondent 12 The respondent was concerned that the awarded contracts did not represent value for money, particularly in relation to the growth outcome in Western Australia.

• Assessments against the first two weighted criteria were conducted by two TAP members.

• The submission was rated ‘competitive’ against the first two criteria and was shortlisted for further consideration.

• In the comparative analysis of ‘price; in Western Australia conducted as part of the value for money assessment, the submission was ranked third of the four that were compared. The respondent’s total tender price was:

− $510,014 more than that of the first ranking tenderer; and

− $465,551 less than that of the preferred tenderer.

• The tender submission was rated as ‘not recommended/competitive’.

• The analysis presented in the tender evaluation report indicated that the proposed delivery model could have an impact on ‘scalability and increasing demand for services’ but this is at odds with:

− the TAP providing a score of 5 against the ‘scalability’ evaluation subcriterion; and

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

94

Tenderer Nature of complaint Extent to which the submission was

evaluated

− the tender submissions proposal to provide 14 facilitators in Western Australia in contrast to the successful tender’s contracted number of nine facilitators.

Source: ANAO analysis of DISER records and submissions made to the ANAO.

Audi

tor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

95

Appendix 7 Non-compliances with the Commonwealth Procurement Rules

Commonwealth Procurement Rule Description of non-compliance

7.12 Relevant entities should include relevant evaluation criteria in request documentation to enable the proper identification, assessment and comparison of submissions on a fair, common and appropriately transparent basis.

7.15 The initial approach to market for a multi-stage procurement must include, for every stage, the criteria that will be used to select potential suppliers, and if applicable, any limitation on the number of potential suppliers that will be invited to make submissions.

10.6 Request documentation must include a complete description of:

a) the procurement, including the nature, scope and the quantity of the goods and services to be procured or, where the quantity is not known, the estimated quantity, and any requirements to be fulfilled, including any technical specifications, conformity certification, plans, drawings, or instructional materials;

b) any conditions for participation, including any financial guarantees, information and documents that potential suppliers are required to submit;

c) any minimum content and format requirements;

d) evaluation criteria to be considered in assessing submissions and, if applicable to the evaluation, the relative importance of those criteria;

e) any dates for the delivery of goods or supply of services, taking into account the complexity of the procurement; and

f) any other terms or conditions relevant to the evaluation of submissions.

• While six relevant evaluation criteria were included in the RFT, the RFT did not disclose to tender participants that the department would employ a staged procurement approach involving the first two criteria being employed as the basis for shortlisting, such that most tenders were not evaluated against the remaining four criteria (including the price criterion) (see paragraphs 2.2 to 2.12).

• The RFT did not identify the relative importance of the criteria to be used in the evaluation. It stated that the criteria were not in any order of importance (see paragraphs 2.3 and 2.10).

• Evaluation criteria applied by DISER were not included in the request documentation for the engagement of the probity adviser (see paragraph 2.72).

5.2 Participation in procurement imposes costs on relevant entities and potential suppliers. Those costs should be considered when designing a process that is commensurate with the scale, scope and risk of the proposed procurement.

• The RFT did not outline to potential tenderers that shortlisting would occur after evaluation against the first two criteria and that there would be a limit on the number of tender submissions that would proceed for further evaluation against the remaining criteria.

• The 53 compliant tender responses invested resources in addressing all six of the published evaluation criteria whereas 39 of those tenders (74 per cent) were shortlisted out without their response to four of the criteria being assessed. Six of those 39 tenders were assessed against only one of the six criteria.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

96

Commonwealth Procurement Rule Description of non-compliance

• The approach to shortlisting was important information for tenderers to know to inform their decision about whether to participate in the tender process (see paragraph 2.11).

9.2 The expected value of a procurement must be estimated before a decision on the procurement method is made. The expected value is the maximum value (including GST) of the proposed contract, including options, extensions, renewals or other mechanisms that may be executed over the life of the contract.

• DISER did not estimate the expected value of the procurements for DMV Consulting (see paragraph 2.19) and Bevington Group (see paragraph 3.48) before making a decision on the procurement method.

10.34 Further consideration must be given only to submissions that meet minimum content and format requirements.

• Eleven tender submissions were progressed to stage 2 evaluations before additional information required to meet the minimum content and format requirements was received by the department (see paragraph 2.22).

4.7 In addition to the value for money considerations at paragraphs 4.4-4.6, for procurements above $4 million (or $7.5 million for construction services) (except procurements covered by Appendix A and procurements from standing offers), officials are required to consider the economic benefit of the procurement to the Australian economy.

• While evaluation against the Commonwealth policies criterion allowed for this CPR requirement to be met:

− the TAP did not document an assessment against this evaluation criterion, including how economic benefit was considered, as part of the overall value for money assessment for any of the tender responses; and

− consideration of the criterion was restricted to a compliance check at stage 2 that tender submissions included the information requested in the RFT (see paragraph 2.41).

6.6 In particular, officials undertaking procurement must act ethically throughout the procurement. Ethical behaviour includes:

a) recognising and dealing with actual, potential and perceived conflicts of interest;

b) dealing with potential suppliers, tenderers and suppliers equitably, including by

i. seeking appropriate internal or external advice when probity issues arise, and

ii. not accepting inappropriate gifts or hospitality;

c) carefully considering the use of public resources; and

d) complying with all directions, including relevant entity requirements, in relation to gifts or hospitality, the Australian Privacy Principles of the Privacy Act 1988 and the security provisions of the Crimes Act 1914.

• The procurements were not conducted in accordance with the guidance provided by the CPRs about how to conduct procurements ethically:

− the department involved the probity adviser it engaged in procurement process activities, an approach that adversely affects the independence of the probity advice (see paragraphs 2.74 to 2.76);

− while a probity adviser was engaged, the procurement of the probity adviser was deficient (see paragraphs 2.71 to 2.73);

− while a probity plan was documented, key elements were not implemented (see paragraphs 2.79 to 2.81 and Appendix 5); and

− incumbency advantages were not transparently managed (see paragraphs 2.87 to 2.97).

• See Appendix 8 for more detailed information.

Appendix 7

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

97

Commonwealth Procurement Rule Description of non-compliance

5.4 All potential suppliers to government must, subject to these CPRs, be treated equitably based on their commercial, legal, technical and financial abilities and not be discriminated against due to their size, degree of foreign affiliation or ownership, location, or the origin of their goods and services.

• DISER did not transparently manage incumbency advantages, in particular:

− there were no specific contractual or payment arrangements in place to govern the department’s engagement of five of the ten existing industry partners to contribute to the redesign and payment for this work (see paragraph 2.91);

− the department did not implement in full the probity risk management measures recommended by its probity adviser (see paragraphs 2.92 and 2.93); and

− departmental records indicate that information about the redesign of the program was revealed to incumbent tenderers involved in the redesign work providing them with a competitive advantage (see paragraphs 2.94 to 2.97).

• The ANAO also observed inconsistent treatment of tenderers throughout the procurement process, such as:

− providing only 14 tenderers of the 30 tender submissions rated as ‘acceptable’ or above to the ‘internal panel experts’ for further technical evaluation (paragraph 2.29);

− providing opportunities for certain tenderers to improve or revise tender submissions (see paragraph 2.67);

− meeting with one tenderer (i4 Connect) to discuss its response to the department’s request for additional information (see paragraphs 2.75 and 2.117); and

− tailoring its approach to a particular tenderer (see paragraph 2.67).

7.2 Officials must maintain for each procurement a level of documentation commensurate with the scale, scope and risk of the procurement.

7.3 Documentation should provide accurate and concise information on:

a) the requirement for the procurement;

b) the process that was followed;

c) how value for money was considered and achieved;

d) relevant approvals; and

e) relevant decisions and the basis of those decisions.

• While appropriate procurement records were partly maintained:

− evidence to support key decisions was not maintained (paragraph 2.102); and

− the tender evaluation report did not accurately reflect the evaluation process that was employed or satisfactorily demonstrate that value for money had been achieved (see Appendix 4 for more detail).

4.4 Achieving value for money is the core rule of the CPRs. Officials responsible for a procurement • DISER’s approach to shortlisting tender submissions following the stage 2 technical

Audi

tor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

98

Commonwealth Procurement Rule Description of non-compliance

must be satisfied, after reasonable enquires, that the procurement achieves a value for money outcome. Procurements should:

a) encourage competition and be non-discriminatory;

b) use public resources in an efficient, effective, economical and ethical manner that is not inconsistent with the policies of the Commonwealth;

c) facilitate accountable and transparent decision making;

d) encourage appropriate engagement with risk; and

e) be commensurate with the scale and scope of the business requirement.

evaluation against the first two criteria did not provide for open and effective competition to be maintained. Only 14 of the 30 tenders assessed as ‘acceptable’ or better against those two criteria were shortlisted (see paragraph 2.106).

• The consequences of DISER’s shortlisting approach were that (see paragraph 2.109):

− three submissions (27 per cent) from non-incumbent providers for growth (three) and innovation (zero) were shortlisted;

− of the three non-incumbents shortlisted for growth, two were for services in Northern Territory and one was for services in Western Australia;

− in New South Wales/Australian Capital Territory, South Australia, Tasmania, Queensland, and Victoria there was no competition from non-incumbents; and

− after further shortlisting there was one non-incumbent — Darwin Innovation Hub — remaining for the evaluation against the ‘corporate and financial viability’ criterion and for the final value for money assessment.

5.5 To ensure that Small and Medium Enterprises (SMEs) can engage in fair competition for Australian Government business, official should apply procurement practices that do not unfairly discriminate against SMEs and provide appropriate opportunities for SMEs to compete. Officials should consider, in the context of value for money:

a) the benefits of doing business with competitive SMEs when specifying requirements and evaluating value for money;

b) barriers to entry, such as costly preparation of submissions, that may prevent SMEs from competing;

c) SMEs’ capabilities and their commitment to local or regional markets; and

d) the potential benefits of having a larger, more competitive supplier base.

• The department’s procurement approach may have disadvantaged SMEs by not giving appropriate consideration to Commonwealth policies relating to SMEs. While 81 per cent of tenders submitted were from SMEs, 20 per cent (9 of 44) of those were shortlisted. Of the 16 tenderers assessed as ‘acceptable’ or better against the two criteria used for shortlisting that were not shortlisted, 15 were from SMEs (94 per cent) (see paragraph 2.111).

7.17 Following the rejection of a submission or the award of a contract, officials must promptly inform affected tenderers of the decision. Debriefings must be made available, on request, to unsuccessful tenderers outlining the reasons the submission was unsuccessful. Debriefings must also be made available, on request, to the successful supplier(s).

• The department did not promptly inform tenderers of the rejection of their submissions. All unsuccessful tenderers were advised on 30 March 2020, including the two tenderers which were assessed as not meeting the minimum content and format requirements on 11 December 2019 and the 39 tenderers which

Appendix 7

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

99

Commonwealth Procurement Rule Description of non-compliance

were not shortlisted on 17 December 2019 (see paragraph 2.114); and

• The department also provided incorrect and misleading advice in writing to two unsuccessful tenderers which did not reflect the evaluation process that was conducted (see paragraphs 2.115 and 2.116). Contributions to the ANAO from other tenderers similarly indicate that they have not been informed by the department that they were not assessed against each of the published evaluation criteria.

10.3 A relevant entity must only conduct a procurement at or above the relevant procurement threshold through limited tender in the following circumstances:

b) when, for reasons of extreme urgency brought about by events unforeseen by the relevant entity, the goods and services could not be obtained in time under open tender.

• In justifying the limited tender procurement approach to the procurement of Bevington Group, the department cited a ‘short timeframe’ as its justification. While ‘reasons of extreme urgency brought about by events unforeseen’ is a condition for limited tender, the ANAO’s analysis is that the short timeframes were not brought about by unforeseeable events given the RFT had foreshadowed that the performance framework would need to be developed (see paragraph 3.48).

Source: ANAO analysis of DISER records.

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

100

Appendix 8 Non-compliance with ethical requirements

1. The Australian Parliament has established requirements in the Public Governance, Performance and Accountability Act 2013 (PGPA Act) including to require the Commonwealth and Commonwealth entities to use and manage public resources properly (section 5). The accountable authority for an entity responsible for relevant money has a duty under section 15 of the PGPA Act to promote the proper use of the money for which the accountable authority is responsible. ‘Proper’, when used in relation to the use or management of public resources, means efficient, effective, economical and ethical (section 8).

2. The Department of Finance PGPA Glossary defines ethical as:

the extent to which the proposed use of public resources is consistent with the core beliefs and values of society. Where a person behaves in an ethical manner it could be expected that a person in a similar situation would undertake a similar course of action. For the approval of proposed commitments of relevant money, an ethical use of resources involves managing conflicts of interests, and approving the commitment based on the facts without being influenced by personal bias. Ethical considerations must be balanced with whether the use will also be efficient, effective and economical.

3. The Australian Parliament has also established, through the Public Service Act 1999 (PS Act), the APS Values set out in section 10. Subsection 10(2) states that ‘The APS demonstrates leadership, is trustworthy, and acts with integrity, in all that it does’. The APS Commissioner has made directions under the PS Act including in subsection 16(f) requiring accountability of APS members by ‘being able to demonstrate clearly that resources have been used efficiently, effectively, economically and ethically’. A mandatory code of conduct is set out in section 13 of the PS Act for APS employees.

4. PGPA Act requirements, including ethical requirements, directly inform key public sector resource management frameworks for specific Australian public sector activities addressed through performance audits. These frameworks contain ethical requirements specific to the activity they regulate. For government procurement, ethical expectations are set out in paragraphs 6.1-6.8 of the Commonwealth Procurement Rules (CPRs), made by the Minister for Finance under the PGPA Act. For the ANAO, in conducting performance audits of procurement activities in entities subject to the PS Act, compliance with the CPRs is assessed against the background of the requirements of the PS Act.

5. In conducting performance audits of entities, the ANAO obtains evidence to inform an assessment of whether the audited entity executes its activities in accordance with the requirement to promote proper use of public money. Findings may be made as to whether the use or management of public money was efficient, effective, economical and ethical. In forming an overall conclusion in a performance audit, the ANAO may also form a view on whether the entity’s activities have been executed in accordance with both compliance with the Rules framework and the intent of that framework, including the requirements of the PS Act for the APS (the entity) to act with integrity in all that it does.

6. Where ANAO findings or a conclusion are made as to whether the use or management of public resources by the entity has been ethical, it is a matter for an accountable authority to assess whether the audit findings in the particular case reflect the broader posture of the entity or relate to individual APS staff conduct.

Appendix 8

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

101

Ethical guidance in the CPRs Non-compliance identified

Officials undertaking a procurement must act ethically throughout the procurement. Ethical behaviour includes recognising and dealing with actual, potential and perceived conflicts of interest.

Not all members of the procurement team signed conflict of interest declarations. Declarations were not provided by:

• the procurement delegate;

• internal legal and procurement advisers who formed part of the specialist advice and support team;

• administrative support personnel; and

• officials providing advice on the footprint from the proposed delivery partner network during the value for money evaluation phase (refer to paragraph 2.83).

Not all conflicts of interest were identified by personnel who supplied conflict of interest declarations. During the course of ANAO audit work it was identified that one of the 'internal expert' assessors was previously employed by one of the industry partners for approximately nine years. While the 'internal expert' completed a conflict-of-interest declaration, nil conflicts were identified (contrary to the probity protocols for the procurement process) (refer to paragraph 2.85).

DISER did not take appropriate steps to manage identified conflicts of interest from the procurement team. No records were kept documenting steps taken to manage declared interests by four members of the procurement team, and identified interests were not included in the evaluation report to the procurement delegate (refer to paragraph 2.86).

Clear records of identification and management of conflicts of interests were not maintained. For example:

• conflict of interest declarations were not recorded in the probity register, and no separate conflict of interest register was maintained; and

• not all conflict of interest declarations were stored within the department's records management system. Six (of fourteen) declarations were added to the department's records management system during the course of the ANAO audit. The audit team located one additional conflict of interest declaration in emailed records.

Officials undertaking a procurement must act ethically throughout the procurement. Ethical behaviour includes dealing with potential suppliers, tenderers and suppliers equitably (CPRs paragraph 6.6(b)).

DISER did not uphold the standards of ethical behaviour set out in the CPRs relating to equitable treatment of tenderers in relation to:

• its management of incumbent tenderers (refer to paragraphs 2.87 to 2.97); and

• the evaluation process for potential suppliers (refer to paragraphs 2.21 to 2.67).

Management of incumbency risks

Incumbency advantages were not transparently managed. Prior to commencing the RFT, DISER undertook a redesign project with the stated aim to improve the operation of the program,

including the delivery mechanism and the structure of the program. DISER engaged five of the ten existing industry partners to assist with the redesign to draw on their ‘deep knowledge of customer needs/program experience.’ In response to a request from the ANAO for advice on the contractual arrangements for engaging five

Auditor-General Report No. 42 2021-22 Procurement of Delivery Partners for the Entrepreneurs’ Programme

102

of the incumbent providers to assist with the redesign, and how much they were paid for this assistance, in March 2022 DISER advised the ANAO that: 'DISER did not enter into new contracts with providers for the redesign project. DISER accessed its existing arrangements with the supplier and paid for their services under those arrangements.' (refer to paragraph 2.91)

Key probity controls designed to manage incumbency risk for the redesign project were not implemented or were eroded during the redesign process. For example, the department’s probity adviser recommended including provisions in the confidentiality deed polls prohibiting industry partner personnel involved in the redesign project from assisting in the preparation of the tender response. DISER removed these provisions, despite the heightened risk identified by its probity adviser. Removal of these provisions occurred at the request of industry partners, who raised concerns that these provisions would unfairly disadvantage their tender preparation (refer to Table 2.4 and paragraph 2.93) and the department did not address that their removal would benefit them at the expense of non-incumbent tenderers.

ANAO examination of DISER records indicates that advice was provided to incumbent providers involved in the redesign project that effectively meant that the work they had performed had been completed and a delivery model decided, rather than simply concept development work to present options for the department to consider and make a decision later on (refer to paragraph 2.95). The breach, the nature of the probity issue and how it was handled (including an ‘urgent’ meeting with the probity adviser) was not transparently documented in the probity register (refer to paragraph 2.96).

The redesign project was recommenced in December 2019, including consultation with one of the incumbent providers (Deloitte), three months before the delivery partners were selected (refer to paragraph 2.97).

Evaluation of tender submissions

The ANAO observed inconsistent treatment of tenderers through the tender evaluation process.

The TAP did not evaluate tender submissions in accordance with the published evaluation criteria and methodology outlined in the RFT (refer to paragraph 2.25 and Figure 2.1).

One tender response (Darwin Innovation Hub) was identified as ‘clearly non-competitive’ and that ‘bid repair’ was required to address non-compliance with the minimum content and format requirements. The TAP later decided to shortlist Darwin Innovation Hub’s tender submission after a panel member reassessed the original rating and the remaining members agreed to shortlist (refer to paragraphs 2.23 to 2.24). This tenderer later benefited from an error in the financial analysis conducted for the department that led to it being ranked more highly than it should have and was, as a result, awarded a contract (paragraph 2.32).

Only 14 of the 30 tender submissions rated as ‘acceptable’ were provided to internal panel experts for further technical evaluation (refer to paragraph 2.29).

The department did not allow sufficient time for all TAP members to conduct individual scoring of each tender submission, followed by consensus scoring. As a result of this approach, not all tenderers were assessed by all members of the TAP. Only four (eight per

Appendix 8

Auditor-General Report No. 42 2021-22

Procurement of Delivery Partners for the Entrepreneurs’ Programme

103

cent) tender responses received an evaluation from all three TAP members; 26 (49 per cent) tender responses received an evaluation from two TAP members; and 23 (42 per cent) tender responses received an evaluation from only one TAP member (refer to paragraph 2.43).

Opportunities were provided for select tenderers to improve or revise their tender submissions (refer to paragraph 2.67).

The TAP and probity adviser met with one tenderer to discuss its response to the department’s request for additional information; this opportunity was not provided to another tenderer that had been asked for additional information (and which was not awarded a contract) (refer to paragraphs 2.75 and 2.117).

Officials undertaking a procurement must act ethically throughout the procurement (CPRs paragraph 6.6).

After receiving the department’s notification of the outcome, two unsuccessful tenderers raised concerns with the department regarding its handling of ‘competitive neutrality’ in the procurement given that the successful tenderer, CSIRO, is a Commonwealth entity. DISER’s advice to the tenderers was misleading and did not reflect the evaluation process that was conducted (refer to paragraph 2.115 to 2.116). Contributions to the ANAO from other tenderers similarly indicate that they have not been informed by the department that they were not assessed against each of the published evaluation criteria.

Source: ANAO analysis of DISER records.

Appendix 9 Portfolio management tool dashboard extract

1. An example of a dashboard contained in the portfolio management tool is set out in Figure A.1.

Source: Delivery partner annual work plan.