Note: Where available, the PDF/Word icon below is provided to view the complete and fully formatted document
Audit Act - Auditor-General - Audit reports - 1994-95 - No. 5 - Follow-up audits - Department of Employment, Education and Training: New Enterprise Incentive Scheme (NEIS): protective security: AUSTUDY


Download PDF Download PDF

THE AUDITOR-GENERAL

Audit Report No.5 1994-95

Follow-up Audits

Department of Employment, Education and Training

New Enterprise Incentive Scheme (NEIS) - Protective Security - AUSTUDY

llBSiii*! mmm ιιιιιιι k

: I gjS wpile

. . . : : ; : **«

* ;■ : | ■ f ■$25|g| s ^ $;||

. . . . ■

M fck m . 5 ‘ · ■ ; r - - la — M B ■ ■ . . :: - ■ .

A u s t r a l i a n N a t i o n a l A u d i t O f f i c e

A u s t r a l i a n N a t i o n a l A u d i t O f f i c e

The Auditor-General

Audit Report No.5 1994-95

Follow-up Audits

Department of Employment, Education and Training

- New Enterprise Incentive Scheme (NEIS) - Protective Security - AUSTUDY

A u s t r a l i a n G o v e r n m e n t P u b l i s h i n g S e r v i c e , C a n b e r r a

®Commonwealth of Australia 1994

ISSN 1036-7632

ISBN 0 644 35100 4

This work is copyright. Apart from any use as permitted under the Copyright Act 1968, no part may be reproduced by any process without prior written permission from the Australian Government Publishing Service. Requests and inquiries concerning reproduction and rights should be addressed to the Manager, Commonwealth Information Services, Australian Government Publishing Service, GPO Box 84, Canberra ACT 2601.

Produced by the Australian Government Publishing Service

Australian National Audit Office Canberra ACT 28 October 1994

Dear Mr President Dear Mr Speaker

In accordance with the authority contained in the Audit Act 1901, the ANAO has undertaken follow-up efficiency performance audits of New Enterprise Incentive Scheme, Protective Security and AUSTUDY in the Department of Employment, Education and Training and I present this report and the accompanying brochure to

Parliament.

The Honourable the President of the Senate The Honourable the Speaker of the House of Representatives

Parliament House Canberra ACT

Yours sincerely

W.G. Nelson

Acting Auditor-General

The Auditor-G eneral is head of the Australian National Audit Office. He and the A NAO are

independent o f the Gover nment. The ANAO assists the Auditor-General to carry out his duties

u n d e r t h e A u d i t A c t to u n d e r t a k e a u d i t s o f p e r f o r m a n c e and f i n a n c i a l s t a t e m e n t s o f

Com m onwealth public sector bodies and to provide independent reports and advice for the

Parliament, the Governm ent and the community. The aim is to improve Com m onwealth public

sector administration and accountability.

A u d ito r-G en eral's reports are available from Commonwea lth G o v ern m en t Booksho ps. Recent

lilies are shown at the back of this report. For further information please contact:

The Reports and P ublication s Officer

A u s t r a l i a n N a t i o n a l A u d i t O f f i c e

G P O B o x 7 0 7

C a n b e r r a A C T 2 6 0 1

p h o n e ( 0 6 ) 2 0 3 7 5 3 7 , f a x ( 0 6 ) 2 0 3 7 7 7 7

A U D IT IN G F O R A U S T RA L IA

iv

Contents

Report Summary vii

PART ONE - NEIS 1

Key Points 3

Chapter 1 - Introduction 4

Chapter 2 - Audit Findings 5

PART TWO - Protective Security 13

Key Points 15

Chapter 1 - Introduction 17

Chapter 2 - Audit Findings 18

PART THREE - AUSTUDY 29

Key Points 31

Chapter 1 - Introduction 33

Chapter 2 - Debt Recoveries 35

Chapter 3 - Turnaround Times 50

Chapter 4 - Post Payment Verification 59

Appendix 1 65

Appendix 2 66

Series Titles 67

Report Summary

Audit focus

1. The AN AO has a policy to undertake follow-up audits of previously tabled reports. The aim of the follow-up audits is to review progress on the implementation of the recommendations resulting from previous audits. The scope and focus of the follow-up audits is generally limited

to an examination of the issues contained in the recommendations.

2. This report contains results for the follow-up audits listed below:

• Part 1 - the New Enterprise Incentive Scheme (NEIS)

• Part 2 - Protective Security, and

• Part 3 - three elements of the administration of the AUSTUDY Program, being debt recoveries, turnaround times and post payment verification.

3. The administration of the above programs is the responsibility of the Department of Employment, Education and Training (DEBT).

4. Field work for these follow-up audits was undertaken in Canberra between May and September 1994. The follow-up audits were carried out in accordance with ANAO Auditing Standards at a total cost of $61 200.

Key findings

5. Findings for each follow-up audit are detailed at the beginning of each Part of this report, however, a summary of the main findings are presented below:

• the Department generally had taken action to address the ANAO recommendations but in most cases there had been lengthy delays in implementing this action, particularly in finalising guidelines and operational instructions (for example, NEIS Guidelines and key recommendations for protective security)

• the ANAO considers that it is important that DEBT regularly evaluate the effectiveness of various initiatives aimed at improving program performance (for example, post payment verification strategies and debt recovery strategies such as mobile review teams and direct deposit arrangements), and

vii

• the AN AO notes DEET’s efforts in setting performance standards (e.g. for debt recovery and turnaround times) but stresses the importance of having a relatively stable set of measures which enable interested external parties to assess the Department’s performance over a number of reporting periods. DEBT acknowledged the desirability of keeping standards constant to allow performance evaluation.

Summary of Recommendations

The ANAO has made recommendations in relation to the elements of the AUSTUDY program as follows:

• that DEBT should regularly assess the cost effectiveness of its debt collection and post payment verification strategies, and

• that, regarding debt recovery action, the Department make a concerted effort with the Department of Social Security (DSS) to finalise arrangements aimed at improving the transfer and reporting of withholdings from DSS benefits.

Summary of DEET’s response

6. The Department was supportive of the comments made by the ANAO in each of the follow-up audits.

viii

PART ONE

New Enterprise Incentive Scheme (NEIS)

M. Golightly A. Thurley R. Hearne

1

Key Points

DEBT has made substantial progress on the implementation of all nineteen recommendations of Audit Report No.39 1991-92. However, many recommendations relied on the development of guidelines and the ANAO was concerned that these guidelines had remained in draft form

for eighteen months.

The guidelines are important as they cover operational requirements for all aspects of the program (e.g. contract administration, requirements for program delivery etc.). During the process of finalising the follow-up audit the guidelines were approved by the Secretary.

The Department was generally supportive of the ANAO’s comments.

3

1 . Introduction

Background

1.1 Audit Report No.39 1991-92 included a review of the New Enterprise Incentive Scheme (NEIS). The audit focused on the need for more rigorous planning, control and evaluation of the operational effectiveness of the program. The audit made nineteen

recommendations, a number of which were implemented during the course of the audit. The recommendations related to program effectiveness, managing agents, monitoring of NEIS participants, training, budgets, NEIS participants’ contracts and business returns and the management information system.

Audit objective and scope

1.2 The objective of the follow-up audit is to review progress on the implementation of the nineteen ANAO recommendations. The scope and focus of the follow-up audit was limited to an examination of the issues contained in the recommendations.

1.3 The ANAO interviewed a number of DEBT officers and reviewed Departmental files and documentation in DEET’s National Office. No interstate fieldwork and no examination· of area office documentation was undertaken.

Changes to the program

1.4 The scheme was initially delivered by the Commonwealth in conjunction with State Governments. Most NEIS places are now delivered by managing agents in conjunction with either the Commonwealth or Commonwealth and State Governments.

1.5 A proportion of NEIS places will be targeted to Job Compact clients each year with a proportion of these places being available through the New Work Opportunities Program.

1.6 Greater emphasis has been placed on the provision of mentoring, which is offered on a fee-for-service basis.

4

2. Audit Findings

Contact with participants

2.1 Recommendation Nos 1 and 2 covered contact with participants. The aim was to improve participant contact in order to better assess program outcomes and therefore gauge its success.

2.2 The Department has taken a number of steps to improve post participation contact. Post Program Monitoring Surveys (PPMS) are conducted for most of DEET’s labour market programs. These surveys collect information on the job and training outcomes of program

participants approximately three months after their assistance ends and is an indicator of the short term effectiveness of the assistance received. NEIS participants were included in the PPMS from December 1993. The activities of managing agents, as detailed in the guidelines, have

been extended to encompass establishing and maintaining a network of current and former participants.

2.3 In accordance with the guidelines the offer of NEIS allowance also seeks agreement from participants, as a condition of receiving the allowance, that they will participate in surveys or evaluations commissioned by DEBT for up to two years after NEIS allowance ceases. Participants must agree with this and other conditions before

receiving the allowance.

ANAO comment

2.4 In principle these measures should improve post participation contact. However, the ANAO notes that the guidelines have only recently been approved after being in draft form for eighteen months. DEBT now needs to distribute these guidelines and ensure their timely implementation. The effectiveness of the guidelines should be evaluated at regular intervals.

2.5 The PPMS has not yet provided useful data for NEIS because an insufficient number of former NEIS participants have been included in the data base. DEBT has now rectified the problem by the inclusion of an automatic program end date for participants in the NEIS Management Information System (NEISMIS). This end date will be amended for

those participants who cease to receive the NEIS allowance prior to the automatically calculated end date. This will allow all NEIS participants to be included in the PPMS. However, DEBT needs to closely monitor the accuracy of the system in relation to these end dates.

5

Disadvantaged participants

2.6 Recommendation No.3 proposed that statistics for disadvantaged participants, including relevant costs, be collected separately. This would allow the separate identification of costs for disadvantaged participants and the outcomes they achieved. Given that the costs are

likely to be higher for these participants it is important that DEBT is able to monitor the associated outcomes.

2.7 DEET did not support the establishment of a separate sub-program for disadvantaged participants since NEIS is not a targeted program. Places under the scheme are being made available under the Job Compact

and DEET have indicated that 50% of NEIS places will be through the Job Compact in 1995-96.

ANAO comment

2.8 The ANAO notes that DEET has collected some data on the involvement of disadvantaged participants in the Scheme. While it is acknowledged that this is not a targeted program, an increasing number of places are being offered to the long term unemployed. The ANAO therefore encourages DEET to continue to collect data on disadvantaged participants and to separately identify the relevant costs for these participants in order to allow assessment of their relative success rate and whether disadvantaged participants require additional support. It will also assist DEET to prepare their budget estimates for the program taking into account the likelihood of additional resources required to support this category of clients.

DEET Response

2.9 The Department has made arrangements to separately identify the cost to the program of Job Compact participants and will consider identifying the costs associated with other disadvantaged groups.

Program effectiveness

2.10 Recommendation Nos 4 and 5 dealt with evaluations, including future evaluations proposed by DEET. Recommendation No. 6 proposed the collection of data on numbers approved for training, numbers approved for support assistance and numbers operating full-time businesses after ceasing support from the scheme. These measures were aimed at improving the assessment of program effectiveness.

2.11 DEET published the report of the 1989-90 evaluation in June 1992 and has subsequently conducted a further evaluation covering all aspects of the scheme including the system of using managing agents. The Report on this evaluation, Evaluation of the New Enterprise Incentive Scheme, Evaluation Management Branch (EMB) Report 10/93 was published earlier in 1994. This report included a range of data relating to program effectiveness, and particularly provided unit costs of successful participants.

6

2.12 DEBT was able to provide other data from a number of

management systems used within the Department to satisfy the requirements of Recommendation No.6.

ANAO comment

2.13 The ANAO was satisfied that data collection and analysis was being undertaken and that, using this data, DEBT was monitoring program effectiveness in terms of outcomes.

2.14 The ANAO notes in the report of the 1993 evaluation of the Scheme that a further review was recommended to be undertaken by the program area to assess the effectiveness of mentor arrangements.

2.15 Given that mentor support is a significant component of the Scheme and separate funding has been provided for mentoring over the past 12 months, the ANAO encourages DEBT to undertake an evaluation as soon as possible in order to assess whether this funding has led to an

improvement in mentor arrangements and therefore program effectiveness.

2.16 In addition, consideration should be given to conducting evaluations of all program elements on a rolling basis to enhance program effectiveness through monitoring outcomes.

DEBT Response

2.17 The Department will consider the ANAO’s suggestions in the context of its portfolio evaluation plan. Surveys of the effectiveness of mentoring by managing agents are already undertaken on a quarterly basis in Victoria. The Department will consider the extension of this

practice as part of a forthcoming investigation of Best Practice.

Managing agents

2.18 Five recommendations (Nos 7, 8, 9, 10 and 13) were concerned with improving arrangements in relation to managing agents with the aim of ensuring effective and efficient financial management of the Scheme. DEBT has developed guidelines which deal with these arrangements.

2.19 Recommendation Nos 7, 8 and 13 related to the initial definition of the budget components by managing agents and any subsequent variations. The draft guidelines viewed by the ANAO clearly specified that a fully costed proposal detailing managing agent support costs,

training provision and mentor services must be included in the tender. The guidelines also specify that managing agents generally are to be paid in arrears.

2.20 Recommendation No.9 related to various aspects of monitoring expenditure and the repayment of unexpended funds. In response DEBT advised that detailed management and monitoring of matters relating to managing agents’ budgets is now undertaken by Executive Officers in

Area Offices.

7

2.21 DEBT acknowledges that monitoring variations to budgets and repayment of unexpended funds would be less of a concern if managing agents were paid in arrears as indicated in the draft guidelines. DEET National Office had no records to indicate what proportions of managing agents were being paid in advance or in arrears. At the ANAO’s suggestion information was collected from area offices. The following Table reports that information and indicates that the majority of payments to managing agents were paid in advance.

Table 1 Payments to managing agents

Managing agents paid in: NSW Vic Qld WA SA Tas NT

(i) in arrears No Approx

one half

Yes Approx one quarter No No No

(ii) in advance Yes Approx

one half

No Approx

three quarters

Yes Yes Yes

2.22 In relation to Recommendation No. 10, the Memorandum of Agreement has been replaced by DEET with a Standard Labour Market Contract. This contract outlines the need for financial acquittals but indicated that audited statements are not required when payments are made to managing agents in arrears or on a fee-for-service basis.

2.23 No information was available within DEET National Office to indicate whether contract conditions for acquittals were being met.

ANAO comment

2.24 As previously stated the ANAO was concerned that the guidelines had been in draft form for eighteen months. In particular, this meant the condition which stated that managing agents generally be paid in arrears had not yet been implemented. The implementation of the condition would obviate a number of problems such as verifying variations to budgets and the need for repayments of unexpended funds.

2.25 Also, given that individual managing agents are paid on average $96 700, the ANAO recommended that DEET ensure the contract and guidelines specify that audits be required for payments made in advance or arrears, including on a fee-for-service basis. DEET has agreed to this suggestion.

2.26 It is further suggested that DEET National Office undertakes quarterly sample surveys to measure the degree of compliance with contract conditions.

8

DEET response

2.27 The Department advised that the guidelines had now been approved. The guidelines (and the CES Manual) now incorporate revisions to ensure clear and proper procedures are in place to monitor payments to managing agents. The guidelines require DEET to visit

managing agents to review performance (including the performance of mentors and training providers). DEET will also collect copies of evaluations of training programs conducted by managing agents.

Mentor support

2.28 Recommendation No. 11 dealt with mentor support with the aim of overcoming deficiencies and inconsistencies in the provision of such support. DEET generally expected that the provision of mentor support was likely to improve the success rate of participants.

2.29 In response, DEET has made funds available to managing agents to provide mentor support and arrangements for the level of payment (of up to $300 per participant) and extent of contact (that is on a one-to-one basis at least six times during the year) are set out in the guidelines. The

provision of mentor support is the responsibility of managing agents who may provide the service directly or recruit mentors.

ANAO comment

2.30 The measures outlined above should address the problem of inadequate mentor support identified by this recommendation. The ANAO is concerned, however, as previously stated, that the guidelines have been in draft form for eighteen months.

2.31 The ANAO has indicated in paragraphs 2.13 - 2.16 that an evaluation of mentor support arrangements be undertaken when they had been operating for a suitable period. DEET’s Report on the Evaluation, EMB 10/93, also recommended a post implementation review of new

mentor support arrangements to gauge their effectiveness. The ANAO suggests that such an evaluation be undertaken in the near future.

DEET response

2.32 The Department’s comments in relation to Program Effectiveness at paragraph 2.17 apply.

Training

2.33 Recommendation No. 12 identified the need for the development of common training course material for use by managing agents with the aim of reducing duplication of effort by managing agents and thereby reducing the cost of training provision.

9

2.34 Subsequently, DEBT had commissioned the development of core modules for the ‘Certificate in Small Business Management’. The core modules have now been accredited and their use will be a mandatory condition of managing agents contracts from 1 July 1995.

ANAO comment

2.35 The ANAO believes the development of core training modules by DEET has satisfied the requirements of this recommendation.

2.36 Given the amount of effort taken to produce the training materials the ANAO suggests that DEET, at a suitable time after the inclusion of the use of this material as a contract condition, undertake a sample check to ensure the materials are being used, as appropriate, by managing agents and training providers.

Participant contracts

2.37 Recommendation No. 14 proposed that the form of acceptance signed by NEIS participants be strengthened with the aim of ensuring that participants fully disclosed information regarding the receipt of other allowances and income.

2.38 This audit found that the Offer of NEIS Allowance clearly indicates that the offer is subject to a number of conditions, including the need for participants to advise managing agents/DEET officers of die receipt of allowances/benefits/pensions at the time of application and of any changes occurring to their income for the period of receiving NEIS allowance. Applicants have to sign an acceptance form indicating that they agree to abide by these conditions.

2.39 DEET has in place a range of procedures to ensure that

participants comply with these conditions. Advice is also provided to the Department of Social Security that participants are receiving NEIS allowance and are not therefore eligible for some other benefits.

ANAO comment

2.40 The ANAO believes action undertaken by DEET satisfies the requirements of this recommendation.

Business returns

2.41 Four recommendations (Nos 15, 16, 17 and 18) dealt with various issues in relation to the business returns questionnaire, and the collection and use of information from these returns. The aim of these

recommendations was to increase the usefulness of information provided in the returns and improve the procedures for collection and analysis of performance data.

2.42 DEET has since revised the Business Return and proposes to collect and use information for means testing purposes and to assess business viability. The revised form has taken into account the ANAO’s

10

recommendations. Managing agents are required to collect business returns from participants every thirteen weeks.

ANAO comment

2.43 While the revised Business Return appears to satisfy the requirements of this recommendation it is in draft form. Considerable time has elapsed since the recommendations were made and the ANAO strongly suggests that the fomi be finalised as a matter of urgency.

2.44 DEBT National Office, at the suggestion of the ANAO, contacted Executive Officers to determine whether business returns were being collected and whether information was being analysed and used. The following Table indicates the proportions of returns being collected.

Table 2 Estimated Compliance Rate for Business Returns per Quarter

Percentage of Business Returns provided

NSW Vic Qld WA SA Tas NT

(i) before follow-up 95% 85% 90% 80% 90% 75% 50%

(ii) after follow-up 95% 95% 100% 95% 100% 80% 100%

Are business returns on NEISMIS None 80% 90% 95% 95% 80% N/A

2.45 While there appears to be a good rate of compliance in providing returns (ANAO was unable to verify within the scope of the audit) there are no indications that the data is being used for monitoring purposes.

2.46 The ANAO notes that returns are being entered on NEISMIS, except for NSW, but does not believe this is a sufficient indication that information is being used to effectively monitor the scheme. The ANAO suggests that as part of the sample survey previously proposed, DEET collect information to gauge whether such monitoring is occurring. This

will be particularly important when the revised Business Return is in use because at that time information from the Returns will be used for means testing and business viability assessment.

2.47 DEET has indicated that NSW gives the entry of data into NEISMIS a low priority because of resource constraints. It is of concern that NSW does not enter information on NEISMIS since it would make the production and analysis of national data impossible. The ANAO

recommends this situation be rectified as soon as possible.

11

DEBT response

2.48 The Department advised that the Business Return form is currently being finalised to be consistent with the approved guidelines in which the requirements for monitoring of returns has been strengthened. The entry of NSW data from the quarterly Business Return will followed-up with the Area Manager of NSW.

NEIS Management Information System

2.49 Recommendation No. 19 of Audit Report No.39 aimed to improve the understanding of the cost effectiveness of the Program by completing NEISMIS, incorporating detailed financial data.

2.50 DEBT advised the ANAO that improvements to NEISMIS had not been approved by the Information Technology Executive Committee of the Department as the required functionality for NEIS payment arrangements would be included as part of the development of an Integrated Employment System (IES).

2.51 However, DEBT was able to provide information to the ANAO to demonstrate that they were able to assess the cost effectiveness of the scheme. The ANAO understands that data is derived from several other management information systems.

ANAO comment

2.52 The ANAO suggests that the NEIS program area ensures their requirements are considered during the development of the IES.

2.53 In addition, since a system covering the whole of DEBT is likely to take a considerable length of time to develop and implement fully, consideration should be given to analysing the costs and benefits of collecting data from a range of systems as opposed to an interim upgrade of NEISMIS.

DEBT response

2.54 The Department advised that it is progressing with the development of a link between NEISMIS and JOBSYSTEM (a monitoring system used by Commonwealth Employment Centres) to satisfy the reporting

requirements of the Operational Plan, enhance the collection of NEIS data and to avoid double keying of data by DEBT staff.

12

PART TWO

Protective Security

M. Golightly D. Mclnnes B. Danaher

13

Key Points

The AN AO found that six of the eight recommendations of Audit Report No. 14 1991-92 had been fully implemented and considerable progress has been made on the remaining two recommendations.

Notwithstanding this progress the ANAO is concerned, given the serious nature of protective security, that it has taken up to two and a half years to implement some of the key recommendations of the audit report.

15

1. Introduction

Background

1.1 Audit Report No. 14 1991-92 included a review of protective security in DEET’s offices in New South Wales.

1.2 The audit focused on the administrative efficiency and the effectiveness of the Department’s security arrangements. The audit findings raised issues which were relevant both at the State and National level. The ANAO made eight recommendations relating to protective

security policy, risk reviews, security education, the involvement of Internal Audit in security reviews and specific security issues.

Audit objective and scope

1.3 The objective of this follow-up audit was to review progress on the implementation of the eight ANAO recommendations included in Audit Report No. 14 1991-92.

1.4 The ANAO interviewed a number of DEBT officers and reviewed departmental files and policy papers. No interstate field work was undertaken.

17

2. Audit Findings

Protective security policy

2.1 Audit Report No. 14 recommended that the Department develop a comprehensive protective security policy and this be incorporated in a policy and procedures manual for distribution to all staff

(Recommendation No. 1).

2.2 The ANAO found that the Protective Security Section has written the DEBT Protective Security Policy Manual (PSPM). It includes chapters on:

• responsibilities and organisation

• document security

• physical security

• information technology security

• personnel vetting, and

• security education and training.

2.3 The first draft was completed in· November 1992. Since then comments and approval has been sought and provided by Departmental staff and the Community Public Sector Union (CPSU). The Protective Security Steering Committee (PSSC) meeting of 28 April 1994 authorised the release of the policy manual and it was officially released on 15 July

1994.

2.4 The ANAO was advised that the release of the policy manual was delayed because of the time taken to obtain CPSU concurrence and the Services and Structure Review conducted by DEET. It was considered essential that CPSU acceptance was obtained and the policies had to be consistent with the Departmental structure and needs subsequent to the Services and Structure Review.

2.5 A copy of the policy document has been provided to all senior personnel (approximately 2000 staff). It will also be accessible on the in­ house mainframe On-line Manual System and the Commonwealth Managers Toolbox. Concurrently, every officer in the Department has been provided with three information booklets:

• The Security Booklet, providing information on responsibilities, document classification, access to sensitive documents, disposal of documents, etc.

• A Guide to Protective Security in DEET; a brief introduction to protective security. It includes the names and phone numbers of staff able to give security advice

18

• Your Guide to Classified Information in DEET·, advice on the classification of materials.

2.6 The Security Section is not issuing a definitive Procedures Manual. Rather, the Section has written procedural guides for inclusion in the Secretary’s Management Financial Instructions (SMFIs), for example, SMFI 33 Use o f Courier and Transport Services. All SMFIs have been vetted and protective security procedures added as required (e.g. SMFI

17, Use of Keys, Keycards and Combination Locks Key). The Security Section also has rewritten parts of the CES operating manual to ensure that procedures are consistent with protective security requirements.

ANAO comment

2.7 The ANAO considers that the Department has developed comprehensive protective security policies.

2.8 The ANAO found that the PSPM provided practical information on a range of protective policy issues. For example, guidelines on classified documents gave classification definitions and details on document handling, movement and destruction. The PSPM includes a complete index which the ANAO found to be particularly helpful, given that most of the users would not frequently refer to die manual and, at times, would not know precisely where to find the desired information. Booklets, such as Your Guide to Classified Information in DEET were

concise and easily understood.

2.9 The ANAO also considers that it is appropriate to incorporate protective security procedures in SMFIs and CES operating manuals. This ensures that a single familiar source provides officers with appropriate advice on procedures for specific activities and programs.

2.10 Notwithstanding the need to discuss the policy with the CPSU and to ensure consistency with Departmental needs subsequent to the Services and Structure Review, the ANAO is concerned tiiat it took two and half years to fully implement the recommendation. Almost a year was

required to produce the first draft and another eighteen months to produce the final policy document. It is considered that the long delay in finalisation of DEET’s security policy had the potential to increase the risks associated with inadequate security.

Protective security risk review

2.11 Audit Report No. 14 recommended that the Department initiate a risk review as a matter of priority (Recommendation No.2).

2.12 In response, the Department contracted the Australian Security Intelligence Organisation (ASIO) to undertake a protective security risk review in December 1992. ASIO provided DEET with a final report on the outcome of the risk review in May 1993.

2.13 As a consequence of the risk review ‘high risk’ areas were identified and plans prepared to deal with potential threats. For example:

19

• a security structure for each DEBT Area is currently being

established, and

• the Department developed Minimum Protective Security Standards for Network Offices in May 1992 and circulated these in October of that year. These standards address inter alia: access control, duress alarms and contingency plans, information technology security, etc.

2.14 The PSPM requires risk reviews to be undertaken every three years. In sensitive areas, such as Parliamentary liaison and areas associated with fraud prevention, a risk review is required every eighteen months.

ANAO comment

2.15 The ANAO considers that the Department has undertaken a satisfactory risk review and developed policies, which when acted on, will ensure that Department's understanding of risks is current. For example, the Minimum Protective Security Standards for Network Offices, together with Part 3 of the Policy Manual (Physical Security), provide a useful guide on access control and the measures required to achieve this. However, it took some twelve months from the tabling of Audit Report No. 14 1991-92 in December 1991 for the Department to contract ASIO to do the review.

Responsibility for protective security arrangements

2.16 Audit Report No. 14 recommended that the Department review operational arrangements for the management of protective security to ensure that they are consistent with the requirements of the 1990 PSM (Recommendation No.3).

2.17 The 1990 Protective Security Manual (PSM) states that there should be ‘appropriate organisational arrangements... for protective security... at Head Office, and at State and regional levels’.

2.18 Compliance with the 1990 PSM requires the establishment of an appropriate organisational structure and staffing, and the definition of responsibilities.

2.19 Therefore, to establish the appropriate organisational requirements:

• DEBT established the PSSC in February 1992

• at the National Office, staff involved in security matters were integrated into one group in die Legal Branch of the Performance and Review Division in November 1993. That is, the responsibility for physical and personnel security was moved from the Staff Services Branch of the Corporate Services Division and responsibility for Information Technology security was moved from the Systems Division, and

20

• approval was sought for each DEET Area to have between two and four officers to handle security problems and implement related policies.

2.20 The responsibilities of various people and organisations are defined in the DEET Protective Security Policy Manual. Included, for example, are the responsibilities of:

• Government security committees (Security Committee of Cabinet, Secretaries Committee on Intelligence and Security, the Protective Security Policy Committee, and the Security Construction Equipment Committee)

• the PSSC, and

• particular Departmental staff.

2.21 At the time of this follow-up audit DEET’s National Office had seventeen staff under the control of the Director of Security Operations. Also, DEET advised that there are now two security advisers located in each Area.

2.22 Comprehensive training has been given to provide a pool of trained people, and further security training is being conducted within National Office Divisions and die Areas. This training will help to achieve the integration of security in DEET.

ANAO comment

2.23 The ANAO considers that an appropriate organisational structure has been developed and the Department has acted to meet the requirements of 1990 PSM.

Building security

2.24 Recommendation No.4 of Audit Report No. 14 recommended that:

• twenty-four hour security arrangements be reviewed by the Department, and

• digital key/combination locks be fitted to all areas in the Sydney Plaza building which require access to be limited due to

processing/retention of classified information.

2.25 In response, the ANAO was advised that the Sydney Plaza building has two guards, twenty four hours a day. By March 1992, digital key/combination locks had been fitted to areas in the Sydney Plaza building which needed access control.

2.26 At a more general level, the ANAO notes that the Minimum Protective Security Standards for Network Offices provide advice on access control (e.g. security locks, alarm systems etc.). The PSPM also covers building security. At a practical level, during the ASIO risk

review, inspections were conducted and risks assessed. Improvements

21

relating to, for example, doors and access control were recommended and many changes have been implemented (e.g. rear doors alarmed in buildings).

2.27 DEET advised that comprehensive security inspections are about to commence in each DEET Area. Building security will be examined during these inspections to identify problems which could impact on, for example, staff safety or information security.

ANAO comment

2.28 The ANAO considers that specific problems flagged in the audit report have been addressed by the Department. At the policy level, the issue featured prominently in the policy manual and minimum standards have been developed for inclusion in the draft Accommodation Standards Manual.

Storage of information

2.29 In relation to the storage of information, Audit Report No. 14 recommended (Recommendation No.5) that the Department:

• undertake an urgent review to establish the extent of material which should be classified

• should review security storage arrangements, and

• investigate the basis on which tax file numbers (TFNs) have been retained with AUSTUDY applications.

Document classification

2.30 Subsequent to ANAO Audit Report No. 14, the Department found that many documents had been misclassified because staff were not conversant with two primary document classification codes, that is:

• National Security Classified Material (NSCM), and

• Sensitive (In-confidence, protected, highly protected).

2.31 The Department considered that before it could effectively classify documents it was necessary to prepare the PSPM, prepare and distribute classification guides, and educate staff. These issues are discussed elsewhere in this chapter.

2.32 The ANAO was also advised that preparatory work included a check on who was authorised to access classified documents. Accordingly, a Designated Security Assessment Position (DSAP) review was completed in July 1992 and a Positions of Trust (POT) review is planned. No firm timetable has yet been set for the POT review.

2.33 The ANAO notes from Departmental Minutes that a data classification review is about to commence in the CES network and at the National Office.

22

2.34 The ANAO recognises that a large amount of preparatory work had to be undertaken before documents could be classified and notes that considerable progress has been made towards compliance with the recommendation. However, the ANAO is concerned that it has taken

two and a half years to get to the present situation.

Security storage arrangements

2.35 The Department considers that a review of security storage arrangements can not take place until documents are correctly classified. In the interim, the Department completed a review of privacy and protective security arrangements at Student Assistance Centres (SACs) in

December 1993, the final draft of the discussion paper being issued in March 1994. SACs were chosen for the initial review because they have sensitive information relating to, for example, homeless students and tax file numbers. The SAC review, inter alia, looked at document storage arrangements and the findings are considered to be relevant to other

DEBT offices.

ANAO comment

2.36 The ANAO acknowledges that it is logical that a document storage review should be preceded by a review of document classification. Otherwise it is likely that some misclassified documents will not be sufficiently protected, or conversely, additional expense will be incurred

in storing other documents which do not warrant that level of protection.

2.37 The ANAO, however, was not able to determine from the SAC Review Report if SAC offices had adequate Security arrangements. While the report provided information on future requirements, it did not have details on the adequacy of the current situation.

2.38 The ANAO considers that progress is being made towards improving security storage arrangements. However, priority should be given to document storage requirements in areas identified as being ‘high risk’. Area security advisers should monitor compliance with the

guidelines and identify areas requiring improvement on an ongoing basis.

AUSTUDY applications

2.39 ANAO Audit Report No. 14 observed cases where AUSTUDY applications were stored on open shelving and many had TFNs attached. In other cases, applications were locked in cabinets which provided minimum security.

2.40 As a consequence of the ANAO audit, DEBT sought a legal opinion on whether or not the TEN needed to be retained. The Department was advised that there were no such legal requirement. The ANAO was advised that, therefore, a new form was designed which

allowed the small portion of the form with TEN information to be detached. The SACs have therefore been storing TFNs separately to the

ANAO comment

23

application form. However, the ANAO notes that the previously mentioned SAC review has recommendations which impact on TFN storage and privacy, including a recommendation that TFNs be removed as early as possible in the processing of the form and that the practice of retention of TFNs be abandoned.

2.41 The ANAO was also advised that the electronic system was changed to improve confidentiality. For example, since 1991, the TFN has not been routinely displayed when outlet staff undertake data interrogations.

ANAO comment

2.42 On the advice given, the ANAO considers that the Department has taken action to improve the confidentiality of TFNs. The ANAO considers that the Department should examine the cost/benefits of the SAC review recommendations with the view of further improving the confidentiality of TFN information.

Jobseeker card records

2.43 Audit Report No. 14 recommended that an assessment of current storage requirements within CES offices be undertaken to ascertain whether sensitive matter is being adequately secured (Recommendation No. 6).

2.44 The ANAO found that at the time of the audit, Jobseeker card records at CES offices inspected were stored in wooden boxes which could not be locked. As the cards had personal information, this was an unsatisfactory situation being in contravention of the Privacy Act 1988.

2.45 DEBT advised that die installation of the computerised Jobseeker system (JOBSYSTEM) commenced in March 1990 and was completed approximately six months later. However, many managers retained the physical Jobseeker cards as a safeguard in case the computerised system failed. Also, the initial computerised version of JOBSYSTEM did not contain all the information recorded on the physical cards, for example, comments and previous employment details. Some managers believed they might need the cards to obtain these details.

2.46 However, the ANAO was advised during the follow-up audit that CES offices no longer retain physical Jobseeker cards as the computerised system now has all the information provided by clients and managers are more familiar with the system.

2.47 The ANAO was also advised that the security of other sensitive materials held by outlets would be improved following the completion of the previously mentioned activities relevant to the classification and storage of physical documents, that is:

• the preparation of the PSPM

• the preparation and distribution of classification guides, and

24

• staff education.

ANAO comment

2.48 The ANAO considers that with the refinement of JOBSYSTEM, the confidentiality of Jobseeker records has improved.

Security education

2.49 Audit Report No. 14 recommended that a regular program of training to ensure staff are aware of their responsibilities and obligations relating to protective security be instituted (Recommendation No.7).

2.50 In response, the ANAO was advised that security education was considered to be a priority activity. Therefore the PSPM, for example, devotes a chapter to security education and training with details on such issues as the design of educational materials and induction training.

2.51 Training is being provided in a number of ways:

• intensive training courses. Over the last year, forty-two

State/Territory staff were given intensive training in security procedures, for example, one course lasting four days provided training on, inter alia: risk reviews, document classification, the Privacy Act and security vetting. Area security advisers engaged in

the future will receive similar training. The training course is similar to those run for Government Departments and Government Business Enterprises by the Attorney General’s Federal Justice Office

• induction programs for all new staff in the National Office include security training. Area security advisers will train State/Territory staff

• security oriented videos: A-Z o f staff security and IT security have been circulated throughout the Department

• circulation of the previously mentioned information booklets: The Security Booklet; A Guide To Protective Security in DEET, Your Guide To Classified Information in DEET

• articles in the National Office Staff Bulletin, Fit to Print and the National magazine, DEET News

• the preparation and distribution by the Security Section of the Security Bulletin, and

• the preparation and distribution by the Security Section of other guides, for example, The Supervisor’s Guide to Systems Security. This is a supervisor’s guide with information on procedures and responsibilities regarding information systems access (ESAS, JOBSYSTEM, PASS etc.).

25

2.52 The ANAO considers that the activities described should improve staff knowledge of security related issues.

2.53 Individual items reviewed by the ANAO were of high quality. The training course for security advisers had a comprehensive curriculum and was based on recognised course material. The Security Booklet, a reference guide for staff who do not have a specific responsibility for Protective Security, was suitably concise and clearly written. It flagged important issues such as the classification of documents and access to classified materials. To facilitate access to more detailed information on a topic, the booklet is also cross-referenced to other documents (e.g. The Supervisor's Guide to Systems Security).

2.54 However, the ANAO noted that an April 1994 Internal Audit report relating to New South Wales found that staff in the offices surveyed did not have sufficient educational materials. Therefore, notwithstanding the quality of the materials referred to, the ANAO considers that there is a need to continue educational programs and ensure that educational materials are distributed to relevant staff.

ANAO comment

Internal Audit

2.55 Audit Report No. 14 recommended that Internal Audit reviews of protective security be incorporated into the Internal Audit program (Recommendation No.8). In Audit Report No. 14 the ANAO had been unable to establish that Internal Audit had been involved in recent

reviews of protective security.

2.56 In response, the ANAO was advised that since 1993 Internal Audit has been involved in security related audits. This year, protective security audits have been completed in Queensland and New South Wales respectively. The ANAO notes that the Internal Audit program for 1994 includes several security related audits.

ANAO comment

2.57 The ANAO reviewed the audit report Protective Security, Draft Report, April 1994, detailing findings for two Areas (South Western Sydney Area, Hunter Western Area Office and State Office - Corporate Services Branch). Issues addressed included:

• protective security arrangements

• staff awareness

• physical security, and

• authorised access.

2.58 The ANAO considers that the involvement of Internal Audit in protective security reviews is a good practice as it provides a useful

26

independent opinion on protective security which complements review work undertaken by the Security Section. However, the ANAO is concerned again about the time taken to implement action relating to this recommendation.

DEBT response

2.59 The Department supports the ANAO’s comments against all recommendations.

27

PART THREE

AUSTUDY

M. Golightly D. Mclnnes L. Bell R. Lapthorne

29

Key Points

The follow-up audit found that appropriate action had been taken in relation to the recommendations of Audit Report No.20 1992-93, although delays in implementation had been experienced.

It was also found that DEET was performing well against most of its turnaround times standards but had not met a number of its internal standards for debt recoveries (however, it is noted that these standards were set as stretching standards rather than minimum standards to be

achieved). The ANAO has made a number of suggestions aimed at improving the effectiveness of standard setting and reporting on performance.

The ANAO recommends that existing debt management strategies be evaluated regularly and implementation plans for new initiatives should take into account appropriate evaluation mechanisms, including measures of cost effectiveness.

31

1. Introduction

Background

1.1 The AN AO report on the administration of the AUSTUDY program (Audit Report No.24 1989-90) was initially followed-up in 1992. Results of that follow-up audit were reported in Audit Report No.20 1992-93. The report focused on three areas of the program being:

• debt recoveries

• turnaround times, and

• post payment verification.

1.2 Report No.20 indicated that DEBT had made significant progress in the improved management of debts but identified some areas requiring further action in relation to debt recoveries. These areas included the need for DEET to:

• reduce the level of current debt and finalise old debt, and

• ensure that it received timely information on withholdings made under arrangements with DSS for cross recovery of overpayments and the prompt transfer of these recoveries.

1.3 The recommendations of Report No.20 for turnaround times related to ensuring greater consistency in the interpretation of application receipt dates and the recording and monitoring of notification letters.

1.4 Report No.20 also found that DEET had generally made satisfactory progress in the post payment verification area although significant delays in implementation had occurred in relation to some recommendations.

Audit objectives and scope

1.5 The objective of this follow-up audit was to review whether DEET had taken action regarding the recommendations of Audit Report No.20 and whether its performance remained satisfactory in relation to post payment verification.

1.6 The AN AO examination was confined to DEET National Office activities. The ANAO interviewed a number of DEET officers and reviewed documents, including standards and statistical information. No interstate fieldwork was undertaken and the ANAO did not examine the

operations of the Student Assistance Centres (SACs).

Program description

1.7 AUSTUDY is one of a number of education income support programs administered by the Department of Employment, Education and Training (DEET). Expenditure for AUSTUDY in 1992-93 totalled $1438m. A total of 458 000 students received AUSTUDY benefits in 1992.

33

DEBT administers AUSTUDY and other student assistance programs through 29 SACs located throughout Australia. The annual processing cycle commences in November and runs until the end of October the following year. Each year the bulk of AUSTUDY applications are lodged by the end of March. During the peak workload period DEBT is under some pressure to provide quality services to AUSTUDY clients and to manage the application processing work efficiently and effectively.

1.8 Since 1989 DEBT has had National Client Service Standards to measure the extent to which high quality services are provided to AUSTUDY clients.

34

2. Debt Recoveries

2.1 This chapter details the audit findings in relation to die

recommendations of Report No.20 concerning debt recoveries. The issues covered by these recommendations were:

• debt management performance

• targeting of debts

• debt recovery through DSS withholdings

• follow-up of outstanding debt

• quality assurance procedures, and

• reconciliation of debt repayment records.

2.2 In addition, some commentary on the setting and achievement of performance standards has been included.

Summary of findings

2.3 The audit found that appropriate action had been taken in relation to all the recommendations, although delays in implementation had been experienced. In particular, DEBT had taken action to reduce the level of outstanding debt including the investigation of various debt reduction strategies and the development of appropriate performance standards. However, DEET’s position in relation to the level of old debt remains a concern.

2.4 While it is recognised that the debt management performance standards were set as stretching standards rather than minimum standards to be achieved, it was noted that the Department had not met a number of its internal performance targets. Therefore, the ANAO considers that

DEBT should ensure that internal reports include a consistent and comprehensive analysis of performance, and continue to closely monitor these reports in order to identify problems and instigate remedial action.

Follow-up of outstanding debt

2.5 Recommendation Nos 5 and 6 of Audit Report No.20 related to the timely follow-up of outstanding current debts and the finalisation of old debt. In response to these recommendations DEBT advised that a number of new debt collection strategies had been introduced to

encourage prompt repayment. These included the use of mobile review teams, private debt collection agents, the introduction of direct deposit arrangements and the provision of additional staff resources specifically for recovery of old debt.

35

Mobile review teams

2.6 In November 1993 DEBT took action to establish a pilot program of mobile review teams in the five mainland States to carry out a range of field activities in respect of the student assistance program.

2.7 Mobile review teams are to perform a range of responsibilities, including enrolment checking, investigation and review of clients, and debt recovery. Their role in debt recovery is to visit debtors to negotiate repayment of debts, with particular emphasis on cases where field contact with clients is expected to be a more effective means of debt recovery than contact by telephone or in writing. Mobile review teams may also be involved in arranging debt collection through garnishee of wages from employers.

2.8 The ANAO believes that the approach was positive in that:

• it was a relatively small scale pilot with a definite timeframe and budget

• in planning for the initiative DEBT had drawn on the experience of DSS which had operated mobile review teams for a number years

• a national workshop had been held to help coordinate arrangements for the pilot, and

• DEBT drafted procedural guidelines for the teams and addressed arrangements for recording and reporting on their activities.

2.9 However, the benefits of this approach could have improved if DEBT had developed and implemented expected outcome measures, such as estimated amount of debt to be recovered and net savings resulting from the operation, to facilitate analysis of the effectiveness of the initiative. While DEBT had identified the need for these targets, no evidence was found to indicate their use. DEBT advised that an evaluation of the mobile review teams would be undertaken at the end of the pilot in preparation for the next budget process. DEBT also emphasised that, in addition to cost/effectiveness issues, the teams also have a beneficial deterrent effect which was difficult to quantify.

Private debt collection agents

2.10 After DEBT has made reasonable initial attempts to collect debts, the debt recovery guidelines provide for the use of private debt collection agents to collect outstanding debts by means of client contact. These

guidelines cover arrangements for selecting agents, referring cases to them, monitoring their caseload and payment of their fees.

2.11 The debt recovery activities of debt collection agents are potentially of benefit to DEBT provided that the Department ensures that the only cases referred to agents are those where DEBT could not otherwise collect the debt cost-effectively. The principal advantage to DEBT from referring otherwise uncollectable debts to agents was that agents bore the

36

costs of case investigation, whether successful or not. DEBT, however, still had some associated administrative costs for the referral of cases, monitoring of agent performance and processing of agent fees.

2.12 DEET debt recovery guidelines state that DEBT would record the quarterly results of debt collection agents in a statistical report, however, DEET did not present statistics on the value of debt referrals to agents or the value of debt collections of agents for the April-June 1993 quarter. DEET advised that the guidelines have now been changed to indicate that performance of debt collection agents will be monitored through invoice

details and recovery information.

2.13 The provision of debtor information to debt collection agents involves some risk of agents retaining that information or disclosing it to other parties, contrary to provisions of the Privacy Act 1988. Agreements with agents do not permit such retention or disclosure, however, the AN AO noted that an internal audit review in NSW completed in January 1994 found evidence of non-compliance with die

retention requirement. However, DEET advised that the contracts with debt collection agencies had now been changed to allow retention of basic debtor identifiers. This enables agents to account for repayments made to collectors in situations where the case has been formally returned to

the Department prior to the payment being received.

Direct deposit arrangements

2.14 At the time of audit DEET had under consideration several proposals for the introduction of Australia-wide direct deposit arrangements for repayment of student assistance debts. The proposals

would involve debtors making repayments directly at a designated service provider, such as a bank, with an extensive network of branches or outlets throughout Australia. The service provider would then electronically transfer repayment details to DEET.

2.15 The AN AO considered that the direct deposit proposals had the potential to provide an improved means of collecting student assistance debts, however, it was noted that the project had proceeded slowly in that arrangements for repayment of student assistance debts through direct

deposit had been under consideration within DEET for some time. The project had commenced in 1991 with earlier proposals dating back to 1987.

2.16 From the documentation examined, it was not evident that Deet had assessed the cost/effectiveness of the limited direct deposit services already in operation when developing the 1991 proposal. Subsequently, DEET has undertaken a review of various direct deposit options

including DEET arrangements in South Australia. However, the report has been in draft form since November 1993.

2.17 Direct deposit arrangements have now been implemented in various forms in Western Australia, Tasmania, South Australia and Queensland. DEET also advised that they were consulting with other agencies such as

37

DSS which have comparable systems under development or in operation.

Additional staffing for debt collection

2.18 In response to the ANAO’s recommendation that it make a concerted effort to finalise old debt, DEBT had approved additional resources for the recovery of old debt.

2.19 The AN AO did not find evidence that DEBT had evaluated the results of deploying the additional staff resources, particularly in relation to the collection of old debts. The ANAO noted documentation indicating that DEBT had originally proposed to evaluate the 1991-92

strategy for collection of old debt on the basis of achievement of targets and analysis of the age structure of student assistance debt. However, no formal evaluation of the exercise was performed. DEBT now advises that an evaluation will be undertaken in preparation for the next budget cycle as funding for these resources expires in June 1995.

Recommendation N o .l

2.20 The ANAO recommends that DEBT regularly assesses the cost effectiveness of its debt collection strategies.

DEBT response

2.21 It should be noted that the Department has informally monitored the outcomes of its strategies involving the use of private debt collectors, direct deposit arrangements and additional staff resources specifically for recovery of old debt and is generally convinced of the value of these strategies. However, its acknowledges the need to formally evaluate these measures on a more regular basis.

Level of old debt

2.22 The ANAO examined the age structure of student assistance debt, particularly DEET’s finalisation of ‘old debt’1 and ‘very old debt’, which are respectively defined here by the ANAO as debts outstanding for a year or longer and for two years or longer.

2.23 The Table below shows the age structure of student assistance debt at the end of December 1993 compared to end of June 1993.

2.24 The figures in the Table below include debts which had previously been written-off but were subsequently reinstated for recovery action. DEBT estimates that debts in this category total approximately $2.8m. This represents a relatively small proportion of old debt.

38

Table 1 Age Structure of Student Assistance Debt, June and December 1993

Age Range (Days)

Outstanding Debt, end June 1993

Outstanding Debt, end Dec 1993

Value % Value %

0-180 $14.3m 29.4 23.

5m

38.9

181 - 364 $14.2m 29.2 10.0m 16.6

365 - 730 $11.4m 23.4 $15.9m 26.4

730 - 1095 $4.9m 10.1 $6.2m 10.3

1096 + $3.8m 7.9 $4.7m 7.8

Total $48.6m 100.0 $60.3m 100.0

Note:

(1) There are unexplained discrepancies between debt totals shown here and in DEBT debt minimisation statistics.

2.25 AN AO analysis of DEBT data in the above Table indicated that:

• DEET’s performance in finalising old debt deteriorated in absolute and relative terms during the first half of 1993-94. The value of old debt rose from $20. lm at the end of June 1993 to $26.9m at the end of December 1993. Old debt as a proportion of total debt increased

from 41% in June 1993 to 45% in December 1993, and

• DEET’s position in respect of the finalisation of very old debt also underwent a decline in absolute terms during 1992-93 and the first half of 1993-94. The value of very old debts rose from $8.7m in June 1993 and $10.9m at the end December 1993. However, the proportion of very old debt remained virtually unchanged at 18% in

December 1993 compared to June 1993.

2.26 It was also observed that:

• the level of outstanding debt as at the end of December 1993 was higher than as at the end of June 1993 for all specified debt age ranges, excepting debts six months to less than one year, and

• DEET’s level of outstanding debt less than six months old rose sharply in absolute and relative terms during the first half of 1993­ 94. Debt less than six months old increased from $14.3m at the end of June 1993 to $23.5m at the end of December 1993. Debt less than six months old was 39% of total debt, as at December 1993

39

compared to 29% in June 1993. These results could indicate improved DEET detection of overpayments that might previously not have been identified, but also could be due to reduced performance of initial payment controls over student assistance benefits and/or a decreased rate of debt recovery.

2.27 DEET advised that the proportion of old debt has fallen from 52% in 1991 to 44% in 1994. Also, the Department has many more debts being repaid through instalment plans rather than lump sum repayments. These debts would appear as old debt but are under active repayment plans.

ANAO comment

2.28 As mentioned above, DEET has initiated a number of strategies to improve the collection of debts (including old debts), however, it appears that these strategies may not as yet have had an impact on the levels of old and very old debt. Given that debts generally become more difficult to recover as they age, it remains important that DEET continue their efforts to develop and implement effective strategies for recovery of old debts.

Targeting of debts

2.29 Audit Report No. 20, noted that DEET was not targeting specific debt categories to maximise recoveries. Debt categories that the ANAO identified for possible review included debt above certain amounts, of a specified age, or repaid to some extent, as well as debt with instalment default. The ANAO recommended that DEET develop and use fixed and

free format reports to help target debt and provide appropriate instruction to staff concerning the production and use of these reports. The ANAO also recommended priority be given to enhancing the computerised student assistance recoveries system (STARS) to identify instalment default cases (Audit Recommendation No.7).

2.30 In response to Audit Report No.20 DEET advised that it was undertaking a review of the STARS report facilities, which would be completed by April 1993. DEET also stated that it had developed a report to facilitate follow-up of debt requiring date-specific action and it was now targeting debtors systematically for review based on specific selection criteria.

2.31 This audit found that DEET had initiated, but not completed, action to improve the design and use of report facilities to target debt categories. At the time of audit DEET was about to introduce new free format reports to help better target debts. DEET completed the review of STARS report facilities in mid 1993 and the proposed improvements were implemented in May 1994.

2.32 Staff were provided with documentation at the time of

implementation. In March 1994 DEET National Office sought local views on staff training needs in respect of debt management operations, including the STARS report facilities, and training was undertaken in

40

September 1994.

ANAO comment

2.33 Although there had been some delay, the ANAO considers that DEBT lias taken appropriate action to address this recommendation.

Debt recovery through DSS withholdings

2.34 In response to concerns raised in the original audit, Audit Report No.20 found that during 1991 and 1992 DEBT and DSS had implemented arrangements to facilitate recovery of AUSTUDY overpayments through withholdings from DSS benefits. However,

Report No.20 identified some areas where further improvement in liaison arrangements between DEBT and DSS was required. DEBT needed to receive information on DSS withholdings in a more timely manner to update its debt records. In addition, DSS withholdings should be

transferred promptly to DEBT to ensure that debts are appropriately accounted for. Recommendation Nos 8 and 9 dealt with information relating to repayments through witltholdings and the prompt transfer of these repayments to DEBT.

2.35 In response DEBT advised that:

• it had reached agreement in November 1991 with DSS for monthly reports on withholdings information and fortnightly reports would be available with the introduction of a new DSS debt management system

• a working party had been established to develop the requirements for the DEBT and DSS computing systems to improve reporting and credit transfer, and

• a proposal had been forwarded to DSS for establishing a charge code accessible by DSS for crediting withholdings.

2.36 While a charge code had been introduced, at the time of the audit little progress appeared to have been made in implementing the other arrangements referred to above. Specific arrangements to provide fortnightly reports on withholdings as well as to improve reporting and

audit transfer through computer enhancements had not been introduced.

2.37 In March 1994 DEBT and DSS met to discuss procedures for cross-recovery of debt and agreed that a computer interface for the exchange of withholdings information would be desirable. However, DSS indicated that unless DEET funded the project, DSS could not

consider the development of the system before September 1995. DEET subsequently sought and received a quote from DSS for the cost of the computer enhancements, however, this quote is now subject to further negotiation.

ANAO comment

2.38 The ANAO considers that DEET and DSS should finalise

41

consideration of the proposal as soon as practicable, and, if cost-justified, proceed with the system development without delay. The appropriate charge code and associated arrangements should be put in place as a matter of urgency.

2.39 AN AO review of progress on these issues disclosed that there continued to be general problems in liaison arrangements between the two departments in respect of DSS withholdings. During 1993-94 DEBT noted that many local DSS offices were not cooperating with DEET requests to initiate withholding action to recover student assistant debts, unless DEET provided local staffing resources to perform the work.

DEET documentation referred to a consequential decline in recoveries through DSS withholdings. DEET advised that, in South Australia for example, the number of debts that could not be recovered through DSS rose from nil to 285 in 1994. This would indicate a significant reduction in repayments, particularly as DSS had collected $8m in DEET debts in

1993-94. This problem is of particular concern and all efforts should be made to rectify this situation.

Recommendation No.2

2.40 The ANAO recommends that DEET make a concerted effort with DSS to finalise arrangements aimed at improving the transfer and reporting of DSS withholdings.

DEET response

2.41 The Department agreed that it is important to resolve this issue with DSS and plans to meet with DSS to progress the development of a fully automated system of cross recovery. Should a system interface not be feasible in the short-term, alternative means of data transfer will be investigated.

Reassessment of entitlement

2.42 Reference was made in Audit Report No.20 to internal control weaknesses in debt management operations in South Australia. The ANAO recommended improved procedures to ensure that DEET followed-up cases requiring reassessment as a result of enrolment and attendance checks. DEET also needed to ensure that cases where debtors failed to respond correspondence concerning outstanding debt were better monitored, using the STARS diary or an follow-up facility. The ANAO also recommended review of procedures to prevent debtors from receiving payments in excess of their annual entitlement

(Recommendation No. 10).

2.43 In response to Audit Report No.20 DEET advised that the benefits control units would check that reassessments were undertaken and procedures had been introduced to identify debtor records where expected

responses had not been received by the due date. In addition, AUSTUDY Regulations have been amended to provide a fortnightly rather than yearly entitlement, thereby removing the possibility of clients

42

receiving a temporary excess of entitlement.

2.44 While DEBT was taking action to improve follow-up of debt, the current audit found that report facilities for monitoring reassessment cases appeared not to be fully used.

2.45 DEET National Office referred cases requiring reassessments after enrolment or attendance checks to local offices by means of paper records. In April 1994 the DEET Benefits Control Section proposed to introduce an electronic facility to transmit case details to assessing

officers, thereby minimising the time taken to action cases. At the time of the audit DEET was developing the technical specifications for this enhancement which will be incorporated in the new payments system scheduled to be implemented in 1995/96. DEET already had a

computerised report facility to record the number of referrals and the completion of action in respect of those cases.

2.46 DEET had also introduced a computerised report facility which provides comprehensive information relating to inactive debts including the number and value of the debts which required follow-up action. In addition, DEET advised that an enhancement to the STARS diary facility

designed to highlight individual cases that officers were due to action was implemented in September 1994.

ANAO comment

2.47 The ANAO considers that the action taken by the Department will address the recommendation, however, there has been delays in finalising the implementation of some of the initiatives.

Quality assurance procedures

2.48 Audit Report No.20 had found a lack of use of the computerised student assistance recoveries system (STARS) by assessors, incomplete evidence of checks by supervisors and non-performance of post-payment and quality assurance checks in 1991. Audit Recommendation No. 11

proposed a number of remedial measures. In response, DEET advised that it would encourage assessors to use the browse access facility and would seek a commitment from staff that supervisor checks would be

undertaken. DEET subsequently stated in its Minister for Finance return in March 1993 that assessors were using the browse access facility and there was staff commitment to the supervisor checks. The quality assurance strategy had been reviewed and checks were now carried out

fortnightly.

2.49 As this follow-up audit was limited to DEET National Office, die ANAO observations below relate to DEET’s progress in introducing revised guidelines for quality assurance only.

2.50 The supporting documentation to the 1994 guide included detailed requirements for quality assurance checks to be carried out in respect of data entry, supervisor and post assessment checks. Target error rates have been set. On a fortnightly basis, DEET National Office generates a

43

2% random sample for post assessment checks. These checks are carried out fortnightly with monthly reports provided to DEET’s Executive along with other performance reports.

2.51 In addition, DEET has recently introduced a quality assurance program for debt management operations. These guidelines have a number of useful features including their coverage of:

• the number and timing of quality assurance checks to be performed

• detailed criteria against which cases were to be checked, and

• the standard format for recording the results of checks.

2.52 However, die ANAO considered that die guidelines could be improved by providing a rationale for the specified number of checks, prescribing procedures for sample selection, describing requirements for reporting results and arrangements for retention of local review records.

2.53 A number of local offices had difficulties with carrying out the first debt management quality assurance exercise in early 1994 due to their different work methods and problems with the format of the check

records. DEET advised that it was reviewing the responses of local offices. DEET is now proposing that the results of the checks be forwarded to National Office every six months (local checks are performed every two months). ·

2.54 In the ANAO view, it may be beneficial for DEET to consider monitoring the results more closely than six month intervals until the quality assurance routines are well established.

ANAO comment

2.55 The ANAO considers that the action taken has largely addressed this recommendation.

Reconciliation of records of debt repayments

2.56 Audit Report No.20 found that DEET had not reconciled debt repayment records and financial ledger records to ensure the completeness and correctness of data in its internal financial systems. The ANAO recommended that DEET ensure regular reconciliation between recoveries received by the Collector of Public Moneys (CPM), the computerised student assistance recoveries system (STARS) and DEET’s financial management system (FINEST) (Audit Recommendation No. 12). In response DEET advised that procedures had been introduced to reconcile the records and that the two systems materially reconciled at

end June 1992.

ANAO comment

2.57 The ANAO found that there had been delays in the implementation of an automated reconciliation facility between STARS and FINEST records, which DEET had planned would be operational in August 1993.

44

At the time of audit DEBT expected that the facility would be available by the end of April 1994. However, DEET had carried out detailed manual reconciliations at State level each month until August 1993 and since that time performed regularly aggregate reconciliations at national

level as an interim control. At the time of the audit the latest aggregate reconciliation showed no material difference between records of CPM repayments in the two systems.

Standard setting

2.58 DEET currently has three main sets of performance standards relating to debt management:

• a national client service standard on debt management for the 1994 student processing year

• debt minimisation standards and indicators developed in May 1993,

• benefit control performance indicators introduced in November 1993.

2.59 Targets relating to each of the above standards are detailed in Appendix 1.

2.60 During the course of following-up the audit recommendations some information on standard setting was collected. Given the focus of the audit, the examination of this area was necessarily limited, however, suggestions for improvement were noted and are discussed below.

National Client Service Standard

2.61 The inclusion of standards related to debt management is a positive feature of DEET’s national client service standards. The 1994 standard appropriately focused on the overall debt base for student assistance programs. This is clearly a key measure of benefits payment controls and debt management. However, in the ANAO’s view, DEET should consider expressing the standard in terms of a specific dollar amount to be achieved by a certain date in the program year as well as in terms of a percentage of program outlays. The benefit of this approach would be

that it is a more direct measure of the level of overpayments.

2.62 DEET advised that performance would be assessed internally against both die absolute dollar and the percentage figures for the 1995 year. DEET also advised that the wording of the standard would be reviewed for 1996.

Debt Minimisation Standards

2.63 The debt minimisation standards and indicators which DEET developed during 1993 covered the major aspects of DEET’s debt management function for the student assistance program. The usefulness of these statistics was enhanced by setting interim State targets for some

standards, which took into account performance variation between States and the Northern Territory. The production of a glossary describing most of the targets, the rationale for target figures and reporting facilities

45

was another positive feature.

2.64 However, a number of improvements are suggested:

• that DEBT consider the absolute value of write-offs in conjunction with other performance information, and

• DEET’s brief narrative reports were deficient in that fortnightly debt management performance was not routinely analysed against the full range of debt minimisation targets and indicators. Also there was no special reporting of debt management performance against the key targets set for the first part of 1993-94. Again, the ANAO considers that this information is important to the effective monitoring of performance.

Benefits Control Performance Standards

2.65 Achievement against the benefits control performance standards was presented in graphical format making it easier for management to note trends in performance, without the need for detailed examination of the statistics.

2.66 The benefits control performance indicators presented performance information in the form of dollar value per client per fortnight of new debt raised as well as debt repaid. However, if used alone, the design of these indicators had several drawbacks: '

• they focused on fewer aspects of debt management performance than the debt minimisation statistics. The ANAO considers that continued reporting of both the debt minimisation and benefits control standards would provide DEBT with more comprehensive information regarding the various aspects of debt management operations

• the indicators used total clients rather than total debtors as the base for measurement of performance. This could distort performance measurement given that the ratio of debtors and clients might vary over time and between SACs. Deet have advised that it is intended that the 1995 reports will use total debtors as the base, and

• this set of indicators appeared to be less target oriented than the debt minimisation statistics and implicitly accepted the previous year’s performance of individual SACs as a suitable benchmark for their current year’s performance.

Staff workload indicators

2.67 DEET lacked performance indicators to measure debt management staff workload, efficiency and effectiveness. In the ANAO’s view performance information, such as the ratio of both debt caseload and debt recoveries to staff numbers, would not only add value to the assessment of the relative performance of SACs but would also assist DEET in its long term resource planning.

46

ANAO comment

2.68 The focus of the client service standards relating to debt management have changed significantly from year to year. It is acknowledged that standards may change to reflect changing Departmental priorities. However, die ANAO believes that it is

desirable for Departments such as DEBT to have a relatively stable set of measures which enable interested external parties to assess the Department’s performance over a number of reporting periods. A stable set of measures would also assist internal trend analysis of the performance of different SACs and Areas against targets. This is particularly important in the development of, and assessing performance against, stretching targets.

2.69 In relation to performance reports, DEBT advised that the format of these reports was to be refined for the 1995 year. While there are obvious benefits associated with less detailed reports, the ANAO considers that DEBT needs to continue to monitor the effectiveness of the

format to ensure that the appropriate level of performance information is provided to each layer of management. Performance against key indicators at any level should also be reported in a consistent fashion.

Actual performance

2.70 DEBT provided the following information on actual performance at the national level.

Table 2 Debt Recovery Performance

1991-92 $m

1992-93 $m

1993-94 $m

New debt notified 44.4 59.2 67.9

Recoveries 30.1 42.9 51.2

Recovery rate 68% 72% 75%

New debt as a % of 3.1% 3.8% 4.2%

outlays

National Client Service Standard

2.71 The above Table shows that DEBT had not reduced overpayments to the level targeted by its National Client Service Standard being 3% of program outlays. Also, over last three years this percentage has increased.

47

2.72 However, the value and rate of recovery of debts have improved. New debt identified has also increased. This could indicate DEBT detection of overpayments that might previously not have been identified, but also could be due to reduced performance of initial payment controls over student assistance benefits or external factors beyond the Department’s control (such as general economic conditions, etc).

Debt minimisation

2.73 As can be seen from Appendix 1, DEBT had established six debt minimisation standards in May 1993, five with specified targets. These are set out in the Table below. The Table also indicates performance against each standard for June and December 1993.

Table 3 Targets and Performance in terms of Debt Minimisation Standards, June and December 1993

Debt Minimisation Standard

Target for end June 1993 Performance, end June 1993

Target for end Dec 1993 Performance, end Dec 1993

Value of debt base

$47m $51m $50m $60m

Average age of debt 9 months 11 months 8 months 10 months

Average value of debt raised

$481 $475 $432 $493

Debt under repayment arrangement

44% 44% 52% 38%

Debt repayments

$40m (annual) $52m (annual) $25m (half year) $29m (half year)

2.74 As mentioned above, DEBT performed well against its target for debt repayments. However, DEET’s overall performance in the six month period to December 1993 generally did not meet the specified internal targets as follows:

• DEBT student assistance debt base rose from $51m at end June 1993 to $60m at end December 1993, compared to the target of $50m by that date. DEBT advised that external factors beyond its control, such as the rising number of AUSTUDY clients and other general economic factors had impacted on the Department’s ability to meet this target

48

• the average age of debt not being repaid remained above the target of eight months. It fell only slightly from eleven months as at end June 1993 to ten months by end December 1993. DEET advised that the decrease in DSS withholdings as discussed in paragraph 2.39 had

impacted on the level of achievement against this target, and

• to reduce the value of amount of debt incurred (in SACs) DEET had indicated that they wanted to reduce the average value of debt raised from $481 to $432 but actual performance indicated that debt increased from $475 to $493.

2.75 While it appears that DEET did not achieve its target of having 52% of debt under repayment arrangements by end December 1993, DEET noted at the time that the result was affected by the inactivation of withholding arrangements after the final 1993 pay run. The AN AO

observed that the proportion of debt under arrangement had been as high as 50% in late October 1993. Following advice from the AN AO, DEET will in future express performance in terms of the last pay date for the calendar year rather than the end of year date.

Benefits control performance

2.76 It can be seen from Table 2 that DEET had improved its debt identification and recovery performance in 1993-94 compared to 1992­ 93, although the Department did not achieve the specific internal target of raising $8 new debt per client per fortnight.

ANAO comment

2.77 It is noted that DEET has improved its performance in a number of areas. However, while it is acknowledged that the standards were set as stretching targets rather than minimum targets to be achieved, DEET needs to continue to monitor its performance against those internal targets

which were not met.

2.78 The ANAO has recently examined debt recovery practice across the Australian Public Service as part of its efficiency audit of cash management in Commonwealth departments.

49

3. Turnaround Times

3.1 This chapter details the audit findings in relation to the

recommendations of Report No.20 concerning turnaround times. The issues covered by these recommendations were:

• recording receipt dates

• application acknowledgment

• early applications, and

• evidence of review.

3.2 In addition, some commentary on the setting and achievement of performance standards has been included.

Summary of findings

3.3 The follow-up audit found that the action taken by DEET since the previous audit substantially met all the recommendations relating to turnaround times. Although the audit noted that DEET had made major improvements to its staff guidelines, their had been a delay in their

finalisation. DEET should ensure that the Guide is issued before the commencement of the processing year.· DEET advised that it was on schedule to issue the 1995 Guide to all SACs before the commencement of the processing year.

3.4 DEET was performing well against most of its National Client Service Standards for turnaround times, although, the performance of some individual SACs was below target.

3.5 While DEET’s processes for developing standards and reporting on performance had numerous positive features, the ANAO has made a number of suggestions aimed at improving their effectiveness.

Recording receipt dates

3.6 Audit Report No.20 reported that in many instances the date of receipt of AUSTUDY applications recorded in the student assistance information system differed from the date the ANAO determined to be the actual receipt date of the applications. The ANAO recommended that DEET monitor that application receipt dates were correctly recorded (Recommendation No. I).

3.7 DEET’s 1993 Guide made detailed reference to the correct recording of application receipt dates. However, only a general reference was made to receipt dates in the 1994 Guide.

ANAO comment

3.8 While the action taken by DEET in 1993 was appropriate, the ANAO considers that there is a continuing need to provide detailed guidance on the correct recording of receipt dates (either in the Guide or

50

its supporting documentation). Without die more detailed guidance diere is a greater risk of inconsistency in operational practices. DEBT advised that detailed instructions regarding the correct recording of receipt dates have been included in the 1995 Guide.

Application notification date

3.9 Audit Report No.20, highlighted the fact that DEBT did not record or monitor the dates on which notification letters were actually dispatched to AUSTUDY applicants. The ANAO recommended that DEBT use the letter’s production date plus three days as the notification date for

performance measurement purposes and monitor that letters were promptly dispatched after production (Recommendation No.2).

3.10 The 1993 and 1994 guides referred to DEBT using the letter production date plus three days to measure application processing turnaround time and required SACs to keep records of the time between

letters’ production and dispatch.

ANAO comment

3.11 The ANAO considers that DEET’s actions addressed the ANAO concerns.

Application acknowledgment

3.12 Audit Report No.20 noted that the Adelaide SAC advised applicants that their applications had been received and the result of the initial assessment of their applications would be notified within three weeks of the date shown on the acknowledgment form. The ANAO considered that this initiative could reduce the number of applicant queries regarding the status of their applications and recommended that

SACs more widely adopt this measure (Recommendation No. 3).

3.13 The 1993 Guide indicated that applicants were to be provided with a tear-off proof of lodgment which also indicated the expected date of notification of the results of their application. Specific reference was not made to this measure in the 1994 Guide, however, as the Department has

now focused on notifying applicants of their eligibility within 21 days rather than just setting a target for sending an acknowledgment of receipt of an application.

ANAO comment

3.14 The ANAO considers that DEBT have addressed this

recommendation satisfactorily.

Early applications

3.15 Audit Report No.20 referred to another source of incorrect recording of the date of receipt of AUSTUDY applications. Where applications were received before the student assistance information

system became operational each year, DEBT assigned to those applications the common receipt date that the system went ‘live’. The

51

ANAO recommended that DEBT investigate the feasibility of making the system available for live processing on the same date as DEBT accepted the receipt of applications (Recommendation No.3).

3.16 DEET guides for the 1993 and 1994 processing years formalised the definition of receipt dates for applications to be the date of system availability. More importantly, DEET brought forward the date of availability of system for processing AUSTUDY applications for the

1994 processing year to 1 November 1993.

ANAO comment

3.17 The ANAO considers that DEET’s actions addressed the ANAO concerns.

Evidence of review

3.18 Reference was made in Audit Report No.20 to the fact that SACs did not maintain evidence of local monitoring of AUSTUDY application processing performance. The ANAO recommended that processing performance reports and evidence of review be retained at SACs and available for independent scrutiny (ANAO Recommendation No.4).

3.19 The 1993 Guide referred to SACs retaining various local records until the end of the following financial year for referral when required (e.g. for audit scrutiny). The 1994 Guide stated that SACs were to collect and retain certain supporting local management information, but it did not specify the expected time period for its retention. Nor did the

Guide require SACs to retain evidence of local monitoring of national performance reports.

3.20 While DEET has taken some action, it is considered that this action does not fully meet the requirements of the recommendation. In particular, the recommendation required the retention of reports and other evidence to illustrate that DEET’s internal review processes for

monitoring application processing performance were operating. Neither the 1993 or 1994 guides specifically require the retention of review documentation.

ANAO comment

3.21 While some action has been taken in relation to this area of concern, the ANAO considers that the Guide, or its supporting documentation, could be further improved by incorporating a requirement to retain evidence of performance reviews and specifying an appropriate retention period.

3.22 DEET advised that the 1995 Guide would include this requirement.

Performance standards

3.23 During the course of following-up the audit recommendations some information on standard setting and achievement against these standards was collected. Given the focus of the audit, the examination of this area

52

was necessarily limited, therefore, this information is presented below as a commentary on DEET’s performance in this area.

Turnaround time performance

3.24 DEET had seven student assistance National Client Service Standards for the 1994 processing year. Two of these standards related to turnaround times (see Table 4). DEET described the National Client Service Standards as the organising principles around which student

assistance staff and management directed their efforts in the provision of quality services to its clients. The standards were regarded as important because they provided the basis to measure the extent to which DEET provided a high quality service to clients. In addition, they were

essential in the context of accountability to clients and the Government.

Table 4 Coverage of DEET Student Assistance National Client Service Standards for Processing Year 1994

Standard Number Issue Covered in National Client Service

Standard (NCSS)

NCSS 1 - To advise eligibility for student assistance within 21 days of first lodgement of an application.

Timeliness of advising applicants of their eligibility for student assistance

NCSS 2 - To advise eligibility for student assistance within 14 days of lodgement of a completed application.

Timeliness of processing completed applications

3.25 In relation to these standards, DEET set three processing targets for the 1994 year (see Table 5). Two of these targets relate to the processing of initial applications and were introduced in the 1994 processing year for the first time. The third target related to the processing of completed applications. Targets for completed applications have been determined for the years 1991-94.

3.26 As can be seen from Table 6, at the time of the audit, DEET was achieving targets 1 and 2 at the national level. The targets were met in most States and the Northern Territory, and in the majority of DEET’s

29 SACs. However, there was a wide range in the achievement of the targets for turnaround times across SACs, for example, the proportion of AUSTUDY applications processed within fourteen days ranged from 22% to 99% (as shown in Chart 1 in Appendix 2). It can also be seen

from this Chart that seven SACs (approximately 33%) had missed this target by more than 5 %.

53

Table 5 DEBT Turnaround Time Client Service Standards and Targets for Processing Years 1991 to 1994

Type of student assistance application

Target for 1991 processing year Target for 1992

processing year

Target for 1993 processing year

Target for 1994 processing year

Initial application

At least 60% to be advised within 14 days

Standard not introduced until the 1994 processing year At least 75 % to be advised within 21 days

Completed application only

80% of all applicants to be notified of initial result within 21_ days

80 % to be notified within 21 days

At least 85% to be notified within 21 days

At least 85 % to be advised within 14 days

100% of applicants to be notified of initially result with 35 days

100% to be notified within 30 days

100% to be notified within 30 days. (This was later

modified internally to 98%)

Table 6 Actual Performance Against Targets

1994 Target Actual Performance

1. 60% advised within 14 days 62%

2. 75% advised within 21 days 79%

3. 85% advised within 14 days 80%

3.27 Standard 3 is the only standard which allows comparisons with the previous year (see Table 7). This Table shows that DEET’s performance against the 1993 targets was good. However, as can been seen from Table 5 the timeframe in the target had been changed from 21 days in

1993 to 14 days in 1994. While it is acknowledged that the 1994 target

54

was a more stringent target, DEET did not meet tire target at the national level (see Table 6). This was also the case in most States/Territories and in most SACs (see Chart 2 in Appendix 2). Again there was a wide range in performance between SACs and as can be seen from Chart 2,

ten SACs missed the target by more than 5%. It should be noted that many of these SACs were the same ones which had difficulty meeting Target 1 (as displayed in Chart 1).

3.28 It is acknowledged that the standards were set as stretching targets rather than minimum targets to be achieved. However, the AN AO considers that DEET should closely monitor the performance of SACs within each area against these targets. While performance of individual

SACS is monitored, DEET advised that it intends to introduce improvements, such as trend analyses, for the 1995 processing year and that the performance of individual SACs and Areas will be more comprehensively analysed to identify problems and the appropriate

remedial action.

Table 7 1993 Targets for Processing Competed Applications

1993 Targets Actual Performance

85% advised within 21 days 97%

100% advised within 30 days 99%

3.29 The time taken to transfer AUSTUDY applications from various DEET outlets (including CES offices) to the appropriate SAC is also a potential source of delay in application processing. However, it was found that DEET’s transfer of applications between DEET outlets during

1993 and 1994 was adequate of the national level (although there was a considerable range in performance of Areas in transferring applications from outlets other than SACs).

Standard setting

3.30 DEET has significantly changed the number of publicly available client service standards for the processing years from 1991 to 1994. As discussed in Chapter 2, it is acknowledged that standards may change to reflect changing Departmental priorities. As previously stated in Chapter

2, the ANAO believes that it is desirable for Departments such as DEET to have a relatively stable set of measures which enable interested external parties to assess the Department’s performance over a number of reporting periods. DEET advised that the 1994 standards will be

consolidated and refined for the 1995 year and that analysis of trends will be undertaken.

55

3.31 DEET’s introduction of a turnaround time standard in 1994 which measured the timeliness of processing all applications, in addition to the standard which measured the timeliness of completed applications only, was of significant benefit in providing a performance measure of DEET’s overall client service in processing AUSTUDY applications.

3.32 DEET’s processes for reviewing National Client Service Standards had a number of positive features:

• the review teams met on a timely basis and were representative of both National Office and the SACs

• DEET’s processes for consultation both during the development of the standards and on proposed implementation of the standards were comprehensive, and

• client satisfaction survey information together with surveys of SAC operational performance were used in the development of the standards.

3.33 However, there were some aspects of the processes for developing the 1994 standards that could have been improved:

• DEBT documentation prepared during the development of new standards should better explain the differences in standards between reporting periods and the rationale for the change of the standards. This would help to ensure that staff fully understand the levels of performance expected, the purpose behind the standards and the

reason for change. DEBT advised that this issue has been addressed in reviewing the standards for 1995, and

• any future SAC survey exercises to help develop standards should be better planned and documented. As mentioned above, DEBT undertook surveys of SAC performance. While the ANAO considers this to be a positive approach to target setting, DEBT had experienced problems with short timeframes, low response rates, inconsistent application of sample selection criteria and inadequate documentation of the conduct of the surveys and compilation of survey results. DEBT advised that the surveys would be managed in

future by the Evaluation and Monitoring Branch within DEBT.

3.34 DEET’s staff guides on the 1993 and 1994 standards were major improvements on DEBT internal statements on standards for previous processing years. The documents provided guidance which would have helped SACs address the deficiencies in operational recording and

monitoring of turnaround time data reported in previous audits. However, there was a delay in their final publication. DEBT should ensure that guides are issued before the commencement of the processing year. DEBT advised that the 1995 Guide will be issued to SACs by the end of October 1994, before the commencement of the processing year.

56

DEBT reporting on turnaround time performance

3.35 DEET’s arrangements for reporting on student assistance performance had a number of positive features:

• DEBT issued comprehensive documentation relating to the reporting format, measurement and data sources for each standard, as well as reporting routines for the forthcoming year

• DEBT adequately defined ‘underperformance’ for reporting purposes and specified requirements for SACs to provide explanations for underperformance

• the reports provided regular, frequent and useful statistical information to monitor turnaround time performance for each fortnightly period and on a year-to-date basis, by SAC, State and nationally, and

• the reports were widely distributed to all levels of DEBT

management.

3.36 However, a number of areas which could have been improved were noted:

• graphic presentation of key performance information such as year-to- date performance against client service standards (for example, see Charts 1 and 2), would assist in providing management with a clear picture of the level of client service achieved nationally in the course

of the current processing year. DEBT advised that 1995 reports would include graphic presentation of key performance indicators

• performance reports should also provide a more balanced coverage of performance in terms of the three turnaround targets, rather than emphasise one particular target and should use year-to-date statistics, rather than current fortnight statistics, as the main type of

performance measure. Year-to-date figures give a clearer picture of performance over the year, are less susceptible to short term fluctuations and are not subject to the statistical distortion that affected DEET’s fortnightly statistics towards the end of the peak processing season. DEBT advised that for 1995 the turnaround

standards had been reduced to two, and performance reporting had been revised to focus on year-to-date statistics

• the commentary accompanying the performance reports should include a comparison of the finalisation rate of applications with the registration rate to give a better indication of the rate of clearance of applications. DEBT advised that future performance reports would

include this comparison where appropriate

• although the performance reports listed individual SACs not achieving particular targets, they did highlight SACs that were performing poorly across the National Client Service Standards during the processing year. They also did not provide a

57

comprehensive analysis of factors affecting general performance or details of remedial action taken or proposed to address problems. As an example, the ANAO examined the performance reports of the SAC with the lowest turnaround times for 1994 (Townsville) and found that the reports:

- tended to understate this SACs low level of service

- mentioned various factors such as workload and staff resourcing issues as factors affecting the level of service without giving an indication the extent of the problem, and

- did not refer to any specific remedial measure being taken to address the problems

• DEBT should ensure relevant and timely reports. DEBT advised that they are currently undertaking a review of the Client Service Delivery Network performance reporting with the aim of being more timely and cost effective, and

• the small (0.6% to 1%) arithmetic discrepancies in some computer generated statistics should be corrected (if resolution is cost effective).

3.37 DEBT advised that the 1995 performance reports would include a detailed analysis of problems facing" SACs together with proposed remedial action.

58

4. Post Payment Verification

4.1 This chapter provides comments in relation to the action taken by DEBT to address the areas of concern raised in the original audit (Audit Report No.24 1989-90) and the first follow-up audit (Audit Report No.20 1991-92). The issues covered are:

• reviews of continuing entitlement

• enrolment checks, and

• other post payment verification checks.

Reviews of continuing entitlement

4.2 The original audit found that the Department was not being advised by many students of a change in circumstances that would affect their entitlement to AUSTUDY and made a number of recommendations aimed at addressing this issue.

Review letters

4.3 Audit Report No.24 1989-90 recommended that DEBT send all AUSTUDY recipients a review letter at least once a year. The initial follow-up audit found that this recommendation had not been implemented due to a number of constraints.

4.4 In 1992 DEBT implemented a strategy which it considered to be an effective alternative to the annual review letter to all students. The strategy targets high risk categories of AUSTUDY recipients requiring them to complete a detailed review form. The remainder of recipients

received an information letter or leaflet reminding them of their obligation to advise DEBT of a change in circumstances.

4.5 The leaflet also contained a ‘whistleblower’ section, inviting information about any person thought to be obtaining AUSTUDY benefits that they may not be entitled to. The leaflet was used in 1992 and 1993.

4.6 The processing costs associated with the leaflet have proven to be high when compared with the identification of possible overpayments. Many recipients were completing the change of circumstances section of the leaflet even when their circumstances had not altered, thereby adding

unnecessarily to the administrative costs associated with the program. DEBT has therefore reconsidered the use of the leaflet. However, the Department is unable to assess the cost effectiveness of the leaflet as separate records of the proportion of changes of circumstances notified

through use of the leaflet have not been maintained except for cases related to the ‘whistleblower’ section. The Department did undertake an assessment of the expected savings from the use of the leaflet in the Northern Territory, however, given the size of the sample (Northern

Territory represents 2% of the client population) these results could not be extrapolated Australia-wide. In addition, separate costs relating to the administration of the leaflet were not available.

59

4.7 DEET also advised that they believed the majority of the savings resulting from the use of the leaflet would have been picked up through the normal compliance regime of enrolment checks, data-matching and spot checks. DEET have indicated that total savings resulting from all methods of post payment verification would be approximately $50.5m with the major portion being saved through enrolment checks (approximately $31.3m).

Personal interviews

4.8 The original audit recommended that reviews could be more effective if they were extended to include personal interviews. DEET had agreed with the recommendation subject to the availability of resources.

4.9 The first follow-up audit found that DEET had determined that verification based on mailed reviews was more cost-effective than personal interviews.

4.10 Personal interviews are only conducted when there is a strong indication that a person is receiving benefits they are not entitled to, and prosecution is likely. At the time of the first follow-up audit the ANAO considered the procedure of supplementary follow-up interviews was satisfactory in view of resource constraints and probable cost effectiveness of mailed reviews in relation to personal interviews in future.

4.11 Subsequently, DEET has piloted and is currently using Mobile Review Teams and envisages these teams will, amongst other things, conduct necessary personal interviews. Mobile Review Teams have been in operation, with a team based in each of the five major capital cities, since January 1994.

Fines for withholding information

4.12 As a further step towards reducing overpayments the Department is proposing to introduce a system of fines. The introduction of the fines is dependent on the enactment of proposed amendments to the Student Assistance Act 1973. The Bill containing the proposed changes is

expected to be introduced in the Autumn 1995 session of Parliament. Under the system, AUSTUDY recipients found to have withheld information on ‘changed circumstances’ will be fined $10 per week until DEET is notified of the correct information, up to a maximum of $1000.

4.13 As well, DEET has designed the original AUSTUDY application form to make it very clear to recipients that they have a duty to report changes in circumstances. Those AUSTUDY recipients who are considered to be high risk are sent a questionnaire to confirm their eligibility. This is done twice a year and in total covers 40% of AUSTUDY recipients. AUSTUDY recipients are required to respond to the questionnaire. In cases where a reply is not provided within the specified time benefits are suspended until a reply is received.

60

Enrolment checks

4.14 The original audit recommended that the Department should review its administrative arrangements and liaise with education institutions to expedite the processing of enrolment checks. It was also recommended that enrolment checks be conducted as near as possible to enrolment confirmation dates to maximise their effectiveness in controlling overpayments. The first follow-up audit found that the Department was

taking action in relation to both these issues. The results of this follow­ up audit with regard to the two issues is discussed below.

Collection of enrolment data

4.15 At the time of this follow-up audit DEET acknowledged that significant problems remained with the collection of data from a small proportion of institutions, particularly with the TAPE sector. A number of steps had been taken to improve the situation. DEET had put a

submission to the National TAPE Chief Executive Committee in February 1994, recommending that TAFEs maintain accurate enrolment records.

4.16 The previously mentioned proposed amendments to the Student Assistance Act 1973 were also to provide the Department with additional powers to obtain information from educational institutions to verify claims made by students. The amendments detail the kind of information

that institutions may be required to give in respect of a student’s personal details, courses in which they are enrolled, and workload level. However, the Privacy Commissioner believes the proposed amendments

would enable the Department to infringe on the privacy of students. The proposed amendments to provide additional powers will not therefore proceed.

4.17 DEET had also taken steps to improve the format in which data is received. Firstly, institutions are being encouraged to provide data electronically. The development of the Core Australian Specification for Management and Administrative Computing (CASMAC) over the next

two years would provide opportunities to increase the standardisation of information provided to DEET.

Timing of enrolment checks

4.18 In 1994 the Department proposed to conduct four enrolment checks for the tertiary sector, taking into account the HECS confirmation dates and the end of semester dates. At the time of the follow-up audit the Department advised that all enrolment checks were being undertaken on

schedule.

4.19 In the case of secondary students checks are carried out once a term, with the most important check undertaken shortly after enrolment in mid to late February. These checks were also being carried out on schedule.

61

4.20 The timing of the enrolment checks as outlined above should maximise their effectiveness in controlling overpayments.

Other post-payment verification checks

4.21 The Department has a policy of carrying out a range of other post­ payment checks to ensure that AUSTUDY recipients are not receiving other forms of support, or income exceeding the allowed limit. These

tests comprise a number of internal and external checks with other agencies to verify independence status, income and receipt of other benefits. The original audit made several recommendations for improving these cross checks and the first follow-up audit found that some action had been taken.

4.22 During this follow-up audit the Department advised that cross checking will be further enhanced by DEET’s expected inclusion, from January 1995, in the Department of Social Security’s (DSS) Accelerated Claimant Matching (ACM) System. Under the arrangement, DEBT will provide DSS with student assistance client data for weekly compliance checking purposes using the ACM system. This will provide both DEBT and DSS with up-to-date and verified information on client details.

4.23 In addition, the Data-matching program has continued to be a significant control against fraudulent claims for student assistance allowances. At 30 June 1994, the Department estimated that for the cost of $1.2m, it has saved $0.5m and identified a further $10.4m in debts. It expects to recover $7.8m of the debt over the next three years.

ANAO comment

4.24 The ANAO found that DEBT has implemented a number of measures to address the concerns raised by the ANAO in previous reports and has a number of other measures under consideration to improve post payment verification. However, the ANAO was concerned that in some cases, for example, the review letters for continuing entitlement, the cost effectiveness of the mechanism could not be established. The ANAO recommends that the Department ensure that the various post payment verification measures in place are routinely evaluated and that appropriate performance indicators are developed and implemented to allow this evaluation to take place. In relation to those measures under consideration, the Department should analyse the measure in terms of expected costs and benefits (including intangible items) before implementing the measure.

4.25 While the ANAO considers that DEBT is aware of the sensitivities related to the introduction of a fines system, the ANAO would particularly emphasise that the Department ensure that all recipients are not only aware of their obligations hut also are aware of the possibility of a fine if these obligations are not met, thereby encouraging the notification of changed circumstances. Otherwise, the fines could be seen by external parlies as a source of revenue rather than a deterrent. DEBT advised that it will take this into account in future form design.

62

Recommendation No.3

4.26 The ANAO recommends that existing post payment verification strategies be evaluated regularly and implementation plans for new initiatives should take into account appropriate evaluation mechanisms, including measures of cost effectiveness.

DEBT response

4.27 The Department agrees with the recommendation and indicated that the performance of all its main post-payment verification strategies are monitored on a regular basis.

28 October 1994 Canberra ACT

63

Appendix 1 DEBT debt management performance indicators

S e t o f p e r fo rm a n c e

in d ic a to rs

N u m b e r o f in d ic a to rs T im in g o f

in tro d u c tio n tim e o f a u d it

S p e c ific m e a su re s

N a tio n a l c lie n t s e r v ic e

s ta n d a r d s re la tin g to d e b t

m a n a g e m e n t

T w o s ta n d a r d s fo r 1992,

th re e s ta n d a r d s f o r 1993

an d o n e s ta n d a r d fo r

1994

A p p lie d to th e

re le v a n t s tu d e n t

a s s is ta n c e

p ro c e s s in g y e a r ,

w h ic h b e g in s in

N o v e m b e r o r

D e c e m b e r o f th e

p r e v io u s c a le n d a r

y e a r a n d ru n s

th ro u g h u n til

O c to b e r o r

N o v e m b e r

1 9 9 4

s ta n d a r d

b e in g u s e d

1 9 9 2

6 0 % o f o v e rp a y m e n t c a s e s to b e le s s th an o n e

R e c o v e rie s an d p r o g r a m s a v in g s to to ta l

$ 5 7 .6 m

1993

A t le a s t $ 5 8 .6 m (f ig u r e to b e n e g o tia te d w ith

D e p a rtm e n t o f F in a n c e )

A t le a s t 7 0 % o f p r i o r y e a r d e b t a s a t 1 J u ly

1 9 9 2 to b e u n d e r a r ra n g e m e n t o r fin a lis e d by

3 0 J u n e 1993

A t le a s t 8 0 % o f c u r r e n t y e a r d e b t (1 9 9 2 -9 3 ) to

b e u n d e r a r ra n g e m e n t o r fin a lis e d w ith in six

m o n th s o f b e in g ra is e d

1994

O v e rp a y m e n ts n o t to e x c e e d 3 % o f an n u a l

p r o g r a m cxiilays

N a tio n a l p e r fo rm a n c e

ta rg e ts fo r b e n e fits c o n tro l

T w o ta rg e ts In tro d u c e d in

O c to b e r 1991

N o lo n g e r in 5 0 % o f p r i o r y e a r d e b t a s a t 1 J u ly 1 1991 to

b e u n d e r a r ra n g e m e n t o r fin a lis e d b y 3 0 Ju n e

1 9 9 2

8 0 % o f c u r re n t y e a r d e b t (1 9 9 1 -9 2 ) to be

u n d e r a r ra n g e m e n t o r fin a lis e d b y 3 0 Ju n e

1 9 9 2

D e b t m in im is a tio n

s ta n d a r d s an d in d ic a to rs

S ix s ta n d a r d s w i t h fiv e

s p e c if ie d ta rg e ts ,

to g e th e r w i t h o th e r

p e r fo rm a n c e in d ic a to rs

re la tin g to d e b t

m a n a g e m e n t

S e t in M a y 1993 B e in g u s e d S ta n d a rd s

T a r g e t d e b t b a s e to b e $ 5 0 m at 3 0 D e c e m b e r

1 9 9 3 a n d $ 4 0 m a t 3 0 J u n e 1994

A v e ra g e a g e o f d e b t to b e 8 m o n th s b y 3 0

D e c e m b e r 1993 a n d 6 m o n th s b y 3 0 Ju n e

1994

T a r g e t a v e ra g e v a lu e o f d e b t ra is e d to b e $ 4 3 2

b y 3 0 D e c e m b e r 1993 a n d $ 4 0 0 b y 3 0 J u n e

1994

D e b t u n d e r a r ra n g e m e n t to b e 5 2 % o f to tal

d e b t b a s e b y 3 0 D e c e m b e r 1993 a n d 6 0 % by

3 0 J u n e 1994

T a r g e t d e b t re p a y m e n t o f $ 2 5 m b y 3 0

D e c e m b e r 1993 a n d $ 5 0 m b y 3 0 J u n e 1994

R e p a y m e n ts to b e a (n o t s p e c if ie d ) % o f d e b t

b a s e s ix m o n th s e a r lie r

O th e r in d ic a to rs

V a lu e o f d e b t ra is e d

W r itte n o f f d e b t a s % o f d e b t b a s e

V a lu e o f d e b t ra is e d a s % o f p r o g r a m o u tla y s

65

Appendix 2 AUSTUDY Applications Processing

C hart 2 A p p lic a tio n p ro ce ssin g C om pleted AUSTUDY a p p lic a tio n s p rocessed w ith in 14 days, SACs, to end A pril 1994

Canningto n (WA) Toowoomba (QLD) B ooragoon (WA) Indooroopilly (QLD)

Perth C en tra l (WA) Mt G ra vatt (QLD) H aymarket (NSW) H obart (TAS)

Caulfield (VIC) Box Hill (VIC) Broome (WA)

Western Sydney (NSW) Liverpool (NSW) Melbourne (VIC) N ewcastle (NSW)

N o rth c o te (VIC) Darwin (NT) F o o ts c ra y (VIC) Wollongong (NSW)

Dandenong (VIC) North ern NSW (NSW) R ockhampton (QLD) Adelaide (SA) C a n b e rr a (ACT/NSW)

Bendigo (VIC) C herm side (QLD) Townsville (QLD) 0 % 2 0 % 4 0 % 6 0 % 8 0 % 1 0 0%

Note: Year to d a te d a t a ro u n d e d to

n e a r e s t p e r c e n t .

C hart 1 C lie n t s e rv ic e AUSTUDY a p p lic a tio n s p ro ce sse d w ith in 14 days, SACs, to end A p ril 1994

Cannington (WA) Indooroopilly (QLD) Booragoon (WA) Perth C en tra l (WA)

H aymarket (NSW) H o b art (TAS) Toowoomba (QLD) Box HIM (VIC)

^ΖΏΞΖΖΖΏΖΖΖ^ΒΖΏΖΖΖ^ί2ΏΖΖΖΒ2ΏΖΏΖΖ2Ζ™ΖΏΖΖΖΏΖ^ΖΖΖΖΖΖ2Ζ2Ώ2ΖΏ2Ζ2ΖΏΖ2ΏΖΖΚ2ΖΖΞΖΖ2Ζ3 86%

Mt G r a v a t t (QLD)

D a r w i n (NT) M e l b o u r n e (VIC) N e w c a s t l e (NSW) N o r t h c o t e (VIC

B r o o m e (WA) C a u l f i e l d (VIC) F o o t s c r a y (VIC)

L i v e r p o o l (NSW) D a n d e n o n g (VIC) W e s t e r n S y d n e y (NSW) N o r t h e r n NSW (NSW)

W o l l o n g o n g (NSW) A d e l a i d e (SA) R o c k h a m p t o n (QLD) C a n b e r r a ( A C T / N S W )

B e n d i g o (VIC) C h e r m s i d e (QLD) To w n s v i l l e (QLD)

6T%

.......................................... . . .

0 % 2 0 % 4 0 % 6 0 % 8 0 % 1 0 0%

Note.· Year to d a t e d a t a ro u n de d to

n e a r e s t p e r c e n t .

6 6

Series titles Titles published in the financial year 1994-95

Audit Report No. 1 Project Audit Other Titles

Commonwealth Scientific and Industrial Research Organisation Annual Report of the Australian National Audit

- Follow-up of an Efficiency Audit Office 1993-94

of External Funds Generation Tabled 24 August 1994

Audit Report No.2 Department of Defence Follow-up Audit Management of Army Training Areas

Preliminary Study Acquisition of Additional F-l 11 Aircraft

Audit Report No.3 Project Audit Wool Tax Australian Taxation Office

Audit Report No.4 Project Audit Special Investigation into Casselden Place Building, Melbourne Department of Administrative Services

Audit Report No.5 Follow-up Audits Department of Employment, Education and Training - New Enterprise Incentive Scheme (NEIS) - Protective Security - AUSTUDY

67