The International Skills Training (IST) Courses Program was designed to address critical competency gaps and support the skills needs of the global workforce. Courses developed under this program were implemented in Fiji, Samoa, Tonga and Vanuatu from 2017 to mid-2019. This evaluation was commissioned to: (i) determine whether there has been a shift to competency-based training in participating countries of the
Pacific in the short and medium term as a result of IST and (ii) recommend how
outcomes can be improved.
What we did
Fiona developed the key evaluation questions and program logic. Based on these, she, developed and implemented (after trialing all data collection instruments in Fiji) an Evaluation Plan underpinned by: Kirkpatrick’s four level model for evaluation of training, contribution analysis, a utilisation focused approach, case study analysis and cross case analysis.
How we did it
The key evaluation questions and program logic were developed through initial stakeholder consultation and document reviews. The underpinning approaches documented in the Evaluation Plan were chosen to maximise learning across the four different countries as the technical and vocational education context of each was different.
The approved Evaluation Plan was implemented, with data collected in more than 150 interviews (including 64% course participants) across all countries (including in rural areas outside the main island). Independent interviews with managers, colleagues and students were able to confirm the changes reported by participants. A survey of all participants was implemented with country return rates of exceeding 70%, except for Vanuatu. Overall, the return rate was 56% of participants. Data was analysed, primarily using content analysis against the program logic.
Case studies were prepared for each country and an aide-memoire provided to the
training provider and client in-country.
Benefit for partners (or the client)
The evaluation was able to identify the impact of the courses on training in each pilot
country. The findings underpinned the decision to expand the program. The
recommendations in relation to course design, commissioning and implementation were adopted to contribute to improved outcomes.
Evaluation of the Australian Department of Employment, Skills, Small and Family Business (DoE) International Skills Training Courses Program, 2019