Evaluation

Evaluation is an essential component of determining the worth and efficacy of projects, programs and services. Evaluation helps to capture how projects, programs and services are performing, the extent to which your goals are being met and what opportunities there are for learning and improvement.

Evaluation is a core component of FPC's work. Our staff possess more than 25 years combined experience evaluating projects, programs and services across a wide variety of topic areas. We undertake process and outcomes evaluations at different scales and varying timeframes. We assist clients in the development of evaluation plans and frameworks, establishing monitoring and evaluation processes. 

A key focus of our work is working closely and collaboratively with our clients to ensure that they own the evaluation process and that the findings and recommendations are useful, practical and relevant to all stakeholders.

Current and past projects

Below are examples of current and past projects across a range of topic areas.

Natural resource management​
​Including agriculture and extension

Evaluation of the Landholders Driving Change Project
NQ Dry Tropics

​The Landholders Driving Change (LDC) project is one of two Major Integrated Projects recommended by the Great Barrier Reef Water Science Taskforce and funded by the Queensland Government. NQ Dry Tropics is delivering the LDC project in the Burdekin region, with the goal of achieving enduring and sustainable land management within the Bowen, Broken and Bogie catchment communities. First Person Consulting has been engaged to work closely with NQ Dry Tropics, various other consultants and key stakeholders to deliver an external independent evaluation of the LDC project. Our work complements extensive internal monitoring and reporting processes, and involves designing and delivering interim evaluations in 2018 and 2019, and a final evaluation in 2020.

Beyond soilcare
​Goulburn Broken CMA

Goulburn Broken CMA has been running the Beyond SoilCare project since a base-line study in 2012 established farmer attitudes and practices regarding soil management. The CMA ran the project to build long-term farmer commitment to soil health and the provision of ecosystem services from soil. Farmers opted in to learn about the capability of their soils by learning about soil types and features. Capability was built through training workshops, field-based activities and visits to demonstration sites. FPC Director Dan Healy, while in a previous role, completed a study in 2013 to determine the project outcomes. In 2016, FPC conducted an impact evaluation of the project to determine the extent to which farmers have been implementing new practices on farm.

Review of the Grazing Best Management Practice Program - Phase II
Fitzroy Basin Association

The Fitzroy Basin Association (FBA) is the regional NRM organisation for Central Queensland, and since 2014 FBA has been managing Phase II of the Grazing Best Management Practice (BMP) Program on behalf of a partnership between FBA, AgForce and the Queensland Government Department of Agriculture and Fisheries. The Grazing BMP Program is designed to improve the sustainability and profitability of grazing enterprises through identifying opportunities for change in agricultural practices, contributing to improved water quality in the Great Barrier Reef through reducing sediment from broad-scale grazing. FPC was contracted to evaluate the effectiveness and value of the Grazing BMP Program, and to provide recommendations to inform future delivery of the Program. This involves assessing the effectiveness of the Program as managed by the FBA, and demonstrating the value of the Program to its funder, the Queensland Government.

Evaluation of stakeholder engagement in pest control programs
Natural resources commission (NSW)

FPC worked with the Natural Resources Commission to evaluate the impact on park neighbours of pest control activities conducted by the NSW National Parks and Wildlife Service. The evaluation explored the success of relationship management through respondents’ satisfaction with the program’s administration, consultation and wider impacts. As part of the evaluation, First Person Consulting surveyed neighbours by phone, letter and email. We also interviewed key community user group representatives and members of Indigenous advisory committees. The surveys contributed to an interim review of the NSW supplementary pest control trial where recreational and voluntary shooters are engaged in ground-shooting of vertebrate pests in NSW parks and reserves.

Evaluation of the Farming for Sustainable Soils Project
North Central CMA

The North Central Catchment Management Authority (North Central CMA) has been delivering the Farming for Sustainable Soils Project (FSS Project) since 2009. The FSS Project is a community-driven initiative that supports new ways of farming for soil health. North Central CMA works with regional groups and community facilitators to encourage the adoption of sustainable land management practices that protect and enhance soil health. Phase 2 of the Program is coming to a close in June 2018 after five years of funding through the National Landcare Program (NLP), and First Person Consulting was engaged to provide evaluation and reporting support to North Central CMA. This involved: developing a series of case studies to demonstrate the outcomes and achievements of the program, proving support at the FSS Conference in March 2018, and drawing on a range of data sources to report on the FSS Project within North Central CMA guidelines and consistent with NLP reporting processes.

Sheep Connect Tasmania Mid-Term Review
​Tasmanian Institute of Agriculture

Sheep Connect Tasmania (SCT) is an extension and communication project aiming to increase the productivity, confidence and resilience of Tasmanian sheep and wool businesses. One of its key objectives is to be recognised as the key conduit for providing valuable, relevant and timely farming systems information to industry. Mid-way through its three-year funding cycle, First Person Consulting externally reviewed the program, providing an update of progress and recommendations for strengthening the program’s monitoring and evaluation.

Evaluation of the Reef Programme (Water Quality Grants)
NQ Dry Tropics

​FPC evaluated the Water Quality Grants component of the Reef Programme, which addressed the threats of declining water quality and climate change to the Great Barrier Reef World Heritage Area. Through the Reef Programme, Water Quality Grants were distributed to land managers to increase voluntary uptake of improved management practiced by landholders to reduce the discharge of sediment, nutrients and pesticides into the Reef. FPC staff worked with NQ Dry Tropics to conduct an interim and final evaluation of the Water Quality Grants - including evaluating the effectiveness of the program in leading to improved farming and resource efficiency practices. This involved large scale surveys and phone interviews with landholders who received grant funding.

Evaluation of the Drought Extension PRogram
Agriculture Victoria, Department of Economic Development, Jobs, Transport and Resources

FPC was contracted to evaluate two key components of the Victorian Government’s Drought Extension Program (DEP). Extension Services and Stock Containment Area grants were delivered by Agriculture Victoria to farming communities around the state affected by drought between 2015 and 2017. FPC provided an independent assessment of the effectiveness and efficiency of these DEP components and make recommendations to inform future program delivery. FPC evaluated the program against its stated objectives and intended outcomes, involving a review of existing reports and data, surveys and interviews with participating landholders and farmers, program delivery staff, industry stakeholders, community members and staff from other comparable programs. The evaluation contributed to a wider evaluation of the Whole of Government Drought Support Package to be undertaken later in 2017.

Northern Illawarra Deer Management Program
South East Local Land Services

The South East Local Land Services, in collaboration with Wollongong City Council, established the Northern Illawarra Wild Deer Management Program (NIWDMP) in 2011. The program addressed the growing negative impacts of deer in the Northern Illawarra region. Central to the program is a regular process of contractors shooting deer on council and private land. First Person Consulting evaluated the NIWDMP with a focus on understanding how effectively it had delivered on its objectives and if and how the program should be continued into the future. We used monitoring data collected as part of the program and interviewed landholders, program staff and members of the program’s stakeholder committee. We also collated information from other broadly similar programs across Australia to help identify lessons for future initiatives. 

Evaluation of four national Landcare program-funded projects
north east CMA

North East Catchment Management Authority (NECMA) is one of ten authorities established by the Victorian Government in 1997. Each Catchment Management Authority (CMA) works with their communities, government and funding organisations to protect and enhance land, water and biodiversity resources. CMA’s deliver a range of on-ground activities that include conservation, restoration and extension and the Victorian Government and the National Landcare Programme (NLP) is a major funder of these works. NECMA engaged First Person Consulting to review four NLP funded projects to determine if they were on track to meet specified outcomes and to capture any lessons that can be used to inform future natural resource management work by the CMA.​

Building Resilience and Sustainability in the Grape and Wine Sector
Australian Wine REsearch Institute

AWRI’s Roadshow is an extension program designed to deliver current research to the grape and wine industry to encourage the adoption of innovative practices. The overarching purpose is to build resilience and sustainability in the industry to the effects of climate change. FPC were contracted to understand and document the effectiveness of the information delivered through the climate mitigation workshop series. This involved interviews and surveys of workshop attendees to ascertain the usefulness of the information and the methods used for delivering the information.​

Development of the Land SErvices Program Evaluation Plan
Western Local Land Services

Western Local Land Services (LLS) is launching a pilot initiative—the Land Services Program (LSP). Its aims are to integrate the range of natural resource management programs that are currently on offer, increase the overall coverage and effectiveness across the community, and establish the organisation as a skills and knowledge provider to stakeholders. First Person Consulting worked with Western LLS to develop a monitoring and evaluation plan to sit alongside the LSP and demonstrate its impact. Our work involved stakeholder consultation, developing a program logic, document review and analysis, and establishing monitoring and evaluation planning guides, tools and resources. The focus of our work was to enable the effective evaluation of program design, effectiveness and implementation, as well as identifying lessons learned and recommendations for future program delivery.​

Enhancing our dandenong creek pilot project review
Melbourne Water

​The Enhancing Our Dandenong Creek (EODC) program is a five-year collaborative pilot led by Melbourne Water, involving community groups, local businesses, councils and EPA Victoria. The pilot trialled an alternative approach to protecting the beneficial uses of Dandenong Creek. The approach included: involving the community in decision-making, focusing on the full suite of risks posed to the creek, and activities that provided the greatest beneficial outcomes. This contrasts with the standard approach of using conventional sewer augmentation to meet Melbourne Water’s legislative obligations. First Person Consulting and collaborators have been engaged to undertake a full review to understand if the pilot has been successful and to identify lessons for future work in this area.

Evaluation of Wildlife Victoria
Wildlife Victoria

Wildlife Victoria has been in operation for 30 years. It is a registered not-for-profit charity that, at its core, provides a wildlife emergency response service. This service provides callers with advice and, where needed, arranges volunteer rescuers and transporters, and links with veterinarians, carers and shelters to get the best help for sick, injured or orphaned wildlife. ​The organisation also advocates for better wildlife welfare and educates the community about how to successfully co-exist with wildlife. Wildlife Victoria received a grant from the Helen Macpherson Smith Trust to undertake an independent evaluation of the organisation to identify the value it provides to Victoria. The evaluation was undertaken through extensive review of existing data and documents, stakeholder interviews and a volunteer survey we found that the organisation leverages an additional $1.9 million above its direct financial expenditure in volunteer time, in-kind contributions of equipment and vehicle costs.

The full evaluation report is available online.

Environment and Community programs' Evaluation Challenge
Department of environment, Land, water and planning

​DELWP’s Environment and Community Program's (ECP's) branch within the Biodiversity Division engaged FPC to undertake a complex body of work known as the 'Evaluation Challenge'. The Evaluation Challenge spanned a range of areas, including undertaking four lapsing program evaluations in line with the DTF Lapsing Program Guidelines, development of an evaluation plan for a $40 million program, review of the Sustainability Fund Evaluation Framework and Biodiversity 2037 Evaluation Framework to identify processes for incorporating into business as usual, and the design and delivery of a suite of evaluation training and coaching to the entire ECP branch. The Evaluation Challenge was set for staff and programs to increase their evaluative capabilities, and to act as an example to the rest of the Department of how evaluation can be incorporated into day-to-day business. 

climate change and sustainability
including energy efficiency

Evaluation Plan for the Victorian Climate Projections 2019 Program
Department of environment, land, water and planning

Victoria’s Climate Change Adaptation Plan 2017-2020 includes commitments to produce ‘better climate projections for Victoria’, as well as ‘guidance on how to use this data’. The Victorian Climate Projections 2019 Program (VCP19 Program) was developed to meet this commitment by working with CSIRO to produce detailed information about the impacts of climate change on a scale of 5km2. These climate projections will improve understanding of the range of possibilities for Victoria under different climatic scenarios to a more localised level. Datasets can then be integrated into spatial tools and decision-making processes to inform adaptation action, including Victorian Government activities. FPC is currently working with the Climate Change Policy Branch within DELWP to develop an Evaluation Plan for the VCP19 Program. The Plan will be developed through document review and a workshop with the Program team. The product will be an Evaluation Plan consisting of a theory of change, evaluation questions, data collection methods and guidance on implementation.

Take2 End of year 1 review
sustainability victoria

TAKE2 is a pledge program through which organisations and individuals commit to taking action on climate change, with both mandatory and voluntary components. Sustainability Victoria (SV) is leading the voluntary component of the TAKE2 program. As the first year of implementing TAKE2 has come to a close, First Person Consulting undertook a review to reflect on the successes and challenges of the program to date, inform future delivery of the program and contribute to the development of a monitoring and evaluation framework. In completing the review, First Person Consulting facilitated workshops with the Climate Change Team and with the Executive Leadership Team, reviewed key documents and data, and delivered an online survey of Pledge Partners.

Evaluation of the climate change mentor initiative
department of environment, land, water and planning

​The Climate Change Mentors Initiative was funded by the Victorian government to support local government in planning for and adapting to climate change. Three mentors worked throughout the state to build capacity in councils and act as a conduit between state and local government policy makers. First Person Consulting conducted the evaluation of the Initiative, which involved interviews and surveys with local and state government stakeholders. The emphasis was on achievements, impacts and key lessons from the Initiative for future programs. 

Reflection of the SV2015 strategy
Sustainability victoria

​First Person Consulting was contracted by Sustainability Victoria (SV) to undertake a reflection on the implementation of SV’s three year organisational strategy, SV2015, across the two priority areas of Integrated Waste Management and Resource Efficiency. After completing extensive document review, stakeholder interviews and qualitative and quantitative analysis, a report was developed which provided lessons learned and recommendations to inform the development of SV’s new SV2020 organisational strategy. This involved synthesis of considerable amounts of data from diverse stakeholders in a range of formats. The final reflection on SV2015 included a range of key lessons and opportunities for SV’s work into the future.

sustainability Strategy evaluation framework
Mount Alexander Shire council

FPC ​developed a 10-year evaluation framework for Mount Alexander Shire Council’s Sustainability Strategy. This included a critique of the logic model used in the sustainability strategy to examine its robustness and its realism (i.e. are the outcomes actually achievable). This informed the development of the evaluation questions and sub-questions, as well as identifying appropriate data sources for use as indicators. A key consideration for the project was the capacity of the council to deliver the evaluation themselves, as such a streamlined analysis and reporting process was developed to reduce the burden and align it to the Environment Team’s regular reporting duties.

Smarter Resources Smarter Business program evaluation
Sustainability Victoria

​In 2016, Sustainability Victoria contracted FPC to evaluate components of the SRSB Program. The SRSB Program was designed to assist SMEs in Victoria improve the efficiency of their energy and materials usage and increase business productivity. FPC evaluated the outcomes of the Resource Assessment and Building Business Capability components of the SRSB Program, which included a review of other resource efficiency programs working with SMEs; surveys and interviews with SMEs; engagement and interviews with industry associations and Program delivery staff; and the development of evaluation findings and recommendations to inform future program delivery. The evaluation team also facilitated a workshop with representatives from other government agencies implementing sustainability and eco-efficiency programs for SMEs around Australia.

Solar for renters Pilot Project Evaluation
Moreland Energy Foundation Ltd.

​The Moreland Energy Foundation Ltd (MEFL) has delivered a range of solar campaigns for homeowners in partnership with local councils, supporting more than 2,500 homes in installing solar over the last four years. Traditionally, tenanted rental properties have been excluded from these campaigns. To help address this gap and align with Moreland City Council’s Zero Carbon Evolution Strategy, MEFL is implementing the Solar for Renters Pilot Project. The Pilot will install solar on 10 rental properties across Moreland through a mechanism that delivers benefits to both landlords and tenants, overcoming the barrier of split incentives. FPC is working with MEFL to evaluate the Pilot, which includes consultation, interviews, data analysis and reporting. This will help to evaluate the outcomes of the model for tenants and landlords, the effectiveness of the project partner’s customer service role, opportunities and barriers for the future, and the potential for wider scale delivery.

Final Evaluation of Victoria's Climate Change Adaptation Plan
Department of Environment, Land, Water and Planning

​FPC, in partnership with Loop & Co, was engaged to evaluate the impacts and outcomes of the Victorian Government’s first Climate Change Adaptation Plan. The Adaptation Plan was released in 2013, and over four years has provided a strategic framework to guide Victoria’s policy responses to the impacts of climate change. Given the scope of the Plan (i.e. a state-wide strategic framework) and the political nature that can sometimes occur with climate change, FPC have taken a multi-faceted approach. This has involved reviewing and weighting (i.e. how reliable is it) data collected from previous evaluations, interviewing and surveying key government staff on their views of the plans efficacy and implementation processes. The results of this evaluation informed the development of Victoria's second Adaptation Plan.

You can find the evaluation report in Appendix 2 of the Climate Change Adaptation Plan, available online.

Monitoring and evaluation of the 'Growing your business with energy savings' pilot project
Department of Environment, Land, Water and Planning

​The Department of Environment, Land, Water and Planning (DELWP) manages the Victorian Energy Upgrades program (VEU) which is designed to reduce business and household power bills, energy consumption and greenhouse gas emissions. However, there has been limited uptake of this program in regional Victoria so far. The ‘Growing your business with energy savings’ pilot project has been created to target implementation of the VEU program within regional business and residential sectors. It is an exploratory project that investigates the effectiveness of trade networking events and other engagement approaches in the Ballarat region in increasing VEU uptake. FPC has been engaged by DELWP to monitor and evaluate the effectiveness of pilot activities. This includes facilitating a logic model workshop, developing an evaluation plan, and designing and conducting a range of data collection activities alongside pilot project delivery.

Households and Small business upgrades program (HASBUP) Mid-term evaluation and process review
Department of Planning, Industry and Environment

​The Household and Small Business Upgrade Program (HASBUP) is a five-year, $52.5 million program that aims to build and stimulate markets for energy efficient products and services in NSW. HASBUP provides financial incentives to help households and small businesses save energy and money through energy efficiency upgrades by partnering with service providers, including through the Energy Savings Scheme (ESS). FPC undertook an independent mid-term evaluation and process review of HASBUP, aligning with the NSW Government Program Evaluation Guidelines and Climate Change Fund Evaluation Framework. The purpose of the evaluation was to evaluate the extent to which program design and delivery mechanisms have been effective and efficient, and to assess progress in delivering intended outputs and outcomes to date. The evaluation involved conducting interviews with the HASBUP team, other Departmental staff, service providers under the ESS, and other program stakeholders, as well as analysis of existing qualitative and quantitative data sources provided to us. We developed findings and recommendations in collaboration with the Department which have informed future program design and delivery.

Latrobe Valley Home energy upgrades program evaluation
Sustainability Victoria

​Latrobe Valley Home Energy Upgrades (LVHEU) program focuses on helping low income households in the Latrobe Valley take up renewable energy and energy efficiency improvements. Delivered by Sustainability Victoria, the program responds to economic changes in the region due to the power station closure, which is also likely to compound existing socio-economic challenges. FPC is working with SV and key stakeholders to provide monitoring, evaluation, reporting and improvement services alongside LVHEU Program delivery. The focus of our work is to demonstrate the outcomes and value of the program, while contributing to ongoing learning and improvement. This involves extensive evaluation data collection and analysis—including participant surveys, stakeholder interviews and energy usage data analysis—and the preparation of midterm and final evaluation reports.

Local Government Energy Saver Program Evaluation
Sustainability Victoria

​The Local Government Energy Saver program (LGES) was funded by the Victorian Government’s Sustainability Fund and delivered by Sustainability Victoria (SV) between 2017 and 2020. The program assisted resource-constrained local governments in regional and rural Victoria to understand their greenhouse gas emissions profiles, implement actions to reduce their emissions and operating costs, and build the capacity of council staff. First Person Consulting, in collaboration with Point Advisory, worked with SV to conduct a final evaluation for the LGES program. This involved developing an evaluation framework, conducting in-depth interviews with council staff and program stakeholders, and analysing a range of existing data to demonstrate the environmental, social and economic outcomes of the program. A comprehensive evaluation report has been provided to SV and the Sustainability Fund, and will inform future program development in this area.

Public health and health promotion

Evaluation of the Social Innovator's Challenge
Movember

​The Social Innovators Challenge (SIC) is a grants-based program supporting the development of innovative and “outside the box ideas” that increase levels of social connectedness and feelings of belonging for men. The SIC was designed to address results of research by beyondblue that indicated men’s lack of social connections lead to social isolation and loneliness which have been identified as risk factors for anxiety, depression and suicide.
​
SIC is a multi-country (Australia, Canada and the UK), multi-year program. FPC is one half of a global evaluation team who will develop an overarching evaluation framework to guide evaluation of the program, provide support to funded projects in the collection of data, and will undertake project-level analysis and reporting.

Evaluation of the Prevention Partnership Program
Victorian REsponsible Gambling Foundation

​First Person Consulting has been contracted by the Victorian Responsible Gambling Foundation (VRGF) as an independent evaluation partner for the VRGF Prevention Partnership Program (PPP). The PPP aims to help prevent and reduce gambling-related harm within local communities by funding 14 projects across Victoria. The 14 projects will receive a combined total of $3.87 million between 2017 and 2019.  These projects will work with diverse community groups in urban and regional Victoria, and will build social connection and resilience to prevent and reduce gambling harm in local communities and among at-risk groups. FPC will be conducting process and outcome evaluations of the Program and its 14 funded projects.

Development of a Monitoring & Evaluation Framework for a health promotion and prevention plan
IPC HEalth

​IPC Health has engaged First Person Consulting to facilitate the development of a Monitoring & Evaluation Framework to support the implementation of their Health Promotion and Prevention Plan. FPC will support IPC Health to develop key evaluation questions and indicators, as well as identifying a diverse range of data collection opportunities throughout the lifetime of the Plan. 

Evaluation of the Ride or Walk to School Program
ACT Health

​FPC have recently been working with ACT Health to evaluate the Ride or Walk to School public health initiative, which aimed to improve rates of active travel by school children in 50 schools across the ACT. This includes reviewing data collection tools created by ACT Health, identifying and developing new tools for collecting data against the key evaluation questions and collating and analysing state (territory)-level indicators of active travel as a comparison point for those schools involved in the program. FPC will be working with ACT Health to develop a peer-reviewed paper from this evaluation in early 2017.

The evaluation report can be found online.

Evaluation of the Early Years Kitchen Garden Program Model
Stephanie Alexander Kitchen Garden Foundation

FPC has been engaged by the Stephanie Alexander Kitchen Garden Foundation (SAKGF) to help develop and evaluate a model of the kitchen garden program specifically for the early years sector. The approach draws on the principles of design and developmental evaluation, and will include working with SAKGF staff throughout the development, piloting and scaling of the model. FPC will also implement an evaluation capacity building program with SAKGF staff to further embed evaluation in the organisation. ​

Evaluation of the Local Prevention Program
Victorian Responsible Gambling Foundation

​First Person Consulting are leading the evaluation of the Local Prevention Program (LPP) for the Victorian Responsible Gambling Foundation (VRGF). The LPP is a $6.5 million grants-program being delivered over three years and is the foundation's flagship public health initiative. The grants are provided to different community-oriented organisations to deliver a variety of primary, secondary and tertiary prevention activities across the state.

FPC's role at the project-level includes the facilitation of logic models and evaluation plans for each of the 17 funded projects and supporting the project teams in the development and implementation of project-level evaluations. At the Program-level, FPC developed the Evaluation Framework guiding the overall evaluation, which includes a synthesised Program logic and defines a process for the synthesis of project-level outcomes data to make sense of the collective impact of the Program.

This is a three year project due to finish in August 2017.

Evaluation of the Entrepreneurs: It's Your move Program
ACT Health

​Entrepreneurs: It’s Your Move (EIYM) is an ACT Government initiative that enables high school students to develop innovative solutions for improving the health of the school community.
 
The EIYM learning materials will be available from 2017 onwards, for years 9 and 10. These learning materials aim to build the health and social entrepreneurs of the future. Over one semester (two terms), students will use design thinking to develop and implement their own health projects. Schools will have access to financial grants, digital tools, classroom resources linked to the Australian Curriculum, online teacher professional learning with TQI accreditation, business and community mentors, networking and ongoing support.

First Person Consulting has been engaged by ACT Health to design an evaluation plan, data collection tools and analysis approach for the Program. The approach taken has helped to ensure the evaluation approach is aligned to the needs of both ACT Health, as well as the teachers and students who will be involved in the Program.​

Evaluation of the School Crossing Supervisor Program
Transport Canberra and City services

First Person Consulting (FPC) has been engaged by Transport Canberra and City Services to undertake an evaluation of the School Crossing Supervisor Program (SCSP). The SCSP involves the implementation of crossing supervisors at 20 sites across the ACT. The SCSP has two primary objectives: to increase safety for children around participating ACT schools; and to lessen parent concerns about safety, which will create school environments that are more conducive for active travel and help to create healthier and safer school communities. Through of a controlled trial, FPC will determine the extent to which the SCSP results in attitudinal change among parents’ of participating schools, and thus the overall effect of schools having access to school crossing supervisors.

Evaluation of the Referral Decision Aid
Healthwest partnership

In 2017, the Pathways for Children with Developmental Delays (PCDD) Alliance developed and launched the Referral Decision Aid in an attempt to create a clear and easy-to-use guide that would assist referrers in Melbourne's west with making appropriate referrals for children under five with suspected developmental delay. HealthWest Partnership, on behalf of the PCDD Alliance, engaged First Person Consulting to carry out a process and impact evaluation, exploring the short-term outcomes of the tool, as well as documenting key lessons learned in the process that can inform future work of a similar nature in Melbourne's west and beyond. 

Evaluation of the dad in progress podcast series
Movember

​It is well recognised that significant events in a man’s life, such as transitioning to fatherhood, can have a profound impact on their mental health. Many social norms de-emphasise the importance of men looking after their mental health at this time and make it difficult to ‘reach out’ if they are having a tough time. Movember, as the leading global charity dedicated to men’s health, wanted to explore the opportunities for digital products to address this need. As a result, Movember, in partnership with the Betoota Advocate, developed the Dad in Progress podcast series. FPC was engaged to undertake an evaluation of the podcast series to explore the ways in which dads benefited from listening to the podcast, and how the podcast can best benefit new dads in the future. This involved several rounds of surveying and three focus groups with new dads and dads of toddlers. Overall, the podcast was very well received with dads reporting that hearing the stories helped normalise their own experience and reassure them that they were ‘not alone’ in their fatherhood journey.

Developmental evaluation of the secondary years kitchen garden project
Stephanie alexander kitchen garden foundation

​The Stephanie Alexander Kitchen Garden Foundation (SAKGF) Program has enabled primary school students to grow, harvest, prepare and share seasonal fresh food since its inception in 2008. FPC is working with the Stephanie Alexander Kitchen Garden Foundation to evaluate their Secondary Years pilot project. This pilot project is funded by AstraZeneca and aims to explore and test the ways in which a kitchen garden program could be implemented in secondary schools and what a Stephanie Alexander Kitchen Garden Foundation program for secondary schools could look like.  FPC walked alongside the pilot project, collecting data in cycles, with insights and reports shared with the organisation as the pilot program developed.  

Evaluation of the Victorian Tumour Summits Project
North East Integrated Cancer Service

​The Victorian Tumour Summits Project (VTS) is comprised of clinician-led, tumour-based forums that aim to identify unwarranted variations in cancer care outcomes and practices. FPC was engaged to evaluate the extent to which VTS is identifying and reducing unwarranted variations in cancer care. It is envisaged that the evaluation will inform future iterations of the VTS as a mechanism for engaging clinicians in collaborative state-wide work.

Financial Literacy, financial counselling and financial assistance

Evaluation of the REcovery Assistance Program
Victorian responsible gambling foundation

The foundation has engaged First Person Consulting to examine the appropriateness and effectiveness of the Recovery Assistance Program model within the context of the new Gambler's Help Services model and other programs in Victoria. The Recovery Assistance Program provides material assistance to those who have experience gambling-related harm and are in recovery. The review incorporates both process and outcome evaluation elements, as well as a literature and practice review of similar programs and an environmental scan of agencies and services in Victoria offering similar or complementary services. 

'You're the boss' Financial literacy program evaluation
The Salvation Army

​For some groups of people, financial literacy is a barrier to their making effective use of financial services and taking control of their lives. Potential flow on impacts can include decreases in personal wellbeing, social inclusion and the ability to handle situations when 'life happens'. The Salvation Army delivers the financial literacy program ‘You’re the Boss’ to over 2,000 people per year who come from very different backgrounds and situations. First Person Consulting has been engaged to conduct a process and outcomes evaluation; a process evaluation examining what works, potential improvements and documenting the experiences of those involved, and an outcomes evaluation examining financial literacy changes among participants. 

Financial Literacy Program evaluation and mentoring services
Wingate avenue community centre

​The Wingate Avenue Community Centre recently secured a grant from Financial Literacy Australia to deliver financial literacy programs to newly arrived migrants and refugees. Over 18 months the project aims to reach 800 newly arrived migrants and refugees through the delivery of 320 classroom sessions within existing courses and community forums. FPC is mentoring staff from Wingate Avenue Community Centre, which commenced with a workshop developing an evaluation plan and involves ongoing mentoring support in the development of data collection tools, data analysis and reporting. FPC analysed and reported on the data on Wingate's behalf.
​
The report has been published online here.

B-empowered financial care and coaching program
Crossway Lifecare ltd

​B-Empowered is a confidential, personalised financial coaching program focused on helping individuals take control of their finances. It is a free service open to anyone in the community, and typically involves one-on-one meetings with a financial case worker. FPC is working with Crossway LifeCare Ltd. on the evaluation of the Program, examining the effectiveness of the program and what works for who and why in the delivery of the program content. Resulting from this project will be a report that captures the strengths and lessons of the program and the impact on participants.

The evaluation report is available online.

Waste management and Litter

Final Evaluation of the Litter Hotspots Program
Metropolitan Waste and REsource Recovery Group

Between 2014 and 2017, the Litter Hotspots Program has supported 25 community organisations and local governments in Victoria—influencing leading litter prevention practice and yielding strong partnership, community engagement and litter prevention outcomes. First Person consulting worked with the Metropolitan Waste and Resource Recovery Group (MWRRG) to undertake a Final Evaluation of the Litter Hotspots Program in line with the Department of Treasury and Finance's Lapsing Program Guidelines. This involved a document review and gap analysis, engagement with representatives from funded organisations, Sustainability Victoria, Melbourne Water and other key stakeholders, as well as data analysis and synthesis, leading to the development of key insights, lessons learned and recommendations for future program delivery.

The evaluation report can be found online here.

​MONITORING, EVALUATION, REPORTING AND IMPROVEMENT FOR THE OFFICERS OF THE PROTECTION OF THE LOCAL ENVIRONMENT (OPLE) PILOT
EPA VICTORIA

Between 2017 and 2020, FPC worked with EPA Victoria to develop and implement a MERI framework for the Officers for the Protection of the Local Environment (OPLE) Pilot. The Pilot trialled the placement of EPA authorised officers within 23 local councils to manage small-scale pollution reports that had not been addressed by either local government or EPA. The FPC team worked closely with OPLE Pilot staff and program stakeholders to develop a program logic, evaluation questions, key performance indicators, and data collection and analysis processes. We undertook a series of interviews, surveys and workshops with multiple stakeholder groups at key points throughout program delivery. We delivered three interim reports that contributed to ongoing improvements and adaptive management throughout the Pilot. Our MERI activities resulted in an overall evaluation report on the effectiveness of the OPLE Pilot, demonstrating the value of the approach and identifying lessons learned for state-wide roll-out going forward.

Evaluation of the Metro Fund
​Metropolitan Waste and Resource Recovery Group

​​FPC worked with MWRRG to undertake a full evaluation of all three funding rounds of the Metropolitan Local Government Waste and Resource Recovery Fund (Metro Fund). This is a $5.5 million Victorian Government initiative that aims to assist councils in metropolitan Melbourne to implement best practice waste collection and management systems. Patrick Gilmour led the team to undertake an in-depth evaluation of the 33 council projects and 15 MWRRG-led strategic projects supported by the Metro Fund, focusing on project strengths, weaknesses, achievements and opportunities for adaptation. The evaluation also improved understanding of the value of the Metro Fund among the MWRRG, Department of Environment, Land, Water and Planning, and Sustainability Victoria, including whether the fund achieved its objectives, supported grant recipients and further insights regarding future implementation of similar programs.

The full evaluation report is available online.

Emergency management

Evaluation of the councils and emergencies project
Local government Victoria, department of environment, land, water and planning

​The Councils and Emergencies Project (CAEP) aims to enhance the emergency management capability and capacity of local government, and is delivered by the Department of Environment, Land, Water and Planning, Local Government Victoria (LGV). The project design involves a three-phased approach that spans four years, from 2015 until 2019, focused on defining the roles and responsibilities of councils in emergency management (Phase 1), understanding councils’ capability and capacity for emergency management (Phase 2), and developing tailored strategies to enhance the capability and capacity of councils based on the outcomes of Phase 2 (Phase 3). FPC’s role was to undertake an evaluation of the CAEP progress to date and to provide DELWP with reasoning and justification for delivering the project through to completion as per the Department of Treasury and Finance’s (DTF) Lapsing Program Guidelines (LPG). The methodology for the evaluation involved review of key project documents and consultation with stakeholders from across state and local government who were directly involved in Phase 1 and Phase 2 of the CAEP.  

Development of a Monitoring, Evaluation and reporting framework for safer together
Department of Environment, land, water and planning

The key recommendations from the Inspector-General for Emergency Management’s 2015 review of bushfire management on public land highlighted the need to move to risk-based bushfire management. Safer Together is a program that involves moving towards this risk-based management approach. Safer Together has four priority themes; FPC is involved with the 'measuring success' priority theme. FPC is currently developing a monitoring, evaluation and reporting framework (MERF) for the Program, which involves facilitating workshops with DELWP staff to inform the MERF's content.One of the key recommendations from the Inspector-General for Emergency Management’s 2015 review of bushfire management on public land highlighted the need to move to risk-based bushfire management. Safer Together is a program that involves moving towards this risk-based  management approach. Safer Together has four priority themes; FPC is involved with the 'measuring success' priority theme. FPC is currently developing a monitoring, evaluation and reporting framework (MERF) for the initiative.

Evaluation of the Emergency Management cluster pilot and the Municipal emergency resourcing program
Local government victoria, department of environment, land, water and planning

Councils across Victoria have a range of responsibilities around emergency management and planning. In recognition of the potential burden this places on some councils, Local Government Victoria (LGV) and the Municipal Association of Victoria (MAV) provide targeted assistance and support to councils to meet these obligations.

​First Person Consulting, together with Point Advisory, has been commissioned by LGV to evaluate two approaches for providing this support: 1) the Emergency Management Cluster Pilot; and 2) the Municipal Emergency Resourcing Program. The evaluations aim to help LGV and MAV improve the support they are providing to the local government sector. These two reports are public.

The Emergency Management Cluster Pilot evaluation and the Municipal Emergency Resourcing Program evaluation.

evaluation of the st andrews conversations
Department of environment, land, water and planning

The impact of bushfire, particularly Black Saturday, have emphasised the need for a paradigm shift in how community-based disaster preparedness and emergency management is operationalised. The St Andrews Conversations (SAC) is an example of this Community-Based Bushfire Management (CBBM) approach, which seeks to involve communities in decision making around bushfire management and planning drawing on the principles and processes of community development. The SAC involved a series of place-based, facilitated conversations among residents of the St Andrews community. Over a series of conversations, it was intended that participants would learn more about each other, how bushfire management planning, response and recovery affects them, what they can contribute and how future events might be managed better. FPC was engaged to undertake an evaluation of the SAC using a desktop-based case study approach to document the process and outcomes of the SAC for sharing with other CBBM practitioners across Victoria.

strategic bushfire management planning consultations - analysis and reporting
department of environment, land, water and planning

Strategic bushfire management planning (SBMP) brings together land and fire managers, communities and stakeholders across regions to inform appropriate responses to bushfire management. DELWP, on behalf of the Metropolitan Region Strategic Bushfire Planning Coordination Committee (M10) has been undertaking community consultation across the region to gather feedback on the M10’s objectives for strategic planning, feedback on the strategy options developed and ultimately feedback on the proposed final strategies. FPC has been engaged to produce summary reports at each stage of the engagement process as well as producing an overall summary report of the engagement process.​

Community-based bushfire management Evaluation
Country fire authority and the department of environment, land, water and planning

The CFA has engaged FPC to undertake an evaluation of community development programs that aim to increase understanding of bushfire risk and involvement of community in disaster preparedness, preparation, response and recovery. FPC has developed an evaluation plan and conducted a range of interviews and workshops with program staff, other agency staff and community members. This project worked with regional and remote community members to understand the success of community development programs that aims to increase resilience to bushfire.

wye river-jamieson track fire evaluative case study
department of environment, land, water and planning

FPC was engaged by DELWP to do an evaluative case study of the community development work at Wye River, Separation Creek, and Kennett River and how that has impacted the bushfire response and recovery to the Wye River-Jamieson Track Fire, and further bushfire preparation and preparedness. FPC have conducted in-depth interviews with community members who were involved in bushfire preparedness, response and recovery efforts and discussed the impact of the community development work on their resilience to disaster. 

Annual Brigade review pilot project evaluation support services
Country fire authority

​FPC is currently contracted to support the Country Fire Authority (CFA) Victoria through provision of specialist evaluation advice in relation to the Annual Bridge Review (ABR) Pilot Project. The ABR Pilot Project is designed to assess the health and capability needs of volunteer bridges to inform capability planning, tailor support services and ultimately improve risk reduction outcomes. This consultancy includes collaborative development of a data framework to support evaluation, facilitation of focus groups to gather evaluative data, and review of data coding and analysis done by the ABR team. FPC is providing independent oversight to ensure the ABR evaluation process is sound and contributes to ongoing compliance with Section 29(b) of the Country Fire Authority Act 1958.  

FIRE ready Victoria PROGRAM EVALUATION
COuntry Fire Authority

Fire Ready Victoria (FRV) is an information-based community bushfire safety program delivered by the Country Fire Authority (CFA). The program targets stakeholders in high bushfire areas and special interest groups through community meetings delivered by trained facilitators in local venues — with the aim of increasing participants' understanding of bushfire risk, preparedness and planning, and encourage them to seek further information and to implement necessary preparedness activities. FRV had been operating for over ten years, and FPC were engaged to undertake an evaluation of program implementation, effectiveness and appropriateness for communities. The evaluation was focused on understanding how the FRV program works, for whom and why, and involved conducting an online survey, in-depth interviews, workshop facilitation and analysis of a range of existing qualitative and quantitative evidence. Working closely with the Community Engagement team, our work complemented a concurrent internal program review and informed decisions about future program design and delivery.

​Other sectors

Review of the alpine resorts strategic plan
ALPINE RESORTS CO-ORDINATING COUNCIL

Victoria’s six alpine resorts are managed through Alpine Resort Management Boards (ARMBs). They are supported in this by the Alpine Resorts Co-ordinating Council (ARCC) and guided by the Alpine Resorts Strategic Plan. This five-year plan was established in 2012 under the Alpine Resorts (Management) Act 1997. It expired at the end of 2017, and FPC was engaged by the ARCC to undertake an independent review of the implementation of the plan and delivery against its objectives and actions. This involved consulting with key stakeholders including ARMBs, ARCC, DELWP, other relevant agencies and alpine industry stakeholders, as well as analysing and synthesising existing monitoring data and reports. Through the review, we identifed and documented lessons and key insights that can be used to inform future plans, strategies or co-ordination of the alpine resorts.

You can find a copy of the Review online here.

​'TELLING THE L2R STORY', EVALUATION SUPPORT AND CAPACITY BUILDING
L2R

Established in 2009, L2R is a not-for-profit organisation in Melbourne's west that works to increase the mental and physical wellbeing of at-risk young people through participation in the arts - primarily by providing free dance classes to young people from culturally and linguistically diverse, newly arrived and at-risk communities. L2R engages with the local community to advance local arts and culture, and builds the capacity of young people to be the next generation of leaders, through mentoring, linking young people to services, and creating pathways into employment within the local arts and community sectors. On a pro-bono basis, First Person Consulting is working with L2R staff, volunteers and board members to provide evaluation support and capacity building to help L2R measure, demonstrate and improve the impact of their work.

Conducted energy device training evaluation
Victoria Police

​Victoria Police (VicPol) implemented a pilot training program for Conducted Energy Devices (CEDs) prior to a state-wide roll out. The training program developed and prepared participants knowledge and skills to safely and effectively deploy a CED in an operational capacity. FPC undertook an evaluation drawing on the Kirkpatrick Training Evaluation Model. The approach enabled the evaluation to determine the extent to which the trainees are applying their learning to their work. The approach resulted in identified opportunities for VicPol to reinforce the learning and practice change that resulted for trainees.

Collaborative networks pilot program evaluation
department of economic development, jobs, transport and resources

​The Collaborative Networks Pilot Program (CNPP) was a $1.5 million initiative designed to capitalise on Victoria’s innovation and technology strengths. It funded four networks in the textile and clothing, dairy, carbon fibre and biotechnology sectors to enhance collaboration between industry and researcher and improve the uptake of technology in small to medium enterprises. In collaboration with Point Advisory and Roberts Evaluation,

​First Person Consulting (led by Patrick Gilmour) evaluated the CNPP against the Department of Treasury and Finance’s lapsing program guidelines. Given the diffuse nature of many of the outcomes Patrick Gilmour designed a retrospective evaluation framework to identify gaps in the data collection processes and ensure that sufficient evidence was present to answer each of the evaluation questions. This was important as EcoDev was particularly interested in determining whether the investment was a worthwhile use of public funds.

Evaluation of the outreach to women in the sex industry project
project respect

Project Respect is a not-for-profit organisation that supports women in the sex industry through direct outreach, practical assistance, research, advocacy, community education and support stream referrals. FPC has been engaged to provide an independent evaluation of the Outreach to Women in the Sex Industry Program funded by the Commonwealth Attorney- General’s Department over the last three years. The Program has  involved providing outreach services to support women in the sex industry in Melbourne to assist in recovery from human trafficking, while also raising awareness of trafficking into the sex industry. They also provide support to address co-occurring issues such as family violence, sexual assault, addiction and mental health. FPC will work with Project Respect to evaluate the results and achievements of the program, and identify insights, learnings and opportunities for the future.

Evaluation of the Peter Cullen Trust Leadership programs
Peter cullen trust

​The Peter Cullen Trust (PCT) leadership programs seek to contribute to more sustainable and equitable water and environmental management by improving participant skills and leadership capability. FPC worked with PCT to develop and implement an evaluation plan to explore the impact of the programs at three level: on the individual, the organisations they work for, and intended system level changes relevant to water and environmental management and decision making. Evaluation findings demonstrated the value of the program and provided an opportunity to explore how the program can be improved. Fit-for-purpose recommendations were developed to inform the PCT strategic plan and ensure continued contribution to the goals and mission of PCT.

Evaluation of the Crisis and Comfort care packs project
Restoring hope ltd.

Restoring Hope was established in 2014 to meet a critical gap in the support of children and young people who have been impacted by sexual abuse. To support this work, Restoring Hope received a grant from the Collie Foundation to support the continued distribution of crisis and comfort care packs to frontline agencies supporting children and young people in a time of crisis. FPC was engaged to evaluate the project, and to provide support to Restoring Hope to identify improvements to the process of delivery, and to understand the benefits of the project to those frontline agencies. 

If you're currently monitoring, evaluating or planning for an evaluation please get in touch with our expert team to chat (with no obligation) about how we can help.