ARCHIVED - Integrated Strategy on Healthy Living and Chronic Diseases - Community Based Programming Functional Component
Public Health Agency of Canada
Project Evaluation and Reporting Tool (PERT)
Complete Questionnaire
Canadian Diabetes Strategy
Community-Based Program
2006-07 Funded Projects
Introduction
The Public Health Agency of Canada (PHAC) Project Evaluation and Reporting Tool (PERT) is a form that has been developed to help monitor and document the effectiveness of all participating PHAC programs, and to assess the impact these community-based programs are making on the health of Canadians and Canadian communities.
PHAC recognizes that it is important to collect the same information, using the same methods, in order to provide realistic and accurate portraits of the outcomes of and value created by participating programs. As a result, PHAC has developed a common analysis framework from which questions relevant to all community-based programs have been developed. In addition, individual programs have developed questions specific to particular program outcomes.
The information gathered by this form will be used to measure and assess the implementation, impact, and effectiveness of PHAC-funded programs. Program level results and lessons learned may be shared with the projects, their partners, with researchers, and within PHAC, unless stated otherwise. Access to individual project forms will be controlled – only your PHAC Program Consultant, their Manager, and the Program Evaluation Consultant will have access to individual project forms. If access to project information is requested by another source (e.g., by a researcher), your consent will be sought.
Please refer to the instructions on the following pages before completing this form. Definitions are provided in the User Guide to assist you in answering the questions.
Thank you for taking the time to complete this form. The information you provide is very important to the Public Health Agency of Canada.
Instructions
What is the PHAC Project Evaluation and Reporting Tool (PERT)?
This Project Evaluation and Reporting Tool (PERT) covers 10 categories:
- Target Populations
- Resources and Sustainability – Human, and Other
- Partnerships
- Target Population Involvement
- Education/Awareness/Outreach Activities
- Training for Paid Staff and Volunteers
- Media Coverage
- Action on Policy
- Project outcomes
- Evaluation Reporting and Dissemination
Please discuss the questions with your project team and record your answers in the spaces provided. Please note that the information collected under “General Information” is mandatory.
On-Line Form
The on-line form is not available at this time.
Paper Form and Returning PERT
The Project Evaluation and Reporting Tool (PERT) is available in Word format. Please answer the questions using as much space as required, save, and email the form to your Program Consultant by the date indicated in the covering letter. If you have any questions, contact your Program Consultant.
General Information
The information below will be used to identify your project and the individual to contact in case clarification is required.
Today’s date (month/day/year):
Name of project:
Name of project sponsor:
(The group that signed the grant or contribution agreement.)
Project number:
Name of Funding Program:
Reporting period:
First 6 months
6 months (end of project)
Participation in Completing the Form: Please record who participated in the process of completing this form and their relationship to the project. (Only provide contact information for the lead person.)
Name(s) of those completing this form and their relationship to project:
Lead person:
Lead person’s telephone #:
Lead person’s facsimile #:
Lead person’s email:
Project Planning: (To be completed by the PHAC Program Consultant and finalized by the project.)
Please describe the sources of information used to design and develop the project.
Needs assessment: Please describe.
Project experience: Please describe.
Collaborative planning: Please describe.
Research, literature review, surveillance information: Please describe.
Other: Please describe.
Section 1: Target Populations
Question #1
1a.
Please list your project objectives (please refer to your project work plan).
1b.
Please provide a short summary of project activities during this reporting period.
1d.
Projects are generally designed to meet the specific needs of a particular group within the population. Please indicate the group(s) that will ultimately benefit from your project.
Risk Factor / Disease Status (check all that apply) | Life Stage (check all that apply) | Setting (check all that apply) | Other Characteristics (check all that apply) |
---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Section 2: Resources and Sustainability – Human, and Other
Question #3
In this section, please identify sources of in-kind contributions (3b.).
3b.
In-kind Contributions for this Project: Please estimate the dollar value of all “in-kind” contributions (e.g., loaned staff, meeting rooms, etc.) received in the last six months, rounded up or down to the nearest dollar.
Note: The estimated value of the “in-kind” contribution is the amount that you would have had to pay if you were to purchase the service or item.
In-kind Contributions | Complete name of organization (department, foundation or other organization) | Total $ Amount |
---|---|---|
Loaned staff |
||
Facility |
||
Travel, living expenses |
||
Telephone/Internet |
||
Project materials (videos, food, etc.). |
||
Administrative costs (office supplies, printing, postage, etc.) |
||
Project equipment (office equipment, furniture, etc.) |
||
Other (specify) |
||
Total In-kind Contributions |
Question #4
4a.
Do you anticipate any aspects of your project to continue after the PHAC funding ends?
Yes → go to 4b.
No → go to question 5.
4b.
yes, please check which mechanisms are in place to continue the activities:
Integration of project activities into the organization
Partnerships
Transfer of activities to another organization
Other, please describe.
4c.
Please describe the aspects that will continue.
4d.
PHAC may have the opportunity to undertake follow up on the lasting effects and benefits to a community from a funding program. If aspects of your project are continuing when PHAC project funding ends, can we follow up with you at a later date?
Yes
No
Question #5
5a.
Have any new or additional programs, activities, and/or services been developed in your community as a result of your project?
Yes →go to 5b.
No →go to question 6.
5b.
If yes, please describe.
Question #6
6a.
What is the total number of all volunteers for this project over the last six months?
6b.
Please provide the total number of hours all project volunteers have contributed over the last six months.
6c.
Over the past six months have you recruited project volunteers?
Yes →go to 6d.
No →go to question 6e.
6d.
If yes, how many new volunteers were recruited for this project over the past six months?
6e.
Please provide any additional information you would like to share about your volunteer program.
Section 3: Partnerships
Question #7
“Collaborative Partners” are two or more groups/organizations working together on a common goal.
7.
Please identify your key collaborative partners during the last six months and describe and results that have been achieved from working with these partners.
Name of Partner Organization | Type of Organization (check one only) | Geographic Level/Scope of Partner Organization (check one only) | New or Existing Partnership (check one only) | Primary Area of Focus for Partner Organization (i.e., sector) (check one only) | Results Achieved from the Partnership specifically for this Project |
---|---|---|---|---|---|
|
|
|
|
Section 4: Target Population Involvement
Question #8
“Target population” is defined as the people and/or organizations that a program is intended to reach. Please be as specific as possible.
8a. Over the last six months, please indicate how members of your target population(s) have contributed to the management and/or delivery of this project. Check as many boxes as apply.
- Not applicable. Target population(s) did not contribute to the management and/or delivery of this project.
- Target population(s) were given informal opportunities to express their views or opinions about the project.
- Target population(s) were given formal opportunities to express their views or opinions about the project (needs assessments, project evaluation, interviews, focus groups).
- Target population(s) played a volunteer role in the delivery of programs.
- Target population(s) were directly involved in a committee or group that provides advice to the governing body.
- Target population(s) were directly involved in making decisions about the project (e.g., by sitting on an advisory or governing body).
- Target population(s) were employed by the project.
- Other; please specify.
8b. Please provide an example, or highlight an area of target population involvement in your project over the last six months.
8c. If members of your target population(s) sat on a governing body that made decisions about your project (advisory, steering, and/or board) during the reporting period, what proportion of the governing body was made up of members of the target population(s)? (Check only one response.)
Less than 25%
Between 25% and 50%
More than 50%
Section 5: Education/Awareness/Outreach Activities
Question #9
9a.
check the population(s) targeted by your project. (Check all that apply.)
Individuals or communities who face specific risk condition(s) and/or are living with specific disease(s)
Practitioners, professionals, and/or other service providers
Policy makers
General public
9b.
Please describe the above population(s) targeted by your project. Please indicate how many individuals were reached in each target population during the last six months and an estimate of the total number of potential people in your target population(s). Please provide or describe any information that supports the estimated total number.
Population(s) targeted by your project/initiative (Individuals)
Total number of individuals reached in each target population
Estimated total number of individuals in each target population
(see User Guide for more details)
9c.
Please describe the approaches you used that were successful in reaching your target population(s) (e.g., provided honoraria, provided programs where your target population(s) meet, extended hours of operation in your project).
9d.
If there were barriers to reaching any of the target population(s), please describe.
9e.
Please describe any lessons learned or any recommendations you have related to reaching target population(s), from which other projects could benefit.
Question #10
We would like to learn more about the types of education, awareness, and/or outreach
activities provided by your project.
“Outreach” is defined as going to where community members socialize, gather, or casually walk in. Outreach activities include community events, educational groups, meetings, consultations, presentations, etc.
10.
Please provide the number of each type of education, awareness, and/or outreach activity provided, as appropriate, over the past six months. Describe the activity you have undertaken and its purpose, the setting in which the activity took place (e.g., community, school, etc.), and the number of target population members you reached through that activity.
Types of activities | # | Description and purpose | Setting in which the activity took place | Number of target population reached |
---|---|---|---|---|
Community Events (e.g., health fairs, walks/runs to raise awareness) |
||||
Community Programs (e.g., ongoing activities such as peer support, outreach to people who inject drugs, skill-development programs like cooking classes) |
||||
Workshops/ |
||||
Activities to |
||||
Other; please specify
|
Question #11
11a.
Has your project developed or distributed educational, awareness and/or outreach products over the past year?
Yes → go to 11b.
No → go to question 12.
11b.
If yes, please note the type of product, check it if it was produced by your project, provide the title and description of the product, and fill in the intended audience (including specific target population), number produced, number distributed, and whether or not you know of any other organizations that have used the product:
Type of product | Check (√) if produced by your project | Title/ description | Intended audience | Number produced | Number distributed | Other org’s used this product (Yes/No/ Don’t Know) |
---|---|---|---|---|---|---|
Manuals/
|
||||||
Brochures/
|
||||||
Communiques/ |
||||||
Presentations
|
||||||
Web site
|
N/A |
N/A |
||||
Position Papers/ |
||||||
Other; please describe
|
Section 6: Training for Paid Staff and Volunteers
Question #12
12a.
Have you provided paid staff with any training and/or learning activities/opportunities, specific to the project, over the past year?
Yes → go to 12b.
No → go to question 13.
12b.
If yes, please describe the staff training and/or learning activities/opportunities (e.g., in-house training, conferences, external training) that were provided and the number of people who attended.
Staff training and/or learning activity/opportunity
Number of attendees
12c.
How have staff training and/or learning activities/opportunities contributed to the project?
Question #13
13a.
Have you provided volunteers (including advisory committee/board members) with any training and/or learning activities/opportunities, specific to the project, over the past year?
Yes → go to 13b.
No → go to question 14.
13b.
If yes, please describe the volunteer training and/or learning activities/opportunities (e.g., in-house training, conferences, external training) that were provided and the number of people who attended.
Volunteer training and/or learning activity/opportunity
Number of attendees
13c.
How have volunteer training and/or learning activities/opportunities contributed to the project?
Section 7: Media Coverage
Question #14
14a.
Please provide examples from the past six months of your attempts to get media attention for your project.
14b.
Have you had any media coverage related to your project over the past six months?
Yes → go to 14c.
No → go to question 15.
14c.
If yes, please indicate the type and number of media coverage activities (e.g., number of articles in magazines/newspapers, news stories on radio/television/web sites, etc.):
Type of media coverage | Number |
---|---|
News articles/stories in magazines/newspapers |
|
News articles/stories on radio/TV/web sites |
|
Responses to letters to the editor |
|
Other; please describe |
Please attach some examples of the news articles, news stories, etc. Please indicate here what you have attached.
14d.
Please describe any lessons learned or any recommendations you have about attracting media coverage, from which other projects could benefit.
Section 8: Action on Policy
Question #15
If your project has influenced policy development or implementation or has supported existing policies (e.g., smoking ban, needle exchange program, etc.), we would like to know about it. Some examples of policies and policy areas include methadone access in prisons, nutritional food in schools, minimum wage, etc.
15a.
As an outcome of your project, did you intend to influence policy development, policy implementation, or support of existing policies? (If your project did not intend to influence policy, but you did, you can provide details in question 15e.)
Yes → go to 15b.
No → go to question 15e.
15b.
If yes, please describe the main policy or policy area(s) that your project intended to influence.
15c.
Please indicate whether or not your project used any of the following approaches to increase community capacity to influence policy during the past six months. If yes, please describe what you did and who you reached.
Provided training on how to influence policy.
Yes
No
If yes, please describe.
Developed resources/tools for use by communities in policy analysis.
Yes
No
If yes, please describe.
Developed a new group to work on policy.
Yes
No
If yes, please describe.
Other action to build the community’s capacity to influence policy.
Yes
No
If yes, please describe.
15d.
Please indicate what actions were used to influence policy during the past six months.
Developed a working relationship with a government or community representative who can provide access to the policy process.
Yes
No
If yes, please describe what you did and who you reached.
Held meetings with policy makers such as school boards, town/city councils, public health authorities, and/or community organizations.
Yes
No
If yes, how many meetings and please describe.
Participated in policy development process with policy makers.
Yes
No
If yes, please describe and note the jurisdictional level (e.g., organizational, community, municipal, provincial, federal, etc.).
Presented briefs or position papers (e.g., to decision makers, general public).
Yes
No
If yes, how often, to whom, and please describe.
Prepared and distributed a policy research report.
Yes
No
If yes, please describe.
Took other actions to influence policy.
Yes
No
If yes, please describe.
15e.
Do you have any information that demonstrates that your project activities were successful in influencing policy (whether intended or not)?
(Some examples:
- Policy makers in community and/or government sectors have put a process in place to address your project’s policy recommendations.
- Policy makers have taken specific follow-up action on your project's recommendations.
- Policy makers have implemented policies that take account of your project's recommendations.)
Yes
No
If yes, please describe and attach any relevant documents.
Section 9: Project Outcomes
Question 16 is a question common to all programs. Other questions in this section are specific to the particular program that funded your project.
Question #16
16a.
Over the past year, has your project monitored changes in your target population(s)’ access to health services?
Yes
No
N/A
16b.
If yes, please provide information on what changes your project was measuring, what information was tracked to measure these changes, and what you learned.
16c.
What difference has your project made, that is, what changes have occurred as a result of your project? How do you know (what information was tracked to measure these changes)?
Section 10: Evaluation Reporting and Dissemination
Question #19
19a.
Have you produced a final evaluation report?
Yes
No
19b.
If no, please describe the status of your final evaluation report.
Status:
19c.
If yes, please attach a copy of your final evaluation report.
19d.
How do you plan to use your evaluation findings?
Question #20
In this section, please describe how you actively disseminated the results and learnings of your project and to which audiences. (Examples of methods of dissemination include presenting a paper/report, providing fact sheets, article submissions, conference presentations, hosting workshops or meetings, press releases, etc.).
20.
Please check off the audiences that apply and describe the corresponding distribution method(s):
Actively disseminated to the following audience(s): | Check (√) |
Method(s) of dissemination |
---|---|---|
General population
|
||
Target population(s)
|
||
Partners/collaborators
|
||
Community-based organizations (that are not partners) |
||
Local government
|
||
Provincial/territorial government(s) |
||
Federal government
|
||
Members of your organization
|
||
Other; please identify
|
Interview Guide
Accountable Director of Diabetes
About the Interviewee
1.
Please describe your role and experience in relation to the Integrated Strategy for Healthy Living and Chronic Disease (ISHLCD), and particularly the Community-Based Programming (CBP) Functional Component.
Roles/Responsibilities
2.
In your view, are the roles and responsibilities between the National Office and the Regional Offices in supporting the implementation of the CBP clearly defined and understood by staff in both the National Office and the Regional Offices?
Probes:
- What are the key challenges in delivering the CBP regionally?
- How and where are roles and responsibilities in relation to the CBP defined? Are there key documents which can be referred to?
3.
Are the roles/responsibilities between the National Office and the Regional Offices clearly understood by key stakeholders (P/T governments, NGOs and other potential partners and funding recipients, etc.)?
Coordination / Process
Coordination between the National Office and the Regions
4.
Are you aware of the Community-Based Network? If so, in your view, is the Network effective in supporting the coordination between the National Office and the Regional Offices?
Probes:
- What are some of the lessons-learned from this Network?
Grants & Contributions Solicitation Process
5.
Is the process for solicitation and application for funding well established / formalized and understood clearly across the Regions and by key stakeholders (P/T governments and potential funding recipients)? How do you ensure this?
Probes:
- Can you provide examples of how it is formalized?
6.
Does PHAC have a preference of using grants over contributions within the ISHLCD?
Relevance
7.
Can you identify any overlap and/or duplication between the CBP and other Functional Components, or within the CBP Functional Component?
8.
Are you aware of any programs similar to the CBP within the public, private and/or volunteer sectors? In your view, does the CBP or any of its elements duplicate, or work at cross-purposes with other existing programs or policies either at the federal or provincial/territorial levels?
Probes:
- Should any of the work carried out under the CBP be transferred to another level of government or sector?
Success/Progress
9.
Can you identify any unintended positive or negative impacts of the CBP to date?
Design/Delivery
10.
Can you identify any gaps in the CBP Functional Component?
11.
In your view, is the current monitoring system for the CBP Functional Component effective?
12.
What would you describe as the greatest successes of the CBP to date? What have been the greatest challenges encountered to date?
Interview GuideProgram Managers
(National and Regional)
About the Interviewee
1.
Please describe your role and experience in relation to the Integrated Strategy for Healthy Living and Chronic Disease (ISHLCD), and particularly the Community-Based Programming (CBP) Functional Component.
Roles/Responsibilities
2.
Please describe the number of FTEs in the National Office and the Regional Offices and their general roles and responsibilities, in relation to:
- the Diabetes Community-Based Programming (CBP); and
- the broader ISHLCD.
Probes:
- Is there consistency in the level of FTEs among the Regions proportional to program funding and funded projects?
3.
In your view, are the roles and responsibilities between the National Office and the Regional Offices in supporting the implementation of the CBP clearly defined and understood by staff in both the National Office and the Regional Offices?
Probes:
- Can you provide examples of how the roles/responsibilities are understood by both national office and regional office staff? Is the level of clarity the same for all Regions?
- How and where are roles and responsibilities defined? Can you share examples of these documents?
4.
Are the roles/responsibilities between the National Office and the Regional Offices clearly understood by key stakeholders (P/T governments, NGOs and other potential partners and funding recipients, etc.)?
Coordination / Process
Coordination between the National Office and the Regions
5.
How are the National Office and Regional Offices’ community-based activities coordinated? What are the formal and informal mechanisms that are used to support the coordination and how effective are they?
Probes:
- How do you ensure knowledge and understanding of work underway in other Regions?
- Is the Community-Based Network effective in supporting the coordination between the National Office and the Regional Offices?
- are some of the lessons-learned from this Network?
6.
Is there consistency among the Regional Offices in the delivery of the CBP?
Probes:
- How is consistency ensured in communications with key stakeholders, support and information provided to funded programs as well as potential funding recipients, etc. across the Regions?
Grants & Contributions Solicitation Process
7.
Is the process for solicitation and application for funding well established / formalized and understood clearly across the Regions and by key stakeholders (P/T governments and potential funding recipients)?
Probes:
- Can you provide examples of how it is formalized?
8.
2006-07 Solicitation: Information received from the Northern Region indicates that the PHAC encouraged the use of grants over contributions for 2006-07 funding in order to allow the funding to be carried over beyond fiscal year end. Was this the case for all the Regions (i.e., did the PHAC make the same suggestion to all the Regions)? Was the ability to carry forward the funding the reason why all 41 projects funded in 2006-07 were grants?
Relevance
9.
Can you identify any overlap and/or duplication between the CBP and other Functional Components, or within the CBP Functional Component?
10.
Are you aware of any programs similar to the CBP within the public, private and/or volunteer sectors? In your view, does the CBP or any of its elements duplicate, or work at cross-purposes with other existing programs or policies either at the federal or provincial/territorial levels?
Probes:
- Should any of the work carried out under the CBP be transferred to another level of government or sector?
Success/Progress
11.
What activities have been undertaken under the “Partnership Development & Collaboration” action area of the CBP, as indicated in the Logic Model, since the implementation of the ISHLCD in the fall of 2005? Are there new partnerships or collaboration between the PHAC and its key stakeholders that have emerged as a result of activities under the CBP?
12.
What activities have been undertaken under the “Dissemination” action area of the CBP, since the implementation of the ISHLCD in the fall of 2005? What activities have been undertaken to support the transfer of the knowledge generated from the CBP, including best practices and evaluation findings?
Probes:
- To what extent is the information about current and past funded projects available to stakeholders, including potential funding recipients, in facilitating the identification of the areas where the greatest need for programming exists? How is it made available? How do you ensure overlap or duplication of funded projects is kept to a minimum?
13.
What activities have been undertaken under the CBP to support the uptake of diabetes surveillance data and the sharing of knowledge among stakeholders (e.g., analysis, synthesis and dissemination of knowledge from communities, surveillance systems, evaluation and research)?
14.
Can you identify any unintended positive or negative impacts of the CBP to date?
Design / Delivery
15.
Can you identify any gaps in the CBP Functional Component?
16.
In your view, is the current monitoring system for the CBP Functional Component effective?
17.
In your experience, what have been the key challenges in implementing the activities under the CBP?
Confirmation of Diabetes CBP Proposal Oversight and Approval Process:
Information Collection and Validation from the National and Regional Offices
Region:
Approach to Call for Proposal
Please describe the approach used in your region in soliciting proposals (e.g., invite selected group(s) or broader outreach to key stakeholder groups). If approach has changed over years, please provide details for each of the fiscal years 2005-06, 2006-07 and 2007-08.
Priority Setting
Please describe how program funding priorities are established in your Region, e.g., who is involved and how?
Governance (Proposal Oversight) and Decision-Making
Please describe the structures in place to oversee the proposal review/ selection process? E.g., a steering committee, a working group, an individual, etc.
Criteria for Selection
What are the criteria used to review the proposals? Are they formalized? Can we obtain a copy of these?
Number of Organizations Engaged
Can you indicate how many organizations received the call for proposals
2005-06:
2006-07:
2007-08:
How many organizations replied to the call for proposal (i.e., submitted a proposal)?
2005-06:
2006-07:
2007-08:
Page details
- Date modified: