Effectiveness
Official title: Federal Accessibility Legislation - Technical analysis report
On this page
As noted earlier, consultation participants want the legislation to include clear requirements and robust mechanisms for monitoring compliance and enforcing the law. We also heard that they expect the effectiveness of the legislation to be evaluated on a regular basis and the results made public.
It was also suggested that evaluations look at: objectives achievement, compliance, monitoring and enforcement, and innovation and best practices. There is less agreement, however, on how often the legislation should be evaluated and reviewed, though annually is the most popular view. As discussed below, as well as earlier in this report, there is consensus on the need for evaluations and audits to be carried out by a well-resourced and independent/objective person or office.
Online engagement
The preamble to this final section of the questionnaire included the following preamble:
“The legislation could include a section or sections that detail when and how the Government of Canada would report on the implementation and effectiveness of the legislation, and when and how the legislation itself would be reviewed.”
Question: In relation to the implementation and effectiveness of the legislation, how often would you want the Government of Canada to report to Canadians?
This question received a total of 1,255 responses.
As shown in Figure S, a plurality of responses suggested that the Government should report to Canadians on an annual basis. Approximately one in 10 propose reporting semi-annually, and an equal proportion suggested quarterly reports. About one in five felt that reporting should take place every two or more years.
Text description of Figure S:
| Responses | % |
|---|---|
| Annually | 42% |
| Comments/suggestions for content of the legislation | 11% |
| Every quarter | 11% |
| 3 years or more | 10% |
| Every 6 months (twice a year, semi-annually) | 10% |
| Every 2 years | 9% |
| Other | 4% |
| I don’t know | 1% |
Sub-group analysis found that the comments of younger participants (for example 18 to 34 years of age) are significantly more likely to indicate a preference for quarterly reporting.
Two primary reasons were put forward in support of having the Government report to Canadians at least once a year (example: coinciding with the presentation of the federal budget). First, frequent review and reporting would help to identify gaps and potential remedies, actions that would be particularly useful during the new legislation’s infancy; and second, relatively frequent reporting would help raise awareness of accessibility issues among the public, as well as political decision-makers.
Annually at minimum. Accessibility impacts daily life, so it's very important that we get regular updates.
– Anonymous
The government would need to report quarterly in order to maintain the freshness in the minds of Canadians and therefore improve the effectiveness.
– Anonymous
Collectively, those who thought it best that the Government report to Canadians every two or more years expressed concern that more frequent reporting could lessen the report’s impact (example: by making it seem routine rather than a special event). Some also suggested that the resources that would need to be devoted to more frequent reporting would be more effectively used elsewhere (example: to fund programs aimed at removing barriers). A few responses also suggested that rigorous assessment and evaluation takes time and that it might therefore be best to produce a superior report every two years than lesser ones more frequently.
Question: What kinds of things should this report look at?
This question received a total of 1,129 responses.
Canadians suggested four areas that the report could look at: 1) objectives achievement, 2) compliance, 3) monitoring and enforcement and 4) innovation and best practices.
As shown in Figure T, the most common suggestion was for the Government to report on the achievement of goals and outcomes by measuring change over time in key areas, such as housing, employment and income security. Respondents envisioned the use of benchmarking, and the marshalling of statistics and measures (example: poverty rates, employment levels, participation rates, social isolation measures) to determine whether and where the legislation was having an impact.
It should be comprehensive but focus on the outcomes and results. It should also be easy to understand.
– Anonymous
Look at what is the intent and goals of the legislation. Write them at the beginning of the process, so that they may be measured at the end. Is accessibility better? Are barriers less?
– Anonymous
Text description of Figure T:
| Responses | % |
|---|---|
| Outcomes/objectives achievements (e.g., built environment, employment statistics, income, housing, transportation) | 66% |
| Compliance (e.g., how many organizations are not compliant – kudos to who are, shame those that are not) | 26% |
| Monitoring and enforcement (e.g., number of audits, fines, warnings) | 24% |
| Innovation: identify new initiatives/strategies to reduce barriers | 14% |
| Other | 7% |
| I don’t know | 2% |
Many people suggested that the Government report on compliance levels. That is, the extent to which organizations are adhering to the legislation (example: across regions of Canada, by sector). As part of their suggestion, a number of respondents recommended that a list of non-compliant organizations be made public.
Related to the issue of compliance was the suggestion that the Government report on monitoring and enforcement activities, including its own efforts in this area. The main rationale for including this information in the report was that it would provide a measure of accountability in an area that many respondents view as a major weakness.
Two other areas that were suggested for inclusion in the Government’s report pertain to innovation, including: international comparisons to identify best practices and lessons learned, and highlighting Canadian examples of programs, strategies and other initiatives that have proven successful and worthy of emulation:
… case studies of what has worked - sharing of best practices.
– Anonymous
Question: How often should the legislation be reviewed?
This question received a total of 1,141 responses.
As shown in Figure U, there was a wide range of views expressed on the question of how often the legislation should be reviewed. About 4 in 10 indicated that a review should occur every two to five years. About one in five suggested annual reviews, while one in 10 proposed a more open-ended approach in which reviews would take place periodically, on an as needed basis.
Text description of Figure U:
| Responses | % |
|---|---|
| Within a range of 2 to 5 years | 40% |
| Annually | 21% |
| On-going review (e.g., open to legislative amendment at any time) | 9% |
| Every 5-10 years | 8% |
| Every 2 years | 7% |
| Every 6 months | 4% |
| Other | 8% |
| I don’t know | 4% |
Some of those who suggested that reviews take place yearly, or six months, also noted that as the legislation matured over time, reviews could be conducted less frequently: “Yearly until any changes are fully implemented and every 5 years thereafter. With each report there should be the ability to provide feedback on reprioritization or other matters that become evident in the report or in the observed effectiveness to date.”
– Anonymous
Question: Are there specific considerations for how any such review should be conducted?
This question received a total of 924 responses.
As shown in Figure V, the involvement of Persons with Disabilities, as well as stakeholders (example: advocacy and community groups) was seen as crucial to a useful and credible review process:
Input from all stakeholders (example: people with disabilities, caregivers, allies, family members etc.) should be required and happening! As we often say in the Disability Action Hall, ‘Nothing About Us Without Us’ please.
– Anonymous
A number of comments also advised the Government to ensure that the review process included sufficient regional representation.
Text description of Figure V:
| Responses | % |
|---|---|
| Incorporate online consultations into the review design (e.g., online surveys) | 33% |
| Incorporate in-person consultations into the review design | 32% |
| Multi-stakeholder collaboration (e.g., getting input from communities, municipalities, provinces, etc.) | 23% |
| Review should be conducted by disinterested third-parties | 13% |
| Comments/suggestions on timing of reviews and content of the legislation | 10% |
| Incorporate a mix of in-person, online or other methods into the review design (e.g., technology-based tools, etc.) | 8% |
| No comments/suggestions | 6% |
| Parliamentary committee | 4% |
| Review should be conducted by the Auditor General | 1% |
| Other | 5% |
| I don’t know | 11% |
Most comments contained suggestions for how to incorporate the input of Persons with Disabilities and stakeholders into the review process, including through online consultations, as well as a variety of in-person methods (example: town halls, forums, focus groups).
A review board should consist of senior analysts from the government who specialize in data analysis; professionals who work on a daily basis with people living with long term disabilities such as Social workers and Occupational therapists; Professionals from the corporate sector who are in charge of employment equity and services. The review should include the success rate for employment; housing; communications; quality of life factors. A questionnaire could be sent out annually much in the format used for this questionnaire.
– Anonymous
A number of comments emphasized the need for objectivity in conducting the review, with some suggesting that reviews be carried out by a disinterested third party, such as the Auditor General.
Public sessions
The Edmonton participants touched briefly on the issues of effectiveness. They generally agreed that after it comes into effect, a review of the effectiveness of the legislation should be conducted with the input of Persons with Disabilities and other stakeholders.
National Youth Forum
The question of how to assess the effectiveness of the legislation was not part of the National Youth Forum discussion agenda. It is relevant to note here, however, the importance that participants placed on involving Persons with Disabilities in the development of every aspect of the proposed legislation, including monitoring and compliance (both key performance indicators of effectiveness).
Thematic roundtables
There was a strong agreement that the federal government should conduct regular reviews of the law after it comes into effect. They also indicated that the reviews and evaluations should include stakeholder input from the disability community. Suggestions for how often reviews should take place, however, varied from annually to every four years.
Stakeholder submissions
There was strong agreement from advocacy organizations that the Government should present annual accessibility progress reports, which would include the results of “objective third-party” evaluations and assessments and plans to improving accessibility. As Barrier Free Canada outlined: “It can be helpful for the Government to have to report each year on what specifically it will do in the next years to implement the federal accessibility law, and to remove and prevent accessibility barriers. The Auditor General should audit these reports, to make sure that what the Government announces as promised new action is not action it has earlier completed or promised.”
It was also suggested that the report be written and presented by an independent body, such as the Auditor General, a new commissioner or ombudsperson.
Labour organizations did not address the question of whether and how to assess the legislation’s effectiveness, but UNIFOR did call for independent proactive oversight: “In order to avoid complaint driven processes as the only trigger for possible oversight, an independent committee should be established and should be charged with auditing government and federal employers for compliance with the standards of the CDA.”
Page details
- Date modified: