Site management options: federal contaminated sites decision-making framework

Skip booklet index and go to page content

Appendix B: site management options assessment

This appendix discusses methods for assessing the relative advantages and disadvantages of a variety of remediation or risk management (R/RM) options.

Part 1: the theory

Role of cost/benefit analysis

Cost/benefit analysis is a commonly accepted approach for determining the feasibility of various alternatives being considered to address a particular problem or project requirement. In the context of contaminated sites, this can be used as a basis to determine the optimum approach where a variety of alternatives exist to address site conditions, representing a range of effectiveness, implementability and cost considerations.

In order to conduct the cost/benefit analysis, it is typically necessary to determine the alternatives for consideration, identify the evaluation criteria to be applied, and then conduct the comparative evaluation using an appropriate method. This is discussed in more detail below.

Development of an alternatives array

The formulation of a range of alternatives can be illustrated in an array that identifies the technical options available for addressing the site conditions (identified prior to Step 7). This may include both remediation and risk management techniques, and a combination depending on the site-specific circumstances and the environmental media that need to be addressed.

For illustration purposes, an example alternatives array is included in Table 1. As shown in the table, a total of nine alternatives are included. Each alternative includes specific actions related to individual site-specific areas or units (e.g., equipment/storage vessels, waste disposal areas, impacted soil areas, impacted groundwater areas). The scope of the alternatives progress sequentially from “do nothing,” to limited action (waste material removal, access restrictions, monitoring), then containment, and finally to active remediation through treatment. The potential application of both risk management and remediation methods is an integral part of the array, and a variety of other combinations may be available beyond what is shown in this example. In carrying out the comparative assessment, it may ultimately be decided that the optimum alternative is a refinement of the alternatives initially included in the array. The evaluation process should provide flexibility for this type of decision making.

It is noted that Alternative 1 (no action) may appear inherently unacceptable at the outset. However, its inclusion in the array may be useful for representing a baseline condition to identify the consequences of “do nothing” and for assisting with the justification for selecting one of the other alternatives.

Table 1: example alternatives array
Cost/benefit analysis for remediation/risk management alternatives for federal contaminated sites
Area/Unit 1 2 3 4 5 6 7 8 9
* MNA: monitored natural attenuation
Drums/tanks/piping No action Remove Remove Remove Remove Remove Remove Remove Remove
Waste disposal pit No action Restrict access Cap Cap Excavate/dispose Excavate/dispose Excavate/dispose Excavate/dispose Excavate/dispose
Soil impacted above industrial use guidelines No action Restrict access Cap Cap Cap Cap Excavate/dispose/treat Excavate/dispose/treat Excavate/dispose/treat
Soil impacted above unrestricted use guidelines No action Restrict access Restrict access Restrict access Restrict access Restrict access Restrict access Cap Excavate/dispose/treat
Groundwater at source area No action Monitor Monitor Treat-ment Treatment Treatment Treatment Treatment Treatment
Groundwater plume No action Monitor Monitor MNA* MNA Treatment/ MNA Treatment/ MNA Treatment/ MNA Treatment/ MNA

Potential evaluation criteria

In most situations, the comparative evaluation of alternatives relies on the use of criteria against which each alternative can be assessed relative to other alternatives. These criteria may be either qualitative or quantitative, and will generally consider factors relating to overall protection, effectiveness, implementability, cost, stakeholder considerations and regulatory compliance. A list of potential evaluation criteria is included in Table 2. As shown in the table, various criteria may be applicable under each category, and are further classified according to the type of criteria (threshold, balancing and acceptance).

Table 2: potential evaluation metrics
Cost/benefit analysis for remediation/risk management alternatives for federal contaminated sites
Category/Evaluation Metrics Metric Type
Overall protection Threshold Balancing Acceptance
Protection of human health X    
Protection of the environment X    
Effectiveness Threshold Balancing Acceptance
Short-term effectiveness   X  
Long-term effectiveness and permanence   X  
Reduction of toxicity, mobility, volume   X  
Implementability Threshold Balancing Acceptance
Use of proven technology   X  
Use of innovative technology   X  
Use of sustainable construction/remediation methods   X  
Permits and approvals required   X  
Time required for implementation   X  
Impacts and risks to the environment   X  
Impacts and risks to the public and workers   X  
Cost Threshold Balancing Acceptance
Construction cost   X  
Operation and maintenance cost   X  
Decommissioning cost   X  
Present worth cost   X  
Stakeholder considerations Threshold Balancing Acceptance
Federal government/custodian acceptance     X
Provincial government acceptance     X
Local government acceptance     X
Community/public acceptance     X
Regulatory compliance Threshold Balancing Acceptance
Compliance with regulatory requirements - federal X    
Compliance with regulatory requirements - provincial X    
Compliance with regulatory requirements - local X    
Other Threshold Balancing Acceptance
Sustainable development   X  
Future development potential   X  
Long-term liability   X  
Impact on land value   X  
Impacts on future operations   X  
Compatibility with federal government policies     X
Socio-economic impacts   X  

Threshold criteria include minimum requirements that need to be met in order for the alternative to be considered for selection. Typically, this relates to protection of human health and the environment, and regulatory compliance. Alternatives that satisfy these threshold criteria will be suitable for passing an initial screening and then short-listed for more detailed consideration.

Balancing (or modifying) criteria are those which are used to compare the relative merits of the various alternatives and associated trade-offs. For example, Alternative A might be expected to achieve a permanent solution and unrestricted future site use, whereas other alternatives may achieve an acceptable cleanup level for commercial or industrial site use at a much lower cost.

Acceptance criteria are those that relate to meeting the expectations of various stakeholder groups, including various levels of government and the local community. Acceptance considerations may ultimately be used to make a final selection from a list of several otherwise suitable alternatives, or as a basis for refinement of a preferred alternative.

An alternative that is preferable based on balancing criteria considerations may not ultimately be selected for implementation if it is not acceptable to stakeholders. In some cases (e.g., large/complicated sites), stakeholder input will probably have already been considered in developing the list of alternatives.

The list shown in Table 2 represents potential criteria for consideration. The selection of relevant criteria may be affected by site-specific factors, and it may be decided that some should be deleted, or others added, as necessary and appropriate.

Options for comparative evaluation of alternatives

A variety of methods are available for conducting a comparative evaluation of alternatives in order to identify the most suitable alternative (and hence site management strategy) for implementation. This includes the following examples described herein: ad hoc methods, checklist methods, economic methods, pairwise comparison methods and matrix methods.

Ad hoc methods compare alternatives in narrative terms without using any explicitly stated methods to order the preferences, based on professional judgment. Typically, the use of ad hoc methods, as the name implies, does not necessarily follow an explicit set of evaluation criteria. This approach can be applied to situations in which the scope of the problem is narrow and well defined, and the rationale for selection of the proposed alternative can be readily communicated. However, in more complex situations this method is subject to potential problems such as assuring that each alternative is evaluated in a consistent manner.

Checklist methods compare and evaluate alternatives against a specified set of criteria with no compensatory rules or tradeoffs. Typically, this involves posing a series of questions related to the individual criteria that require a yes or no response, such as:

This approach may be useful for identifying dominant alternatives for screening purposes. For example, if Alternative A is better than Alternative B in at least one respect and no worse than Alternative B in any other respect, Alternative A may be considered dominant.

Economic methods use economic procedures and principles to translate non-commensurable units into monetary units. This methodology relies largely on determining an individual's willingness-to-pay (the amount that individuals affected by the project would be willing to pay for the defined benefits), and the availability of market prices that relate to the benefits. By their nature, many of the benefits associated with environmental improvements cannot be readily determined based on market prices; however, this type of method may have application in situations where the property is being considered for sale and/or redevelopment. In this case, it may be possible to directly relate the cost of implementing an alternative to the beneficial value of land improvement.

Pairwise comparison methods use the sequential comparison of alternatives in pairs as a basis for subsequent ordering of preferences. In its simplest form, the procedure develops a measure of how frequently one alternative is superior to another based on the various evaluation criteria. This is improved using fuzzy set procedures, which is based on subjective interpolation, and is used to identify inefficient alternatives (those that are dominated by other alternatives). In this case, each alternative is numerically ranked for each evaluation criteria. Initially, two alternatives are compared to determine dominance, i.e., which of the two alternatives has the greater number of occasions of dominance. The dominant alternative is then compared to the next alternative, and so on, until one dominant alternative is identified. The method can be based on either a non-parametric or parametric ranking; however, in both cases, the assignment of the ranking values may be subjective. Also, the relative importance of each criterion is not reflected in the procedure unless the criteria are ranked into groups.

Matrix methods use a matrix for the summary, comparison and evaluation of criteria and alternatives, based on professional judgment (as an extension of ad hoc methods). In this case, weight factors are applied to each evaluation criterion to reflect its overall importance, and ranking factors are applied to each alternative (for each criterion). These are multiplied and summed to develop an overall score. In this manner, alternatives that score well can be considered to be superior to other alternatives. This method relies on subjective assignment of the weight and ranking factors, and therefore would need to be supported by the assessor's justification for assigning the factors. It is an improvement over ad hoc methods in that all evaluation criteria need to be considered for each alternative, and it is amenable to sensitivity analysis by examining the effects of changes in the factors. Both the pairwise comparison and the matrix methods are transparent in the identification of the preferred alternative and hence potentially very useful in public consultation.

Expert support tools that may assist the custodian in completing the preceding evaluation include the Guidance and Orientation for the Selection of Technologies (GOST) and the Sustainable Development Tool (SDT). GOST is a technology database that contains individual fact sheets on a host of treatment technologies/approaches. The user is prompted for a series of inputs regarding contaminant and site data (e.g., hydro-geologic conditions), which results in the identification of a number of technically feasible R/RM options. Custodians could consider the use of GOST as early as Step 5 and during Step 7 of the 10-step federal process, primarily to identify potential candidate technologies/approaches for management of their sites. A secondary benefit of GOST is that it provides assistance to the custodian in confirming the necessary data to be collected during the environmental site assessment (ESA), via the required inputs to the model, to support this evaluation.

Once the custodian has identified a suite of potential technologies/approaches using GOST, a secondary evaluation can be conducted using SDT to evaluate and compare up to five separate treatment options from the perspective of the three pillars of sustainability: economic, social and environmental. Custodians can choose from a suite of parameters for all three elements--and further, use weightings for each parameter--to reflect their specific site situation. The output from the model is both graphical and numerical, such that it serves as a communication as well as an analytical tool. This approach allows for stakeholder engagement and incorporation of multi-stakeholder requirements. The intent is that custodians will select the most balanced alternative with the cost in mind; SDT will help them to incorporate sustainability aspects into their evaluation process when identifying the preferred alternative.

Part 2: example alternatives assessment

An example of alternatives assessment using the pairwise comparison and matrix methods based on a contaminated site scenario is presented below.

Use of evaluation metrics to select the preferred remediation/risk management (R/RM) alternative

Table 2 includes the potential evaluation metrics within various categories that can be used as part of the process for identifying the preferred R/RM alternative. Although Table 2 lists a number of evaluation metrics associated with each category, not all evaluation metrics will be employed in an actual evaluation. In practice, it is only necessary to employ the evaluation metrics that are relevant to discriminating between the R/RM alternatives. Hence, only a subset of the potential evaluation metrics will be employed in any particular evaluation.

As apparent from the list of potential evaluation metrics, individual metrics are not measured using the same units, and hence they are not additive. As a result, they cannot be combined in a simple manner. Instead, a means of combining the value of an alternative must be made relative to the various evaluation metrics, to determine which of the alternatives is preferred.

An additional dimension of the evaluation criteria must also be acknowledged; if an alternative does not attain a threshold (e.g., with respect to human health and the environment), that alternative is not acceptable and is not considered beyond the first level of analysis.

Example problem definition

The following example demonstrates how the methodology is applied. Please note that this problem situation has been kept fairly simple in order to focus on the methodology rather than on precise complexities that may arise in practice. Furthermore, the exact details of the preferred strategy are not supplied but are assumed to be consistent with good practice, specifically for a remote site.

Consider the following situation:

A waste disposal pit and an underlying groundwater plume have been identified at a site. The alternatives for the remediation of the disposal pit were identified as capping the pit, or excavation and disposal of the waste.

For the underlying groundwater plume, the alternatives that will be considered are monitored natural attenuation (MNA) and groundwater treatment with MNA combined (treatment/MNA). The duration of these options will vary, since treatment will promote a more rapid reduction in contaminant concentrations. The status quo (“do nothing”) option associated with the groundwater plume should also be considered, as there may be no need to undertake action, and it will provide a baseline for comparison of this scenario.

The individual alternatives, as classified into vertical sets of options, are illustrated in Table 3. Other options may be available, for example cap and treatment/MNA, but are not included in this example in order to maintain simplicity.

Table 3: List of alternatives
Area/Unit EXC* 1 2 3
*EXC: excluded as a viable alternative as it does not attain the necessary threshold levels.
**MNA: monitored natural attenuation
Waste disposal pit No action Cap Excavate/dispose Excavate/dispose
Groundwater plume No action Monitored natural attenuation Monitored natural attenuation Treatment/monitored natural attenuation

These alternatives include elements related to both risk management (i.e., cap and MNA) and remediation (i.e., excavate/dispose and groundwater treatment/MNA). We are now interested in selecting the preferred alternative, where the preferences between the options regarding long-term effectiveness may be different, for example, than cost considerations.

To proceed to the next step, each of the alternatives needs to be considered with respect to each of the evaluation criteria within the categories. This step is accomplished in the following sub-tables, as follows:

(i)The alternative is judged to be unacceptable or excluded in terms of threshold levels and, hence, is no longer considered.

(ii)Table 4(a) summarizes the attributes of each of the alternatives relevant to effectiveness. It should be noted that the only relevant effectiveness evaluation criteria are the long-term effectiveness and the reduction of toxicity, mobility or volume.

Table 4(a): description of attributes of alternatives related to effectiveness criteria
Alternative Long-term effectiveness Reduction of toxicity, mobility or volume
*MNA: monitored natural attenuation
1. Capping and MNA Capping not necessarily effective in the long term, and attenuation of groundwater contamination will occur but will take some time. There will be no reduction of toxicity, mobility or volume of chemicals within the waste disposal pit contents, and reduction of groundwater contamination will occur over time.
2. Excavation/disposal and MNA This represents a permanent solution for in situ wastes, and attenuation of groundwater contamination will occur but will take some time. This will reduce/eliminate the waste disposal pit contents, and will result in attenuation of the groundwater contamination over time.
3. Excavation/disposal and treatment/MNA This represents a permanent solution for in situ wastes, and will result in attenuation of groundwater contamination more rapidly than MNA alone. This will reduce the toxicity of waste pit contents, and will result in attenuation of groundwater contamination more rapidly than MNA alone.

(iii)Table 4(b) summarizes the attributes of each of the alternatives relevant to “implementability.” All of the alternatives involve the application of proven technologies, none are innovative, all involve the necessity to obtain permits, etc. This means that the discriminating factor between the alternatives is the time required for implementation (e.g., MNA requires a lengthy period for site remediation whereas capping is implemented relatively quickly). Impacts and risks to the environment during implementation must also be a consideration (i.e., consider the risks associated with the transport and disposal of the excavated material).

Table 4(b): description of attributes of alternatives related to cost criteria
Alternative Time required for implementation Impacts and risks to environment
1. Capping and MNA Rapid to construct cap. MNA will take time to be totally effective. Technology of capping is understood, and risks to environment by MNA are small, although the possible ongoing source of contamination must be considered.
2. Excavation/disposal and MNA Excavation/disposal relatively rapid. MNA will take time to be totally effective. May be issues in relation to contaminant release during excavation as well as at the disposal site. MNA risks are small.
3. Excavation/disposal and treatment/MNA Excavation/disposal relatively rapid. Treatment/MNA will be more rapid than MNA alone. May be issues in relation to contaminant release during excavation. Treatment/MNA risks are small and less than MNA alone.

(iv)Table 4(c) describes the attributes of the various alternatives in terms of costs. In this application, the costs are determined in terms of present worth (or net present value) and therefore show the combination effect of construction costs, operation and maintenance costs, and discount rate.

Table 4(c): description of attributes of alternatives related to cost criteria
Alternative Magnitudes of costs of each alternative
1. Capping and MNA 2 million + 1 million = 3 million
2. Excavation/disposal and MNA 10 million + 1 million = 11 million
3. Excavation/disposal and treatment/MNA 10 million + 2 million + 0.5 million = 12.5 million

(v)Table 4(d) describes the attributes for different alternatives in terms of the “Other” category. This may be a relevant consideration in selecting between the alternatives in that there is long-term liability for ensuring that MNA functions as intended, as opposed to, for example, the excavation and destruction of the wastes. For MNA, there is some degree of long-term liability associated with the site. Also, the potential impacts on future site operations may be a consideration.

Table 4(d): description of attributes of alternatives related to other evaluation criteria
Alternative Magnitudes of long-term liability Impacts on future operations
1. Capping and MNA Liability exists since capping does not destroy the contaminants, and the time for MNA to be effective is potentially long. Operations: Capping will limit certain land-use activities on site.
2. Excavation/disposal and MNA Reduced long-term liability since the remediation removes the contaminants, although time for MNA to be effective may still be prolonged Excavation/disposal will allow future land uses depending on residual contaminant concentrations.
3. Excavation/disposal and treatment/MNA Minimum long-term liability since the remediation removes the contaminants and time for treatment/MNA is shorter. Excavation/disposal will allow future land uses depending on residual contaminant Concentrations.

The set of Tables 4(a) through 4(d) summarizes how the alternatives are measured with respect to each of the evaluation criteria. The next stage is to identify which of the alternatives is/are the preferred alternative(s). This will be accomplished using the two separate procedures designed for this identification, namely (i) the pairwise comparison method and (ii) matrix weighting procedures.

Identification of preferred alternative

Using the pairwise comparison method
Table 5: comparison of alternatives 1 and 2
Category/Evaluation Criteria Preferred Alternative Rationale
Long-term effectiveness 2 Alternative 2 is more effective since it removes the waste material.
Reduction of toxicity, mobility or volume 2 Alternative 2 is more effective since it removes the waste material.
Time required for implementation 1 Alternative 1 requires less time for implementation and avoids potential
Impacts and risks to the environment 1 Alternative 1 requires less time for implementation and avoids potential impacts
Costs 1 Alternative 1 involves lower cost.
Long-term liability 2 Alternative 2 reduces long-term liability associated with leaving the waste in place.
Impacts on future operations 2 Alternative 2 reduces long-term liability associated with leaving the waste in place.

Alternative 2 has four evaluation criteria in which it is preferred to Alternative 1, and there are three criteria in which Alternative 1 is preferred to Alternative 2. In this situation, Alternative 2 moves on to be compared with Alternative 3. It is noted that this comparison suggests that there is little difference between Alternatives 1 and 2.

In a more complete assessment of impacts, other considerations such as off-site impacts like transportation of excavated material and liability/risk associated with disposal might also be evaluated with regard to each alternative. Please note this type of evaluation does not give weight to the evaluation criteria; it only allows a preference for one alternative method or another. A weighted matrix example is explained later in this appendix.

Table 6: comparison of alternatives 2 and 3
Category/Evaluation Criteria Preferred Alternative Rationale
Long-term effectiveness 3 Alternative 3 is more effective since it reduces the contaminant mass through treatment.
Reduction of toxicity, mobility, or volume 3 Alternative 3 is more effective since it reduces the contaminant mass through treatment.
Time required for implementation 3 Alternative 3 requires less time to reach acceptable contaminant levels because it reduces contaminant mass through groundwater treatment.
Impacts and risks to the environment 3 Alternative 3 requires less time to reach acceptable contaminant levels
Costs 2 Alternative 2 involves lower cost.
Long-term liability 3 Alternative 3 shortens the time that groundwater contamination persists.
Impacts on future operations No Difference --

In this comparison, Alternative 3 is preferred to Alternative 2 with regard to five evaluation criteria, whereas Alternative 2 is preferred to Alternative 3 in only one criterion. This indicates that Alternative 3 is the preferred alternative remediation option.

Based on the above, it could be concluded that Alternative 3 is the preferred alternative, if all evaluation criteria were considered to have equal weight (or importance), as is the case with this method. Weightings are applied in the matrix method discussed below.

Using matrix weighting procedures

Two sets of weighting factors are required:

The selection of the weighting factors needs to consider the viewpoints of the interested parties, recognizing that different stakeholders may be more sensitive to specific evaluation criteria than others. However, the procedure does allow sensitivity testing to determine differences in the analysis resulting from changes in the weight factors.

Ranking of one alternative relative to another

In the example matrix, each of the alternatives is ranked relative to the others using non-parametric means, such that the best of the three alternatives associated with each of the evaluation criteria receives a 3, the second-best gets a 2, and the third-best gets a 1. In the event of a tie, the average of the two is assigned to both.

Simple matrix weighting calculations are summarized in the table below, which shows that Alternative 3 is the preferred alternative of the three (i.e., has the highest score).

Table 7: scoring calculation for alternatives using matrix weighting procedure
Category/evaluation criteria Factor weight Ranking score alternative
(1, 2, 3)
Weighted Factor Priority group weight Weighted factor alternative
(1, 2, 3)
*Note: red numbers refer to alternative 1; blue numbers refer to alternative 2; and purple number refers to alternative 3.
Long-term effectiveness 0.7 1 2 3 0.7 1.4 2.1 x 0.2 0.2 0.4 0.6
Reduction in toxicity 0.3 1 2 3 0.3 0.6 0.9
Total weighted factor 1 2 3
Time required 0.5 3 1 2 1.5 0.5 1 x 0.3 0.6 0.45 0.75
Impacts of risks 0.5 1 2 3 0.5 1 1.5
Total weighted factor 2 1.5 2.5
Present worth 1 3 2 1 3 2 1 x 0.3 0.9 0.6 0.3
Total weighted factor 3 2 1
Long-term liability 0.7 1 2 3 0.7 1.4 2.1 x 0.2 0.2 0.46 0.54
Impacts on future operation 0.3 1 3 2 0.3 0.9 0.6
Total weighted factor 1 2.3 2.7
Final alternative score 1.9 1.91 2.19
Ranking of each alternative on a scale of one to ten

Another option is to rank the values on a scale from one to ten using parametric means. This allows the assessor to determine, for example, the magnitude of the differences between the alternatives for individual evaluation criteria.

Additional considerations

The example problem was kept very simple to allow the primary focus to be on the selection procedure for the preferred alternative. However, it should be clear that the process may be considerably more complex in a real situation. Examples of the challenges that could arise include the following:

The procedures are straightforward to apply and test the sensitivity of the selection by allowing different methods to arrive at the same conclusion.

There is merit in completing evaluations using one or more procedures, for example pairwise comparison or matrix weighting comparisons; if the results are the same, it demonstrates that the findings are robust.

Page details

Date modified: