Science Policy Note: Choosing a Percentile of Acute Dietary Exposure as a Threshold of Concern

July 28, 2003
ISBN: 0-662-34730-7
Cat. No.: H113-13/2003-1E-PDF
(SPN2003-01)

The following document is a policy/guidance document that reflects the United States Environmental Protection Agency's (U.S. EPA's) recent dietary risk assessment science policy/guidance paper entitled, Choosing a Percentile of Acute Dietary Exposure as a Threshold of Regulatory Concern (March 16, 2000; U.S. EPA 1999a).

The Pest Management Regulatory Agency (PMRA) has adopted the policy and guidance outlined in the U.S. EPA document as part of efforts to harmonize dietary risk assessment procedures for determination of the safety of pesticide residues in domestic and imported treated foods.

This endeavour, to harmonize methodologies, is part of the North American Free Trade Agreement (NAFTA) goals within the Pesticides Technical Working Group Subcommittee.

The U.S. EPA has taken the lead in developing science policies related to the U.S. Food Quality Protection Act. Harmonization of these policies between our agencies has been key to our ability to do joint reviews. Such policies play an increasingly important role in the evaluation and assessment of risks posed by pesticides and improve the regulator's ability to make decisions that fully protect public health and sensitive subpopulations. These policies are vetted by the NAFTA Technical Working Group on Pesticides and have been approved for adoption, only after extensive consultation by scientific experts from governmental, academic and all nongovernmental interested parties. The consultation process utilized by the PMRA for science policy notices is outlined in a memo entitled: Memorandum to Registrants, Applicants and Agents, (January 25, 2001) and may be obtained from the PMRA web site.

The following policy document is intended to provide guidance and information to PMRA personnel and decision-makers, and to the public. As a guidance document, the policy in this document describes the process used by PMRA scientists in dietary risk assessments. Stakeholders remain free to comment on the application of the policy to individual pesticides. The PMRA will carefully take into account all comments that are received.

The PMRA is responsible for regulating the nature and amount of pesticide residues in food under the Food and Drugs Act and Regulations (FDAR). Sections 4(a) and 4(d) of the Food and Drugs Act (FDA) authorizes PMRA to set a maximum residue level (MRL) within Regulation B.15.002(1) of the Food and Drugs Regulations (FDR) or an exemption from the requirement of a MRL under Regulation B.15.002(2) of the FDR, if the Agency determines that the residues would be "safe". The Agency performs various types of risk assessments to evaluate the safety of pesticides in food, including analyses to determine the nature and the amounts of pesticides that people might be exposed to over a single day. This paper discusses how PMRA generally applies the statutory safety standard to acute dietary risk assessments as to pesticide residues in foods.

The safety standard, described herein, is termed the Threshold Of Concern (TOC) and is defined as the threshold at which dietary exposure from aggregate food residues is considered safe. That is, the potential daily intake (PDI) at the 99.9th percentile compared with the acute reference dose (ARfD) is less than 100%, or stated another way, the TOC is the point at which the aggregate exposure from food residues, at 99.9%, is equal to the acute reference dose. This concept is the basis of this policy/guidance document.

The PMRA intends to use the 99.9th percentile of the distribution of estimated acute dietary food exposures for calculating a TOC when probabilistic assessment techniques are used to model the distribution. The PMRA would compare this percentile of estimated exposure to the ARfD, a value that reflects an amount of a pesticide to which a person may safely be exposed in one day. This document explains the PMRA's policy and details some of the various concerns that have been raised, additional associated public health-related issues, as well as, the PMRA's plans for further evaluation and implementation. This policy has broad applicability to many pesticides.

The PMRA's current approach with respect to assessing and regulating the food uses of pesticides, when using a probabilistic method of estimating acute dietary exposure, is as follows: If the 99.9th percentile of acute exposure from food, as estimated by probabilistic (e.g., Monte Carlo) analysis, is equal to or less than the ARfD for the pesticide, then PMRA would generally consider its threshold of concern in applying that the safety standard of the FDA (sections 4(a) and 4(d)), not to be exceeded with respect to acute risk from food. However, if the analysis indicates that estimated exposure at the 99.9th percentile exceeds the ARfD, the PMRA would generally conduct a sensitivity analysis to determine to what extent the estimated exposures at the high-end percentiles may be affected by unusually high food consumption or residue values.

To the extent that one or a few values from the input data sets seem to "drive" the exposure estimates at the high end of exposure, the PMRA would consider whether these values are representative and should be used as the primary basis for regulatory decision making. In either scenario, the PMRA would consider submissions by interested parties that question the appropriateness of the use of the 99.9th percentile in calculating the TOC for the particular risk assessment in question or question its use generally.

It is important to note here that the above position refers to the 99.9th percentile of exposure and not consumption. The 99.9th percentile of exposure represents the joining of each individual's consumption data set with randomly selected residue values from the residue data set. The consumption values associated with the 99.9th percentile of exposure do not necessarily represent the 99.9th percentile of consumption since it is both the selected consumption value and residue concentration which is responsible for determining exposure.

The PMRA's current policy is used only with daily exposures to a single chemical through the food pathway only. Estimates of exposure through drinking water and residential uses are not sufficiently developed to warrant inclusion in a probabilistic assessment. Establishing the threshold of concern for the food pathway using the 99.9th percentile of exposure is considered to be a "first step" toward regulation of exposures on an aggregate, and then cumulative, basis.

The PMRA recognizes that different types of risk assessments will generally be needed for aggregate and cumulative evaluations and that these assessments might also be associated with different regulatory thresholds. Although the PMRA is moving toward regulating on the basis of probabilistic aggregate and cumulative exposure assessments, a decision has not yet been made regarding how the appropriate threshold of concern should be calculated for these types of assessments. When exposures through drinking water and residential uses are sufficiently refined to be incorporated into probabilistic evaluations, they will be aggregated and assessed, and may use a different population percentile.

Table of Contents

PMRA's present practice and policy for acute dietary risk assessment

Introduction

Under the Food and Drugs Act and Regulations (FDAR), the PMRA may authorize a maximum residue limit (MRL) or exemption from the requirement of a MRL, to allow a pesticide residue in food, only if the Agency determines that such residues would be "safe" (FDA, sections 4(a) and 4(d)). The term "safe" is defined as a reasonable certainty that no harm will result from dietary exposure to the pesticide chemical residue.

To determine whether food is safe to eat, the PMRA must assess the potential risks from pesticide residues in food. The size of the potential risks depends on the toxicity of the pesticide (how much harm, if any, is caused by specific amounts of the pesticide) and the magnitude of the exposure to the pesticide. Exposure to a pesticide in the food supply depends, in turn, on two factors: the amount of the pesticide present in food and how much food a person eats. It is impossible to know precisely how much food every individual in the country consumes, either over a lifetime or even on a single day. Similarly, it is impossible to know how much residue each specific item of food contains. Thus, the Agency must use available, reliable and representative data to develop estimates of such exposure.

In evaluating the potential risks from pesticides in the diet, the PMRA assesses both chronic (long-term) exposure and acute (short-term) exposure. For chronic exposure, the PMRA estimates the average amount of pesticide residue a person might consume over extended periods, potentially ranging from several months to a lifetime. For acute exposure, the PMRA is instead interested in the amount that might be ingested on a single day. To evaluate acute dietary exposure, the PMRA now uses a probabilistic exposure modelling technique, an example of which is "Monte Carlo analysis".

For the purpose of discussion, this paper will use the term "Monte Carlo" keeping in mind that other probabilistic techniques may be used as well. This probabilistic assessment technique estimates the different levels of exposure people experience as the result of differences in the types and amount of foods they eat, as well as variations in the level of pesticide residue that may be present, among other factors.

Over the last several years, the PMRA has been working to expand its capability of evaluating acute dietary exposure and risk using probabilistic techniques of assessment. The PMRA has established the current policy based on an earlier policy regarding the use of risk assessment techniques. Probabilistic analysis techniques, given adequate supporting data and credible assumptions, can be viable statistical tools for analyzing variability and uncertainty in risk assessments.

The Agency has set a number of conditions to be considered in judging the acceptability of a probabilistic analysis for review and evaluation; these conditions relate to transparency, reproducibility and the use of sound scientific methods. Agency policy supports Monte Carlo analysis as a probabilistic technique that has been accepted so far but would be open to considering other probabilistic techniques. When probabilistic exposure assessments for acute dietary risk are possible, the PMRA would refer to the 99.9th percentile of estimated exposure in making its risk-management decisions. In general, the PMRA would compare this level of exposure to a safety benchmark, i.e., the ARfD, in determining whether a particular regulatory action would be consistent with the statutory safety standards.

Previous review of PMRA's interim policy

The PMRA has followed and participated in the evolution of dietary risk assessment (DRA) with that developed by the U.S. EPA. The NAFTA Technical Working Group for Pesticides has worked towards harmonized DRA procedures/methodologies since 1997. The following discussion reflects the evolution of U.S. EPA and PMRA policy concerning the use of the 99.9th percentile in DRA. In March 1998, the U.S. EPA brought its interim policy to a panel of scientific experts called the Scientific Advisory Panel (SAP). The SAP generally agreed with the proposed probabilistic approach. They considered, among other things, the use of a 99.9th population percentile of exposure and expressed divergent views on whether using the 99.9th percentile is an (adequately) conservative approach. They noted that, in their view, if the 99.9th percentile is utilized, a percentage of the population could still be exposed daily to estimated levels that exceed the threshold of concern. They further noted that, even though the percentage was small (0.1%), the number of people represented by that percentage was very large because the exposed group is potentially the entire population of the country. The following additional remarks were made by the Panel.

  • To judge whether any given percentile criterion is conservative for acute effects or not, it would be necessary to consider the margin of safety which is already incorporated into the toxicological portion of the risk evaluation.
  • To identify the level of risk, variability not only in exposure levels but also in human thresholds for the toxic effects under consideration would be needed. That is, a probabilistic "toxicity" component of a risk assessment should be incorporated into the analysis as well.
  • By recognizing and separately modelling subpopulations, it may be possible to choose a lower, less statistically tenuous percentile in calculating a threshold of concern for one or more of these subpopulations. This lower percentile may also be warranted, they indicate, if the risk assessment contains a number of "conservative" assumptions that might result in overestimates of risk even at the 99.9th percentile.

Following the SAP review and after considering public comment, the U.S. EPA revised portions of its interim policy document on probabilistic exposure assessment. This revised document incorporated many of the changes recommended by the SAP in its March 1998 meeting discussed above. Subsequently, the U.S. EPA published a revised document as a draft science policy paper entitled Guidance for Submission of Probabilistic Human Health Exposure Assessments to the Office of Pesticide Programs (U.S. EPA 1998a).

As its title indicates, the science policy paper contained guidance on conducting dietary exposure assessments; it also stated that the U.S. EPA would separately present an explanation of its policy decision to refer to the 99.9th percentile of estimated acute dietary exposure in making its risk management decisions. The current document addresses this latter issue.

The United States Department of Agriculture (USDA) has commented that the use of databases that contain too few data points to project high-end percentiles of consumption of a particular food or levels of residues in a specific commodity with statistical confidence raises questions about the estimates of high-end exposure developed using probabilistic assessment techniques.

The USDA's comments have been considered and revised this paper to explain and address its concerns.

PMRA's current approach to dietary risk assessment

Chronic versus acute exposure and risk assessment

The PMRA typically performs a dietary exposure assessment for two different exposure time frames-short term or "acute" exposures and long-term or "chronic" exposures; each assessment is calculated differently. In chronic exposure assessment, the risk assessor is attempting to estimate a person's average dietary exposure over the long-term (e.g., several months to a lifetime). Consequently, the use of both average (or mean) residue value for each food commodity and average (or mean) consumption of food commodities is generally regarded as appropriate. Estimates of exposure through drinking water are subsequently combined with these estimates of exposure through food to calculate combined dietary exposure through food and water.

In acute dietary exposure assessment, however, the risk assessor is trying to estimate the range of exposures that individuals could encounter on a single day and determine the exposure to which "high-end" persons could be subjected (where "high-end" is defined as a plausible estimate of exposure for those individuals at the upper end of the exposure distribution). The PMRA is using Monte Carlo techniques (and its current 99.9th percentile approach) for these acute food exposure assessments only. The PMRA is not using Monte Carlo techniques at this time for chronic exposures due to the limitations of the existing food consumption data. The PMRA, U.S. EPA and USDA, however, are exploring statistical techniques that may allow such analyses in the future. The Monte Carlo Guidance document (U.S. EPA 1997a) and a subsequent U.S. EPA memorandum (U.S. EPA 1997b) provide additional information regarding the tiering process used in acute assessments, for both probabilistic and nonprobabilistic assessments (see also U.S. EPA 1998a).

The risk equation

Dietary risk can be expressed as a function of toxicity and exposure:

RISK = f (toxicity, exposure)

That is, to determine risk, which can be either acute (one-day) or chronic (long-term), one "combines" a value representative of the toxicity for the pesticide with the amount of pesticide to which an individual is exposed. The notation above is not meant to imply that the two quantities represented by "toxicity" and "exposure" are necessarily multiplied together, just that toxicity and exposure are two quantities which together determine riskFootnote 1.

The toxicity part of the risk function is typically expressed as an ARfD (in units of milligrams (mg) per kilogram (kg) body weight (bw) per day). An ARfD is an amount of toxicant (in mg/kg bw/day) to which a person can be safely exposed for one day. In general, an ARfD is set at a level at least 100 times smaller than the no observed adverse effect level (NOAEL, in units of mg/kg bw/day), if the NOAEL used is from a controlled toxicological study in laboratory animals.

What does the term Distribution mean?
Think back to the classic bell curve we learned about at some point in our school days. When grades were being determined, some of us had scores that were either on the low end or high end of the range while most of us had scores in the middle. If the frequency of occurrence were plotted, the resulting distribution of grades would resemble the bell curve:
bell curve
Essentially what the bell curve tells us is that most things are near the middle; there are far fewer occurrences at the extremes.
The distribution of the pesticides residues does not typically follow this bell-curve shape. Instead, the curve is generally right skewed with a long tail to the right. In the above diagram, it would be as if the vast majority of grades were A's and A-'s with very few C's, D's, and F's.
For an additional explanation of how this exposure distribution curve is interpreted in risk-based decision-making, see the Appendix to this document.

The NOAEL is defined as the largest amount of toxicant (in units of mg/kg bw/day) that produces no observed adverse effects in a test animal in a controlled toxicological study. The factor of 100 is a generally applied adjustment, (sometimes called a "safety factor" or, more frequently, an "uncertainty factor") to account for the potential that humans could be more sensitive to the toxic effects of a compound than laboratory test animals (10×) and that some humans could be more sensitive than others (10×). Additional uncertainty and/or safety factors could be applied for potential sensitivity of the young, severity of the endpoint and completeness of the database.

The basic equations for acute dietary food risk assessment are as follows:

Exposure (mg/kg bw/day) =
Consumption (kg food/kg bw/day) × Residue (mg pesticide/kg food)

ARD (mg/kg bw/day) =
(NOAEL (mg/kg bw/day)) ÷ (uncertainty + safety factors)

The value of the % ARfD reflects the relative size of the ARfD and the estimated exposure, termed the potential daily intake (PDI). If the estimated exposure is less than the ARfD, the value will be below 100%. Conversely, if the exposure is estimated to exceed the ARfD, the value will be greater than 100%. Traditionally, if the % ARfD is less than 100%, the estimated exposure is considered "safe".

The safety standard, described herein, is termed the TOC and is defined as the threshold at which dietary exposure from aggregate food residues is considered safe. That is, the PDI at the 99.9th percentile compared with the acute reference dose is less than 100%, or stated another way, the TOC is the point at which the aggregate exposure from food residues, at 99.9%, is equal to the acute reference dose. This concept is the basis of this policy/guidance document.

Databases used in probabilistic dietary exposure estimates

Currently, the PMRA is developing acute, probabilistic dietary exposure assessments using Monte Carlo techniques that require data on (1) the distribution of daily consumption of specific commodities (wheat, corn, apples, etc.) by specific individuals (in g commodity/kg bw/day), and (2) the distribution of concentrations of a specific pesticide in those food commodities (in µg pesticide/g commodity).

The latter information is generally obtained from crop field trials, the USDA's Pesticide Data Program (PDP) data, CFIA and USFDA monitoring data, market basket surveys conducted by the registrants, Health Canada (HC) or U.S. FDA, and other sources while the former is collected by USDA in its Continuing Survey of Food Intake by Individuals (CSFII). These two input data sources, the USDA CSFII and the residue data sources, are discussed below.

Primer on interpretation of exposure distribution curves

Traditionally, the PMRA has selected a regulatory TOC (e.g., an individual cancer risk of no greater than a range of one in a million or a % ARfD of no more than 100%) that could not be exceeded. The risk threshold was derived by calculating a high-end (or bounding) point estimate of exposure using certain high-end exposure assumptions and combining it with the toxicological endpoint to determine whether a hypothetical high-end individual exceeded the regulatory TOC. If so, exposures were deemed to be unacceptable and mitigation actions were generally sought. However, it was not known whether the PMRA's high-end exposure estimate represented the 95th, 99th, 99.9th, or 99.999th percentile individual or if the high-end exposure estimates were well beyond the exposures received by even the maximally exposed individual (i.e., if high-end exposure estimates were above the 100th percentile).

With the advent of Monte Carlo analysis, the PMRA is no longer limited to assessing exposure and risks to the population using methodologies which produce only a single high-end point estimate. Monte Carlo analyses permit the risk assessor to not only produce more accurate estimates of exposure but to produce estimates of exposure across the entire population that incorporate the probabilities of being subjected to these exposures. This distribution of exposures can be represented graphically as a probability density function similar to the classic bell curve. An example of one of these curves is illustrated in the U.S. EPA document Choosing A Percentile of Acute Dietary Exposure as a Threshold of Regulatory Concern (March 16, 2000; see pp. 38 and 39).

Food Consumption: USDA Continuing Survey of Food Intakes by Individuals

The following information provides detailed background information on the food consumption data used in the PMRA dietary risk assessments (DRAs). The PMRA has determined that these data, collected from surveys conducted in the United States (U.S.), reasonably reflect the food intakes for Canadian populations.

The food survey data used in the PMRA's and U.S. EPA's probabilistic exposure and risk assessments are collected by the USDA and are currently from the 1996-1998 CSFIIFootnote 2. The CSFII surveys are conducted as separate 1-year surveys and were designed to measure what Americans eat and drink.

In accordance with federal data reporting guidelines, the USDA identifies and cautions users of its databases about the lack of adequate numbers of data points for certain statistical projections. For example, some of the commodities for which the U.S. EPA sets tolerances are eaten so infrequently that the USDA cautions against using the survey data to estimate high-end percentiles of consumption of such commodities, e.g., the 95th percentile or greater.

The CSFII data were derived from information provided by individuals (e.g., 15 128 individuals in the 1991 survey) who participated in the survey. One- to three-day food and nutrient intake data for individuals of all ages were collected between for each survey year. Individuals who took part in the survey were asked to provide three consecutive days of dietary data. The first day's data were collected in a personal in-home interview using a 1-day dietary recall. The second and third days' data were collected using a self-administered 2-day dietary record. Intake amounts were reported, and energy and nutrient intakes were calculated using the USDA Nutrient Database for Individual Intake Surveys. Subject to the cautions about statistical treatment of data, the data collected for such large numbers of survey participants, who have been scientifically selected so that results could be projected from the sample to the U.S. population, constitute a reliable and representative national sample.

Residue data sources: Field trials, monitoring, and market basket surveys

In addition to the food consumption data provided by the USDA's CSFII, information on the distribution of residue levels in foods is necessary in order to calculate exposure and risk in a probabilistic manner. Data on the distribution of residues on foods for use in PMRA's probabilistic exposure and risk assessments can be obtained from a variety of sources including:

  1. crop field trials;
  2. CFIA and U.S. FDA enforcement monitoring;
  3. USDA PDP monitoring;
  4. specialized market basket surveys (sometimes termed total diet studies, conducted by a pesticide registrant, HC or U.S. FDA); and
  5. studies on the effects of commercial processing, peeling, washing, cooking or other activities that may affect residue levels.

Crop field trials are experimental trials, usually performed by a pesticide company, in which the maximum usage scenario (with respect to application rate, number of applications, preharvest interval (PHI), etc.) is simulated. These PMRA-required experimental trials are conducted according to Agency guidelines, primarily to determine maximum residues that may be present in fruit, vegetable, grain and other food and feed crops at the earliest point where these food commodities could enter commerce. These data are used to establish legally enforceable pesticide MRLs.

In contrast to the pesticide residue data collected during the experimental field trials, CFIA, U.S. FDA and USDA pesticide monitoring data (as well as registrant-sponsored, market basket survey data) represent residue data in crops collected from commercial trading channels (wholesalers, warehouses, distribution centers, retailers, etc.). These data better represent pesticide residues to which consumers are actually exposed, because they measure residues in food in commercial channels that is closer to the consumer than food sampled following experimental field trials conducted under maximum application scenarios.

The PMRA prefers to use data from CFIA, U.S. FDA or USDA PDP monitoring data or market basket surveys, when available, in calculating pesticide exposure estimates. However, these data are not always available or appropriate for use; when this is the case, the PMRA uses pesticide residue data collected from the experimental field trials.

As the field trial data represent residues resulting from a maximum application scenario to which only very few crops are actually subjected, the PMRA may refine these data to take into account other factors such as residue degradation as a result of transport or storage, or variabilities in farming practices such as use of longer than label PHIs and lower than label application rates (see U.S. EPA 1999b). In addition, the PMRA's exposure estimates can be modified or adjusted, as appropriate, to take into account decreasing or increasing concentrations in processed commodities as a result of commercial processing practices or (generally) decreased residues as a result of cooking or in-home preparation such as washing, peeling, coring, etc. (see U.S. EPA 1999c). Finally, information on the percentage of the crop that is treated, if available, is also used to adjust the probability of encountering a treated commodity.

Dietary Exposure Evaluation Model

Until 1998, the PMRA used a deterministic assessment model to conduct its acute DRAs for pesticide residues in foods. Acute assessments assumed that 100% of a given crop with registered uses of a pesticide was treated with that pesticide and that all such treated crop items contained pesticide residues at the MRL.

The resulting acute risk estimates were considered high-end or "bounding" estimatesFootnote 3. However, it was not possible to know where the pesticide exposure estimates from the older software fit in the overall distribution of exposures due to the limits of the tools being used. Thus, risk-management decisions were being made not only without a full picture of the distribution of risk among the population but also without full knowledge of where in the distribution of risk the risk estimate lay.

The PMRA is now using the Dietary Exposure Evaluation Model (DEEM) computer software program for its dietary exposure and risk assessments for pesticide residues in food. Like the previous model, DEEM calculates acute and chronic risk using the inputs of pesticide residues in and on food, food consumption and toxicity.

Also, like the old model, DEEM is able to calculate an estimate of the risk to the general population in addition to two population subgroups, including five subgroups for infants and children:

  • General population
  • General population, spring
  • General population, summer
  • General population, autumn
  • General population, winter
  • All infants
  • Nursing infants
    (<1 year)
  • Non-nursing infants (<1 year)
  • Children 1-6
  • Children 7-12
  • Females 13-19, not pregnant or nursing
  • Females 20+, not pregnant or nursing
  • Females 13+, pregnant/not nursing
  • Females 13+ nursing
  • Males, 13-19
  • Males 20+
  • Females 13-50
  • Seniors 55+

REGIONS

  • Northeast
  • Midwest
  • Western

Unlike the older exposure model, DEEM can generate probabilistic assessments of acute dietary food exposure. DEEM uses a mathematical technique called Monte Carlo analysis to generate estimates of the distribution of pesticide dietary exposures. That is, it uses all the individual food consumption and pesticide residue level data points included in a data set to determine the combined (or joint) distribution of exposures (and associated risk).

At this time, the PMRA uses the DEEM probabilistic (Monte Carlo) model to develop probabilistic exposure estimates only for acute assessments. The Monte Carlo technique provides a relatively new tool in exposure assessment for more accurately estimating the complete distribution of exposures and provides probabilistic and statistical assessment of dietary risk using more refined information than was used previously. This analysis uses the actual distribution of pesticide residue levels from either the experimental field trials performed by the registrant or monitoring or market basket surveys, whereas in the older model only a single, high-end residue value was used. Also, it can incorporate information on the percentage of the crop that is treated. That is, it includes the actual distribution of possible consumption and residue values and weighs these possible values by their probability of occurrence. Using Monte Carlo techniques, the PMRA does not assume (as was previously done with the older model) that 100% of the crops with registered uses are treated with the pesticide of interest or that all residues are present in crops at maximum legal (MRL) levels. Rather than the crude high-end, single-point estimates provided by the older model, Monte Carlo assessment provides more accurate information on the range and probability of possible exposure and their associated risk values.

Monte Carlo techniques are, in and of themselves, neither more conservative nor less conservative than the older system they supplement: the "conservatism" is determined, in part, by the risk manager when he or she determines the appropriate percentile of the model's output distribution (e.g., 99.9th percentile) to be used for regulation in conjunction with the nature of inputs selected and assumptions used. Monte Carlo and probabilistic techniques are simply tools that potentially allow the risk assessor and manager to see a more accurate distribution of risks among the general population and subpopulations.

95th Percentile Dietary Food Exposure Versus Monte Carlo 99.9th Percentile Dietary Food Exposure

The Agency has in the past used the estimated 95th percentile of exposure in calculating a TOC with an acute dietary risk analysis. Concerns have been raised about what is seen by some as a significant "raising of the bar" by now choosing to refer to the estimated 99.9th percentile of exposure from a Monte Carlo analysis. While it may appear at first that the Agency is taking a more stringent approach, this is actually not so.

Estimated exposure at the 99.9th percentile calculated by DEEM probabilistic techniques is significantly lower than exposure calculated by DEEM using older, nonprobabilistic assumptions at the 95th percentile for most cases reviewed by the PMRA to date. There are several reasons for this. An acute dietary risk analysis assumes that residues are present at MRLs in all crops that have registered uses and 100% of the crop is treated at maximum label rates and harvested at minimum PHI.

In general, Monte Carlo techniques will provide lower and more realistic estimates of exposure than previous techniques when:

  • a lower percentage of the crop is treated (e.g., 10% rather than 100%);
  • the bulk of residue values from crop field trials are present at low levels and there are only a few high values; and
  • a greater number of crops are registered (e.g., 10 crops instead of 2 crops).

For example, a given food item (e.g., cherries) can have several dozen or more individual residue values generated from experimental field trials for a certain pesticide. In an acute old model analysis, only the highest residue value (or MRL level) would be used, and all registered crops would be assumed to be treated and contain these high residue values. In a Monte Carlo run, the entire set of actual residue data points generated in the crop field trials and the percentage of the crop which was treated would be considered.

The differences between the estimated exposure numbers generated by these two techniques can be substantial, with the Monte Carlo generated estimated exposures (at the 99.9th percentile) frequently many times lower than deterministic estimated exposures (at the 95th percentile).

Table 1 illustrates some of these extensive differences in exposure estimates for a widely used agricultural pesticide which was recently evaluated by the PMRA.

Comparison of DEEM 95th percentile exposure and % ARfD estimates from a Tier 1 analysis with Monte Carlo 99.9th percentile exposure and % ARfD estimates from a Tier 3 analysis for one widely used agricultural pesticide (Expressed on a per capita basis using 1989-1991 CSFII data)
Population Subgroup Exposure (mg/kg bw/day) % ARfDFootnote a
DEEM 95th percentile estimate (Tier 1) DEEM Monte Carlo 99.9th percentile estimate (Tier 3) DEEM 95th percentile estimate (Tier 1) DEEM Monte Carlo 99.9th percentile estimate (Tier 3)
General population 0.0192 0.0013 770 50
Infants 0.0375 0.0007 1500 38
Children 1-6 0.0402 0.0017 1610 67
Females 20+/np/nnFootnote b 0.0126 0.0011 510 45
Males 20+ 0.0119 0.0014 480 55
Footnote 1

The % ARfD represents the portion of the acute "risk cup" which is occupied. The % ARfD is obtained by dividing the estimated exposure at any given percentile (e.g., 95th or 99.9th percentile) by the ARfD. Comparison of the estimated exposure to the resulting ARfD is then done to determine the acceptability of that exposure.

Return to footnote a referrer

Footnote 2

Females 20+, not pregnant, not nursing.

Return to footnote b referrer

As can be seen, estimated exposures (and corresponding % ARfDs) are significantly lower at the 99.9th percentile DEEM/Monte Carlo analysis than they are at the 95th percentile using a deterministic dietary risk analysis; this is almost invariably the case. In fact, at all comparable percentiles, the exposure estimates derived from DEEM/Monte Carlo are lower than the corresponding older estimates. The advantage of this probabilistic technique is that it can refine the exposure and risk estimates by more fully incorporating all available information and minimizing reliance on values chosen more for their regulatory and administrative convenience than their scientific merit.

DEEM/Monte Carlo analysis tends to provide a lower, more accurate and more reliable estimate of actual exposure in exactly those situations where older analyses are least realistic. The PMRA will continue to use the estimated 95th percentile of exposure in calculating a TOC when actual tolerance levels and 100% crop treated assumptions are used during exposure assessment but recognizes that this approach can significantly overestimate actual exposure levels. In those cases where exposure estimates at the 95th percentile using less refined assumptions are greater than the regulatory TOC, the PMRA's policy is to use Monte Carlo techniques to assess estimated exposure at the 99.9th percentile, using more refined data.

In practice, risk assessments done at the estimated 99.9th percentile using more refined data almost invariably result in lower estimated exposures (and corresponding estimated risk) than assessments performed at the estimated 95th percentile of exposure using less refined data.

Issues related to the methodology and databases used in acute dietary risk assessment

Concerns have been raised among the academic, public health, industry and grower communities with regard to the appropriateness of the estimated 99.9th percentile of exposure as the default decision point for regulation when using probabilistic techniques for acute dietary risk assessment. Specifically, these concerns include: the presence of "outliers"Footnote 4 in the pesticide residue and food consumption data; the representativeness of the data sets used in Agency risk assessments; the limited size of the input databases; the reliance on "uncertain" consumption values that fall at the extreme tails of the distribution when generating exposure estimates; potential variability/uncertainty in the recipes used to convert the reported food items (on an "as consumed" basis) to agricultural commodities used in the DEEM software; and the degree to which the PMRA's 99.9th percentile estimate incorporates conservative default assumptions.

Because of these areas of concern, issues have also been raised about the interpretation of the output developed by the Monte Carlo technique. Some contend that, if the input data are not reliable and representative, neither are the outputs of any technique using such data. Therefore, they contend that the Agency should not use the 99.9th percentile of estimated exposure as a starting point for regulatory decision making and/or should make adjustments in the data sets which are inputs to the exposure assessment.

Treatment of high-end consumption values ("Outliers") in the USDA CSFII Survey

Concern has been expressed that the USDA's food consumption data have not been properly evaluated to identify potential errors in the data sets or to assess the potential impacts of outliers on the estimated 99.9th percentile of exposure. As a result, some contend that errors are propagated throughout the PMRA's Monte Carlo analysis resulting in distributions that inappropriately and artificially inflate estimates of risks at the upper ends of the distribution. Consequently, some believe that tests for outliers should be conducted, and outliers should be removed from the data set, so that the high end of estimated risk is not defined by the outliers. They state that the PMRA's failure to do this means that the results may not be reliable or scientifically based.

The Agency shares this basic concern and does not want to use data that are not reliable measures of food consumption. By the same token, the PMRA does not want to ignore data which measure real, but perhaps relatively infrequent, consumption events. Anytime a survey as extensive as the USDA's CSFII is conducted, high consumers of a particular food item will be found and reported. It is also true that some portion of the food consumption reports in the initial database may be erroneous. Thus, outliers may be present in the raw survey data. Given that very high food energy intakes do occur in the American population (even though they are not common), considerable judgment is required to determine whether a high-end value should be declared in error and discarded or should be retained.

To ensure that the 1996-1998 CSFII database was free of erroneous or unreliable data points, the USDA extensively validated and cross-checked any questionable survey results prior to their insertion into the 1996-1998 CSFII database.

All reported high intake values retained in the 1996-1998 CSFII database have been checked by USDA and resolution or adjudication of values outside specified ranges have been accomplished. Thus, the USDA 1996-1998 CSFII database has been properly evaluated and contains accurate and reliable consumption values that the PMRA will use in assessment of human dietary exposure to pesticide residues. The PMRA has similar confidence in the 1996-1998 CSFII survey data.

The PMRA does, however, recognize that unusually high intakes can potentially "drive" calculated exposure and risk estimates and believes that it may be inappropriate to base risk management decisions on unusual consumption values, particularly if these consumption values dominate high-end exposure estimates. Therefore, the PMRA has decided that risk characterizations will include a "sensitivity analysis" which will take advantage of a recent upgrade to the DEEM software program which is now capable of generating a "Critical Exposure Contribution" (CEC) analysis when run in the acute Monte Carlo mode. The CEC provides insights into the sources contributing to the exposure estimated for the most highly exposed people in the exposure distribution. This listing contains a detailed exposure analysis for individuals having a total exposure greater than a user-specified "CEC exposure value" (at present, typically around the 99.9th percentile of exposure) in the user distribution profile. The display includes key demographic information (gender, age, body weight), the food(s) consumed, amount consumed, the residue value, the total daily exposure estimate, and the exposure estimate by food. Thus, the CEC provides the PMRA with comprehensive information on foods (and food forms) that account for the largest portion of the person's estimated exposure. If the PMRA finds that the high-end exposures are principally driven by suspect high-end consumption values, the PMRA's risk mitigation decisions can appropriately consider and weigh these factors.

Treatment of high-end residue values ("Outliers") in crop field trial or monitoring data

As with the food consumption data, some have stated that the residue data included in Monte Carlo assessments have not been properly evaluated to identify potential errors in the data sets or to assess the potential impacts of these outliers on the estimate of the 99.9th percentile risk. The PMRA acknowledges that it is not uncommon, when field trial residues comprise the data sets used in a probabilistic assessment, that these data include one or more residue values which are significantly higher than the other measured concentrationsFootnote 5. Just as with food consumption data, it is important to assure that these data are as accurate as possible. Retaining an erroneous highend value may result in overestimating exposure, but discarding accurate high-end values may lead to an underestimate of exposure.

Even though the PMRA may previously have reviewed and relied on a data set, each pesticide residue point in the residue data sets included as input to any Monte Carlo analysis is carefully reviewed and verified by PMRA staff scientists because of the recognized potential impacts outliers could have on the high-end exposure estimates. The PMRA's longstanding approach to outliers has been articulated in the recent U.S. EPA draft document Guidance for Submission of Probabilistic Exposure Assessments to the Office of Pesticide Programs (U.S. EPA 1998a). The decision to discard an outlier is based on a scientific or quality assurance (QA) basis and is only made with extreme caution, particularly for environmental data sets which can often contain legitimate extreme values. The PMRA believes that statistical tests can be used to identify possible outlier data points that require further investigation, but that it is inappropriate to eliminate outliers from analysis on this basis alone unless further review of the suspect data points reveals a significant mistake in protocol, which renders a generated residue value irrelevant to label conditions (e.g., wrong tank mix concentration, mistaken application rate, too early a PHI, too many applications, etc.) or there is some other basis to conclude that the data point is not appropriate for use. This is particularly true in cases where the data points in question have been used by the Agency in establishing a tolerance or other regulatory limit.

Occasionally, high-end values may be found among the data from the CFIA, USDA or U.S. FDA monitoring programs. The Agency relies on the extensive quality assurance/quality control (QA/QC) procedures followed by the CFIA, USDA's PDP program and the U.S. FDA program to determine which data points should be retained. Therefore, the PMRA normally does not discard any of these values without other evidence of their invalidity.

For example, the monitoring data provided by USDA's PDP are collected under rigorous QA/QC procedures which include method validation (determination of limit of detection (LOD) and limit of quantitation (LOQ) for each pesticide/crop combination), confirmation of residue identity by alternate detection system, use of blanks, spikes, and internal and external standards, as well as verification of the analyst's performance (check samples, audits, etc.). Similarly, the U.S. FDA and CFIA use official analytical methods that include blanks and spikings and require confirmation of residues of regulatory significance by use of an alternate detection system and verification of results by a different analysis. These measures are intended to ensure integrity of monitoring data from the sample collection to data reporting.

As with high-end consumption values from the USDA food consumption survey, the PMRA scientists will, as part of the risk characterization, inform the risk manager if high-end residue values are driving the upper ends of the exposure using DEEM's CEC analysis. If specific pesticide residues on certain crops substantially contribute to the majority of the exposures above the specified percentile, the risk manager can incorporate this information into the risk decision and determine an appropriate Agency response.

Representativeness

The PMRA combines both residue data (from crop field trials, monitoring programs, and market basket surveys) and food consumption data (from USDA CSFII surveys), using the DEEM computer software program to generate estimates of the distribution of daily exposures to pesticide residues in food for the general population and 22 specific subgroups within the general population. The reliability of the estimate of the distribution of exposures depends on the quality of the data used in the model. The data sets used should be sufficiently representative to support reliable estimates.

The relationship between representativeness and the reliability of estimates of the distribution of exposure is easy to understand. Even if all of the values in a database are accurate (see the discussion in the previous two subsections on outliers), the use of a data set in a probabilistic assessment will produce unreliable exposure estimates to the extent that the data sample is unrepresentative of the larger population it purports to represent. For example, if no one from low-income groups were interviewed about their eating habits, the survey results would miss the very real impact that income has on dietary choices.

The food consumption data used by the PMRA are collected by the USDA (and as stated previously, are considered representative of the Canadian general population and all subpopulations) through a survey that is carefully designed to assure that the results would be representative of all populations. The survey design specifically requires that samples be collected from people who differ in ways that could affect the types and amounts of foods they eat.

For example, the survey covers people of different ages, genders, ethnicity, regions of the country, and socioeconomic status. People who are selected for interviews are contacted on different days of the week, scattered throughout the year to capture differences due to the time of year or day of the week. A number of other aspects of the survey are also controlled in order to maximize the prospect that the results are representative not only of the entire general population, but also particular subgroups, including those for which the PMRA generates acute dietary food exposure distributions.

While the USDA food consumption surveys are designed to be generally representative of the U.S. population, it is clear that some factors that can influence dietary choices are not addressed in the survey design. For example, the CSFII surveys do not purport to be representative of people in institutional living arrangements (colleges, nursing homes, etc.) or of different religions or health status. In addition, concern has been expressed about how "representative" the survey results are at the high ends of consumption. This concern, in effect, involves the size of the food consumption databases. The Agency addresses this concern in The following subsection of this paper and presents a summary of the results of further analysis of the methodology in the section, "Issues Related to Public Health Policy".

The various databases on pesticide residues in food raise different set of issues with respect to "representativeness". If market basket or monitoring data are not available, the PMRA will use residue data sets generated by the registrants and submitted to the Agency for MRL-setting purposes. The field trial studies are designed to follow the directions on the product labeling and are required to be performed in different areas of the country where the crop on which the pesticide is being used is grown. Multiple field trial sites are required if the crop is a significant component of the diet (e.g., wheat, corn, tomatoes, etc.) and if it is grown in geographically and climatically distinct regions. Of necessity, these are conducted at maximum label rates and minimum label PHIs in order to establish MRLs (tolerances) on food. Thus, the data set resulting from the required field trials represents the distribution of residues that are likely to be found in a particular raw agricultural commodity following a maximum label application scenario. However, due to the design of these field trial studies, the data are not likely to be representative of the residue values in food, as consumed. As discussed in the section, "Databases Used in Probabilistic Dietary Exposure Estimates", adjustments can be made to these data to better represent the amount of pesticides actually used (incorporating, for example, the range of typical application rates and typical PHIs and percent crop treated). Further adjustments and refinements can also be made to better reflect actual exposures; these can include cooking studies, residue degradation studies, washing/home processing studies, etc.

An even more representative picture of the amounts of pesticides in food to which the general population is exposed can be obtained when the PMRA uses data from market basket surveys or from USDA PDP or U.S. FDA monitoring. These data sources are considered to be more "representative" of actual exposure to consumers than field trials conducted under conditions using maximum rate, minimum PHI, and other use conditions likely to lead to the highest lawful residue. Market basket surveys, for example, are statistically designed and are conducted on a single-serving basis at the point of sale to consumer. These types of studies, thus, best reflect those residues to which consumers are actually exposed. The USDA exercises great care to assure that the food items sampled in their programs are representative of the large majority of that type of agricultural commodity consumed in the country. These monitoring data are also designed to be statistically representative of commodities that are typically available throughout the year, except that they represent 5-lb (2.3 kg) composite samples (and not single-serving items) collected at distribution points just before release to supermarkets and grocery stores. In addition, commodities are washed, peeled, de-stemmed, or cored, as appropriate, prior to laboratory analysis to represent typical consumer practices. CFIA and USFDA surveillance monitoring data are geared more to MRL/tolerance enforcement and not toward the PMRA's risk-assessment needs. Collection occurs as close to the farm gate as possible, and the program is not designed to generate statistically representative samples for use in risk assessments. Due to sampling and collection methodologies, residues measured under the CFIA and U.S. FDA surveillance monitoring program likely overestimate pesticide residues to which consumers are exposed. Nevertheless, they are considered more representative of residue levels to which consumers are exposed than the experimental field trial data submitted for tolerance-setting purposes.

Size of the input databases

In addition to being accurate and representative, the data sets should also be sufficiently large to permit characterization of the overall exposure of the population of interest. As noted earlier, each person in the USDA's CSFII survey currently being used by the PMRA was asked to contribute information for one to three different days. The CSFII database used in the DEEM software collectively represents food records for thousands of unique person days. Moreover, since each person typically eats many different types of food during a day, there are a great many data points for consumption of specific foods in the database. Table 2 summarizes some of the 1989-1996 survey information for several of the 20 populations that the PMRA evaluates in its dietary exposure assessments.

Size of 1989-1991 and 1994-1996 CSFII (with SCS) Database
Size of 1989-1991 CSFII database (all three days) Size of 1994-1996 CSFII database with supplemental children's survey (both days)
Population Group Number of Respondents Number of Intakes Number of Respondents Number of Intakes
General population 11 912 35 736 20 607 41 214
Infants <1 202 606 1486 2972
Children 1-5 1067 3201 6487 12 974
Children 6-11 1172 3516 1913 3826
Females 12-49 3459 10 377 3021 6042
Males 20+ 3381 10 143 4751 9502

NOTE: The PMRA is currently using the 1996-1998 CSFII data set, but expects to start using the combined 1998-2000 CSFII database and the Supplemental Children's Survey in the second quarter of calender year 2002.

Despite the overall large scope of the USDA CSFII database, some contend that USDA survey population sample sizes are of insufficient size to provide reliable estimates at the high end of exposure and risk and that there is a need for specific criteria for a minimum number of samples before an estimate is derived and used in establishing policy.

The major focus of this discussion has been the small number of data points at the extremes of the consumption distribution for any given commodity. In other words, a very small number of people may have reported having eaten a food containing a particular commodity. Some state, in particular, that for many infrequently consumed commodities or for small population subgroups, an adequate number of individuals is not available to calculate a high-end consumption percentile. They say, therefore, that the percentile exposure represented by high-end consumers of infrequently eaten foods is highly uncertain. They further note that the USDA has identified minimum population sample size criteria for estimating various percentiles of food consumption and recommend that the USDA flag estimates that do not meet these criteria. They believe that the agency should not use data points that would fall at a percentile that would be flagged by the USDA. Rather, they argue that such high-end (and "uncertain") values should be discarded (or otherwise adjusted) prior to using the data set to perform probabilistic exposure analyses.

The PMRA recognizes that there are limits with respect to the USDA food consumption database that would affect the reliability of estimates of high-end consumption of particular commodities. In particular, for many infrequently consumed commodities and for small population subgroups, an adequate number of individuals may not be available to calculate a reliable high-end consumption percentileFootnote 6. However, the ability to define a high-end consumption percentile for each commodity in an exposure assessment is not necessarily critical to ensuring that the exposure assessment can define high-end exposure. As discussed more fully below, the concern about the size of the input database is not directed at whether the data sets are adequate to define high-end percentiles of pesticide residue levels in a particular food or consumption of a specific food form. Rather, the concern is whether the databases are sufficiently large to characterize accurately the distribution of daily pesticide exposures from all foods which an individual eats in any given day. The distinction between estimating high-end percentiles of exposure and high-end percentiles of consumption (or residue) for a particular commodity is an important one. It may be that the part of the exposure distribution that is derived from (or includes) any single (presumably uncertain) upper-end USDA consumption value does not necessarily produce an invalid exposure value. Many of the upper-end exposure estimates might not contain upper-end USDA consumption values, and thus, these uncertain USDA consumption values may not be driving the high-end Agency exposure estimates at all.

Although this concern might need to be heightened when PMRA's probabilistic exposure assessments involve only a single commodity with few residue data points, one or even a few very high food consumption values do not appear likely to be the primary driver(s) of exposure and disproportionately influence the outcome of the DEEM exposure estimates (see the section, "Issue Related to Public Health Policy", for a brief summary of the testing results. The more commodities that are included in the analysis, the more unlikely it is that the upper-end exposure values are driven by upper-end USDA consumption values.

Finally, as discussed above with respect to the accuracy and representativeness of the input databases, the PMRA will perform a sensitivity analysis on all probabilistic assessments of dietary exposure. The CEC module will identify the critical input data points, and the Agency can decide whether to rely on the estimates of high-end exposure in its risk management decisions.

Potential variability/uncertainty in recipe translations

Another area of expressed concern is potential variability/uncertainty in the recipe translations used to convert foods on an "as eaten" basis (e.g., pizza) to foods on an agricultural commodity basis (e.g., wheat flour, tomato paste/puree, milk, beef, etc.). In other words, the USDA CSFII survey requests individuals to report the foods consumed (as consumed) on any given day, but these foods have to be subsequently expressed on an appropriate agricultural commodity basis so that processing factors, appropriate matching and other considerations can be incorporated into the analysis. This recipe translation information is an intrinsic component of the DEEM software, and concern has been expressed about the potential for these standard recipe translations to inadequately reflect the full range of actual recipes used in practice. For example, if the standard recipe for 100 g of pizza is assumed to contain 15 g of tomato paste instead of only 10 g (which may have been present in the particular piece of pizza eaten), then tomato paste consumption would in this instance be overestimated. The concern is that it is impossible for a single standard recipe to be universally applicable and to account for all "real-world" variations which may exist.

At present, the recipe translation files used by the DEEM software are proprietary and not available for review. The USDA and PMRA have, however, collaboratively developed recipe translations, which are expected to be publically released early in the year 2000 and to be incorporated shortly after that time into the DEEM software. These translations have been peer reviewed by government and private industry food and nutrition experts.

While the PMRA acknowledges that it is impossible for one standard recipe to reflect the variability that exists in ingredients selected and quantities used in the many kitchens and food manufacturing facilities across the U.S., the PMRA does not expect these to be significant sources of error in its exposure estimates. Firstly, in many instances examined to date it is the fresh agricultural commodity, such as raw apples and tomatoes, that are found to be the primary risk drivers, and commodities in these forms are not subject to translation error (they are not translated). Only rarely would canned baked apples (as in apple pie) or tomato paste/puree (as in pizza sauce), for example, be expected to be risk drivers. Secondly, even considering the variability that exists in the multitude of actual recipes which are used, this variability is unlikely to be particularly significant or to introduce substantial error in our exposure estimate. If, for example, the standard recipe for 100 g of pizza dough contains 70 g of flour, it is unlikely that the "true" amount differs from this by more than 20-30% as there are defined limits as to how far a recipe used in practice can deviate from a standard recipe and still produce an identifiable and edible product. Finally, variability in recipes can result in either overestimating or underestimating consumption amounts and exposures and the nonsystematic nature of this error (for each of many translations) is not expected to contribute to large errors, which occur predominantly in any one direction in estimated exposure.

Impact of Agency default assumptions on the choice of a percentile exposure estimate for the threshold of concern

Some contend that the use of conservative default assumptions by the PMRA, in its treatment of pesticide residue data, results in the estimate of the 99.9th percentile exposure being significantly higher than the actual 99.9th percentile exposure. They point specifically to the PMRA's use of maximum rate/minimum PHI field trial data in the exposure assessment, the PMRA's treatment of nondetects (NDs), and its use of 95th percentile data from monitoring studies. The PMRA agrees that these three assumptions would lead to an overestimate of dietary exposure, but the PMRA uses these assumptions only in its early tiered, screening assessments. The PMRA's policy is to rely on a screening estimate of exposure only if the estimate indicates that risk would be acceptable. Because such screening estimates overstate exposure, the PMRA refines its exposure and risk assessments using more realistic data. These refined, higher-tiered estimates do not use the conservative default assumptions likely to overestimate risk. For more information on these tiers, see the U.S. EPA document, Guidance for the Submission of Probabilistic Human Health Exposure Assessments to the Office of Pesticide Programs (U.S. EPA 1998a).

With respect to residue data, the PMRA prefers to use data (when appropriate and available) from market basket surveys, or USDA PDP, CFIA or U.S. FDA surveillance monitoring data in conducting its pesticide exposure assessments rather than from field trials. However, these market basket or monitoring data are not always available or appropriate for use. When exposure data are obtained from field trials, these data can be modified or adjusted to take into account decreasing or increasing concentrations in processed commodities as a result of commercial processing practices or decreased residues as a result of cooking or in-home preparation such as washing, peeling, coring, etc. The PMRA is also able to incorporate information about lower than label-specified application rates and longer than label-specified PHIs, if available (see U.S. EPA 1999c).

Concerning the treatment of NDs, PMRA has issued a document containing guidance for handling these type of data: Assigning Values to Nondetected/ Nonquantified Pesticide Residues in Food. The Agency describes its policy to use one half the LOD (in place of the full LOD or the LOQ) for treated commodities in cases where the LOD has been adequately documented. As explained in the science policy paper, empirical data indicate that it is not unreasonable to assume that treated NDs contain residues equal to ½ LOD. The Agency will also be performing sensitivity analyses which can determine whether the assumed residue value assigned to the ND values is driving the risk estimate. In this document, the PMRA has also presented a method for dealing with residue data sets in which many of the observations are below detectable levels. The PMRA considers that these refinements, in the treatment of NDs, will alleviate many of the perceived overly conservative biases in exposure estimates with regard to the assigning of values to NDsFootnote 7.

With respect to concern about use of the 95th percentile residue value from monitoring data, this policy has recently been revised. Prior to the availability of the probabilistic software, a registrant could, for blended commodities such as corn, soybean, wheat, etc., use either average crop field trial concentrations (which reflect the maximum label use scenario) adjusted for percent crop treated or 95th percentile CFIA, U.S. FDA or USDA monitoring data in its acute risk assessment. Either of these would be entered as a point estimate for use in an acute risk assessment. The use of 95th percentile monitoring data had been introduced as an alternative to the use of average field trial concentrations, since the PMRA believed that this would be a more realistic (but still conservative) estimate of actual exposures that would take into account actual use practices. Because probabilistic software is now available, the PMRA need not rely solely on point estimates of residue values in its acute dietary risk assessments, and this policy has been revised accordingly. The PMRA no longer uses a 95th percentile point estimate from monitoring data for blended commodities but, instead, uses the entire range of monitoring data and, therefore, incorporates the entire distribution in its exposure assessment using all of the available monitoring data. Thus, the concern about the PMRA's reliance on upper-end (95th percentile) monitoring data for blended commodities in its risk assessments is no longer justified; the PMRA uses the full set of monitoring data, thereby fully incorporating the most refined concentration data available.

Issues related to public health policy

MRLs are set or retained for a pesticide, only if the Agency determines that there is "a reasonable certainty of no harm" from dietary and other nonoccupational sources of exposure. The PMRA will use available and reliable scientific information to characterize the toxicity and exposures of a food use pesticide in deciding whether a particular pesticide meets the safety standard. Put simply, the PMRA's goal is to regulate pesticides in such a manner that everyone is reasonably certain to experience no harm as a result of dietary and other nonoccupational, as well as, occupational exposures to pesticides. The PMRA has to make a risk management judgment about what level of pesticide residue in food is consistent with this standard. The reasonable certainty of no harm standard informs PMRA judgment on the overall risk management decision as well as on component parts of the decision. The PMRA has decided to express its risk management judgment for acute dietary risks in quantitative scientific form, as a TOC. By a TOC, the PMRA means that exposures below the threshold generally would not be grounds for pursuing risk mitigation but that exposures above the threshold would, at a minimum, have to be seriously examined to determine whether they exceeded the statutory standard.

A TOC for acute dietary risks, when based on probabilistic exposure estimates, has two elements:

  1. a percentile (or proportion) of the population for use in estimating exposure to a pesticide residue; and
  2. a benchmark for judging a safe level of exposure.

Due to differences in the various inputs to exposure assessments, the PMRA will vary the population percentile of exposure used to estimate exposure so as to ensure that, in attempting to protect the general population and significant subpopulations (i.e., by not underestimating the exposure to such groups), the PMRA does not unreasonably overprotect these groups (i.e., by unreasonably overestimating exposure). For exposure estimates using deterministic high-end assumptions, risk assessors should generally use the 95th percentile of exposure as a reasonable high-end exposure. For probabilistic assessments using more realistic inputs, assessors should generally use in the first instance the 99.9th percentile of exposure. The ARfD will be used as the benchmark of safety. The rest of this section discusses more fully the PMRA's rationale for choosing the 99.9th percentile and the concerns that have been expressed about that choice.

In adopting this policy, the PMRA recognizes that the choice of the population percentile for formulating a TOC involves a balancing of a number of factors. The PMRA considered a variety of factors in formulating the policy and this policy recommends that these factors be considered in decisions regarding the population percentile in individual risk assessment. The first consideration is the size of the exposed population and the proportion that might receive daily doses above the benchmark of safety, the ARfD. A second consideration is the level of confidence the PMRA has in its exposure estimates and the extent to which such estimates may overstate (or understate) potential exposure, because they incorporate conservative assumptions or rely on atypical and unrealistic data. A third consideration is the degree to which individual exposures would be estimated to exceed the ARfD to the extent this can be understood. A final consideration is the degree of public health protection incorporated into the determination of the ARfD.

Because of the need to balance a variety of factors in selection of a population percentile for calculating a threshold of concern, the PMRA is issuing its views regarding population percentiles as nonbinding policy guidance rather than as a binding rule. Complex risk-assessment and risk-management issues such as those involved in this policy seldom can be reduced to meaningful rule-style commands. Rather, the scientist and risk manager need to have flexibility in considering a variety of factors and outcomes. This policy is intended to focus the analysis on factors deemed most critical without barring consideration of other factors that may be found to be relevant. As a policy, this guidance does not-in fact, as a legal matter, cannot-draw brightlines or preclude reconsideration of basic principles. The PMRA would retain the option to depart from the policy. Furthermore, affected parties remain free to challenge the specific application of the policy or the underpinnings of the policy itself.

In initially determining where to establish the threshold of concern, PMRA considered a number of issues and past practices, including the U.S. EPA document: Guidelines for Exposure Assessment (U.S. EPA 1992b). These Guidelines established a broad framework for Agency exposure assessments by describing the general concepts of exposure assessment and by providing guidance on the planning and conduct of an exposure assessment, including the characterization of uncertainty. Specifically regarding the use of high percentile values, the U.S. EPA in these exposure assessment guidelines has stated the following: Although the USEPA has not specifically set policy on this matter, exposure assessors should observe the following caution when using simulated distributions. The actual percentile cutoff above which a simulation should be considered a bounding estimate may be expected to vary depending upon the size of the population. Since bounding estimates are established to develop statements that exposures, doses, and risks are "not greater than...", it is prudent that the percentile cutoff bound expected exposures for the population being evaluated. For example, if there are 100 persons in the population, it may be prudent to consider simulated exposures above the 1 in 500 level [sic] or 1 in 1000 level (i.e., above the 99.5th or 99.9th percentile, respectively) to be bounding estimates. Due to uncertainties in simulated distributions, assessors should be cautious about using estimates above the 99.9th percentile for estimates of high-end exposures, regardless of the size of the population. The Agency or individual program offices may issue more direct policy for setting the exact cutoff value for use as high-end and bounding estimates in simulations.

Although the USEPA has not specifically set policy on this matter, exposure assessors should observe the following caution when using simulated distributions. The actual percentile cutoff above which a simulation should be considered a bounding estimate may be expected to vary depending upon the size of the population. Since bounding estimates are established to develop statements that exposures, doses, and risks are "not greater than...", it is prudent that the percentile cutoff bound expected exposures for the population being evaluated. For example, if there are 100 persons in the population, it may be prudent to consider simulated exposures above the 1 in 500 level [sic] or 1 in 1000 level (i.e., above the 99.5th or 99.9th percentile, respectively) to be bounding estimates. Due to uncertainties in simulated distributions, assessors should be cautious about using estimates above the 99.9th percentile for estimates of high-end exposures, regardless of the size of the population. The Agency or individual program offices may issue more direct policy for setting the exact cutoff value for use as high-end and bounding estimates in simulations.

Taking this U.S. EPA guidance into account, and giving significant weight to the size of the exposed population, the PMRA uses as a TOC, the 99.9th percentile of estimated daily acute exposure (from food only) using probabilistic exposure estimation techniques. The calculated exposure for residues from all foods and for all subpopulations should be equal to or less than the ARfD. Under this policy, when the 99.9th percentile of estimated exposure is equal to or less than the ARfD, the estimated exposure of the vast majority of people would not exceed safe levels. Only those individuals whose exposure is estimated to be in the very high end of the exposure distribution might theoretically receive amounts of pesticide in their food that even approached the level where concern would exist.

Based on the PMRA's experience reviewing Monte Carlo acute dietary exposure estimates, it appears that those with significantly lower exposures (i.e., at lower percentiles of estimated exposure) would be consuming levels of pesticide in their food potentially several orders of magnitude below the ARfD.

It may be argued that the PMRA's policy is not sufficiently protective, even at the 99.9th percentile of exposure. Because the group eating food containing pesticide residues is very close, if not equal, to the entire population of the country, currently about 30 million people, they argue that if the 99.9th percentile of exposure is equal to the ARfD, very large numbers of people, including many children, could be exposed at levels which exceed the ARfD each day. Thus, one could argue for additional protections to be provided in the risk-assessment or risk-management process.

While the PMRA recognizes that, under this policy, a large number of people-including infants and children-would theoretically be exposed at levels potentially exceeding the ARfD, the Agency believes, for several reasons, that allowing this level of estimated exposure would not raise public health concerns. Firstly, the PMRA believes that actual exposure is unlikely to be greater than that estimated and, in most cases, would actually be somewhat lower than the estimates based on data currently available. As discussed in this and other papers, the PMRA uses a tiered approach and the best available data to develop estimates of exposure to pesticide residues in food. Monte Carlo techniques are used in the PMRA's highest, most refined tiers-the tiers that are designed to provide the most realistic estimates of exposure. Nonetheless, at the present time, because of data limitations, even the most refined estimates of exposure may still include residue values that are higher than people actually consume for one or more commodities. For some pesticide-commodity pairs, for example, the PMRA may not have residue monitoring data, such as the USDA's PDP data. In such cases, therefore, the estimated exposure at the 99.9th percentile (or any other percentile) may overstate potential exposure, and some portion of the most highly exposed 0.1% of the population would, in actuality, be exposed at levels less than the ARfD. Nevertheless, the PMRA does not believe that this significantly overstates exposure, and in any case, the sensitivity analysis of CEC module will identify the extent to which high residue values for specific commodities account for the upper end of exposure. In those instances where data limitations result in an exposure estimate that is highly conservative, it may be necessary to generate better data for the specific commodity of concern. Thus, just as when the PMRA uses the 95th percentile with nonprobabilistic exposure assessments the PMRA is not suggesting that the PMRA is leaving 5% of the population unprotected, the PMRA is not by choosing the 99.9th percentile for probabilistic exposure assessments concluding that only 99.9% of the population deserves protection. Rather, it is the PMRA's view that, with probabilistic assessments, the use of the 99.9th percentile generally produces a reasonable highend exposure such that if that exposure does not exceed the safe level, the PMRA can conclude there is a reasonable certainty of no harm to the general population and all significant population groups.

Secondly, exposure at a level above the ARfD would pose public health concerns only to the extent that such exposures might result in harm. Certainly it would be difficult to justify allowing individuals to get doses of a pesticide at such levels, if the PMRA expected all such exposures to result in harm. From information about the general shape of the distribution curve of dietary exposures, the PMRA expects the vast majority of individuals estimated to be exposed to residues over the ARfD would be exposed only to levels slightly greater than the ARfD. The PMRA believes that its risk estimation methods incorporate sufficiently conservative (health protective) approaches, so that the overall approach provides sufficient protection for the small percentage of people (those above the 99.9th percentile) who may be exposed at levels slightly above the ARfD.

The PMRA sets the ARfD well below (usually 100-1000 times lower than) the appropriately chosen NOAEL in the most relevant laboratory animal toxicity study. Due to conservative modeling assumptions, it is possible that no actual person is exposed to levels predicted by the model. This is why the PMRA has included sensitivity analysis, commodity contribution analysis and other techniques to critically examine the results of the analysis prior to making regulatory decisions.

Thirdly, given the size of the exposed population, the occurrence of an "exceedance" would (at the 99.9th percentile level) be very infrequent for the typical individual. For example, at the 99.9th percentile, the time between exceedances, on average, would be once every 2 to 3 years. Depending on an individual's diet, an exceedance may occur more or less frequently.

Collectively, this information gives the Agency confidence that its approach to protecting people from risks associated with single-day exposure to pesticides in their diet is adequately protective. It may be perceived that the PMRA's policy is overly protective, e.g., that the PMRA's exposure methodology significantly overestimates actual exposure to the extent that the underlying databases include outliers or unrepresentative (and unrealistically high) field trial residue data. A closely allied concern is that exposure estimates overstate exposure when the methodology uses unrealistic, conservative assumptions. These concerns are addressed in the "Issues Related to the Methodology and Databases used in Acute Dietary Risk Assessment" section of this paper and, as discussed there, efforts are made to use only reliable, realistic data. Because of the careful QC measures taken by the CFIA, USDA and U.S. FDA in the generation of food consumption and residue monitoring data, the PMRA typically accepts those data sets compiled by the respective agencies as being reliable and realistic.

The PMRA conducts its own review of residue data from field trials and adjusts these data to better reflect actual residues on food. As discussed in the "Impact of Agency Default Assumptions on the Choice of a Percentile Exposure Estimate for the Threshold of Concern" section of this notice, the PMRA believes that it does not use overly conservative assumptions. In sum, the PMRA does not believe that the databases used, and the ways in which they are used to develop probabilistic exposure estimates, will produce significant overestimates of exposure at the 99.9th percentile.

Another concern is that the databases available for use in probabilistic exposure estimates yield estimates at the 99.9th percentile that are unacceptably uncertain, and because of the uncertainty, the PMRA should use a lower percentile (e.g., the 99.5th, 99th, or 95th percentile) of exposure in its expression of the TOC. While the PMRA agrees that the estimates of the 99.9th percentile of exposure have some uncertainty due to the use of high-end consumption and/or residue data, it does not know whether the probabilistic assessments understate or overstate actual exposure at the 99.9th percentile nor does PMRA expect that one high-end residue or consumption value will "drive" high-end exposure estimates. In order to evaluate the possible impact of high-end values, the PMRA will perform sensitivity analyses at the TOC to determine what data account for the largest part of the estimated exposure.

Recognizing that there was considerable concern about both the scientific and regulatory judgments underpinning the policy, the PMRA has performed further analysis of the methodology to provide both the Agency's staff and the public with a better understanding of the most critical elements of the methodology. Many of the scientific concerns resulted from questions about how this relatively new approach to assessing acute dietary exposure would be performed and what aspects of the methodology had the greatest impact on the outcome. In fact, reliance on probabilistic exposure modeling techniques in regulatory decision making has very few precedents. These analyses were conducted to evaluate a variety of statistical attributes of the distributions produced using the Monte Carlo technique. The activities were designed to provide a better understanding of the most critical elements of the methodology, several of which addressed the issue of the high consumption individual and his effect on the tails of the distribution. Briefly, PMRA drew the following conclusions as a result of these analyses.

  1. For even a reasonably large data set as would be expected for a major agrichemical with extensive nationwide use, approximately 1000 iterations of the DEEM software are adequate to produce reasonably stable exposure estimates at the 99.9th percentile (generally varying less than 1-3%).
  2. Given an adequate number of iterations, exposure estimates from the DEEM software are reasonably reproducible (i.e., any randomly selected run is unlikely to be more than 2% from the "true value" where the "true value" represents the DEEM exposure estimate which would result if an infinite number of iterations were performed).
  3. Extreme consumption events are not pervasive in the USDA's CSFII survey and are unlikely to have a significant effect in controlling the exposure estimate at the 99.9th percentile (i.e., acting as primary "risk drivers"). The PMRA will, in any case, use the capabilities of the DEEM software to identify any extreme eating occasions that might occur and fully characterize any exposure estimates that appear to be driven by high or unusual reported consumption.

If the PMRA adopted a policy that relied on a percentile of exposure lower than the estimated 99.9th percentile, it would need to justify its decision in public health terms as being consistent with the statutory standard to prove safety to human health. As indicated above in the discussion of whether the estimated 99.9th percentile of exposure is adequately protective, the choice of any percentile less than 100% assumes that, to the extent that estimates understate or correspond to actual, real world exposure, some portion of the exposed population could receive an amount of pesticide in excess of the ARfD. As lower percentiles are considered, the estimated size of the population potentially exposed to levels greater than the ARfD increases. Furthermore, if a lower percentile of regulatory concern were selected, a greater proportion of the population would be exposed more frequently to a one-day intake of pesticide residue that exceeds the ARfD by a greater margin.

For example, at the estimated 99th percentile of exposure, on average, individuals would experience an exceedance roughly once over several months. Moreover, the size of the exposed population potentially exceeding the ARfD at the 99th or 95th percentiles would be 10 and 50 times larger, respectively, than the number at the 99.9th percentile. In the PMRA's view, the above analysis raises concerns about routinely using a lower percentile than the 99.9th. If, because of uncertainties associated with using the 99.9th percentile, the PMRA decided to use a lower percentile of exposure as its TOC, it would still have some uncertainty in its assessment of acute dietary risk from pesticide residues in food. At the lower percentiles, the predicted incidence of these exceedances is quite high, and there is a substantial possibility that some significant number of people would be receiving doses that are considerably higher than the ARfD.

However, the PMRA would not have much certainty about either the number of people above the ARfD or, more importantly, how close to the ARfD or NOAEL their exposures might come. Therefore, if the PMRA chose a lower percentile as its TOC, it would also need to consider whether other steps (e.g., use of an additional safety/uncertainty factor) would be needed to assure that the safety standard was satisfied.

Nevertheless, the selection of the estimated 99.9th percentile of exposure for use in calculating the "threshold of regulatory concern" should not be regarded as an immutable "bright line" from which deviation is not possible, regardless of the nature of ancillary data and information. The PMRA retains some discretion to choose a different percentile for regulatory concern if the conditions or situation warrant.

A risk assessment should fully characterize the nature of the analysis for consideration by the risk manager and his or her selection of an appropriate regulatory threshold. A number of criteria should be considered including, for example, the exposure estimate's perceived degree of conservatism considering in particular the identity of the risk "drivers", the reliability and characteristics of the input data, the size of the affected populations, the results of a sensitivity analysis, etc. In this manner, the risk manager can evaluate how supportable the 99.9th percentile exposure estimate is and evaluate whether or not it is appropriate to deviate (up or down) from the 99.9th percentile.

Specifically, a full and adequate characterization of the risk estimates might include a review of the following (in approximate order of relative importance):

  1. whether a "high-end" consumption value actually acts as a "driver" in the risk assessment (in many cases, high-end consumption values may not be actual "drivers", i.e., significant contributors, in the risk assessment and, thus, may not be the primary reason behind high estimated exposures at the tails of the distribution);
  2. how extreme the upper tails of the consumption curve are (e.g., Is the 95th percentile consumption value greater than four times the mean consumption? Is the 99th percentile value greater than six times the mean consumption?);
  3. how far the presumed high-end consumption value is from where it would be expected to be given the pattern (or distribution) of reported consumption values in the lower percentiles (e.g., if a distribution can be reasonably established for the reported consumption values in the lower percentiles, e.g., 70th through 95th percentiles, how extreme would the suspected outlier be in an appropriate Q-Q or other statistical plot);
  4. the size of the affected subpopulation and how likely exposure estimates for the subpopulation would be subject to undue effects of outliers (a high-end value would be expected to have more influence on the upper-end exposure estimates in a small subpopulation than it would in a large subpopulation);
  5. from a dietary standpoint, how likely the high-end value is to be a valid reported consumption event (e.g., although they may be equally extreme from a probabilistic standpoint, consumption of three gingko fruits in a day might be considered more reasonable than consumption of 10 apples);
  6. the nature of the inputs both in the overall assessment and (particularly) for the drivers (this would include, for example, whether input residue data included field trials versus PDP data versus market basket survey data; the use of default versus actual processing factors; the extent to which single-serving values are measured versus established by "decompositing"Footnote 8, the nature and degree of percent crop treated data, etc.); and
  7. comparison of exposure and consumption estimates using different years consumption survey data (if different CSFII data sets produce similar estimates of exposure and contain similar extremes of consumption, it is more likely that highend reported consumption is indeed an actual value or at least not affecting the exposure assessment in any significant way).

In summary, the PMRA believes that the risk assessor should adequately characterize the nature of the assessment (including any biases and uncertainties) and perform a sensitivity analysis, where appropriate, such that the reasonableness of the upper-end percentile estimates (including the 99.9th ) can be properly gauged. Any risk assessment performed by the PMRA should characterize the effect of any high-end points (on the consumption) on the regulatory percentiles of possible regulatory interest. Likewise, it is important for the risk manager, in turn, to consider the entire set of data and information available in deciding if the 99.9th percentile is an appropriate demarcation point for use in regulation. In particular, any risk management decisions should consider the effect of any high-end data values (consumption or residue) or other relevant factors and, when appropriate, be flexible with respect to the regulatory threshold selected. Nevertheless, based on the several dozen risk assessments and sensitivity analyses we have performed to date using probabilistic techniques, we do not expect this review to warrant a departure from the 99.9th percentile in the vast majority of cases.

List of abbreviations

ARfD
Acute Reference Dose
CEC
Critical Exposure Contribution
CFIA
Canadian Food Inspection Agency
CSFII
Continuing Survey of Food Intake by Individuals
DEEM
Dietary Exposure Evaluation Model
DRA
Dietary Risk Assessment
FDA
Food and Drugs Act (Canada)
FDAR
Food and Drugs Act and Regulations (Canada)
FDR
Food and Drugs Regulations (Canada)
HC
Health Canada
LOD
Limit of Detection
LOQ
Limit of Quantitation
MRL
Maximum Residue Limit
NAFTA
North American Free Trade Agreement
NDs
Nondetects
NOAEL
No Observed Adverse Effect Level
PDI
Potential Daily Intake
PDP
Pesticide Data Program
PHI
Preharvest Interval
PMRA
Pest Management Regulatory Agency
QA
Quality Assurance
QC
Quality Control
SAP
Scientific Advisory Panel
SCS
Supplemental Children's Survey
TOC
Threshold of Concern
U.S.
United States
USDA
United States Department of Agriculture
U.S. EPA
United States Environmental Protection Agency
U.S.FDA
United States Food And Drug Administration

References

  • U.S. Environmental Protection Agency (U.S. EPA). 1992a. Memorandum from H. Habicht to Assistant Administrators. Guidance on Risk Characterization for Risk Managers and Risk Assessors. February 26, 1992. U.S. Environmental Protection Agency, Washington, D.C. U.S. Environmental Protection Agency (U.S. EPA). 1992b. Federal Register Notice. Final Guidelines for Exposure Assessment. 57 FR 22888. May 29, 1992. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA). 1997a. Guiding Principles for Monte Carlo Analysis. Appendix A: Policy for Use of Probabilistic Analysis in Risk Assessment at the U.S. Environmental Protection Agency. Office of Research and Development. March 1997. EPA/630/R-97/001.36. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA). 1997b. Memorandum from W. Wood to D. Patton. Proposed Policy for Use of Monte Carlo Analysis in Agency Risk Assessments. January 29, 1997. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA). 1998a. Federal Register Notice. Guidance for Submission of Probabilistic Human Health Exposure Assessments to the Office of Pesticide Programs. Draft. 63 FR 59780 November 4, 1998. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA).1998b. Federal Register Notice. Assigning Values to Nondetected/Nonquantified Pesticide Residues in Human Health Dietary Exposure Assessments. Draft. 63 FR 67063-67066. November 4, 1998. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA). 1998c. Federal Register Notice. A Statistical Method for Incorporating Nondetected Pesticide Residues into Human Health Dietary Exposure Assessments. Draft. 63 FR 67063-67066. December 4, 1998. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA). 1999a. Federal Register Notice. Choosing a Percentile of Acute Dietary Exposure as a Threshold of Regulatory Concern. Draft. 57 FR 16962. April 7, 1999. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA). 1999b. Federal Register Notice. Guidance of the Conduct of Bridging Studies for Use in Acute Dietary Probabilistic Risk Assessment (dated 7/29/99) and Guidance for the Conduct of Residue Decline Studies for Use in Probabilistic Acute Dietary Probabilistic Risk Assessment (dated 7/29/99). Draft. 64 FR 42372. August 4, 1999. U.S. Environmental Protection Agency, Washington, D.C.
  • U.S. Environmental Protection Agency (U.S. EPA). 1999c. Federal Register Notice. Data for Refining Anticipated Residue Estimates Used in Dietary Risk Assessment for Organophosphate Pesticides. (dated 3/26/99). Draft. 64 FR 16967. April 7, 1999. U.S. Environmental Protection Agency, Washington, D.C.
Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: