ARCHIVED - Executive Summary: Look-Alike Sound-Alike (LA/SA) Health Product Names Consultative Workshop

Date 2003-10-20

Contact

A Look-alike Sound-alike (LA/SA) Health Product Names Consultative Workshop was held on October 20-21, 2003 with 38 stakeholders in attendance representing industry associations, government, healthcare professionals and non-government organizations including patient and consumer groups.

The objectives of this workshop were to:

• inform/educate interested and affected parties about LA/SA health product names and the current policy development process;
• ensure accuracy and completeness of the issues identified;
• provide an opportunity for interested and affected parties to give feedback regarding policy options and proposed recommendations; and
• demonstrate an ongoing commitment to relationship-building with affected parties.

After an initial address from Diane Gorman (Assistant Deputy Minister, Health Products and Foof Branch) and welcome from Julia Hill (Director General, Biologics and Genetic Therapies Directorate), the following were presented over the two days:

1. Look-alike Sound-alike (LA/SA) Health Product Names: Developing a Common Understanding
   Michèle Chadwick, Biologics and Genetic Therapies Directorate (BGTD), Health Products and Food Branch, Health Canada
2. Proprietary Name Evaluation at the Food and Drug Administration (FDA)
   Captain Thomas Phillips, Office of Drug Safety, Centre for Drug Evaluation and Research, U.S. FDA
3. Look-alike Sound-alike (LA/SA) Health Product Names: Developing a Comprehensive Policy Recommendation
   Michèle Chadwick, Biologics and Genetic Therapies Directorate (BGTD), Health Products and Food Branch, Health Canada
4. Computer Asssisted Decision Analysis in Drug Naming
   Dr. Bruce L. Lambert, Department of Pharmacy Administration, Department of Pharmacy Practice, University of Illinois and Chicago
5. Automatic Detection of Confusable Drug Names
   Dr. Greg Kondrak, Department of Computing Sciences, University of Alberta
6. Phonetic Orthographic Computer Analysis (POCA) System
   Dr. Rick Shangraw, Project Performance Corporation
7. Med-E.R.R.S. Name Review Process
   Susan M. Proulx, Pharm.D., Med-E.R.R.S.
8. Risk MonitorPro^TM and Look-alike/Sound-alike Errors
   Jerry Seibert, ^rL Solutions

Please refer the PowerPoint presentations that are posted on this website for further details on these presentations.

In attempts to ensure accuracy and completeness of the issues identified and provide an opportunity for stakeholders to give feedback regarding policy options and proposed recommendations, a number of questions were asked of stakeholders (Appendix A)

The feedback from this consultation and that with the Therapeutic Products Directorate (TPD) Advisory Committee on Management will be analyzed along with comments received on the draft LA/SA Issue Analysis Summary (IAS) which has been posted on the Biologics and Genetic Therapies Directorate (BGTD) website. The outcome of these consultations will be considered as part of the policy recommendations put forth to senior officials in the Health Products and Food Branch for their discussion and decision-making.

The issue

The Proposed Problem Statement:

1. Look-alike sound-alike (LA/SA) health products refer to names of different health products that have orthographic similarities and/or similar phonetics. These similarities may pose a risk to health by contributing to medical errors in prescribing, dispensing or administration of a product.
2. These medication errors may be more likely to occur because of contributing factors such as identical doses, dosage forms or routes of administration, similar packaging or labelling, incomplete knowledge of drug names, illegible handwriting, verbal order errors and even lack of appropriate knowledge base.
3. A specific safety issue involving the potential for confusion between two approved biologic drugs, as well as longstanding unresolved issues relating to LA/SA health product names has prompted the Biologics and Genetic Therapies Directorate to request a review and analysis of the issues associated with health products and to recommend an appropriate course of action.

The Alternate Problem Statement:

• Look-alike sound-alike (LA/SA) health products refer to names of different health products that have orthographic similarities and/or similar phonetics (i.e similar when written or spoken).
• These similarities may pose a risk to health by causing errors in prescribing, dispensing or administration of a product.
• These errors may be more likely to occur because of contributing factors related to health products themselves or system errors.

Questions

1. 1. Based on your experience and the problem statement as currently written, what components of it would you change to improve its clarity and accuracy? Are there any details you feel are necessary that are missing?
   2. Based on your experience and the alternate problem statement as currently written, what components of it would you change to improve its clarity and accuracy? Are there any details you feel are necessary that are missing?
2. The proposed scope of the issue includes:
   • similarities in brand names
   • similarities between brand names and generic names
   • product line extensions
     
     From your perspective, does this encompass the majority of LA/SA errors? What examples of LA/SA errors come to mind when you reflect on this issue?
3. How would you prioritize (1 being of highest priority) the implementation of the LA/SA health product name project among the following (currently listed in alphabetical order):
   • medical devices
   • natural health products
   • over-the counter-drugs for human use;
   • prescription drugs for human use; and
   • veterinary drugs.
     
     Please explain why you prioritized the list in this way?
4. Health Canada developed rating criteria to help select recommendations. Please refer to this list (LA/SA IAS, pg. 19) Are there any other criteria that should be considered?
5. What do you like best about the recommendations(s) (LA/SA IAS, pg. 25)? What do you like least? From your point of view, is there an alternative recommendation that should be considered ?
6. Having looked at the pre-market and post-market actions identified by HC that could be taken to address LA/SA health product name issues), what would be your recommendation for action? Please mark a Yes beside the elements you support, a No beside the ones you do not support or a Maybe beside the elements you would like to discuss further. If you marked No or Maybe, please explain why you believe this is not the most appropriate course of action.

Pre-market
HC policy
company/sponsor provides name analysis
prioritized list provided by company/sponsor
Complex Computer Application Screening
HC name review
HC name review committee

Post-market
HC policy
Monitoring
Promotion of known LA/SA health products

Look-alike Sound alike (LA/SA) Health Product Names: Developing a Common Understanding

Date 2003-10-20

Contact Policy and Promotion Division

October 20, 2003

Michèle Chadwick
Policy and Promotion Division
Centre for Policy and Regulatory Affairs
Biologics and Genetic Therapies Directorate

Day 1 Objectives

• Inform/educate interested and affected parties about LA/SA health product names and the current policy development process

• Ensure accuracy and completeness of the issues identified

LA/SA Medication Errors

• Research has identified that one of the most frequent causes of pharmacy dispensing errors (29%) is failure to accurately identify drugs because of LA/SA drug names

• Confusing drug names are responsible for 10 000 patient injuries each year in the U.S.

• More than 770,000 people are injured by medication errors at a cost of $177 billion each year.

Case #1

• Confusion between Seroquel and Serzone

• Due to similarities in names, verbal and written prescriptions were incorrectly interpreted, labelled and/or filled

• Contributing factors included:
  
  • overlapping strengths (100 mg and 200 mg)
  • dosing interval (BID)
  • stocked close together in pharmacies
    

• 3 patients hospitalized and 4 patients required emergency room visits

• One 25-year-old female experienced fever and respiratory arrest after taking Seroquel for 3 days instead of Serzone and eventually died.

Case #2

• Lisa, a 40-year-old Montreal woman with bipolar disorder

• Lamisil prescribed instead of Lamictal

• Contributing factor included:
  
  • appearance of the drug (round white pill)

• Lack of efficacy

Case #3

• vincristine instead of vinblastine

• Contributing factor:
  
  • both are cancer drugs

• Death by overdose (several cases)

Other cases

Primaxin I.V. and Primacor

(death)

Taxotere and Taxol

(death)

Lamictal and Lamisil

(hospitalization)

Serzone and Seroquel

antipsychotic incident)

Tobradex and Tobrex

(glaucoma)

LA/SA Errors - more than just Drugs

• Diovan (valsartan) and Diavan (herbal remedy for diabetes)
• lab test (anti-factor Xa expressed as Anti-Xa) and Arixtra (factor Xa inhibitor)
• Lamisil (terbinafine hydrochloride) and the medical device Limicel (Osmotic Cervical Dilator)

HPFB LA/SA WG Members

• Gloria Mah Cawthorn, BREC, BGTD
• Julie Clare, SMD, CPRA, BGTD
• Basanti Ghosh, SMD, CPRA, BGTD
• Michèle Chadwick. PPD, CPRA, BGTD (chair)
• Ruth Hansson, PPD, CPRA, BGTD (secretariat)
• Supriya Sharma, MHPD
• Bill Leslie, MHPD
• Micheline Ho, PID, SMAB, TPD
• Marilyn Schwartz, SIPD, BOS, TPD
• Michael Wood, SIPD, BOS, TPD
• Stephanie Pereira, NHPD
• Kerry Reinhard, NHPD
• Vicky Butz, VDD, HPFB
• Michelle L. Boudreau, Legal Services
• Catherine Yen, HPFB Inspectorate
• Julie Desrosiers, Health Policy and Communications Branch

LA/SA Action Plan

Phase I

• Establish a WG, Develop draft ToR, Define problem and scope, Development of Issue ID paper

Phase II

• Phase II: Develop Action Plan, Obtain Approval of Action Plan, Draft Policy Process Review/Evaluation Plan, Draft Public Involvement (PI) Plan, Analyze Issue, Identify Potential Options, Draft Issue Analysis Summary, Implement Phase I and II of PI Plan, Select a Strategy, Develop Feasibility Study (computerized options), RPF

Phase III

• Develop Implementation Strategy, Implement Phase III of PI Plan and Evaluation Report

Problem Statement (draft)

• Look-alike sound-alike (LA/SA) health products refer to names of different health products that have orthographic similarities and/or similar phonetics.

• These similarities may pose a risk to health by contributing to medical errors in prescribing, dispensing or administration of a product.

• These medication errors may be more likely to occur because of contributing factors such as identical doses, dosage forms or routes of administration, similar packaging or labelling, incomplete knowledge of drug names, illegible handwriting, verbal order errors and even lack of appropriate knowledge base.

Current Process

• case by case basis
• no consistent or formal process in place to review look-alike sound-alike health product names
• current computer systems are not set up to flag identical or similar names
• subjectivity
• perception of questionable authority (Can the Food and Drugs Act be used to support a request for a name change?)
• sponsors are encouraged to consider changing their product name when a look-alike sound-alike drug is identified (success has been mixed)

Act and Regulations (pre-market)

• Section C.08.002 and Section C.01.014.1 of the Food and Drug Regulations require that a drug's name be provided in a drug submission as part of the information required to assess safety and effectiveness of a product.

• These sections allow HPFB to adopt a pre-market requirement that names not be confusing with one another.

• If confusion is considered likely and could result in safety concerns, HPFB need not issue a DIN (old drugs, new drugs) and/or NOC (new drugs only).

Act and Regulations (post-market)

• Section C.01.013 of the Food and Drug Regulations permits the Director to require a manufacturer to submit evidence sufficient to establish the safety of a drug under the conditions of use for which the drug is recommended by a specified date.

• When evidence is not provided or is not sufficient the Director may, by notifying the manufacturer, direct the manufacturer to make no further sales of the drug.

Risks of No Action

• the morbidity and mortality of Canadians due to medication errors

• the risk that public trust may be lost

• If decisions are made inconsistently, HPFB risks a legal challenge based on arbitrariness when trying to act in accordance with the HPFB mandate.

• HPFB's potential risk of liability for failure to meet responsibilities outlined in its mandate if beneficiaries of the program suffer morbidity and mortality due to medication errors.

Objectives

• A consistent and formal process is required within HPFB to respond to LA/SA health product names. Pre-market and post-market processes must be developed to respond to LA/SA issues of similar brand names, brand names that are similar to generic names and product line extensions

Scope

A priority

• similarities in brand names including the sub-issue of using abbreviations in brand names
• brand names that are similar to generic names
• product line extensions including the sub-issue of marketing products with the same brand name yet different active ingredients

Less of a Priority

• similar generic names-preventative measures only (i.e. labelling changes)
• similar brand name sub-issue of creating brand names by adding company abbreviations (APO)

Not within the Scope of the Working Group

• similarities in labelling or packaging

Not an Issue

• different brand names for the same active ingredient within the same company

LA/SA Product Prioritization

• prescription drugs

• over-the-counter drugs

• natural health products, veterinary drugs, medical devices

Proprietary Name Evaluation at FDA

Date 2003-10-20

Contact Policy and Promotion Division

Jerry Phillips, RPh
Associate Director for Medication Error Prevention
Office of Drug Safety
October 20, 2003

What is a Medication Error?

• Any PREVENTABLE event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional, patient, or consumer.

• FDA focuses on medication errors related to the safe use of a drug product. This includes the naming, labeling and/or packaging of a drug product.

What is a Proprietary Name?

• A name owned by a company or individual and is used for describing its brand of a particular product.

• Also known as a "Brand Name" or "Trademark"

How Serious Is The Problem with Names?

• 700 name pairs (both proprietary and generic names) reported to USP and FDA for sound-alike or look-alike confusion

• 25,000 medication error reports received by FDA

• 12.5% of errors related to names

Mortality Data from 1993-1998

AJHP -Vol 58; Phillips, et al; October 1, 2001

• 469 Fatalities - Med Errors
• 16% of deaths were due to receiving the wrong drug
• 5% of deaths were caused by proprietary name confusion
• 4% by generic name confusion

Causes?

Similar Labels/Labeling

Avandia and Coumadin

What Is FDA Looking For?

• Sound-alike/Look-alike Properties
  
  • To currently marketed & unapproved drug names
  • To other Medicinal Products
  • To commonly used medical abbreviations, medical procedures, and/or lab tests
• Promotional/Misleading Claims

What information is needed for the FDA Risk Assessment?

• Proprietary and Established Name(s)
• Strength(s)
• Dosing Schedule
• Use/Indication
• Labels/Labeling
• Working device model
• Formulation and Packaging proposed

DMETS Proprietary Name Analysis

• Expert Panel

• Computer Analysis
  • Orthographic/Phonetic
  • Search of other External Databases
• Rx Studies (Simulated)
  • Verbal Orders
  • Outpatient Orders - written
  • Inpatient Orders - written
• Overall Risk/Benefit Assessme

FDA Expert Panel

• Approximately 12 DMETS Safety Evaluators (Physician, Pharmacists, Nurses)
• 1 DDMAC representative (pharmacist)
• Facilitator is randomly selected & rotated
• Each expert member reviews reference texts and provides a relative risk rating for each name PRIOR to the meeting
• Overall group discussion and consensus for each name

Rx Studies

• Outpatient prescriptions - written
• Inpatient prescriptions - written
• Verbal prescriptions (Inpatient or Outpatient)

The Rx Study Design

• Study prescriptions are developed for failure mode from discussions/concerns of similar names at the Expert Panel
• Various staff members are asked to write sample prescriptions for each name
• Marketed Drug or a Control Rx is included
• Rx is scanned and e-mailed to a subset of FDA health care workers
• Results/interpretations are e-mailed back

Sample Size

• About 130 FDA physicians, nurses, pharmacists volunteers respond by e-mail with their interpretation and comments
• To eliminate any one reviewer from reviewing a name more than once we divide the entire group into thirds (n = 43) to review each verbal order, written outpatient and written inpatient orders . Response rate is approximately 70%.

Handwriting Samples

Verbal Orders

• Randomly selected DMETS staff asked to record a verbal Rx
• Example: This is Dr. Dee Mets and I'm calling in a prescription for Jane Doe for Novicar 40 mg daily. Give #30 with 2 Refills
• Verbal Prescriptions are recorded by DMETS on a voice messaging system and sent to the assigned FDA reviewers for interpretation
• Results are e-mailed back to DMETS

Phonetic and Orthographic Computer Analysis (POCA)

• A set of phonetic and orthographic algorithms for use in an automated and computerized method of evaluating proprietary names for sound-alike and lookalike properties
• Prototype completed in July of 2003 - should be operational in October and used routinely in all future DMETS reviews
• POCA provides a percentage ranking of orthographic and phonetic similarity between the proposed name and the databases of existing proprietary names.
• POCA also considers similar strengths and dosage forms when looking at a name

Safety Evaluator Risk-Analysis

• Examines the data from Expert Panel, Rx Studies, computerized searches, and POCA to establish any risk for confusion
• Evaluates the potential safety risk associated with two identified drugs being confused with each other due to similarity
• Also examines appropriate Post-Marketing pADE Data, Clinical and Regulatory Experience, & literature reports

Some Contributing Factors for Name Confusion

• Similar indications
• Same patient population
• Identical formulations
• Overlapping strengths or directions
• Stored in same areas

What is the Potential for Harm?

• What are the consequences if the patient misses the pharmacological action of the intended drug?
• What are the pharmacological actions and toxicities of the unintended drug?

Final Review

• Trademark is re-reviewed 90 days before action on the application
• Extensive evaluation is not repeated. Only reviewed for any confusion with names that have come on the market since the original review

Look-alike Sound-alike (LA/SA) Health Product Names: The Developing a Comprehensive Policy Recommendation

Date 2003-10-20

Contact Policy and Promotion Division

October 21, 2003

Michèle Chadwick
Policy and Promotion Division
Centre for Policy and Regulatory Affairs
Biologics and Genetic Therapies Directorate

Day 2 Objectives

• Inform/educate interested and affected parties about LA/SA health product names and the current policy development process

• Provide an opportunity for interested and affected parties to express their concerns and/or endorsement regarding policy options and proposed recommendations to address LA/SA health product names

Proprietary Name Review at FDA

European Agency for the Evaluation of Medicinal Products (EMEA)

• Guideline entitled Guidelines on the Acceptability of Invented Names for Human Medicinal Products Processed Through the Centralized Procedure.
• EMEA attempts to ensure that a medicinal product not bear an invented name that could be confused with that of another medicinal product (safety issue)
• EMEA believes that it is crucial that consistent, non-arbitrary criteria are applied when reviewing the acceptability of proposed proprietary names
• The invented name should only consist of one word and should avoid qualification by letters and numbers including the use of short forms and abbreviations.
• The sponsor can provide the EMEA with up to three proposed names per market authorization application.
• The Name Review Group reviews proposed invented names

Proposed Pre-Market Options

• General
  • Status Quo
• Policy
  • SOP/Policy /Guideline
  • Regulations/Legislation
• Computer related
  • use of current Drug Submission Tracking System (DSTS)
  • use of current integrated Records Information Management System (IRIMS) system
  • development of new computer application in-house
  • basic computer application
  • LA/SA-specific complex computer application with added features
• Review process
  • Foreign Review (FDA/EMEA etc)
  • Review by Third Party
  • Name review (HC)
  • Name review committee
• Sponsor filing requirements
  • Sponsor provides a prioritized list of name choices
  • Sponsor does search/analysis for LA/SA similarities
  • Require trademarks
• Other
  • Combination of options

Post-Market Options

• General
  • Status Quo
• Policy
  • SOP/Policy /Guideline
  • Regulations/Legislation
• Computer related
  • Bar coding - product ID and verification
  • electronic prescribing (printed scripts)
• Industry Requirements
  • require company to change name of the health product
  • require company to modify label (i.e. lettering on label)
• Monitoring (environmental scans)
  • Incorporate LA/SA errors into ADR reporting
  • LA/SA error reporting (from other jurisdictions) - look back (i.e. Anonymous FDAMedWatch reporting)
  • Foreign reviews
  • ISMP-watchdog
  • use pre-market system to look for LA/SA approved health products
• Health promotion/stakeholder awareness
  • increase awareness of documented LA/SA health products to stakeholders (e.g. info line, fact sheets, Dear Health Care Professional Letters, comments in ADR newsletter, LA/SA website, education of LA/SA health products in medical schools)
• Other
  • Combination of options

Criteria Used to Assess Options

• WG agreed to use both a quantitative and qualitative approach to assess options
• The quantitative analysis included the use of a decision analysis technique to review options in which criteria used to assess options were grouped into screening criteria (must haves) or comparative criteria (nice to have)
  If options did not meet screening criteria, they were eliminated from further consideration. After assigning a weight to comparative criteria (1 to 10), options were ranked according to how well they met the other comparative criteria (using a 10 point scale).
• The qualitative technique required that the LA/SA WG members vote on preferred options.

Recommendations

Pre-market

• 1st choice: Policy/Complex Computer Application/name review/name review committee/company provides name analysis and prioritized list
• 2nd choice Policy/Computer Application/name review/name review committee/company provides name analysis and prioritized list
• 3rd choice Policy/Build internal system & name review and name review committee/ company provides name analysis and prioritized list
• 4th choice Policy/DSTS system (enhanced with some DPD support) & name review and name review committee/ company provides name analysis and prioritized list

Post-market

• 1st choice Policy/Promotion & Monitoring
• 2nd choice Policy/Promotion

Next Steps

• Post-Workshop report will be posted on the Health Canada website and mailed to stakeholders.
• The LA/SA WG will consider and analyze all feedback provided in the consultation period and will revise the Issue Analysis Summary, where necessary.
• Approval of revised IAS and recommendations by senior management and the WG
• Proceed with computer feasibility study, RFP and Phase III of this project

Computer Assisted Decision Analysis in Drug Naming

Policy and Promotion Division

Date 2003-10-20

Bruce L. Lambert, Ph.D.
Department of Pharmacy Administration
Department of Pharmacy Practice
University of Illinois at Chicago
lambertb@uic.edu

Overview

• How can computers be used to objectively measure similarities/differences between name pairs?
• What evidence is there that these measures are valid?
• How do we evaluate a name-retrieval system?
• What are the next steps, challenges, limitations

Preface: Need to Change Focus!

• Names are not enough: Must focus on drug products not drug names
• Similarity is not enough: Must focus on similarity and frequency(of prescribing)
• Error reduction is not enough: Must focus on harm reduction
• Pairs are not enough: Must focus on overall confusability/intelligibility of individual names
• How to balance public risk against private/corporate benefit?

Why Do These Errors Happen

• Similarity- and frequency-based errors in cognitive processing
• Memory (recall and recognition)
• Perception (visual and auditory)
• Motor control (picking wrong drug from dropdown menu)
• Poorly designed systems (e.g., handwritten orders, oral orders, no CPOE, etc.)

Naming Decisions Must Be Grounded in Best Available Scientific Evidence

• Public policy decisions must be based on validated scientific evidence, not marketing claims
  • Peer reviewed publications
  • Relevant evidence
  • Validated measures
  • Transparency, full disclosure of methods, reproducibility, objectivity

Different Types of Similarity

• Name similarity
  • Orthographic (spelling)
    • Predicts visual perception errors and some motor control errors
  • Phonetic or phonological (pronunciation)
    • Predicts auditory perception errors and short-term memory errors
    • Need to account for dialect and foreign language accent
• Product similarity
  • Composite of name and non-name (e.g., strength, dosage form, route, schedule, color, shape, etc.)
  • Predicts variety of practical confusions and moderates effects of other types of similarity

Phonological Similarity Not Enough

Avandia 4mg p.o. ?
Coumadin 4mg p.o. ?

Tequin 400mg p.o. ?
Tegretol 4mg p.o. ?
http://www.ismp.org/msaarticles/a072600safety.html

http://www.medmal-law.com/illegibl.htm

Plendil or Isordil?

• Isordil^® prescribed
• Plendil^® dispensed
• Cardiologist found negligent
• $450,000 damage award
• First ever award for bad penmanship!

Different Types of Similarity

• Different types of similarity are associated with different types of errors
• We want to prevent all types of errors
• Therefore, screening process must use validated measure of all three types of similarity
  • Orthographic, phonetic, product
  • Handwritten (cursive, print), spoken (dialect, foreign language accent)

Objective Measures of Name Similarity

• N-gram: based on proportion of n-letter subsequences that two names have in common
  • Bigram: two-letter subsequences (e.g., Premarin=Pr, re, em, ma, ar, ri, in)
  • Trigram: three-letter subsequences (Pre, rem, etc.)
  • Dice coefficient: Proportion of n-grams in common
• Edit distance: the number of insertions deletions or substitutions needed to transform one name into another
• Alignment distance methods
  • Kondrak's ALINE
  • Fisher's AL-DIST
• N-gram and edit distance measures can be used on any formal representation of the name (spelling or phonological)
  • Use phonetic alphabet (e.g., International Phonetic Alphabet or ARPAbet)
  • ARPAbet: Zyprexa: z ay p r eh k s ax
  • Phoneme bigrams: z ay, ay p, p r, r eh, eh k, k s, s ax

Objective Measures of Similarity

• There are many variations on these basic measures
  • Add spaces before or after to emphasize beginning or ending of name
  • Use different weights depending on position of letters
  • Use different equations to compute numerical similarity (Dice, Hamming, etc.)
  • Allow approximate matches between letters (e.g., m=n, a=e=i=o=u)
• Power of simple descriptive analyses
  • Ten most common three-letter prefixes in US brand names:
    • Pro-, Bio-, Car-, Tri-, Vit-, Pre-, Nut-, Ult-, Con-, Per-
  • Lambert BL, Chang KY, Lin SJ. Descriptive analysis of the drug name lexicon. Drug Inf J. 2001;35:163-172.

Distribution of Distance Scores

Lambert BL, Chang KY, Lin SJ. Descriptive analysis of the drug name lexicon. Drug Inf J. 2001;35:163-172.

Distribution of Similarity Scores

Lambert BL, Chang KY, Lin SJ. Descriptive analysis of the drug name lexicon. Drug Inf J. 2001;35:163-172.

Objective Measures Do Predict Probability of Human Error

• Similarity accurately distinguishes between known error pairs and non-error pairs
• Greater objective similarity correlated with higher rates of recognition memory errors by laypeople and pharmacists
• Greater similarity correlated with lower rates of free recall errors
• Objective similarity correlated with subjective similarity (for experts and laypeople)
• Similarity neighborhoods predict visual perception errors

Similarity accurately distinguishes between known error pairs and non-error pairs

Histogram of trigram string similarities for 969 error pairs and 969 control pairs. Vertical axis is on a logarithmic scale. Light bars represent error pairs. Dark bars represent control pairs. Values at ends of vertical bars are frequencies. Values on the horizontal axis represent the bins for the histogram. For example, (0, 0.1) means "greater than 0 and less than 0.1," and [0.1, 0.2) means "greater than or equal to 0.1 and less than 0.2." From: Lambert: Am J Health Syst Pharm, Volume 54(10).May 15, 1997.1161-1171

Figure 1. Effect of spelling similarity on pharmacists recognition memory errors

Lambert BL, Chang KY, Lin SJ. Effect of orthographic and phonological similarity on false recognition of drug names. Soc Sci Med. 2001;52:1843-1857.

Figure 2. Effect of spelling similarity on pharmacists free recall errors.

Lambert BL, Chang K-Y, Lin S-J. Immediate free recall of drug names: effects of similarity and availability. Am J Health-Syst Pharm. 2003;60:156-168.

Figure 3. Effect of phonological similarity on pharmacists' free recall errors.

Lambert BL, Chang K-Y, Lin S-J. Immediate free recall of drug names: effects of similarity and availability. Am J Health-Syst Pharm. 2003;60:156-168.

Figure 4. Relationship between objective similarity and lay people's subjective dissimilarity.

Lambert BL, Donderi D, Senders J. Similarity of drug names: Objective and subjective measures. Psychology and Marketing. 2002;19(7-8):641-661.

Neighborhoods Matter

• Concept of similarity "neighborhood" is key part of modern theories of visual and auditory perception
• Neighborhood characteristics
  • Frequency (of prescribing)
  • Density
  • Radius (i.e., how close does a name have to be to be in another name's neighborhood?)

Neighborhood Illustration

Dense Neighborhoods: High and Low Frequency

Examples

• High log SF names (log SF > 7): Ventolin^®, Dyazide^®, Provera^®
• Low log SF names (log SF < 3): Vistazine^®, Antispas^®, Protaphane^®
• Name from a sparse neighborhood: Flexeril^® (no neighbors in NAMCS/NHAMCS)
• Name from a dense neighborhood: Dynabac^®, Synalar^®, Rynatan^®, Dynapen^®, Dynacirc^®, Dynacin^®, Cynobac^®

Figure 5. Effect of similarity neighborhood on RPh visual perception of drug names.

Lambert BL, Chang K-Y, Gupta P. Effects of frequency and similarity neighborhoods on pharmacists' visual perception of drug names. Soc Sci Med. in press.

Objective Measures: Conclusions

• They work.
• They are not perfect.
• Better on population basis than on individual basis
• Better for public health than for legal wrangling
• We should be using them.

Software Demonstration

• Name searching
  • Spelling
    • N-gram, edit distance
  • Pronunciation
    • N-gram, edit distance
• Product Searching
  • Name, dosage form, strength, route
  • Each weighted for importance
• ^{Lambert BL, Yu C, Thirumalai M. A system for multi-attribute drug product comparison. Journal of Medical Systems. in press.}
• ^{Lambert BL, inventor. Apparatus, method, and product for multi-attribute drug comparison. US patent 6,529,892. March 4, 2003.}

How can computer resources be used to calculate weights for various elements in name similarity?

• One way is to calculate a composite similarity score using multiple distinct similarity measures
• Use multiple measures to predict error probability or some other outcome
• Expert = 0.69 - 0.01^*Editex - 0.30^*NED + 0.22^*Trigram2b - 0.02^*EditSoundex

Figure 6. Using multi-measure regression model to predict expert similarity judgments.

Lambert BL, Yu C, Thirumalai M. A system for multi-attribute drug product comparison. Journal of Medical Systems. in press.

Actual Name Retrieval Results for Query Name: Curosurf^®

See: Lambert, B. L., Yu, C., Thirumulai, M. (2004). A system for multiattribute drug product comparison. Journal of Medical Systems, 28(1), 29-54.

Evaluating a Drug Product Search/Retrieval System

• From the FDA's RFQ 223-02-5618: Empirical comparisons of the system's ability to identify at least 75% of the potentially confusing names that have been identified by all the safety evaluators and included in the review of the proposed proprietary name. This will be done with the most recently completed 100 proposed proprietary name reviews.
  • This is not the appropriate test. Easy to identify 75% of confusing names, just return a list of all names, and it is guaranteed to include 100% of confusing names.
  • Trick is for search system to retrieve confusing names without lots of false positives.
• From the FDA's RFQ 223-02-5618:
  • System's capability to identify 95% of the known and published known drug name pairs. (Presumably referring to error pairs.)
  • Pairs can be useful unit of analysis in assessing association between similarity and risk, but not correct unit of analysis in assessing retrieval performance.
  • Published lists do not distinguish between actual errors and near misses or pairs that simply raised concerns. These lists contain voluntary reports that are known to miss > 99% of actual errors.
  • Regulatory agencies must approve individual names, not pairs of names.
  • N pairs of names yields [N(N-1)]/2 pairs. Positive predictive value of tests based on rare events from large populations is very poor.
• Appropriate measures are recall and precision
  • Recall: the number of relevant names retrieved divided by the total number of relevant names in the database
  • Precision: the number of relevant names retrieved divided by the total number of names retrieved
  • Relevant names identified by method of pooled relevance judgments (see http://trec.nist.gov)
  • ^{See: Lambert, B. L., Yu, C., Thirumulai, M. (2004). A system for multiattribute drug product comparison. Journal of Medical Systems, 28(1), 29-54.}

Example of Recall/Precision Curve

http://www.itl.nist.gov/iaui/894.02/works/presentations/bcs-irsg/sld012.htm

What level of performance can be expected? Variety of tasks.

http://www.itl.nist.gov/iaui/894.02/works/presentations/bcs-irsg/sld014.htm

What level of performance can be expected?

Figure 9. Precision of editex retrieval method at 11 levels of recall (mean precision = 17.4%).

See: Lambert, B. L., Yu, C., Thirumulai, M. (2004). A system for multiattribute drug product comparison. Journal of Medical Systems, 28(1), 29-54.

How Well Do Human Experts Do?

Figure 13. Precision of expert rating retrieval method at 11 levels of recall (mean precision = 26.7%).

See: Lambert, B. L., Yu, C., Thirumulai, M. (2004). A system for multiattribute drug product comparison. Journal of Medical Systems, 28(1), 29-54.

How can computer resources be used to calculate weights for various elements in name similarity?

• Compute distinct similarity score for each product attribute
  • Name, Dosage Form, Strength, Route of Administration, Schedule
  • Indication, Shape, Color, etc.
  • Use equivalence classes for approximate matching of attribute values (e.g., tablet and capsule)
• Use regression or other modeling techniques to assign weights to various attributes
• Use ISMP/MERP error data to estimate the importance of various attributes

Can computer assisted pattern recognition support the decision process to determine name/name similarities?

• Yes, but .
• Problems
  • False positives
  • False negatives
  • Reliability of data for modeling (too often based on voluntary reports)
  • Determination of a threshold beyond which a name is "too confusing to approve"
  • Validation of predictive models
  • Which database to search?

Summary

• Computers can be used to objectively measure differences between name pairs, but need to assess all dimensions of similarity.
• Computers can be used to calculate weights for various elements in name similarity, but good evidence about the relative importance of non-name attributes is lacking
• Please contact me to discuss further.
• lambertb@uic.edu
• Thanks.

Automatic Detection of Confusable Drug Names

Contact: Policy and Promotion Division

Date 2003-10-20

Greg Kondrak, University of Alberta
Bonnie Dorr, University of Maryland
October 21, 2003

Overview: Drugname Matching

Method
Orthographic
Phonetic

Distance
Edit distance
Soundex

Similarit
DICE, LCSR
ALINE

Dice coefficient

Double the number of shared bigrams and divide by total number of bigrams in each string.

Examples:

• Similarity between {za,an,nt,ta,ac} &{co,on,nt,ta,ac} is (2 · 3)/(5+5) = 6/10 = .6
• Similarity between {za,an,nt,ta,ac} & {xa,an,na,ax} is (2 · 1)/(5+4) = 1/9 = .22

LCSR

Double the length of the longest common sub-sequence and divide by total number of chars in each string.

Examples:

• Similarity between zantac & contac: (2 · 4)/12 = 8/12 = .67
• Similarity between zantac & xanax: (2 · 3)/11 = 6/11 = .55

Edit distance

Count up the number of steps it takes to transform one string into another.

Examples:

• Distance between zantac and contac is 2.
• Distance between zantac and xanax is 3.

Soundex

• Transform all but first letter to numeric codes and truncate to max. four characters.
• Character conversion: 0 =(a,e,h,i,o,u,w,y) 3=(d,t) 1=(b,f,p,v) 2=(c,g,j,k,q,s,x,z); 4=(l) 5=(m,n) 6=(r)
  - king and khyngge: d(k52,k52) = 1.00
  - knight and night: d(k523,n23) = 2/4 = 0.50

Phonetic similarity

Example: Osmitrol and Esmolol

• Identifies identical pronunciation of different letters.
• Identifies non-identical but similar sounds.

DNA alignment

CG-CACAT-AGTC-CGAGA-GA-TAGGCAAG

CGGCACATTCGTCTCGAGATGACTAGGC-AG

A protein scoring scheme

An elementary similarity function

Similarity scheme based on multi-valued phonetic features

The vocal tract

Places of articulation

Some dimensions of sounds

Validation: Comparison of Outputs

Validation: Precision and Recall

• Precision and recall against online gold standard: USP Quality Review, Mar, 2001.
• 582 unique drug names, 399 true confusion pairs, 169,071 possible pairs (combinatorically induced)
• Example (using DICE):

Validation: Precision of Techniques with Phonetic Transcription

Conclusion

• ALINE: Highest interpolated precision; easily tuned to the task.
• DICE,LCSR,EDIT: Match names with shared letters/bigrams.
• Solution: combined approach that benefits from strengths of all algorithms.

Phonetic Orthographic Computer Analysis (POCA) System

Date 2003-10-20

Contact Policy and Promotion Division

Presented by

Dr. Rick Shangraw
Project Performance Corporation

Outline

• Overall System Design
• The Medical Repository
• System Demonstration

System Objectives

• Accessibility
  • Web-based
  • Varying levels of permission for varied tasks
  • Ease of use
• Efficient Use of Expert Time
  • Decrease paper production
  • Increase analysis time while decreasing search time
• Scientifically-Based Analysis
  • Reliability of Algorithms (Recall)
  • Predictive Validity of Algorithms (Precision)

System History

• Pilot System Development Funded by the U.S. Food and Drug Administration (FDA)
• Development Team
  • Project Performance Corporation
  • Bonnie Dorr, University of Maryland
  • Greg Kondrak, University of Alberta
• Pilot System will be Tested by FDA

Safety Evaluation Process - Before

Before Computer Analysis:

System Architecture

Safety Evaluation Process - Computer Assisted

With Computer Analysis:

Medical Repository

• Oracle Database
• Features
  • Populated with FDA Corporate Database
    • Approved and Unapproved Proprietary Names
    • Additional Factors (drug strengths, dosing intervals, dosage forms and routes of administration)
  • Ability to Import and Manually Update the Repository
• Next Steps
  • Biologics
  • Supplements and Herbals
  • Medical Terminology

System Demonstration

• Login / Security
• Search Capabilities
  • Phonetic
  • Orthographic
  • Other Factors
• Workflow
  • Saved Searches
  • Watch List
• System Administration

Med-E.R.R.S. Name Review Process

Date 2003-10-20

Contact Policy and Promotion Division

Susan M. Proulx, Pharm.D.
President, Med-E.R.R.S.
October 21, 2003

Med-E.R.R.S.

• Wholly-owned subsidiary of the Institute for Safe Medication Practices (ISMP), incorporated in 1997
• Med-E.R.R.S. Board of Trustees
• Works with the pharmaceutical industry in premarket phase to evaluate product labeling, packaging, and nomenclature for safety
• Works in post-marketing phase to help monitor and evaluate potential and actual medication errors that have been reported Approximately 45- 50% of medication errors reported to the USP-ISMP Medication Error Reporting Program (MERP) and FDA MEDWatch Program are related to problems with product labeling, packaging or nomenclature.

The Med-E.R.R.S. Process

• A two-part process using the ERRS^TM model, a variation of Failure Mode and Effects Analysis (FMEA)
• Takes into account practitioner input and expertise of the Med-E.R.R.S. staff

Client Input

• Safety testing performed in latter stages of trademark development
• Clinical information should be available
• Names should be narrowed down through database searches
• Ten "finalists"
• Clinical information about the product - indications, dosage, route, etc.
• Proposed trademarks
• Pronunciation (syllable breaks, accent marks)

Project Coordination

• US and International
• Project dependent
  • Pharmacists (most commonly used group)
  • nurses
  • physicians and other prescribers
  • others as appropriate
• For US studies, all aspects are coordinated by Med-E.R.R.S. staff
• Project coordinators in Europe, Middle East, South America, Australia and Asia help coordinate international reviews

Data Collection Tool

• Client information compiled into data collection tool
• Proposed names are scripted and scanned into graphic files
• If provided, pronunciation guides are included
• Data collection tool sent via e-mail and/or FAX or completed via the Internet

Practitioner Input

• Review of trademarks for visual and phonetic issues
  • Similarities to drug products
  • Similarities to medical terms or abbreviations
  • Likelihood of confusion (clinical context)
  • Likelihood of patient harm (clinical context)

Failure Mode and Effects Analysis (FMEA)

• A systematic assessment of how and where a product or system may be vulnerable to failure
  • Set up process flow
  • Determine failure modes
  • Rank likelihood of occurrence, severity of outcome
  • Where effects of errors are judged unacceptable, action may be taken to minimize potential for errors

The ERRS^TM Model

• An application of FMEA to the assessment of pharmaceutical trademarks
• Various clinical environments in which a product most likely will be used (stored, ordered, transcribed, dispensed, etc.) are considered
• The goal is to bring problem areas to the surface, so that actions can be taken to minimize or eliminate possible errors

Consider the process flow including:

• Who purchases?
• Where stored?
• Who prescribes?
• Ordering process?
  • handwritten, verbal, telephone, computer
• Where used?
• How does it get to site?
• Who administers?
• Who/how monitored?
• Who adjusts therapy?
• Recording of administration?
• Reordering?

Med-E.R.R.S. Analysis

• Collates information from data collection tool
• Performs comprehensive FMEA
• Look-alike and/or sound-alike confusion with:
  • other drugs
  • medical terms
  • abbreviations
• Likelihood of error
  • clinical context
• Ease of pronunciation, potential offensiveness of name, overall concerns for vulnerability
• Trademark candidates are reviewed from the perspective of the FDA based upon information made public by FDA
• Trademarks are evaluated using USAN criteria
• For international projects, the requirements of other world authorities (e.g., EMEA, WHO) must be considered

Report to Client

• On average, standard US projects completed within 15 working days
• Expedited US projects completed within 10 working days
• Six to eight week turn-around time for international projects
• Executive summary with tables
• Scoring methods
  • each name rated on five point scale
    • 5 = "low vulnerability"
    • 1 = "high vulnerability"
  • any score of 2 or lower is considered unsafe due to the potential for confusion

Key Points of Trademark Safety Testing Process

• Importance of both practitioners and experts
• Analysis of product in its clinical setting through the use of Failure Mode and Effects Analysis (FMEA)
• Review of medication error literature
• Qualitative process

About Us

Text

Date 2003-10-20

Contact Policy and Promotion Division

Our Vision:

A world where the customer experience in healthcare is second to none

• Solution provider since 1993
• Canadian owned
• Offices in Toronto, Florida, and Australia
• Specialize in Healthcare risk management and customer service
• Over 200 implementations

• Examples of actual incidents reported through Risk MonitorPro^TM

• How front-line staff report incidents
• Incident management
• Analysis

Incident 1669: A near miss

Incident 2663: A near miss

Incident 2340

Demo

Benefits

• Improves responsiveness
  • online submission, alert notification, email
• Increases productivity
  • single data entry, form letters, auto reports
• Promotes effective incident reporting
  • forms - simple, customized
• Enhances analysis & communication
  • drill-down, scanning
• Facilitates regulatory compliance

Structure