Monday, August 9, 2010

FDA – Personal Genetics: Is it safe? It’s a marathon, man…

It’s nearly 10 years now and still there is no clarity about the position of personal genetics in the regulatory framework. Maybe that’s going to change soon with the FDA activity and the recently published HGC Principles. It would be good to get it settled one way or another, the uncertainly doesn’t help anyone except those who exploit it to exploit the gullible.

Some elements:

  • DTC vs. DTMD (via physician) – I will argue that DTMD is actually higher risk and needs closer scrutiny
  • Is it medicine? I think this question is a waste of time, it will not be resolved, the definition is too broad, medicine is practised everywhere by everyone - if I take my son’s temperature, put a plaster on a cut or administer medication I am practising medicine.
  • What is the FDA duty bound to do and what will they decide?
    • No regulation – more or less the current situation
    • Tight regulation – medium/high risk requiring pre market approval (PMA)
    • Somewhere in between – the “light touch”

The questions are what regulation, oversight and quality controls are necessary (and possible) to get the maximum benefit, for all

What is a medical device?

SEC. 201. [21 U.S.C. 321]
(h) The term "device" (except when used in paragraph (n) of this section and in sections 301(i), 403(f), 502(c), and 602(c)) means an instrument, apparatus, implement, machine, contrivance, implant, in vitro reagent, or other similar or related article, including any component, part, or accessory, which is--

(1) recognized in the official National Formulary, or the United States Pharmacopeia, or any supplement to them,

(2) intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease, in man or other animals, or

(3) intended to affect the structure or any function of the body of man or other animals, and which does not achieve its primary intended purposes through chemical action within or on the body of man or other animals and which is not dependent upon being metabolized for the achievement of its primary intended purposes.

This unfortunately is very broad, “prevention of disease”. That can apply to soap and a toothbrush and certainly applies to any publication that contains any form of health advice (if you stay healthy it is because you are preventing disease, right?). This is so broad that what it really boils down to is what the FDA decide. In 2007 they decided it was not, now they say it is. According to the definition it always has been – so this is very much a decision of the FDA, it is something they could, or could have avoided.


What are the FDA saying?

In their letters to 23andMe and others they said:

The 23andMe Personal Genome Service™ is a device under section 201(h) of the Federal Food, Drug, and Cosmetic Act (the Act), 21 U.S.C. 321(h) because it is intended for use in the diagnosis of disease or other conditions or in the cure, mitigation, treatment, or prevention of disease, or is intended to affect the structure or function of the body.

A few days ago Alberto Gutierrez and Elizabeth Mansfield of the FDA gave a revealing (taped!) interview to Mary Carmichael as part of her intriguing DNA Dilemma series in Newsweek here are some relevant snippets (whole interviewinterview plus comments, see also Daniel MacArthur’s post on the subject):

AG (to Mary Carmichael):  The law requires us to clear devices or approve devices before they go into the marketplace when they make medical claims…Now clearly the claims they are making are medical claims [re 23andMe]

AG: what is the risk of an undetected false result, what would be the medical action taken, and whether there’s high risk there or not… there might be some claims there that we may consider to be PMA-like [pre market approval]

EM: they have a variety of claims that span our normal risk classification. On a claim-by-claim basis, there are some that might be low, there are some that might be moderate, there are some that might be high

MC: So, would a company need to be approved just to provide a raw SNP list to people?
EM: They would if they made medical claims about that data. If they don’t make any medical claims about that data, then they’re free to provide information as far as we’re concerned.

So:

1. The “law requires”, they have no choice if a device makes medical claims.
2. FDA have decided that they are making medical claims and are to be regulated as devices
3. The type of regulation will depend on risk (class I, II or III)
4. Just providing genetic results is not a medical device

While many, including me, would prefer to have the FDA step back and allow a system of self-regulation, market forces and transparency (with mandatory genetic testing registry) to control the field it looks like the FDA have crossed the Rubicon with (1) and (2). This makes risk classification very important as it determines the regulation (see here) and whether expensive and time consuming pre-market approvals are required. Note that there is no distinction by the FDA on DTC vs. DTMD, a device is a device, but it would affect the risk.

The green light to the genotyping data is helpful because it means we can focus just on the actual interpretation and for this I am going to look at parts of three different services:

1. WebMD a medical website written for an audience that includes general public. It includes gene variant related information which would be relevant and personal to an individual who has already been genotyped.

2. 23andMe – DTC personal genetics service from genotyping to interpretation

3. Existence Genetics – DTMD personal genetics.

Note that like 23andMe “WebMD does not provide medical advice, diagnosis or treatment.” But they do say things like “Your 23andMe results can help you make more informed decisions about your health” or “Better information. Better Health”. So they do (both) expect you to make decisions based on the information presented on the website – is it safe?


Common diseases where current genetic knowledge is low impact, e.g. Heart Attack:

1. WebMD has a large section on Heart Disease including “Tips for Reducing Heart Disease Risk”. There is also an article on the SNP analysed by 23andMe - 9p21

2. 23andMe: “20.9 out of 100 men of European ethnicity who share Greg Mendel's genotype will get Heart Attack between the ages of 40 and 79 (average is 21.2 out of 100)”  in the disease risk overview it says “your risk is 20.9%” and describes it as a typical risk (and is .99x lower risk). The advice is general, very much like webMD – plenty of background about the role of genetics and environment with advice on prevention and keeping healthy

3. Existence Genetics. Much less info available but some sample pages for heart attack among others:

a. Your lifetime risk = 61%,
b. General lifetime risk = 32%
c. Equal to 90% increased risk
d. It says that this disease is very important to my health and wellness and that preventative measures can be taken and lists some specific interventions, what to monitor and also how my genetics may affect medication.

All are services intended for the “prevention of disease”. In my opinion all are low risk (I would actually say no risk given current knowledge of gene impact) and would not merit any more than class I classification. Risk is classified, if I understand what AG was saying (not 100% here!) on

a) What action a person might take
b) What would be the consequence of a false positive result
c) What would be the consequence of a false negative result

Common disease - Potential Risks:

For WebMD false positive or false negative would be applied to the correctness of the advice. For the genetic companies there are additional possibilities of error: the genotyping, the software delivering the interpretation based on the genotype, the interpretation itself. It’s hard to see how an individual can come to any harm by following the advice on either of the consumer services and for EG the actions will be mediated via a medical professional (who though needs to be confident that the EG service itself is correct). For now a false result is not a risk, the predictive impact of genetics on heart disease is too low for any serious consequences. A false positive may lead to extra attention to diet & lifestyle and false negative will leave the situation unchanged – as long as it is clear that typical or reduced risk does not mean NO risk, and that in fact for these common diseases, with what we know now, the lifetime risk for anyone is always going to be significant. (If the impact of the genes is so low and uncertain, what’s the point of the service? Doesn’t matter, that’s for the buyer to decide and has no bearing on risk or FDA classification).

Serious diseases where genetics have v. high impact, e,g. cancer involving BRCA mutations:

WebMDHaving one first-degree relative with breast cancer approximately doubles a woman's risk, and having two first-degree relatives increases her risk 5-fold [this is DTC genetic risk information!]… An estimated one in 800 women carry the BRCA1 gene (the number of carriers of BRCA2 remain unknown). Women with inherited changes in either of these genes have up to an 80% chance of developing breast cancer in their lifetime.

23andMe: (it looks like you need to create a demo account and log in to get this info) “Carrier for the 185delAG BRCA1 mutation. Lifetime risk of breast cancer for women is increased from 13% to 81% and risk of ovarian cancer is increased from less than 2% to 54%. May significantly increase risk of prostate cancer in men. There is also an increased risk for breast cancer in men”. Of course it also tells me if I have the mutations or not. [Note that this section is “locked” the results are released only after reading some specific warnings and giving consent]. They also state “Please remember that the BRCA mutations covered by this report are only three of hundreds in the BRCA1 and BRCA2 genes that can cause cancer. Their absence does not rule out the possibility that you may carry another cancer-causing variation in one of those genes.”

EG: no info available

What are the risks? What action would be taken for each of the different possible results?

WebMD may make a person go and take the genetic test, especially if there is a first degree relative.

23andMe:

a. Positive: I don’t know what it says as I have not seen a positive result, but it should say something like “this service is for information only, go to your doctor and order an FDA approved clinical test for BRCA1 and BRCA2 mutations”. Even if it doesn’t the most likely outcome is a trip to the doctor who should then decide to order the approved clinical test. Would a doctor take action based on the 23andMe report? He/she should not and would not if competent – the FDA risk assessment does not take into consideration actions of incompetent professionals – there is nothing in the 23andMe report, as far as I can see, that would make the doctor think that this is an approved clinical test. Outcomes along the lines of demanding an immediate mastectomy and not stopping until finding a surgeon prepared to perform it are inconceivable, and anyway would still require the intervention of an incompetent or unscrupulous professional.

b. Negative: I think we have to base our assessment on the assumption that the customer does read the whole report so it should be clear that the customer realises that there is still a slight chance of having some other mutation and if worried should order full clinical test especially if family history. On the web page (need to log in) they say at the top: “Please remember that the BRCA mutations covered by this report are only three of hundreds in the BRCA1 and BRCA2 genes that can cause cancer. Their absence does not rule out the possibility that you may carry another cancer-causing variation in one of those genes”.

c. False positive: This would obviously cause a lot of anxiety until the result is revealed as false by the follow up clinical test. Do the FDA incorporate the likelihood of a false result, I don’t know? Genotyping is very accurate as far as we know - it would be helpful to have this controlled and published. The interpretation algorithm is very unlikely to be wrong. Overall there is a very low possibility of a false positive, and given that the benefits of detecting the mutation, when it is present, include saving lives, I would say that they outweigh the risks of a few weeks of extreme worry for some individuals.

d. False negative. This is tricky. A false negative could be catastrophic and much worse than a false positive. Negative is the most common result by far and it is not likely to be questioned (unless there is a lot of family history, in which case the clinical test should be taken anyway). A false negative for cancer is particularly bad because the disease is often not detected until it is too late for simple therapy. The consequences would be severe – this is the same for any provider of this test. What are the steps taken with Myriad to ensure that false negatives almost never occur?

For carrier status of other serious diseases (cystic fibrosis etc) I would describe the consequences more or less the same as for the BRCA mutations.

I would classify as low risk this part of the 23andMe service because I cannot see any serious consequences as happening more than very infrequently based on any of the outcomes (except (d), come to that in a minute) and one reason is precisely because it is a DTC test and clearly not to be used as the basis for clinical decisions, in my opinion, reduces the risk, not the opposite. I would definitely classify the Existence Genetics service as medium or high risk (as I would the Myriad test) – because these tests are giving information to clinicians that is designed influence serious clinical decisions, for this reason I would want to be very confident of the quality of the services (and paradoxically, until now, the physician only services have been far less transparent and far less subjected to scrutiny (see here).

Much has been said about the inclusion of BRCA mutations and other serious diseases being a game changer and that now they do need extra regulation. I don’t agree (we need confidence and transparency, but that’s a different issue). Not only are DTC services low risk, they will also be life savers. How often are BRCA mutations found in women after they have been diagnosed with cancer, i.e. too late? That’s the potential number that could be saved by encouraging screening – even though DTC would only be a “first draft” to be confirmed by clinical screening. If FDA involvement is any more than a “light touch”, if it involves anything that would reduce, restrict or remove DTC access then that action would have a high risk of having life threatening consequences for the public – the risks involved in over-regulating are much higher than those of proceeding slowly and with extreme caution. It’s not the other way around.

The false negative worries me somewhat, whether for BRCA or for carrier status of very serious genetic diseases like cystic fibrosis. However it needs putting in perspective. Genotyping as far as we seen so far, at least among the transparent companies, is very accurate. I would like to have a formal public measure of that though, an ongoing and public quality control on any genetics tests for serious mutations, this include Myriad, Counsyl etc. How accurate is Myriad, how many false negatives are they likely to have produced? Some FDA or other office should be set up that regularly sends blind control samples to all the companies (the companies can pay for it with an annual fee – wouldn’t cost much, 24 samples per year?)

Conclusions

DTC
Overall, with a few caveats, I think that the service offered by 23andMe is comparable to that of WedMD – I think it is very low risk and should not be more than Class I, there should be almost no time, effort or expense required to get it cleared by the FDA. Notification should be enough. The FDA may have bound themselves now by saying that they are medical devices, but they have not yet decided on things like Class, PMA etc, so there is still the opportunity to regulate this area in a sensible way, and even, ideally, to develop a novel future-proof framework for this new and disruptive field.

Caveats: The obvious differences are that the 23andMe services have more steps, compared to WebMD, where errors are possible (genotyping, software algorithms, and the actual interpretations). The FDA is not concerned with genotyping quality, that is CLIA’s domain (although the FDA, at the same time, are pressuring Illumina, and still at the same time are saying genotype data on it’s own is OK - contradictory). The quality control of the software is equally important – making sure that the data from the lab is not jumbled, that each result is assigned to the correct gene and the correct person. That each interpretation and each calculation based on each SNP, is working correctly. I don’t know what quality standards are required here, but some should be. In my opinion (and experience) there is more chance for error here than in the genotyping itself, much more. We also have to remember that here we are talking about 23andMe, deCODE, etc – they are transparent and have been scrutinised a lot by many experts. We have no idea who is going to come along next though. Companies will come along that will simply be bad, incompetent, unscrupulous or whatever. The regulatory net needs to be fine enough to catch them but without tangling up the good players. If all of the elements of the calls for transparency, the NIH GTR and the HGC Code of Practice were put together into an effective system then that would satisfy the needs to guarantee that DTC personal genetics services are at the lowest risk – basically do what at is proposed by Dan Vorhaus and stir in the HGC.

DTMD
The situation with Existence Genetics, Counsyl and may be Pathway & Navigenics, and all others who sell to medical practitioners is different. They are all selling a service to a medical professional. A genetic service delivered to clinicians who will take clinical action is something that has higher risk simply because decisions will be made and actions will be taken. The risks of serious consequences are higher, especially when results are in error. DTMD needs tighter controls than a DTC test that is clearly marked “for information only”. If a DTC service did appear that actually did claim that the results can be used to make clinical decisions then that would be need similar controls, in addition to the decision as to whether it should be available DTC or not.

I can get legal or financial information from a website, newspaper, journal etc, and also from lawyers and financial professionals. The difference is clear as are the differences in regulation and accountability.

The debate is the wrong way around – deal with the DTC via the transparency etc methods and deal with genetics services to clinicians as medical devices (I believe that the Roche Amplichip is class II). Once you allow genotyping you cannot block the information flow (SNPedia, Promethease, 23andMe, etc) and if the information is marked as “not for clinical” that has to be accepted.

Maybe not everyone will agree with these conclusions, maybe some of the analysis is mistaken – but regarding personal genetics, I think it is safe.



NB: I need to add something about pharmacogenetics:

The 23andMe result for Greg Mendel is Slightly increased warfarin sensitivity. May require decreased warfarin dose.

It does say at the top…”Only a medical professional can determine whether warfarin is the right medication for a particular patient. The information contained in this report should not be used to independently establish a warfarin regimen, or abolish or adjust an existing course of treatment”.

I bolded independently because I interpret it as meaning that the user should not make a decision alone but can make a decision after consulting a doctor.

I think this is or is close to saying that this information can be used for clinical decisions and I really think that 23andme need to modify the advice. They should say that it needs confirming via a class II regulated medical device and until then NO change in medication should be made.

I agree that the chances of the advice being wrong are small, but we have to think beyond 23andMe and the other high profile well managed companies.

Other recent comments on the FDA re personal genetics:

GeneticFuture: The FDA doesn't plan to regulate access to raw genetic data;

FDA Blog: FDA states cost 'not considered in any of our reviews' and In wake of FDA offensive, genomics entrepreneurs look overseas

8 comments:

  1. There are some problems with comments, I have received a couple via email that have not appeared here, I'll post them soon myself. Looks like a Blogger problem that has affected other sites as well - Keith

    ReplyDelete
  2. Good article but I don't understand your comment:

    "A genetic service delivered to clinicians who will take clinical action is something that has higher risk simply because decisions will be made and actions will be taken."

    So you don't think that decisions will be made and actions taken on DTC tests? Isn't that a bit naive? Especially since 23andMe is now offering their tests in doctor's offices as well (https://www.23andme.com/about/press/20090427/)

    ReplyDelete
  3. Keith,
    You know what they say about assumptions......

    You assume a hefty amount here which the FDA cannot.

    "It’s hard to see how an individual can come to any harm by following the advice on either of the consumer services and for EG the actions will be mediated via a medical professional"

    Ok, try this one

    On 23andMe's site https://www.23andme.com/health/Heart-Attack/howitworks

    "Smoking, excessive alcohol use, and not getting enough exercise can damage arteries and lead to atherosclerosis. High blood levels of C-reactive protein, the amino acid homocysteine, or fibrinogen are also known to be associated with clogged arteries and heart attacks

    Patient assumes that they should lower homocysteine, not because 23andMe says so, but because they don't say not to. Nor do they say that there is no evidence that clearly indicates lowering homocysteine works. FYI, I would say precisely this in a consult....

    Result: Consumer takes too much folate and promotes colonic tumor growth. Or even worse, takes folate with an MVI and gets toxic on it.

    I once had a patient take Vitamin D and get Vitamin A toxic, because they didn't see the small print which said "with naturally occurring Vitamin A"

    In America, people have a significant problem with health illiteracy. Tell me if the sample report reads well.

    BRCA Sample report
    "Please remember that the BRCA mutations covered by this report are only three of hundreds in the BRCA1 and BRCA2 genes that can cause cancer. Their absence does not rule out the possibility that you may carry another cancer-causing variation in one of those genes."
    Flesch-Kincaid Grade level: 22
    Flesch-Kincaid Reading Ease score: 15

    FYI Reading ease score, legal documents are often at a 10, while a comic is at a 90...

    So the warning is above most consumers heads...

    Now, I ask you. Do you think they will pay attention to that? Probably not. This is another big assumption

    Further,

    " Even if it doesn’t the most likely outcome is a trip to the doctor who should then decide to order the approved clinical test. Would a doctor take action based on the 23andMe report? He/she should not and would not if competent "

    The 23andMe test is done in a CLIA approved lab. Why would I not take the MultiSite test as accurate? Would that make me a horrible doctor if I trust 23andMe? What about LabCorp?

    I disagree, LabCorp should be held to the same standard here whether as a clinical or not.

    Playing fast and loose with BRCA results is sketchy at best.

    Keith, I think DTCG, when selling genetic marker testing that is highly clinically useful be regulated as such. I.E. Medicine as medicine

    Just because it comes in a pretty box and isn't ordered by an MD doesn't make it less dangerous.

    People do stupid things. I see it everyday.

    This regulation will be done on a marker by marker basis regardless of whether the MD orders or not.

    It is nice to see all the DTCG support shift the argument to "See it's safer when those crazy stupid doctors with medical degrees and 12 years of training don't get involved" "Let the consumers figure it out on their own. What's the worst they could do?"

    Yeah, that will fly like a lead balloon to regulators.

    The answer is simple, on a case by case basis determine if the gene in question has clinical use and potential risk to paient based on correct or incorrect findings.

    Regulate accordingly.

    The biggest risk are the markers which can be used clinically

    PGx and BRCA1/2. Highest risk
    Predispo. Intermediate Risk, except ARMD
    Family/Ancestry/Novelty Traits Low risk

    If you don't want 23andMe to be mistaken for a clinical lab, make them stamp a huge sign on the side of the box that says,

    "These results are useless for your doctor, if you test "positive", the doctor will need to draw your blood to double-check"

    I wonder what most consumers would say?

    ReplyDelete
  4. @anonymous

    Would a doctor make a decision based on a 23andMe test? Yes I think that is likely but my argument is that only decisions with low risk consequences would be made – the doctor might discuss the health and lifestyle aspects if heart disease, cancer, arthritis, diabetes etc and give advice on diet, exercise etc. I would not expect a doctor to make decisions with serious consequences without a confirmatory clinical test – prophylactic surgery, embryo screening because of parents disease carrier status, drug selection and dose (but 23andMe need to change their test, as above).

    ReplyDelete
  5. @Steve

    I’ll address your “stupid doctors” comment first – the focus of my argument is on DTC being low-risk (under the right circumstances) and that DTMD is higher risk. Exactly what I am saying is that the tests and information should be designed and regulated to reinforce this, to make sure that any high risk decisions are made with professional help, so it’s not “stupid doctors”, on the contrary.

    1.Heart disease information, folate, homocysteine - There is nothing here that is any different from thousands of websites, magazines, newspapers, etc. How is the vitamin A/D anecdote relevant? (The risks/benefits of folate has to be a different discussion)

    2.BRCA and reading skills: If the reader cannot understand that statement (which of course is possible) then the reader is unlikely to understand much at all from the breast cancer pages – I don’t see any dangerous consequences because of that

    3. The CLIA part covers the genotype data, what you would be acting on is the interpretation – I don’t think that (the current) personal genetics companies are using CLIA as a smokescreen to imply that they are selling medical tests are they? You would not be a good doctor if you ordered prophylactic surgery based on 23andMe results

    4.“I think DTCG, when selling genetic marker testing that is highly clinically useful be regulated as such. I.E. Medicine as medicine” The “is it medicine” part is dealt with in the post

    5. “stamp a huge sign on the side of the box” – Useful for tobacco I don’t think that this is necessary yet for DTC, the usual wording, which is not just in the T&C should be enough.

    6. "These results are useless for your doctor, if you test "positive", the doctor will need to draw your blood to double-check" – yes I think it’s a brilliant use for these tests, they are much cheaper than clinical tests and can point you to only those that you may need. Let the consumer decide – absolutely

    Steve, a question for you – do you have any info on the false pos or neg rates of the Myriad BRCA test? It’s not a provocative question, it’s real curiosity

    ReplyDelete
  6. I've tried to stick to logical arguments in the post but here I will say that for nearly 10 years many and various people have been looking for evidence of harmful consequences, the result so far is nil and the consequence is that we get the unsavoury spectacle of an FDA director using scare stories "out of context" and a Washington Post reporter misrepresenting interviewees.

    ReplyDelete
  7. Keith,
    good argument. But it does not pertain to the current laws.

    ReplyDelete
  8. Steve - which current laws? The fact that they are medical devices according to the strict definition of the FDA, and therefore need to be regulated as such? If so that is not a problem - I've included that, if the FDA have to get involved they can classify the devices according to risk and regulate accordingly. Notification vs. pre market review is a big difference. Also the FDA have the opportunity to develop the innovative procedures required, or do they not have that possibility, are they just simple enforcers of the rules?

    ReplyDelete