Biomarkers: A Tale of Two Labs
Source of FUD: Citizen's Petition
Summary of FUD claims: Simufilam failed a Phase 2b biomarker trial in May 2020. The company, later, attributed the failure to an outside lab's faulty analysis. Again, according to Cassava, it was the fault of an external force and not the product itself. Two months later, the data completely flipped after being analyzed in a different lab. The drug showed incredible biomarker reductions across the board.
Contributing Authors: Cotdt and GlazedPizzaDonut
There are many reasons for invalid lab results. Assays from separate “batches”, lots, and vendors can have significant variance, so proper protocols for establishing baselines are essential. There is more than one method of establishing a baseline and retesting throughout trials based on the volume of individual samples and the frequency of sample testing. Add to this the fact that testing for AD biomarkers is a very niche specialty and few labs in the world have experience doing this, and the chances of erroneous results increases from lack of standard protocol. One misstep or error in performing protocols can result in data requiring retesting. All this to say, retesting biomarkers and obtaining a different set of results, in and of itself, is not suspicious.
Clinicians are taught to “treat the patients, not the labs”. It is not at all uncommon to obtain labs that do not fit the clinical picture. In such situations, retesting is the norm. Cassava Sciences obtained results which did not fit the clinical picture, and thus samples were retested.
In the case of Simufilam, even within the placebo group, the biomarker numbers were random with numbers all over the place. This inconsistency invalidated the results from the first lab that analyzed the 2b biomarkers because this isn’t what one would expect from placebo biomarkers (clinical picture did not fit the biomarkers). Additionally, this level of variation isn’t likely to happen after 28 days without any treatment. This randomness makes little sense and points to a mistake in analyzing the plasma and CSF.
Cassava Sciences stated in their 10-Q from November 2020:
“In May 2020, we announced that an outside lab with whom the Company had no prior work experience had generated an initial bioanalysis in which our Phase 2b study missed its pre-specified primary outcome. The data set from the initial bioanalysis showed unnaturally high variability and other problems, such as no correlation among changes in levels of biomarkers over 28 days, even in the placebo group, and different biomarkers of disease moving in opposite directions in the same patient. Overall, we believe data from the initial bioanalysis can be interpreted as anomalous and highly improbable. With its validity in question, the initial bioanalysis serves no useful purpose.”
To explain this further in layman’s terms: If a patient is sick and measurements A,B,C, increase during sickness, then one would expect ‘A,B,C’ to increase on testing a sick patient. The initial lab reported ‘A’ was elevated, and ‘B’ was low, and ‘C’ was normal. Because these values are inconsistent with reasonable expectations, the next obvious step would be to get samples retested.
It is also important to point out 11 of the original 17 biomarkers in question were tested on 50 patients twice over 6 months. Those results were consistent with the retested data. So we have 5 tests. 1 of the tests do not correlate, 4 of them do. It is reasonable to assume the 4 tests are correct, and the 1 test is an outlier/erroneous.
Please see Dr. Burns’ comments on Research Square for more information on the testing procedure which strengthens validity of the data.
“Worldwide Clinical Trials Bioanalytical Sciences developed and holds the method for testing.
Samples were analyzed in triplicate; the usual is duplicate. But that is not your question. We first sent samples to a lab that produced highly variable data, evident in dramatic changes over the one-month treatment period in placebo. Further, individual patients showed changes in opposite directions in different biomarkers. In fact, the average correlation between biomarkers in change from baseline (in the placebo group only) was nonexistent: 0.06. Our consultants agreed that this was not possible and was assay variability or perhaps sample handling. We did NOT reanalyze this data. We sent BACKUP CSF samples to ANOTHER lab, who -- again analyzing samples BLIND to treatment group -- analyzed the samples and produced data that showed no dramatic changes in placebo and a high average correlation between biomarkers in change from baseline (in placebo): 0.96. Night and day. This high correlation between biomarkers in placebo validated the second analysis of samples. Analyzing in triplicate does make the data tighter, but that is not the whole story between the two labs.”
The critics state that Dr. Wang’s lab was this other lab, and that because he is affiliated with SAVA (and co-discoverer of simufilam) that he is biased. What they are neglecting to point out is that these labs are part of a randomized control study and therefore blinded, so Dr. Wang wouldn’t know which samples were part of the treatment group vs the placebo group. Even if he wanted to fabricate the data, he would not know which patients should get the “improved” data.
We also know that Quanterix, an independent lab, generated P-tau181 serum levels for SAVA. Their data correlated with biomarker data across both P2 trials. To clarify, there is “Lab 1” (with erroneous data), “lab 2” (which retested the samples), and Quanterix which tested for serum P-tau 181 levels. Quanterix and “Lab 2” data correlate. “Lab 1” and quanterix data contradict each other. Additional evidence that the “Lab one” supplied erroneous data.
Lastly, all the above data was reviewed in depth at an “End of Phase Two” meeting between SAVA and the FDA. Considering Cassava Sciences was granted a special protocol assessment (SPA) with the FDA for phase 3 trials, one could conclude that the FDA did not find retesting of the erroneous data as a sign of misconduct.
An interesting observation to neuroscientists is that Aß22 is decreased in AD patients CSF. It was debated how the AD disease state impacts this biomarker. Then, it was found that Simufilam increases this Aß22 level to normal with treatment. This lends further support to the likely effectiveness of Simufilam.
Of note, Dr. Hoau Yan Wang should be feeling validated by this data. His Western Blots were accused by the Citizen’s Petition and by Dr. Elizabeth Bik of being manipulated data. While the vast majority of these western blots are unrelated to Simufilam, one of them relates to p-Tau181. As we see here, Wang's ultimate work on p-Tau181 and amyloid monomers analysis/quantitation was independently reconfirmed by Quanterix. Thus, two independent labs, using different methodologies and different samples (plasma vs cerebrospinal fluid) confirmed each other. It does not get any more credible than that.
Molecular Biologist rebuttal to the Citizen Petition western blots:
and if you're really interested in the specific details, there's a Part 2: