If you torture the data long enough, it will confess…..Ronald Coase (economist)
Carole is backtracking from PostGrad qualifications in Coaching, to undertake an undergrad Psychology degree. I’m overwhelmed by the depth of statistical expertise expected of her, and despite having a GradDip in Clinical Research I draw a blank on something called factor analysis. Psychologists doing investigations gather together factors which may be influential on the patient outcome, and search for interactions in the data. Medicine doesn’t do this. In fact post-hoc analyses are anathema. Data dredging – shame! The example often given is from 1988, in the Lancet, when studying the benefit of aspirin after a heart attack it was found that subgrouping by starsign significantly affected recovery. [Laughter 🙂 ]
Actually, this is worth thinking about. If you’ve been told since birth the behaviours expected from a Taurean, it’s quite possible you need additional counselling to subdue the inner beast. And not simply a different med dosage. But everyone except the doctors mines data nowadays. The social sciences statistical packages are being heavily adopted by business to glean profitability trends, most notably since IBM acquired SPSS.
All clinical trials have a single purpose, ie to test a hypothesis, even if multiple outcomes are considered and when multi-arm interventions (factors) are being tested (see Bonferroni). If the analysis isn’t declared upfront in the protocol, the ensuing report will be discredited by colleagues. Worthless even, since physicians’ distrust of their peer’s integrity leads to a presumption of bias – doing unethical selective analysis so as to claim ‘Eureka’ for something, anything! Earnest conferences churn out checklists for marking studies – GRADE, CONSORT, SPIRIT, PRACTIHC, STROBE, and even specialty specific guidelines such as PEDro (for Hispanic physiotherapists?). All seemingly ensuring transparency in the system, but somehow we’re forever growing the numbers of malcontents who claim that the regulatory oversight is broken.
Myself included. The problem arises from the cartel of institutional research, and I’ve written here often about our delusional confidence. The investment in years to attain a medical qualification, followed by the personal sacrifice of a research-entitling doctorate leaves medicos with little choice but to play the game. I don’t have evidence as to whether the psych’s datamining or the physician’s approach to test a hypothesis yields more fruit but am concerned that despite their claims to foster creativity, the universities stultify nonconformists as we make progress by degrees. Just getting funding is enough grounds to claim a breakthrough.
This month saw the publish of the ‘Handbook of Academic Integrity’, 72 chapters and starting price $USD400. To prove there’s no sanctimony on my part, here’s a sneaky free link to half a dozen chapters.
Everyone’s guilty of wrongdoing, sin is in our nature. Doing something even more wrong here: