u/GratefulGrapefruite

Hello, IBKC family! I'm a 15+ year, PhD-level practicing psychologist specializing in trauma, and I've been reflecting on my thoughts and feelings about the Body Keeps the Score episode since I listened to it yesterday. I had a few things to push back against from the episode (not necessarily in defense of van der Kolk/the book, but around trauma research and practice in general), and I wanted to share one particular argument with you about the nature of evidence for broad claims in psychology, especially as it pertains to trauma.

(TL;DR: research on psychological practice is limited, flawed, and prone to bias; because of this, I don't think it justifies confident declarations about what constitutes best practices, even though such confident guidelines exist; as such, from a trauma-informed perspective, the boys could have been a bit more tentative in their debunking and referenced sources outside academia/academia-based practice guidelines [i.e., patient advocates, race/gender critical analysts, etc.] to provide a more nuanced, validating, holistic perspective in their critique.)

From my limited vantage point, it seems that there's a pretty significant discrepancy between what academic psychologists say vs. what practicing psychologists think about what best therapeutic practices are, especially regarding trauma. Research in our field poses unique difficulties (I'll name some below) that make it less reliable than research in the "harder" sciences. Because of this, I think the medical model of what "gold-standard" evidence is (usually meta-analyses of high quality double-blind studies, or as close to that as you can reasonably get) for best practices doesn't meaningfully apply in our field.

There are reasons why psychological research is very difficult to do "well" as measured by the medical model, contributing to my field having significant replication problems and causing a split between what academics find in their research and what practitioners find in their practices.

- It's impossible to "blind" a study when studying therapeutic modalities, so virtually every study is influenced by the expectations of both the researcher and the participants (the psychological equivalent of the placebo effect)

- Research subjects often have vast differences in their histories (of past experiences with therapy, trauma histories, protective factors, etc.), symptom manifestation/persistence, and personal contexts (i.e., what environment are they working in all day? what kind of environment do they go home to every day? are they experiencing ongoing social, economic, political stressors in their community?) that are very rarely factored into therapeutic research

- The majority of factors that contribute to client outcomes (top two being client factors [motivation, readiness to change, etc.] and therapeutic alliance [relationship between client & therapist]) tend not to be factored into therapy modality research and are likely to be significant confounding variables

- The data we collect are inherently subjective - almost all data is self-reported by the participant (not the most objective/reliable) and are gathered using quantitative symptom questionnaires that quantify things thay many people prefer to describe qualitatively (to illustrate: when given the opportunity to ask questions as they fill out symptom questionnaires, my clients often ask clarifying questions and describe their symptoms to me in stories that I then guide towards whatever the closest answer might be on the questionnaire; in studies, I'm certain those conversations aren't happening, and if they were, they would be a confounding variable)

- There are substantial criticisms that have been levied against research in our field based on gender, racial, and socioeconomic critiques, given that the majority of psychological/trauma research is based on participants who are either college students or veterans, leaving significant populations of people commonly impacted by trauma (witnesses/victims of domestic violence, witnesses/victims of community gun violence, survivors of significant adverse childhood experiences [ACEs], survivors of sexual abuse/assault, impoverished people, minoritized people, etc.) significantly underrepresented in the literature

- Research usually doesn't account for drop-out rates (which are up to 35-40% for CBT-based trauma therapies like PE and CPT) at all, as participants who drop out of therapy studies are typically not followed up with and are not counted in the results; this likely hides negative outcomes and doesn't address how many people might actually benefit from a given therapy by being able to tolerate it long enough to benefit from it in the real world

More specific to research on therapeutic modalities:

- The majority of studies on the efficacy of specific therapeutic modalities lack a meaningful control group and are often compared against no treatment at all; as such, when working groups/panels conclude that "x treatment is better than y", people may assume that x and y were tested against each other to see which one led to the best outcomes, but that is almost always not the case. Rather, the recommendations are based on comparing the size and quality of the body of research behind each modality (again, when tested against no treatment at all), which is influenced by how many years/decades the modality has been available for study, how easy it is to research, how likely the research on that modality is to get funded, etc.

- Because client outcomes are so strongly determined by client factors and therapist factors, only ~7% of client outcomes are determined by the therapeutic modality (ie, CBT, EMDR, etc.) itself; because of this, working groups/panels that rank therapeutic modalities against each other tend to overstate the differences between them

Because of all these challenges and limitations, the recommendations that stem from working groups/panels on the "best practices" for the treatment of specific conditions tend to come to the same conclusions: CBT is the top choice for nearly every condition in the DSM. If you went to your doctor for depression, anxiety, OCD, PTSD, eating disorders - almost any diagnosis in the DSM - you will likely be referred for CBT therapy, and this is because CBT has the largest and most high-quality body of research behind it. This is partly because CBT is relatively easy to research (it's more manualized and therefore consistently applied between therapists than, say, psychodynamic therapy, narrative therapy, etc., which makes doing large-scale studies - the closest to the "gold standard" - much easier), and it has been around the longest (often by decades) out of most of the modalities currently being widely used in research and practice in North America (others like psychodynamic therapy and behavioral therapy have been around longer, but these aren't as widely practiced here anymore).

Herein lies the discrepancy between research and practice: what is best for large-scale research doesn't necessarily translate to what's best for an individual client, especially given how relatively weak the evidence has become in order to make it so generalizable, compared to how individualized therapy has to be. All this makes it very hard to definitively either "prove" or "debunk" a lot of things in my field, especially around therapy practice. It also means it's important to include the perspectives of clients and critical analysts when considering these things, because so much about the expeeience of trauma and its treatment isn't captured by the research. It also makes it especially important to hold well-informed, critical, nuanced understandings of psychological conditions and practices, and to lead first with empathy and understanding for clients/people living with trauma.

This is where some of the episode irked me, I think. I feel like the boys didn't quite hit the right balance on this one, at least on some points. Van der Kolk was due for a critical review, with his low grade evidence, questionable vignettes, and overly-certain, biased conclusions. But I think the certainty with which the boys refuted some things wasn't quite justified by the quality of the evidence available, just given the nature of research in this area as outlined above. And the cost of getting it wrong is kinda high, IMO, as they risk leaving some readers who felt benefitted by the book feeling shamed/invalidated (as I'm sure some of us felt about their critiques of our own previously appreciated books [*cough* Freakonomics *cough*], but we only felt intellectual shame in those cases - much lower stakes than with this book). It may have been worthwhile for the boys to consider additional information outside of academia and psychology experts, such as the contributions of mental health advocates and people who have engaged in critical analyses around trauma conditions/therapies, to round out their critique in a more nuanced, holistic, affirming way.

Congratulations on getting to the end of this thesis! Thanks for indulging me on this. ❤️

Pussytoes Question

Trauma Therapist on Psych Research