Thank you for your help! Mental fatigue questionnaire study update — how the scale was put together and what this round is doing
Last month I posted here recruiting for my PhD study developing an acute and chronic scale measuring mental fatigue in sport. First off, I want to say a big thank you to everyone who took the time to fill it out. The responses have been brilliant and a few of the comments made me see the problem through a different lens, especially as my background is in climbing and weightlifting so seeing it from a runners perspective was really helpful. A few people also asked how the questionnaire was put together and what this round is doing, and I should have laid that out from the start. Full references are in a separate comment below.
Where the items came from
The scale is being developed following Boateng et al.'s (2018) framework for scale development, which is a detailed primer explaining how to develop and validate scales in behavioural and health science. The starting point was a wide search of the literature. I pulled items from two main sources: 16 measures of mental fatigue and mental load used in the general adult population (identified through Diaz-Garcia et al.'s 2021 systematic review), and 19 measures used in sport-specific contexts (identified through my own systematic scoping review of mental fatigue and mental load measurement tools in sport. I am looking to publish this soon). On top of that I added items developed from my readings of six papers that describe how athletes experience mental fatigue and what drives it (Van Cutsem et al., 2017; Martin et al., 2018; Pattyn et al., 2018; Russell et al., 2019; Gantois et al., 2020; Habay et al., 2021). That gave me a pool of 462 items.
Those 462 items went through a deductive thematic analysis (Braun & Clarke, 2006) at the latent level to identify the underlying domains, which reduced the pool to 105 items. During a team review with my supervisors, it became clear there were two distinct constructs that needed separating: acute mental fatigue (the momentary state right now, before or after a session) and chronic mental fatigue (the longer pattern that builds over weeks and months). The five themes that came out of the analysis were inputs contributing to mental fatigue, motivation, perception of effort, decisional balance, and the influence of mental fatigue on behaviour.
Expert review with subject matter experts
The next step was getting six subject matter experts to review every item. The panel was deliberately mixed: researchers in mental fatigue, an exercise physiologist, a cognition specialist, someone with scale development expertise, a professional coach, and an athlete as end-users. Putting athletes on the panel was important, because items that make perfect sense to experts can land badly when you try to use them in a training context. Each item was rated on appropriateness, representativeness, and clarity using Hardesty and Bearden's (2004) sum-score decision rule, and items that didn't make the cut got removed. Some items were reworded based on expert feedback (for example, "tiredness" was changed to "fatigue" across several items to keep the construct clean). That process left 43 items for acute and 51 for chronic, which is what's currently being distributed.
What this round is doing
This round is about dimensionality and item reduction phase. The data from everyone who fills it in goes into an exploratory factor analysis, which takes that wide item pool and works out which items group together and load cleanly onto meaningful factors, then cuts the ones that don't. The finalised scale is a much shorter and captures the underlying structure without the redundancy. The goal is well under 20 items total across both acute and chronic. Although I am at the mercy of the analysis as to what the final number will be.
As such the current length isn't an accident. Starting wide and cutting based on real participant data is the only way to do this properly. But I'm fully aware that it makes the experience heavier than the final tool will be, and that's a trade-off I'm asking participants to accept to achieve high rigour.
I received feedback that some items felt unclear or hard to map onto their own experience. I want to be upfront that this is useful information. Items that don't sit naturally with athletes tend to be exactly the ones that don't load cleanly in factor analysis. So, they should be removed through this process naturally.
What comes after
Once the analysis is complete and the scale is reduced, there's one more round after this focused on validation, looking at concurrent validity (does the new scale correlate with established measures of mental fatigue) and test-retest reliability (does it produce stable results across time). I'll be writing this round up as a paper either way, and I'll come back here with a summary including which items survived and what the final scale looks like. Happy to answer questions in the comments on the methodology or mental fatigue research in general too.
For anyone who hasn't filled it in yet, the link is below. It takes 10-15 minutes to complete and will help us get 1 step closer to understanding how work impacts the sport we love.
https://derby.questionpro.eu/t/AB3vCJoZB3waVr
Cheers.
Cam