Sanity check: I want to ban discussion of sample characteristics from article critiques this year.
One of my assignments in my research methods class is leading a class discussion of an empirical article. It seems like, even before people could ""read"" with AI and definitely after, there are two types of critiques that go down in U.S. college classes:
- U.S. Sample: critique immediately hones in on issue of WEIRD samples, beats the dead horse for 50 minutes
- Non-U.S. Sample: everyone concludes study is meaningless due to only being from Guatemala, or Korea, or Sweden, and dismisses it entirely; bonus points for any outright xenophobic comments
Like, there's never anything about sample size and statistical power, or about sample recruitment and demand characteristics. No one even brings up potentially meaningful differences between the U.S. and China (commerce, internet moderation, attitudes toward family, etc.) or concerns about translation, or even a meaningful explanation of why WEIRD matters. It's all either "WEIRD is Bad because it is" or "what could we possibly have to learn from Chinese students?"
Anyway, I'm thinking of announcing on the first day that they're more than welcome to share opinions on the articles, but criticisms of sample characteristics are banned unless run by me first. Am I depriving them of a whole swathe of important and valid critiques out of pique, or could this actually work to cut down on the casual ethnocentrism?