Discussion about this post

User's avatar
Joseph Conner Micallef's avatar

What's particularly frustrating is that in a lot of ways hackish research is MORE of a threat than deliberately false research! False research is usually discovered and un-published; it's harmful but more temporal. hackish research lives forever because it's not TECHNICALLY wrong and can really damage discussion of public policy issues for a generation or more. There's serious serious serious ethical questions surrounding research done using garbage methods and/or garbage data that almost always ends up being shoved off into the corner because a cadre of people want to make laziness and conformation bias some kind of shady conspiracy to undermine their own pet beliefs. We as a political science and public policy community can't have a desperately-needed discussion about how massively damaging p-hacking, garbage data, lazy analysis, and overconfident conclusions are to the field when every opportunity to have those discussions becomes a story about fabrication instead.

Expand full comment
Joshua's avatar

As a physicist, I'm always curious about how radically different research practices are in the social sciences, and for good reason. Obviously the kind of highly quantitative, repeated, high-sample-size projects we do in physics are simply not practical in the social sciences. But many times it seems to be simply a matter of resources. There are many questions in the social sciences where there are hundreds of individual, uncoordinated studies that each have low SNR and many confounding factors. But if you were able to devote all the resources across all those studies into a single, coordinated, large longitudinal study, you could get something much more transformative. But academic funding just isn't set up for that. You would need every professor and grad student at hundreds of institutions to forego leading their own projects to participate in a huge research network.

But the science benefit would be huge.

Am I wrong in thinking of that as a way forward for complex social science? Is it a problem of the structure of how social science operates in that way, or are there more fundamental issues where scaling up the work would not actually address the problems of signal size, confounding variables, and replicability?

Expand full comment
8 more comments...

No posts