Google's 540B language model that was announced this week is incredibly impressive (deciphering jokes, inference chains). Don't think it would be too much of a leap to train an AI to be able to read methodology sections to ascertain statistical method being used and whether it's questionable given the research question at hand. Training set would probably be a collection of widely-considered high quality papers along with a reference table of statistical methods to generally-appropriate applications.
Not saying it's readily doable now, but could be a good automated way to attempt and flag what obviously needs greater scrutiny.
I’m not sure what you’d use as the training set there but am open to ideas. Peer review, as currently deployed, seems to be kinda shitty to be honest.
Google's 540B language model that was announced this week is incredibly impressive (deciphering jokes, inference chains). Don't think it would be too much of a leap to train an AI to be able to read methodology sections to ascertain statistical method being used and whether it's questionable given the research question at hand. Training set would probably be a collection of widely-considered high quality papers along with a reference table of statistical methods to generally-appropriate applications.
Not saying it's readily doable now, but could be a good automated way to attempt and flag what obviously needs greater scrutiny.