Why I Would Like the Media to Please Stop Talking About Science Ever Kthanxbye

This morning, someone linked to me an article about girl toys and boy toys on Cracked.com. (I don't have the link handy, but I'm sure you can find it.) In it, one of the things they discussed was that gender bias may be inherent after all, because of "a study" that showed girl chimps playing with dolls and boy chimps playing with trucks.

This set off a little "Ah-OOGah!" siren in the back of my head, and it's one that I'd like to see ingrained into the heads of every single journalist ever. (Actually just shutting up about science ever was hyperbole.) Because the first and foremost thing to understand about science is that when "a study" shows something, that is not proof. That is not the end of the scientific process. That is the beginning.

Because anyone can run a study. There is no International Board of Scienceness that monitors the design, protocols, and data selection for a scientific experiment to ensure that the scientist got it right, and scientists are not all inherently free of bias, ideology, error, hidebound thinking, stupidity, mendacity, fraud, and general cranial-rectal interactions. A study, even one that makes it into a peer-reviewed journal, is only worth something once other scientists read it and successfully, consistently replicate its results.

In the case of our chimps with trucks study, there's a lot we don't know. We don't know how many chimps they used. We don't know how long they spent with the chimps. We don't know how diligent they were in recording their activity, or what they considered "play" to be. We don't know whether the chimps were aware they were being observed, and whether they received any kind of different treatment, consciously or unconsciously, if they "played" in the manner the researchers expected. All we know is that "a study" was run, and its results happened to confirm the scientists' gender stereotypes.

This is not a basis to report something as fact.

And more significantly, while I'm using this as an example, it is the common way that science is reported in the media. "A study today said...", the article generally starts, without commenting on who ran the study, what its research protocols were, whether there was any possibility of bias on the part of the people commissioning the study (hey, the tobacco industry just reported smoking is good for you! Wow, who could have expected?) or whether the results have been replicated anywhere else. This is like reporting that a seed was planted yesterday, and expecting dinner on the table that night.

So please, journalists everywhere--only report on "studies" whose results have been consistently replicated after being reported in peer-reviewed journals, and leave off on authoritatively citing random studies out there to make it sound like your point has been proven by SCIENCE! for a while? Kthanxbye.