In my previous post I discussed how the Serial podcast reminds me of doing science: starting with an initial straightforward question that rapidly becomes a murky mess, poring over ambiguous evidence that provides few clear answers, and so on (read the post here). Today I’m going to continue with this analogy and discuss what the search for truth in science really means.

One of the most frustrating aspects of Serial is that a lot of the evidence comes from witness statements, and you never know who’s lying. Whether you ultimately believe that Adnan is innocent or guilty has a lot to do with who you think is telling the truth, from Adnan to Jay to the other witnesses. But it’s impossible to be sure whose story to believe. You can tell this is one of the things that drives host Sarah Koenig crazy throughout the podcast.

At least in science, our experimental subjects aren’t intentionally lying to us. (Unless you work with humans, in which case it’s possible.) Our subjects can betray us in other ways, though. Whether we’re working on molecules, cells, fruit flies, or mice, our subjects often change their story. My fruit flies can behave differently depending on the day or season, subtle differences in their food, or which other flies they’ve been hanging out with in their vials. Basically, a nearly infinite list of things that I can’t control.

Even the neurons that I study like to change their story, despite being individual cells and not conscious beings. In one fly they might be perfectly quiet, and in the next they’re firing like crazy. I used to think that these variable results meant that I was doing things wrong, until I realized that most everyone else is seeing the same type of thing too—they just don’t like to talk about it.

Is there truth?

All this equivocation from my experimental subjects has made me doubt whether a concrete truth really even exists at all in biology, especially neuroscience. Biology isn’t like physics; there aren’t universal laws that always apply. For every tenet of biology that’s been discovered, there are exceptions to the rule.

Remember learning the “central dogma” of molecular biology, which states that information is transferred from DNA to RNA to protein? Well, that’s not always true. For example, some viruses transfer information from RNA to DNA.

How about the fact that neurons receive signals through their dendrites and send signals through their axons, which is probably the first thing you ever learned about neuroscience? There are plenty of exceptions to that rule, too: sometimes dendrites send out signals and sometimes axons receive them.

What about the long-discredited Lamarckian notion that your life experiences can be passed on to your offspring, helping them adapt to the environment? While this idea may have been ridiculed by your high school biology teacher, in recent years scientists have discovered that sometimes life experiences can indeed change your DNA in a heritable way, opening up a whole new field called epigenetics.

Don’t get me wrong—general principles are still central to our understanding of biology even if they aren’t universally true. But my point is that any biological phenomenon you discover is only going to be true in certain cells or organisms under a certain set of conditions. If it’s true in 99% of cases, that’s awesome. Even 50% isn’t so bad. But what if your “big discovery” only ends up being true for the very specific set of conditions that you happened to use for your experiment? That would kind of suck. And then when people can’t repeat your results, they may cast doubt on the validity of your work even if you did nothing wrong. That really sucks.

So that’s why I say that sometimes, even in science, there might not be a single right answer. Different researchers might get different results after performing ostensibly the same experiment, because there are so many factors that influence how biology works. And since every level of biology provides a foundation for the level above it (e.g. from molecules to single cells to populations of neurons), this problem gets magnified the “higher up” you go.

By the way, I’m not trying to make you guys too depressed about science or anything. I just want to highlight how science is really hard and confusing, and not simple or clear-cut like non-scientists probably think.

I imagine that no one listening to Serial was surprised that it can be hard or even impossible to uncover the truth in a murder case. But I’m guessing that a lot of people think science is totally different, that there’s some concrete truth that we can discover by performing objective experiments that produce objective results. I’d argue that in biology, not only are the experimental results subjective, but that there may not even be an absolute truth to discover in the first place.

Spinning your story

The fact that biology is messy, that an absolute truth may not be attainable, isn’t necessarily a bad thing. It’s just how things are. The problem is that a lot of scientists haven’t come to terms with it. Nowhere does this denial manifest more clearly than in the peer review process.

Let me start by going back to Serial. There were one or two episodes that specifically discussed Adnan’s trial and the arguments made by the prosecution and defense. These were the episodes that I could barely get through, because it infuriates me how our justice system allows lawyers so much liberty in spinning their stories—to cherry-pick whatever evidence supports their side and pretend like the rest doesn’t exist; to portray the defendant as either a monster or a saint; to invent a narrative based only in wild speculation.

Even if you believe Adnan is guilty, I don’t see how anyone believes the prosecution’s story that he was a crazy ex-boyfriend and a terrible son; there’s just no evidence for that. The defense took plenty of poetic license as well, like trying to cast blame on the poor streaker dude who found the body. I’m sure some of you will think I’m being naive, but I don’t understand why the justice system can’t be more of an objective search for the truth (much like the Serial investigation). Instead, our justice system consists of two opposing sides trying to convince an audience that they’re right by any means possible, whether it’s actual evidence, bombastic rhetoric, emotional appeals, or legal technicalities.

The thing is, science is actually not so different—especially in today’s competitive environment when a high-profile publication can make or break your career. Sure, you can’t rely on emotional appeals and your story has to be more entrenched in data, but you’re still trying to convince an audience to believe your narrative rather than objectively presenting the data.

These days, it’s impossible to publish a high-profile paper unless you have a “clean” story. Reviewers want to see a relatively simple, satisfying conclusion backed up by flawless data. So if you’ve got results that conflict with your main conclusion, you’re just not going to put them in the paper. It’s the equivalent of “bad evidence”, which former homicide detective Jim Trainum discusses on Serial: evidence that “go[es] against [the police’s] theory of the case”. Referring to police detectives’ role in investigating a murder, he puts it bluntly: “Rather than trying to get to the truth, what you’re trying to do is build your case, and make it the strongest case possible.”

This isn’t so different from what scientists do when they’re writing up papers. You’ll rationalize your decision to leave out the “bad evidence” by telling yourself that every experiment has potential confounds, that there are plenty of legitimate reasons why those troublesome results might not be correct. But if that’s true, shouldn’t you be able to publish them alongside the rest of your data and let readers draw their own conclusions?

For a high-profile journal, the answer is no: if you put in a bunch of conflicting data, your paper just isn’t going to get accepted. That’s why nearly every paper that gets published in these journals reads like a perfect story, a scientific fairytale where everything works out just as it’s supposed to.

Take my grad school experiments that I told you about earlier, the ones where both activating and silencing the same set of neurons produced the same result instead of having opposite effects. These results may be counterintuitive, but there are plenty of plausible explanations. And in any case, I can’t actually control what results I get, I can only report what I find and interpret them the best I can.

Well, that wasn’t good enough for the reviewers who read our paper. They couldn’t wrap their head around these unexpected, complicated results; they wanted things to be simple. They also were offended by other “complicated” findings, such as different fruit fly strains behaving differently from one other. (Apparently in reviewer-land, all individuals of the same species are supposed to act identically, like little automatons.)

The main criticism of our paper that led to its rejection, as stated by the editor, was that “many of the results are hard to interpret”. Now, I didn’t necessarily expect the paper to get accepted at that journal, as it’s one of the most selective in the field, but do I think there’s something wrong when high-profile journals only want to publish results that are easy to interpret. All that does is increase our incentive to work on simple problems, discard conflicting data, and spin the story in a way that we think the reviewers will like.

As common as those practices are, I think we can all agree that that’s not what science should be about. Science should be about about exploring difficult questions, not just the simple ones. Science should be about conducting careful experiments and reporting the results as you find them, regardless of whether they’re “hard to interpret”. Science should be about following those results wherever they take you, even if it’s somewhere unexpected or confusing. Science should be about searching for the truth, as elusive or multifaceted as it might be.

Hopefully someday we’ll live in a scientific world where these ideals are reinforced, rather than undermined, by the reality of the scientific process.