When it comes to political activism, “pro-choice” and “pro-life” have definitive, clear-cut meanings. Pro-choicers stand for increased access to all forms of reproductive health care, including contraception and abortion. Activists who call themselves “pro-life” want to ban abortion and take a dim view of contraception access. (This is why pro-choicers call them “anti-choice,” because it’s frankly a more accurate description.) But when you step out of the world of those who are pushing for specific legislation or policy changes, you’ll find that a lot of people don’t think about this issue in such a binary way. Their views could be charitably described as “gray,” or more accurately as “muddled.” This is a fine way to view abortion when it comes to your own personal choices, but creates problems when we’re talking about policy.

Usually, Gallup polling—the most frequently referenced source on these issues—tends to ask average people to choose whether they’re “pro-choice” or “pro-life.” A recent poll performed by Vox, however, tried to get at a bit more nuance. When given more flexibility in how to answer, Vox found, four in ten Americans shunned simple categorization. Eighteen percent identified as “both” pro-choice and “pro-life”; another 21 percent identified as “neither.”

As Vox writer Sarah Kliff noted, it got even more complicated than even that. For instance, 28 percent of Americans agreed with the statement, “Abortion should be legal in almost all cases.” But far more were on board when they were reminded that women are involved; a full 37 percent agreed with the statement, “Women should have a legal right to safe and accessible abortion in almost all cases.”

The easy way to interpret this data is to decide that a lot of people love to opine on issues they’ve probably not given much thought to. But the reality is a bit more complex. The problem is that abortion is so often framed—in both the media and in everyday dinner-table discussions—as a moral issue instead of a health-care or legal issue. Because of this, when average people are asked about it, they tend to think in moral terms, often to the point where they forget that this is a legal question at all.

When you ask when abortion should be “allowed,” people often mistake the question for one asking them when abortion is right. And so they treat the prospect much like you would if someone asked you when it’s OK to lie or to break up with someone—as if it’s a question of hypothetical morality, as opposed to a legal one.

Vox’s findings demonstrate how true this is. A lot of people wanted to spitball their ideas to Kliff about when abortion is justified and when it is not, without even considering whether their elaborate framework would be definable, much less enforceable, in the law. “I guess if you’re raped or in a desperate situation, then abortion would be the way to go,” one woman told her. “But if you’re just being careless and irresponsible, then I don’t think it’s the right decision.”

“I don’t agree with her decision, because she was being reckless,” another said about a friend who aborted, before going on to say she nonetheless understood that she couldn’t have the baby.

“I think in situations where it could be proved that the woman had control of the situation, they should not be allowed to abort the baby, because they were part of that conscious decision,” another man on the street told Kliff.

What becomes clear in these interviews—and in research conducted by Planned Parenthood that shows a similar “it depends on the reason” attitude toward abortion—is that what people are really passing judgment on is not the abortion itself, so much as the sex that caused the pregnancy in the first place. What one might delicately call “nuance” is, in reality, just the same old sex policing: a paranoia that women are escaping the “consequences” of having too much sex or the wrong kind of sex.

It’s the same story with the sentiment, “I could never do it, but it should be allowed,” which pops up in basically every casual discussion of abortion, whether it be online or off. In fact, a woman in the Vox video says it in so many words. While this has the virtue of being more generous and clear-headed on the legal issues, the essential premise is the same: The speaker wants you to know that while she’s pro-choice, she doesn’t want to be mistaken for one of those kinds of girls.

None of this is really that surprising, if you look at the larger context. Americans are confused, generally, about how we feel about sexuality, especially female sexuality. Women are supposed to be sexual but not too sexual, sexually available but not slutty. There are no hard-and-fast rules on the subject of respectability: Instead, where the “line” is and who has crossed it is entirely dependent on the gut feelings of the person passing judgment.

Naturally, prejudice strongly influences whether or not the self-appointed judge in question feels someone has crossed the slut barrier. Anyone who has had “slut” thrown at her by a man who was calling her an angel right up until she told him she wasn’t interested could tell you as much. How much sex is “too much” depends on whether your assessor likes you, how they feel about themselves, and probably their mood at that moment. Class and race, too, strongly impact who gets deemed sexually loose, or, as Kliff’s respondents might say, “careless,” “reckless,” or “irresponsible.”

With all this in mind, Kliff writes that abortion “is as complicated and personal an issue as they come.” Of course it is. All issues regarding sexuality are, especially when you invite people to compare themselves to others and pass judgment.

The problem is that the blunt instrument of abortion law has no interest in personal judgment and isn’t equipped to deal with complexity. The law cannot decide if someone’s forgoing of the condom was “reckless,” to use one of Kliff’s interviewee’s words, or understandable passion. The law is not interested in whether or not you think your friend is too quick to go to bed with a new guy and therefore should have felt worse about her abortion. The law is all about external, simplified parameters: How many weeks pregnant you are and whether you jumped through enough legal hoops to get the procedure. In fact, even though legislators sometimes try to delineate “acceptable” abortions by building rape exceptions into restrictions, the law is terrible at sorting out who gets one of these exceptions from who doesn’t.

None of these legal limits is remotely capable of differentiating whether a woman is “desperate” enough or whether others might deem her sexual encounter reckless or responsible. It’s natural for Americans in this muddy middle to assume that having some restrictions on abortion will sort the supposedly undeserving from the righteous, but the real world shows that is not the case at all. Still, unless they’re reminded of that fact, it’s unlikely that they’ll be compelled to come out swinging as legislators chip away at access.

There’s no quick-and-dirty solution to this situation. When you force people to think of abortion as a public health issue by asking questions or showing data that frames it that way, most people are pretty smart about the issue, opposing most of the restrictions anti-choicers would place on abortion. But then we all go straight back to talking about abortion as an abstract moral issue: a specialty of anti-choicers, who dwell endlessly about their personal values, as if those should matter one whit when it comes to someone else’s decision-making.

It’s absolutely true, then, that the recent surge in abortion story-telling is an important part of the long-term solution. After all, if people can stop stigmatizing the procedure, they’ll lose interest in supporting restrictions on it out of a misplaced hope that doing so will keep the “wrong” women from getting abortions. But in the meantime, it doesn’t hurt for pro-choicers to remind everyone, as much as possible, that whatever your personal feelings on abortion may be, basic human rights are at stake here.