When Harvey Weinstein’s predatory behavior made national news, it sparked a long-overdue public reckoning about sexual violence, rape culture, and the ways in which survivors are silenced. In its wake, we’ve seen power players in tech, politics, business, academia, and the restaurant world ousted from their respective positions for sexual abuse.

Now, however, has come the inevitable blowback: an overture of columnists, pundits, and tweeters, crying “sex panic” and warning against a world where anyone accused of sexual harassment can lose their job or place at school “without due process.”

Talking about due process in this context reveals a fundamental misunderstanding of basic constitutional law. A conversation about where sexual violence of all kinds fits in with the legal landscape of our justice system is an important one to be had—but misrepresenting due process’ place in workplace firings or school expulsion serves only those arguing that the #MeToo movement is going too far.

Examples of this misunderstanding have emerged on both sides of the political aisle and from a range of figures. In her recent column for the New York Times about what she argues are the failures of #MeToo, Daphne Merkin opined, “In our current climate, to be accused is to be convicted. Due process is nowhere to be found.” Bari Weiss used her New York Times column to victim blame, and conflate legal and personal justice, saying “The insidious attempt by some women to criminalize awkward, gross, and entitled sex takes women back to the days of smelling salts and fainting couches.” And in another Times opinion column about former Sen. Al Franken’s fall from grace, Zephyr Teachout of Fordham Law argued, “Zero tolerance should go hand in hand with two other things: due process and proportionality …. Due process means a fair, full investigation, with a chance for the accused to respond.”

The right to due process is guaranteed under the Fifth Amendment, which says that no one shall be “deprived of life, liberty, or property without due process of law.” Professor Paulo Barrozo, who teaches law at Boston College Law School, explained to Rewire, “In its most general sense, due process refers to sufficient procedural and substantive safeguards afforded individuals before they may legitimately be deprived of or refused certain rights, protections, or goods that they ordinarily would be able to enjoy.” In short, one’s due process is violated when the government takes away a right.

“The first important thing to note is that due process is not guaranteed in private settings—rather, it is a legal concept that applies to government action,” said Andrea A. Curcio, a professor of law at Georgia State University College of Law. “Thus, there is no such thing as a legal ‘due process right’ when it comes to getting fired by a private employer.” She pointed out that due process claims can only be filed against government entities.

As Constance Grady recently wrote in Vox, the standard of proof for a conviction in a criminal court is much higher than in a civil court and internal corporate investigations—which makes criminal convictions for sexual violence hard to come by. She points out that “for cases involving sexual violence, these kinds of lower-stakes alternatives are invaluable because sexual violence is very difficult to prove beyond a shadow of a doubt.”

“At a private company, firing someone because they are a suspected sexual predator is not a deprivation of due process,” said Colby Bruno, senior counsel at the Victim Rights Law Center.

Bruno also notes that even when employers are well within their rights to fire an employee—meaning they are not doing so on discriminatory grounds—they sometimes have to defend those actions in a lawsuit.

“An employer will have to sink a lot of money defending [an] action,” she says, “so the idea that employers engage in a witch hunt is absurd, not true, and an argument used by rape apologists.”

Curcio told Rewire that those who are concerned about “due process” are, in fact, likely concerned that the accused are getting fired without an opportunity to defend themselves or to present their side of the story. As she points out, that is the essence of due process—and those calling for it are likely using the term colloquially, as opposed to literally.

Still, that argument is flawed.

“Remember, due process does not mean that you have the right to be believed, just that you have the right to be heard,” said Curcio. She says it is likely most employers engage in some type of investigation when sexual harassment claims are brought. As Grady points out, many did face the scrutiny of internal investigations before getting the ax.

Bruno also says that an employer can and should conduct an investigation into allegations of sexual violence in the workplace; they must also “consider the safety and security of the other employees.”

“An employer needs to do a risk calculation, ‘Should I keep this person at this job knowing that something happened … or should I fire this person because they are a danger?”

Alexandra Brodsky, an attorney with the National Women’s Law Center and co-founder of Know Your IX, also argues that the impression that men are being removed from their jobs without a proper investigation may be due in part to the number of men who have resigned before an inquiry into their behavior.

“So many public figures who have been accused, have stepped back to avoid investigation,” she says, “which feeds this idea that there is a lack of due process.”

Furthermore, if the concern in these instances is false accusations leading to wrongful terminations, Curcio rightly noted that “the stereotype of the ‘false accuser’ is a prevalent one when it comes to sexual harassment claims—yet the data suggests that very few accusations are false; much more prevalent are the women who fail to report the abuse.” Roughly two out of every three sexual assaults will go unreported, and out of every 1,000 rapes, only six will yield a conviction. And only an estimated 6 percent of rape accusations are false.

When is termination unlawful?

All of this is not to say though, that getting fired is never a matter of violation of your rights. Barrozo points out that while the law gives a good amount of latitude to employers when it comes to terminations, there are occasions when it is, in fact, unlawful. Although getting fired from your job for being a sexual predator is not a violation of your due process rights, for instance, survivors of sexual violence in the workplace do have statutory protections that entitle them to redress under the law.

As Curcio explains, Title VII of the Civil Rights Act of 1964 was enacted to remedy specifically sex- and race-based discrimination at the workplace, and it eventually came to include sexual harassment as a form of sex-based discrimination. Using the example of a boss telling an employee she has to sleep with him to get promoted, Curcio said, “that [is] a form of sex based discrimination, because the job now hinged upon compliance with sexual demands that were based on the person’s gender.”

Title IX provides similar protections for students: It “ensures that schools need to prevent and respond to known sexual harassment, including assault, and part of that response has to include that the student needs to continue to learn safely in the wake of violence,” as Brodsky said.

Bruno, who is an expert on Title IX, says Title IX is “deeply intertwined” with fundamental due process rights.

Under the law “in a state school, both parties should receive due process rights, and in a private school, both parties should receive due process rights,” she says. This means that both parties must be notified of the complaint, have an opportunity to present evidence, and should both be treated fairly.

Conversations around Title IX often center on discipline, but Brodsky said that while it is “certainly true that sometimes schools need to take action against a wrongdoer to make sure that they don’t hurt others and to understand the severity of what happened,” she says that in her experience, many survivors report in order to access support, like mental health counseling. Bruno says this also includes requiring that survivors and perpetrators have the same access to education—for example, if they are in a class together, Title IX says that either the survivor is entitled to switch classes, or the perpetrator can switch.

Bruno says there is a marked shift for the better in how campuses are addressing these issues. In recent years, she has seen a push to educate schools and students about their Title IX rights, and responsibilities. It’s not that women are experiencing sexual violence more or less now, she said, but that there has been a “small cultural shift into the belief that that is no longer acceptable.”

Still, there is room for improvement when it comes to these laws. Curcio argues, though, that change needs to go beyond the legal framework.

“Could the laws be better?” she asked. “Probably. Would better laws change a culture of sexual violence? Not necessarily.”

“Legal standards can help shape cultural understanding of societal wrong,” she continued. “However, true change only occurs when we educate ourselves and others about the harms caused by both overt and subtle sexually aggressive conduct, decide that conduct is socially unacceptable, and call out those who engage in unacceptable behaviors.”

Who are people in power supporting?

If anything, Brodsky says, there is a demand for more scrutiny, more diligence, and more benefit of the doubt for the accused, when it comes to sexual violence in the workplace. “You rarely see these kinds of arguments for people being bad at their job,” she said, with regard to individual firings. “Suddenly, you’re talking about sexual harassment and people want to have a full criminal trial,” she continued.

Writer and activist Ijeoma Oluo credits this to a longstanding prioritization of men and devaluing of women in society.

“If you always have something, even if it’s something you never should have had, taking it away feels like an injustice,” she explains. “So much of our national dialogue prioritizes men, and that plays to privilege. But now that security has been taken away.”

Oluo, who wrote recently about the long overdue need for due process for survivors, argues that women in the workforce are devalued in a way that makes it harder to fire a powerful man—even if he’s a well-known predator—than to lose a woman. And Brodsky sees this trend mirrored in her work with girls and young women in public school. Though Bruno was hopeful, Brodsky says that in the cases she’s been involved in, administrations have never conducted investigations without lawyers getting involved. The student filing the claim often ends up being effectively punished, she says: sent to an alternative school, or cyber school.

These issues are both personal and societal, says Oluo, and so should the response to them be.

“We have to take a multilayered approach, because sexual harassment and sexual assault are both personal and societal evils. Just like racism in America, or any of these things that are built off systemic oppression, happen on two separate levels … you need personal accountability and societal accountability,” she said.

On a personal level, she said, “There is no way to move forward without justice even on an individual level, to say you made something right unless you pay. We really do need personal atonement; you have to say that you paid for what you’ve done and you worked to be a better person.”

Meanwhile, on a broader scale, she continued, “You can’t have strong HR policies that are well enforced in a society that doesn’t care about women. We have to start looking at all the ways in which we do de-prioritize women, and make women unsafe.

“We have to start asking, ‘How do we make women of color, and trans women and disabled women even more unsafe?'” she continued. “It’s important to say that you believe women are worth this much, that victims of harassment are worth this much.”

In order for this moment to be successful, to yield real change, we need to address not only the predators in our workplaces, our friend groups, and our classes—we need to push those who excuse and enable to do better, to be better. And while the #MeToo movement will have to parse with how best to handle sexual harassment in a legal and private framework, that conversation needs nuance, not reactionary diatribes.

The gradual tilt towards a better understanding of sexual violence and survivors’ needs feel foreign, even threatening for some. So not only must we confront the sexual predators and the cases before us, we must consider how best to correct the social mores that led us to a society that enables and excuses abuse, and punishes survivors—and this includes ensuring survivors get due process.