Michigan voters in 2006 approved Proposal 2, a measure that told the state’s public colleges and universities they could no longer “grant preferential treatment” on the basis of race in their admissions policies. Last summer, the Supreme Court overruled a federal appeals court and decades of jurisprudence and ruled Prop 2 was constitutional in Schuette v. Coalition to Defend Affirmative Action.

“This case is not about how the debate about racial preferences should be resolved,” wrote Justice Anthony Kennedy for the majority in Schuette. “It is about who may resolve it.” And the “who,” in the balance between citizen voter initiatives and the Court, would tilt in favor of the citizens.

Fast forward to almost a year later: If there is a way for the Roberts Court to screw up a seemingly likely win for marriage equality, it will be because of Schuette.

In Obergefell v. Hodges, the marriage equality case, there are two legal questions before the Court. The first is the marriage question: whether states can refuse to grant marriage licenses to same-sex couples. The second is the recognition question: if states that ban same-sex marriages must recognize legally performed marriages from elsewhere.

But if those were the legal questions before the Court, the political question weighing on the minds of the conservative justices during Tuesday’s oral arguments was evidently “who gets to decide” what the definition of marriage should be.

Justice John Roberts told Mary Bonauto, one of the attorneys representing the couples, that if the Court rules that states must issue marriage licenses to same-sex couples, it would “redefine” marriage, noting that “every definition I looked up until about a dozen years ago” defined the institution as between heterosexual couples. Meanwhile Justice Antonin Scalia sniffed, “You are asking us to decide for this society [whether to allow same-sex marriage] when no other state has had [same-sex marriage] until 2001?”

Justice Anthony Kennedy picked up Justice Roberts’ historical concern for the institution of marriage, declaring that the definition of marriage as a union between one man and one woman had been around for millennia. He then pivoted to Justice Scalia’s point, noting that it was only recently that the Court ruled Texas could not criminalize sex acts between consenting adults like sodomy bans. The point, Kennedy pressed, was that “it’s very difficult for the Court to say we know better.”

That was just the kind of statement from Kennedy marriage equality opponents were hoping for: a signal that the justice was at least considering upholding the state marriage bans on similar grounds to Schuette by deferring to the will of the people rather than the judgment of the Court. In Schuette, Kennedy wrote, “Our constitutional system embraces … the right of citizens to debate so they can learn and decide and then, through the political process, act in concert to try to shape the course of their own times and the course of a nation that must strive always to make freedom ever greater and more secure.” The question of race, and the role of racial preferences may be a difficult one, Kennedy reasoned, but voters should be trusted to be up to the task to engage in that debate themselves.

This was the theme purposefully repeated by opponents of same-sex marriage Tuesday. In their brief to the Supreme Court, attorneys for the State of Michigan quoted heavily from Kennedy’s opinion in Schuette, arguing that it was up to the people, and not the Court, to evolve on the issue of same-sex marriage. Pundits and lawyers widely believe Kennedy to be the swing vote on marriage equality, in large part because of his opinion last summer in United States v. Windsor that struck the federal definition of marriage in the Defense of Marriage Act (DOMA) as between one man and one woman. In that case, Justice Kennedy wrote emotionally about the respect same-sex couples and their families deserve, and the animus expressed in laws intended to keep them out of the institution of marriage.

Kennedy’s language in Windsor may have been heartening. Still, given his opinion and the outcome of Schuette—where, again, the Court left it up to citizens to decide who gets full civil rights and who does not—it seems increasingly less likely that he will rule to strike down state-level same-sex marriage bans. That is, unless you factor in race.

In order to support Prop 2 in Schuette and sweep aside decades of constitutional jurisprudence that protected the use of race in some policy decisions in the name of advancing equality, Kennedy and the rest of the justices in the majority determined that Prop 2 was passed without “racial animus.” In other words, the Court concluded, Michigan voters decided to ban affirmative action plans for some reason other than an intent to discriminate against racial minorities.

It is not surprising that a Court that gutted the Voting Rights Act on the grounds that the country was “post-racial”—and therefore voting protections were unnecessary—would believe that a ban on affirmative action policies could be based on anything other than racial bias. Kennedy even insisted that Schuette was not about the wisdom or legality of race-based admissions policies, but rather whether “the courts may not disempower the voters from choosing which path to follow.”

Those declarations were put to the test in the marriage equality cases Tuesday. Conservatives opposed to marriage equality played up as best they could Kennedy’s anti-federalism in Schuette, making their case all about the voters evolving on the issue of same-sex marriage through the democratic process. Opponents of the marriage bans argued forcefully for the dignity of their clients, relying both on the law and personal anecdotes to drive home the fact that people’s lives and families were before the Court.

So which path will Kennedy take?

Despite being a reliable member of the Court’s conservative voting bloc, Justice Kennedy has authored the Court’s most recent and expansive rulings on gay rights. And they haven’t been terrible. Under his guidance the Court has struck sodomy laws and parts of DOMA, as well as beginning to articulate a line of jurisprudence that recognizes the fundamental human rights and dignity of LGBTQ persons. In fact, during his confirmation hearing, Kennedy, a President Reagan appointee, articulated during his confirmation process what pundits named the “dignity doctrine.” When asked to describe what concepts are protected under the Constitution’s liberty guarantee, Kennedy responded: “A very abbreviated list of the considerations are: the essentials of the right to human dignity, the injury to the person, the harm to the person, the anguish to the person, the inability of the person to manifest his or her own personality, the inability of a person to obtain his or her own self-fulfillment, the inability of a person to reach his or her own potential.”

Kennedy’s dignity doctrine should spell good news for marriage equality supporters. During Tuesday’s arguments Kennedy expressed real skepticism at the state’s justifications for same-sex marriage bans. And as in Windsor, it was clear that the dignity of same-sex couples and their families matters to him.

But Justice Kennedy also has real concerns over the “dignity” of the state and the ballot initiative process. This much was clear in Schuette, as was the fact that Kennedy picks and chooses to whom this dignity doctrine applies and who it does not. In Schuette, the dignity of those impacted by Prop 2 were brushed aside for the “dignity” of a ballot process steeped in racism. And that may be enough for conservatives to reign in a broad ruling from the Court in favor of a narrower one that refuses to mandate states issue marriage licenses to same-sex couples but requires states to recognize marriages of same-sex couples legally married elsewhere. Justices Roberts and Antonin Scalia searched for this compromise position, leaning heavily on the rhetoric of Schuette to try and bring Kennedy to their side. At the end of the arguments, it wasn’t at all clear whether they had convinced Justice Kennedy to sign on.

If they do, and the Court issues a split or more limited decision, then we’ll know Kennedy tried to balance his anti-federalism with his belief in the dignity of LGBTQ persons. But if the conservatives on the Roberts Court are not successful and Kennedy does side with the liberal wing to support marriage equality broadly, then it will be incumbent on progressives to not just talk about Kennedy’s improving record on the dignity of LGBTQ persons, but also his abysmal record on the dignity of non-whites.