In the book The Nine: Inside the Secret World of the Supreme Court, Jeffrey Toobin wrote that Justice David Souter wept after the Supreme Court’s nakedly political, legally indefensible and bitterly divided decision in Bush v. Gore. And apparently it’s been downhill from there.

I’ve been hearing for years that Justice Souter would be the next retirement from the court, even ahead of the 89-year-old Justice John Paul Stevens and Justice Ruth Bader Ginsburg, a cancer survivor. The skinny, as I’ve heard it, is that he just doesn’t like the life of a Supreme Court justice. He’s a New Hampshire loner, an introspective man, more enamored of the simple life at his farmhouse outside Concord than the Washington social circuit. In his 19 terms on the court, he’s proved himself to be quite brilliant, and of course he became the stealth nominee who didn’t turn out quite the way President George H.W. Bush’s handlers expected. In fact, Justice Souter emerged as one of the strongest and most principled moderate voices on the court (don’t let anybody fool you—there are no real liberals on this court).

One of the most interesting aspects of Justice Souter’s tenure on the court is that he’s demonstrated how a white, male Republican Northerner from a mostly rural state can become educated about, and even a champion for, the legal rights of minorities, women and the poor. If you listened to his questions during Wednesday’s oral argument in North Austin Municipal Utility District Number One v. Holder, the case that may result in the dismantling of the 1965 Voting Rights Act, you heard the contribution of someone who’s been paying attention to and learning about the interplay of race and politics in Southern states, and who’s taken the time to learn something of the history that forms the critical context in which the Voting Rights Act was enacted and reauthorized. In this sense, Justice Souter’s tenure on the court is a real success story – not for liberals or moderates, but for the way the court should work.

At its best, the court can be a place where the interaction among justices of different backgrounds and viewpoints produces dynamic and informed decision-making. When Justice Thurgood Marshall retired in 1991, Justice Sandra Day O’Connor was particularly eloquent about the effect of Justice Marshall’s storytelling in the justices’ conferences. Marshall’s experiences as a black man growing up in segregated Maryland and as a civil rights lawyer traveling through the South opened up a world for his colleagues on the court, a world to which they had virtually no exposure. Justice Byron White later wrote that Marshall told them “things they knew, but didn’t want to know.” The stories Marshall told were not just his own, but those of the hundreds of clients he’d represented – black teachers and schoolchildren, criminal defendants wrongly accused, black lawyers, black Army privates facing court-martial in Korea, civil rights protesters, laborers, mothers and Pullman car porters. This is one of the (many) key differences between Justice Marshall’s contribution to the court and that of Justice Clarence Thomas. Marshall knew and could relate multiple and varied experiences of black life. Justice Thomas talks about black life with his colleagues and in his opinions, but almost always only about his own life.

Which brings us to thinking about Souter’s possible successor on the court. A good deal of the chatter already has focused inordinately on how selecting a nominee will play as move for President Obama. Others have begun to speculate about how Sen. Arlen Specter’s defection to the Democratic Party this week will affect the dynamics of the Senate Judiciary Committee, which Democrats lead and where Specter has been in the past a very forceful ranking leader (think the Thomas hearings and his treatment of Anita Hill). Having switched parties, he’ll now have no real power role on the committee. Interesting, yes. But the appropriate question should be, what does the court need? What are the perspectives and experiences that are missing from the court? Three things strike me as critically important.

First, the court lacks an African-American justice who can articulate and literally “bring to the table” the unique perspective of how law interacts in the lives of black people. It’s an important perspective because it is the basis upon which almost all of our civil rights laws were enacted. Without a force at the table during deliberations who can provide context and meaning to these important statutes – and who can do it with moral and intellectual power – we may find ourselves on the precipice of the end of federal civil rights law. If you look back at oral argument in the voting rights case that came before the court this week, and the almost hysterical solicitude several of the justices demonstrated for what they seemed to regard as a statutory unfair singling out of Southern jurisdictions for special monitoring under the Voting Rights Act, the danger is clear.