Freeman Walker is a story told by a mulatto slave, Jimmy Gates, freed by his owner-father when he is 7-years-old, separated from his mother and everything he holds dear. After receiving an unforgettable talk by his father about the rules of life he will no doubt discover on his journeys, and a copy of the Declaration of Independence, he is sent to England to get an education. Jimmy, in the first of the novel's great ironies, has had a blissful, loving childhood and never understood he wasn't free until his new "freedom" enslaves him miserably. Despite his loneliness for home, he learns fast and well and makes himself a good and popular student. Four years pass, and while he is waiting for his father to visit for the first time, he learns that his father's ship has sunk and his father has drowned at sea. Bereft of financial support, mourning still his long lost mother and now his father's death, Jimmy is sent to a London workhouse where he spends six years making saddles, reading heroic novels to his companions, being sexually abused by the proprietor, finding the comfort of prostitutes, and discovering the inspirational speeches of an Irish revolutionary named Cornelius O'Keefe, or O'Keefe of the Sword.

When he is 18, dreaming himself a warrior and a hero, he returns to the States to rescue his mother. While looking for his mother in northern Virginia--he discovers that if he wears a hat he can pass for white--he gets caught in a major battle. Jimmy is overjoyed to be able to take part, but is soon overwhelmed by its horror. Untrained, and unattached to any unit, he nevertheless has a chance meeting with O'Keefe of the Sword, who is now a Union General leading a brigade of Irishmen. Jimmy saves O'Keefe on the battlefield, but later is captured himself by Confederate forces, and again made a slave, spending the next two years attached to a confederate regiment digging graves. When his unit is overrun and he is found shackled in a root cellar with his friend, a Yankee officer presents to him a terrible choice, stay locked up, or commit an atrocity and go free. He chooses to walk free.

He changes his name to Freeman Walker and as he reinvents himself once again and makes his way into the mythic territory of the Great American West, the novel begins to change. He hopes to live peacefully by getting rich, and he does live peacefully and get rich, for a while. But his race catches up again, and he is lynched, and he loses his treasure, and he surrenders to the mud on the side of the road, and looks forward to the coming winter and his own demise.

But into the territory that winter rides the new territorial governor, none other than his childhood hero, Cornelius O'Keefe, who the war has turned into a pacifist. Freeman's life changes once more as he becomes O'Keefe's secretary, and the two of them, joined by a half-breed captain named Felix Belly--three outcasts--form the only government in the Territory, a wild and savage place run by vigilantes. Their quixotic attempt to stop the vigilantes from a campaign of terror against the Natives spurs a terrible but noble adventure and brings Freeman a kind of rebirth in which he finally comes to understand the meaning of moral freedom.