Derby was a soldier, cartographer, humorist, engineer, and practical joker. He surveyed the Sacramento Valley, the Tulare Valley, San Diego Harbor, the mouth of the Colorado River. His humor concepts were used and "expanded" by Mark Twain. He knew and interacted with General Winfield Scott, Robert E. Lee, Ulysses S. Grant, and William Tecumseh Sherman. He was a classmate of George Pickett, George McClellan, and Stonewall Jackson. Derby lived in San Diego from 1853-54 in the Derby-Pendleton House, now located on the Whaley House grounds.