Sharpe was born in Kingston, Ulster County, New York. He graduated from Rutgers in 1847, then studied law at Yale College. He was admitted to the bar in 1849 and practiced law at the firm of Bidwell & Strong (now known as Cadwalader, Wickersham & Taft). Then he traveled to Europe and served 1851-52 as Secretary of Legation at Vienna. After his return in 1854, he practiced law until he joined the Union Army in 1861 as captain of Company B of the 20th New York State Militia (known as the "Ulster Guard") for three-months service.
In January 1863, Sharpe assumed the intelligence role for Hooker that Allan Pinkerton had performed for McClellan. His estimates of enemy troop strength proved to be far more accurate than those of his predecessor.
In 1867, on request of William H. Seward, Sharpe became a special agent of the U.S. State Department and went to Europe to locate and investigate Americans who might have been involved in the assassination of President Abraham Lincoln. Seward was particularly interested in finding John Surratt, whose mother Mary Surratt had been hanged as one of the assassination conspirators. Surratt was brought back to the United States and put on trial in a civilian court. The trial ended with a hung jury, and Surratt was soon set free, never to be tried again.
Sharpe was married to Caroline Hasbrouck, daughter of Abraham Bruyn Hasbrouck, and their children were Severyn Bruyn Sharpe, a county judge, Henry G. Sharpe, a U.S. Army officer, and Katherine Lawrence Sharpe who married Ira Davenport. He died while visiting the Davenport's residence at 31 East 39th Street in New York City. He was buried at Wiltwyck Cemetery in Kingston, New York.