William Harrison Scarborough was born on November 7, 1812 in Dover, Tennessee. While his family heritage was English, the Scarboroughs had been living in America for two generations at the time of William's birth. Growing up, William's parents encouraged each of their thirteen children to pursue their educations, and at sixteen he left home to explore the South and seek out educational opportunities. He initially studied both medicine and art, but after quickly discovering that he did not have the stomach for surgery, he dedicated himself to becoming an artist.
In 1828 Scarborough began his serious study of art at the Littleton Museum in Cincinnati, Ohio with Horace Harding and Henry Inman. In October of 1833 he married Sarah Ann Grimes and they lived in Tennessee. Two years later his young bride died giving birth to their son, John, and Scarborough soon left the state wanting to make a new life for himself and his infant son. After a brief time in Alabama they settled in Charleston, South Carolina, a cultural center of the antebellum South. In 1836 Scarborough made the acquaintance of John Miller, a planter and lawyer who commissioned many portraits of his family which included seven daughters, one of whom Scarborough married in 1838.
Charleston was not the only wealthy and culturally rich area of the Carolina coast and in 1839 the family moved to nearby Darlington District. Scarborough found a patron in Colonel Edgar Welles Charles and he painted many members of the colonel's family. Through his friendship with the Charles family, Scarborough met many influential people from the District and was able to obtain enough commissions to live very comfortably. Scarborough's career escalated, however, once he moved to Columbia, South Carolina in 1843. There he painted the most prominent members of South Carolina society, including governors and high ranking military officials. The following thirty year period marks the pinnacle of Scarborough's career and most of his well known works were painted during this time. William H. Scarborough died in Columbia in 1871 at the age of 58.