Watson Brown was a son of the abolitionist John Brown and his second wife Mary Day Brown, born in Franklin Mills, Ohio. He was married to Isabell "Belle" Thompson Brown, and they had a son Frederick W., who died of diphtheria at age 4, and is buried at what is now the John Brown Farm State Historic Site in North Elba, New York.

Mary Ann Day Brown was the second wife of abolitionist John Brown, leader of a raid on Harpers Ferry, Virginia, which attempted to start a campaign of liberating enslaved people in the South. Married at age 17, Mary raised 5 stepchildren and an additional 13 children born during her marriage. She supported her husband's activities by managing the family farm while he was away, which he often was. Mary and her husband helped enslaved Africans escape slavery via the Underground Railroad. The couple lived in Pennsylvania, Ohio, and in the abolitionist settlement of North Elba, New York. After the execution of her husband, she became a California pioneer.