Stirling was born in Baltimore, Maryland, on 6 May 1843, the son of Archibald Stirling and the former Elizabeth A. Walsh. He and his wife, Ellen, had seven children. Their son, Yates Stirling, Jr., also became a rear admiral in the Navy, making them the second family in the history of the U.S. Navy to have a father and son both become rear admirals. Another son, Archibald, was a captain in the Navy.