There's a debate going on in the UK at the moment about whether men can or can't write romantic fiction. The debate, it is true, has been stirred up to publicise a new TV programme about romantic fiction called Reader, I Married Him, but it brings home the fact that the Regency period had a wealthy of literary figures writing romance.
The most famous male author of his day was most probably Sir Walter Scott. Scott was famous for writing poetry in the early years of the nineteenth century, so when he wrote his first work of prose fiction, Waverley, which appeared in 1814, he published it anonymously. Subsequent books came out as being by 'the author of Waverley'.
Waverley was a huge success, selling out in 2 days. It was well received critically as well, although John Wilson Croker, writing in the Quarterly Review, questioned the mixing of history and romance. 'Romance' was used by Croker in its wider sense, to mean a story dealing with the exciting adventures of chivalric heroes, but Scott's books also contain what we would today think of as romance, and so it could be said that Waverley marks the birth of historical romance.
In particular, Scott can be see as the father of all the Scottish historical romances, with romantic moors, lochs and men in kilts proving immensely popular up to the present day. Rob Roy came out in 1817, nearly 200 years ago, and is still a great read even today. Scott's language takes a bit of getting used to for the modern reader, but it's no more difficult than Austen, so if you haven't yet discovered books like Waverley, The Talisman, Ivanhoe and Rob Roy, you've got a treat waiting for you!