For years, I meant to read 'Arabian Sands', Wilfred Thesiger's account of two punishing camel journeys during the late 1940s across Southern Arabia's Empty Quarter. Now that I have, I can sheepishly join the chorus of those who revere the book as one of the half dozen greatest works of modern English travel writing.
Do what you love. Do what is important to you and your family. I just want to encourage people to just go for it and do what makes sense to you.