Another factor is the decision, made in 1976, to sharply divide the FBI and the foreign intelligence agencies. The FBI would collect within the United States; the foreign intelligence agencies would collect overseas.
I like to write sad songs. They're much easier to write and you get a lot more emotion into them. But people don't want to hear them as much. And radio definitely doesn't; they want that positive, uptempo thing.