Spanish city loved by Brits under threat as 'thousands of homes left empty'