Harry and Meghan warned their 'empire is crumbling' after latest move