Divided Labour fear party is fading to Gray