Charles leaves Clarence House as aides insist 'zero chance' of Harry's return