Former Post Office chairman rejects Kemi Badenoch's accusations over his dismissal