Louis Tomlinson Says Harry Styles Romance Rumors “Irritate Me a Little”