The comedian told ‘The Late Show With Stephen Colbert” it would have been “impossible” to make jokes about both Republicans and Democrats.