When do impressions stop being funny and start being mean?

Impressionists say exaggeration is part of the job, but some warn that jokes shouldn’t punch down.