I think you're vastly underestimating the implications that 54% of the American public disagreed with letting the President keep his job.
Put it into electoral context. If 54% of voters voted against Trump in November (to remove him), the Democratic challenge would be earning the largest share of the popular vote since 1984 (even Obama in '08 only got 52.9%).
Now obviously that isn't how it will shake out, and popular vote doesn't matter (though at 54%, you probably win the EC anyways), but it's still hugely significant.