So they call you a racist. It’s what they do. Don’t let it stop you telling the truth or giving your opinions. The whole point of calling you a racist is to shut you up. So whatever you do don’t shut up. If you do shut up it only reinforces the bad habits they have got themselves into.
It is just a word. Don’t be afraid of just a word! It doesn’t mean that you are a bad person just because they want to stop your opinions being heard. The bad people are those who want to silence all expression of views they don’t approve of.