Most of what you're wanting to call racism isn't racism at all. It is more correctly called tribalism where "tribe" can be a very ephemeral thing.
And the big thing about tribalism is people want to feel empowered, and they'll often seek that empowerment by means of identifying with a tribe.
After their tribe has been defined, that search for empowerment takes the next step. They seek to denigrate and degrade those of any opposing tribe they can identify.
Racism is simply the simplest expression of tribalism because skin color and other physical characteristics are not easily changed. But it isn't the only form that tribalism can take.
Modern day hate groups in the United States don't operate along racial lines anymore. They now operate along ideological ones. And there is plenty of historical precedent for that. Just ask any Mormon, Catholic, or Jew about that just that.
There is so much wrong with that brief soliloquy that I have to hold back my comments. But first, let me assure you that I am not a racist. You can take my word for that, because I just told you. My friends also would say I'm not a racist, and I can attest that they aren't racists, either.
It's also probably true that most Jews in Germany wouldn't say that the Nazis were anti-semitic racists. In fact, most of them tried to fit into society just like the Nazis, which proves that they were all part of the same tribe. It didn't work out, but only for purely technical reasons, not because the Nazis were anti-semitic racists.
In modern America, what you call "hate groups" aren't hateful and certainly aren't tribalists, and they would tell you that if you ask them. The religious groups that we might think hate blacks would explain to you, calmly I might add, that they are only following "God's order" and that all people, not just blacks, but also whites, should know and respect their proper place in society. You don't have to be a racist tribalist to see that.
It's really just a matter of empowerment, and there's just not enough power to go around. It's just common sense that you would reserve as much power for yourself and your friends, your family, your church, your social caste as possible.
It just makes sense, and no offense intended.