Obviously being beaten up, harassed and intimidated is not fun. But taking the violence out of the equation, can you really compare a bunch of black people calling a white guy "peckerwood" or "whitey" (or, for that matter, "honky") with a bunch of white guys shouting "nigger"? There's a whole different context at play here.
Part of the reason for hate crime laws is that there is inequality in the first place. While they may seem "unfair" when applied to specific cases, such as this one, the broader idea is to prevent the majority from abusing their advantage over a minority. If you're white, and someone calling you a "honky" makes you upset, you have very very thin skin. That's because it's someone coming from a historical position of disenfranchisement taking a shot at someone who has traditionally come from a position of power.
Obviously members of a minority group can commit a hate crime against white people, but what makes it tricky is that people who are at an economic or social disadvantage are more likely to commit crimes, period, and minorities tend to be at an economic or social disadvantage. So hate crimes in the broader sense are harder to determine, and also less neccessary. It's possible a, say, Latino mugger might only want to rob white people, but the basic motive there is financial. "Hate" assumes that racism or prejudice was the primary reason for the attack, and tends to imply someone at an economic or class level equal to or higher than the victim, or possessed of some form of social power over the victim.
At any rate, since part of the point of hate crime laws is to act as a deterrant, I'd say it's less important to bring them in in cases where the assault came from a member of a minority group. It's kind of like drawing a distinction between a terrorist and a guy who shoots up a McDonalds. Both are serious crimes, but there's a political element to the former that needs to be taken into account.