When I was growing up the use of the word by White people was definitely a sign that they were not smart successful or educated. It was definitely a class thing. If my mother had heard any of my class mates using it, she would have told me not to play with them. If I had used it, she would have washed my mouth out with soap, literally. It was like "shit" or "fuck", a completely forbidden word that nice boys never said. (she would have forbidden me to to play with boys who used those words, too.) The banning of the word was motivated not so much by sympathy for black people as by prejudice against southern white people.
That doesn't mean my family was free of racism. The few black people we encountered (usually maids and janitors) were the only adults we were allowed to call by their first names. I never thought about that when I was growing up, but it is obviously racist to me now. These days I never call a black person, especially an older black person, by their first name unless I am given implicit permission to do so. (The "older part" is not so important now, as most people older than me are dead.) That got confusing here in California, as everyone calls everyone by their first name. But now I define "implicit permission" as calling me by my first name. This is not something I try to force other people to do. In fact, it's not something I was really aware I did until I wrote this post. It probably makes me sound like I have a stick up my ass sometimes, but I think people appreciate the respect it implies.