On the other hand, you could just as easily be worried if you found that this white guy had dated lots of other black girls. That would prove he was fetishizing you, that he was only interested in you because you are black. I have read a lot of medium articles by black women complaining about this.

And even if you decide you only want to date white guys who have dated other black girls: Every single one of those guys once dated a black girl when they had never dated a black girl before. That's just a truth of logic. Maybe you are only the first of many more. I don't know why you should find that comforting, but that is the only alternative if you object to you being his first. It may be that you are the first because he sees you as being very special, which inspired him to do something he has never done before.

Of course I have no direct experience of being victimized by racism. But what I have learned from reading black writers is that almost any action (and its opposite) will probably be interpreted as racist by someone. When you have to live with real racism, a painful side effect is that you will often see it when it isn't there. I quote several black authors discussing this issue in this article

https://medium.com/@teedrockwell/why-reverse-racism-is-sometimes-justifiable-1bc9e4747b54