This is a right that every author has and ought to have. The original version of the Alladin story (in the Arabian Nights) takes place in China. It is however, a China in which most people are Muslims, the men wear turbans, and is ruled by "the Sultan of China-Land." And it's a great story because the author's imaginary China is a compelling mixture of the Author's ignorance and imagination. That what story tellers do. They make stuff up.
That's your conclusion. You just don't want to admit it to yourself because you want to continue to badmouth white writers without putting the same standards on yourself. There is nothing wrong with so-called cultural appropriation. It's a meme that deserves to die, and never should have existed in the first place.