There are several different senses of ownership, and other kinds of membership. My copies of the Harry Potter books are my books, because I went through the necessary legal procedures to have them legally recognized as mine. (I bought them.) But the books and articles I have written are mine in a very different sense, which is independent of any legal recognition of this fact. Even if someone published my book under their name, the book would still be my book, and not theirs. In the same sense, enslaved African-Americans are Americans because of the fact that they built this country. This is what Frederick Douglas said in response to people, black and white, who said that enslaved African-Americans should go back to Africa, because they were really Africans, full stop.