There is one context in which the term POC is relevant. All POC were to some degree victims of the White Supremacy doctrine that justified colonialism. The idea that White people are superior to all the other races justified the view that All POC were unfit to govern themselves. You can see this doctrine stated explicitly in Rudyard Kipling’s poem “the white man’s burden”, and even in Darwin’s book on human evolution. It also was the justification for the enslavement of Africans, and the genocide of Indigenous Americans and Australians. You don’t find this doctrine of racial superiority in earlier imperialist cultures. That’s largely because these cultures didn’t have the idea that all people have equal rights that arrived with the European enlightenment. Once you have the idea of universal human rights, the only way you can justify imperialism is to claim that the people you are conquering are in some sense subhuman. That’s why all POC are to some degree marginalized in cultures that are seen as white.

Nevertheless, I think you are right that the modern use of the term often causes more confusion than clarity. It prompts numerous POCs to be clueless about their own privilege, and enables the new bigotry that has become acceptable in Woke circles. The widely accepted bigotry against people perceived as white is often ignored by many SJWs, as is the widespread animosity between Asians and African-Americans, The conquest of the Tibetans and Uighurs by the Chinese, the conquest of Korea and Okinawa by the Japanese, and the still widely prevalent caste system in India.