I think you are confusing "is" and "ought" here. Most of the time people don't get the lives they deserve. Privilege is by definition, opportunities and benefits that people don't earn. But just because people don't get something doesn't mean they don't have a right to it.