Ethic, I'm sorry you had to go through that, I don't agree with how it was handled, except the pass at the end. And I was bullied, too, so it's not like I'm basing my opinions only on what I see other people going through.
I think cases where individuals are getting bullied should be handled in a much better fashion than I'm guessing they're handled now. But I don't see any reason to believe that PE is special as a breeding ground for bullying. All we have here is your experience, which conflicts with mine. You can say it's non-debatable, but, we're kinda debating it right now. I just don't agree.
You could just make PE an elective, but many kids who could benefit from it would skip it entirely. I think that's a bad solution.
Another idea for kids that really struggle in PE could be an elective class that satisfies the requirement and involves individual exercise, especially if the exercise is what provides the benefit found by studies (rather than competition or working with teammates or whatever). Add that to better and quicker action by teachers and administrators when bullying does happen, and I think you have something that helps a lot more than it hurts.
I'm not ignoring the difficulty some kids have in PE, I just think there's more to it than that.