It is an awfully grey area.
There are way too many things to take into consideration. Without knowing how they make their decision on whether or not you get a permit we can't even guess as to what it entails, and for all we know the individuals, or companies, or medical professionals, or whoever they are who decide this may or may not be biased, and variations most likely occur letting some people have a permit who may not necessarily need it all the time, and others who do deserve one miss out.
For some people walking with an aid may not be a problem, but for others BAD pain can cause you to be immobile.
Of course there would be certain people who should get one without a doubt, if you have lost a limb for example.
Regardless, if someone parks in a disabled spot with a legitimate permit, then it really isn't anyone else's business to decide as to if they look disabled or not, or should be parking there or not; for all we know they could have a kid, or a partner with a bad disability in the car (who gets out, I feel like they shouldn't be parking there if the disabled person isn't getting out of the car, that's just being lazy).
Now if they don't have a permit, like many people we see out there who park in these spaces, that's a different story.