I know where you're coming from and I would probably agree. I have to have a medical to teach or work underwater, yet I can drive a taxi/bus etc. with no medical/test whatsoever. Which is more dangerous?
I am a great believer in keeping fit (any level of fitness is better than none) and would like to see fitter divers. The HSE medical in the UK has a fitness test which is why I would like to see divers having a medical, not every year but maybe every 5 years.