All this "speed doesn't kill" and "I should be able to drive as I please on quiet roads" is all a bit silly really. It's a bit like saying bullets don't kill - it's people standing in front of guns that cause death. If you have a licence for a gun and a private, safe place to shoot it, then you wouldn't wander around a country lane firing it just because you feel it's safe, would you?
Laws are there for a reason - sometimes they may seem over the top and sometimes they may seem inconsistent, but they still apply to everyone. Follow the law - no fines, no points, simple. If everyone followed the law there would most likely be less accidents too. If you do enjoy driving fast, fine, do it at a place that provides that service such as a race track.
