Self driving cars will only work if all cars are self driving. Put a human in the mix and they have slower reactions, unpredictable behavior and a "I'm better than a robot" mentality that they will be the ones that will be causing the issues. If you take people out of the equation, traffic will flow.
In a perfect world.
People like to drive and be in control. With anything. Sometimes automation is good, but for the most part, people like to feel that they are in control of what ever it is they are doing.
A drawback of technology is also people. Software is written in a language that people can understand, that's why there are so many issues with it, too many words and symbols. If a computer could write it's own programming, the syntax would be a fraction of the size it is now. People get in the way of technology, but if technology had it's way, would we be living in The Matrix?
I love driving too, but sometimes a car that would drive itself would be handy.