Mike, they've done studies and determined that talking on a cell phone is indeed more distracting than having a passenger in the car. A passenger is aware of what's going on around the car and (hopefully) knows when to STFU because you need to concentrate on traffic. A passenger will (usually) tell you if you do something stupid. Not so when you're talking to someone who isn't in the car.
One study found that a driver on a cell phone is four times more likely to miss an exit than a driver with or without a passenger. (link below)
That said, I think it's unrealistic to ban cell phones while driving.