Hand-held units pose their own problems, in that you only have one hand 'free' for the controls. Especially problematic in a vehicle with a manual transmission.
Note, however, that the same is true in using a hand-held microphone for a two-way radio. This, BTW, _is_ one of the reasons law-enforcement vehicles are almost invariably equipped with automatic transmissions. :)
There is also an interesting 'human factors' aspect that comes into play -- the 'minimal-seeming' act of having to "push the button" to talk on a two-way radio has a _major_ effect on the thought-process, and serves (indirectly) to greatly reduce the 'distraction' effect of a 'juicy' conversation. "Push the button" requires an overt and conscious decision (and effort), which triggers an awareness of the diversion from the 'primary mission' of operating the vehicle. The logic seems a little silly on the face of it -- the fact that you have to work harder at it makes it _less_ distracting -- but it _does_ work that way.