Some places switched cold turkey. Some places switched gradually, such as going from a full name to just the two letters, and then giving new customers all numbers while existing ones kept their letters. Some places, like Philadelphia (the last to switch) were resistant.
As ANC progressed, some people objected to that and the increasing computerization of 1960s society. In hindsight, service was far more manual back then than today. But not all new computer billing systems worked very well, with many stories of receiving a $1,000,000 bill for monthly electricity in a small home.
(I personally got burned in those days when I changed jobs within an employer. My pay was hourly and I switched to a per diem pay, but the computer kept my old pay rate. So instead of getting paid for 40 hours, I got paid for only 5 hours because I worked 5 days and that was entered. Was very hard for them to straighten that out, for months I got "interim" manually prepared estimated checks.)
Unfortunately, too many people today are too far distant from that usage and wouldn't understand what you were giving them. In Phila it's been 25 years since the conversion and that's the newest.
I will note however that just yesterday (March 2006) I saw construction trucks still marked with DE 3 of the contractor's phone number. I noticed some newer units had 333 so he finally switched over, but much of his fleet remains. I'd love to get a picture, but it is a highway project and no place to park and walk over. (And they'd be suspicious of me if I were taking pictures and call the cops).
Here and there there's an old building with an old phone number painted on it and still in use. I saw some in Oyster Bay LI a few years ago. Some stores still have an old neon sign with their old style number.
Larger businesses can move or go centrex and lose their original number over time.