I had the same problem with a Cisto ATA-186 (or even X-Ten Lite) and the el-cheapo ZTE router provided by my DSL ISP. My conclusion was that the router was a piece of crap (and you've got a D-Link, so that conclusion likely applies), so I bought an SMC Barricade and that's been the end of my troubles. The SMC has never crashed, VoIP works great, and even my singing voice is better.
miguel