How do you handle PMTU discovery - or do you prevent segments with the DF bit set leaving your network, or do you mangle the headers and remove the DF flag, or do you just accept that some sites on that Internet may not be reachable from nodes on your network, or do you rely on Windows rather inefficent "PMTU Blackhole discovery" feature working ?
If you don't allow *any* inbound ICMP and don't implement effective work arounds then you (or your network users) would have some problems with all of my locally hosted servers - but then you don't have access anyway, so you maybe you can live with the fact that your implementation is broken ;-)
PS - You are not alone in your screwed up thinking - the company I used to work for adopted a similar policy, and it effectively caused all my VPN connections from work to home to fail. Easy to 'fix' since I controlled the 'home' end of the VPN, but not necessarily quite so easy to fix for an arbitary site on the Internet.