Much of the 'traffic' is harmless, and much of it is not. Hackers constantly are scanning, looking for open machines.
Set up your firewall to block everything. Then start allowing access only to the software you are using that needs access in/out. After that, bar *everything* else. That way you won't see all the crap that constantly tries to access your machine unless you manually go through the "alert" log. This method keeps you from getting aggravated and paranoid. :)
You are either new to the Internet, or you have belatedly discovered what has been happening to the rest of us for years. Hundreds per day? That's pretty minor. Last time I bothered to log the noise, I was seeing around
20000 per day.
What - did someone try to stroke your port 22 with a thousand (or so) attempted logins using dictionary passwords? Your headers say Debian - so figure out where a command line is, and run the command '/bin/netstat -tuan' and see if you need all of those ports open. If you are a home user, there should be NO ports open - or at worst, port 113/tcp if needed by your ISP. If you have decided that you need SSH open, restrict it using your firewall to those addresses (or address blocks) where you have some reasonable expectation that you will actually want to make a connection from.
All the time.
"It's only called paranoid when they AREN'T after you."
"When they _are_ out to get you, always check your paperwork."
The only reason you are receiving attention is that you have something a spammer or skript kiddiez wants - bandwidth. See that you are not offering services to the world. See that your system is kept current. Then stop worrying about non-events.