Your router needs to be configured to accept DHCP from the Internet provider and should also be setup to get DNS from your ISP.
DO NOT use: My ISP Requires a username and password
WE DO NOT use PPPoE. (If you select this, your router will not work correctly)
In many cases you will not be able to use the auto configuration option on the router for setup and will need to set it up manually.
Your router should also be setup to hand out DHCP on the local network.
When you setup your Wireless or WiFi, be sure to choose a unique name and don't leave it as a default such as 'Linksys' or 'Belkin', something like 'TonysApt101' or custom is important.
Also be sure to setup WPA or WPA2 security on your wireless network. This is setting up a password on your network so others aren't getting on your router and seeing everything you do or taking your bandwidth.
Here is a video explaining how to connect a router and hook it up to your Modem: