There are two scenarios here.
1) you have a modem with a router built in. The cable (usually coax, like you used to put on a TV) attaches to the modem for your internet signal. You can then connect to the router using WIFI. Some routers also have Ethernet ports, which you can use a LAN cable to connect devices such as desktops and DVRs.
2) You have a modem and a separate router. Again the coax cable should connect to the modem, however now there is an Ethernet cable which connects from the modem to the router. The Ethernet cable will go into the Ethernet port n the back of the modem and will connect to the router in the port on the back which is labeled something like "Up-link". Once the router is configured for your home network, you can connect wirelessly to the router OR use an Ethernet cable to wire devices in direct. Note, you don't want to connect a device into the modem or to the router's UP-LINK port. That is only for the modem.
Chances are, your blue cable is an Ethernet cable. You can tell because it LOOKS like a phone connector, only instead of 2 or 4 wires in the connector, it has 8. You can turn it over and look into the clear connector to count the wires.