rlinks is a command line tool for checking that links in websites are live, allowing also to dump chose links for later use. It has a number of configurable options, including level of parallelism, user agent setting and the ability to ignore urls by regex.