A Basic Web Crawler

a simple web crawler that grabs all the URLS on a website, for a coding exercise.

to run type

go run main.go -c 5 -url "http://www.acky.com" 

go run main.go -c 5 -debug false -url "http://www.acky.com"