I need a large list of domains/websites. I can accept a large list if you already have it, or some spider-program that you can write and does the search/collect for me, (if it is threaded and can use all my bandwith)
Or any other idea that you have that might help me.
The larger the list the better it is.
Được trao cho:
7 freelancer đang chào giá trung bình $36 cho công việc này
Hello, i already created a multithreaded desktop spider program that can get urls by searching google, and then crawls them with up to 2 levels of recursion. i can easily modify it to return only urls.
I can easily make you a program that could pull all websites from google and or yahoo relating to a certain word or phrase. Not sure if thats what you would need. Pm me if interested.