I am looking for a developer to create Java program that does this:
1. Connects to a database of websites urls.
2. Crawl each URL to get the META information of that Website
3. Update the Database with the meta information of the related url.
The program should allow configurable number of threads for the crawling.
The Database is located on hosted server.
The program has another module that takes a big CSV file (over 800k rows, around 25mb), fetch row by row and insert it to the database.