I am looking for a .net developer that can build me a search engine scraper
The task will be to build a multi threaded application that will query google and scrape urls -
Notes about the project
I will provide some search strings that will be used to enter into [login to view URL] - The software will visit google and perform a search for this - it will then scrape urls that are shown and save them to a file
Its important that the software is multi threaded
its important to provide proxy support for both private and free proxies and to auto rotate proxies if a proxy does not work or it is being blocked
I will provide a number of search strings that should be selectable in a drop down menu in the software
the software will have a text area where I am able to enter additional strings to the ones already provided
For example - the string I might provide will be somethng like this
"powered by wordpress" and in the text box I might enter the keyword "Insurance"
The software will now search google for "powered by wordpress" + "insurance"
it will then get all the urls and save to file
It is important to provide control to how many pages should be scraped in one run
Please bid fairly and in your bid please add the following notes so I know this is not an automated bid - My bid for URL harvester
Please note I am reposting this project because the last freelancer sent me an application that was not multi threaded and it worked by opening a chrome browser window - This needs to be coded using httwebrequests showing no browser and its an urgent project that needs to be completed asap due to the last freelancer taking over 5 days to tell me that they didnt understand the project - no time wasters please
14 freelancer đang chào giá trung bình $210 cho công việc này
Hello, i have checked your project and would like to have your project. I am confident to fulfill your requirements in timely manner, along with quality work. Please check your pmb.
My bid for URL harvester This should very easy considering i can just use the api, and ill provide a nice UI with it as well, simple simple simple, if you have any questions just ask!