Improve a crawler based on websites content patterns
$250-750 USD
Đang triển khai
Đã đăng vào hơn 11 năm trước
$250-750 USD
Thanh toán khi bàn giao
Hello,
I need someone really smart in coding (PHP, Perl, Python, or whatever working on the web) improve a small web crawler (to fill a classifieds search engine).
The software already exists, but it needs some improvments to make it faster. I also have new websites to crawl.
*****TASK1*****
I want to that the script can trigger some crawls, something like [login to view URL] that contains 10 websites and lauch the crawl when a brower open [login to view URL], [login to view URL] that contains 5 websites and lauch the crawl when a browser open [login to view URL], etc...
*****TASK2*****
The software consists in detecting a the pattern of the content of given websites, and then, take the content and send it to a API interface that will store it in a database. The software doesn't access directly to the database. It just send requests to another software in charge of storing all in database.
I will prove specifications for every website to crawl.
In your response, provide a bid, AND a price per website in the text (a website can contain many pages to crawl, but always with the same pattern), because, I need to know my budget.
You can evaluate what is the work to do by looking inside the actual script (see attached files).