Đã hoàn thành

131353 Scraper Script Update

Vlad the script still isn't working but I have an idea I think will work. The main url to spider is really just an index of url's to scrape. Can you modify the script to just hit this url [url removed, login to view] begining with the CustomerID=1 and auto incrementing. When there isn't a listing for the id the page displays "The requested Member Profile was not found.".

Can you make it skip the empty pages and scrape ones that have relevant data. It should count up to 200k. Have it retrieve a page every 30 seconds. Lemme know if this is feasible.

I forgot to mention the reason I think this will work... the main url is what loads very slow and is 1.5mb. The actual pages we want load very fast. I don't think the site operator is sophisticated enough to block proxies, identify us as a bad bot etc. and I doubt this script even takes enough resources to be noticed.

Kĩ năng: Bất kì công việc gì, MySQL, PHP

Xem nhiều hơn: just the begining, think bot, update listing, auto update, scraper mysql php, update url, retrieve url, auto data scraper, auto etc script, auto index script php, scrape data mysql, net scrape data, script aspx, feasible, working auto script, script load, script auto work, script auto mysql, auto script page, auto scrape data, auto fast page, auto index php script, page fast bot, fast bot page, auto scrape

Về Bên Thuê:
( 8 nhận xét ) RENTON, United States

ID dự án: #1877522

Được trao cho:


Ok. I will rewrite it to download 200 000 profiles.

$35 USD trong 1 ngày
(305 Đánh Giá)