Đang Thực Hiện

161398 Run my html scrape for me

I am scraping a large data set from the web. To make things go faster, I need to recruit a few people willing to run the script on their computer.

The task is INCREDIBLY easy. It will require you to have Ruby and PHP installed, but after that all you have to do it run the scripts for various inputs and then.... wait. The downloads are throttled at 1 per second, and i'm looking to downloading about 500000 entries. So it will take patience. But it's really easy. I need distribute this task because if I don't distribute this load it will take me much longer downloading from a single point.

1) run the script for the requested input ranges (I will provide these). If the script fails because the connection breaks, you will have to restart the script from the file where the connection last worked (easy).

2) You will also break up the downloaded html and jpg files into ranges of 10000, so the directories are traversable without windows system explorer choking on the high file count.

3) Finally you will zip the files up and send to me via ftp.

You have ~2 weeks to complete the task for 500000 entries.

Kỹ năng: Bất kì công việc gì, Python

Xem thêm: things people, html web scripts, go ruby, run, python task, python ftp, my entries, php load html, web script python, run faster, python html script, python send data, html load php, 10000 downloads, run windows, file explorer python, python load, python file explorer, html data load, python restart, 500000, python web script, html jpg php script, connection python, faster load

Về Bên Thuê:
( 5 nhận xét ) Cambridge, United States

Mã Dự Án: #1907587

Đã trao cho:

ynabilsl

I can do it.

$20 USD trong 14 ngày
(0 Đánh Giá)
0.0