Đã Hủy

Web scrape of 5-7 online retailers using Ruby Mechanize/Watir/Nokogiri (repost)

I run a data analytics consultancy in Sydney, Australia. I have a requirement to setup weekly scrapes of at least 5 large (50,000+ SKUs) online retail websites. The project will be split to scrape a single website to start with, extending to the others assuming satisfaction with the work completed.

Technical requirements:

- Scrapes must be written in Ruby using Mechanize/Nokogiri or Watir as appropriate

- Where possible, avoiding JS and scrape via direct HTTP requests using Mechanize - i.e requires strong skills in tracing/debugging HTTP requests/responses

- Alternatively, load page using Watir webdriver then pass off cookies to Mechanize

- Last resort would be brute force Watir webdriver approach (scroll, point, click, etc)

- Implement utilities for use of proxy / IP rotation where required (access to pool of anonymous IP addresses will be provided)

- Scheduling via cron job to run at least weekly

- Must use object oriented approach and design in a DRY manner so that code can be leveraged for future scrapes of additional sites and ensure future code manageability

- Must use Activerecord to persist records scraped

- Must implement logging (production logging to file + verbose debug option to STDOUT)

Personal requirements:

- Must be a great communicator both in terms of coding concepts and project management

- Good at clearly laying out the workplan, milestones and progress

- Must be available for 2x daily stand-up calls with Sydney AUS (timing TBC at mutually agreed time)

- Must be able to demonstrate prior experience with ruby web scrape projects similar to above

- Code must be well-structured and well-documented / commented

Opportunity for ongoing Ruby backend and RoR web app build if this project is a success.

Kĩ năng: Django, Python, Ruby, Ruby on Rails, Web Scraping

Xem nhiều hơn: Web development using Ruby on Rails, 8 5 x 7 5 print online brochure, web scrape 5 sites, quick app site web scrape using proxies, simple web application using ruby, web scraping using ruby rails, online sharing using web service, creating portal using ruby, web scrape asp login, online bookstore using php, flash web scrape, example online form using php, create online admission using php javascript html, php web scrape script, online questionnaire using asp

Về Bên Thuê:
( 0 nhận xét ) Australia

ID dự án: #15865779

21 freelancer đang chào giá trung bình $37/giờ cho công việc này

bestit4u

Hello,sir I am very interested in your project, because I have much experience in such projects. I have good skills with the program language including C/C++, C#, java, php, asp.net, python, VB.NET. So I have expert Thêm

$41 AUD / giờ
(47 Nhận xét)
6.1
yingli1987

Hello, I read your job posting carefully and I'm interesting in your job. My major language for scraping is python with some packages(scrapy, scrapyD, lxml, BS, selenium packages). I can make scraping script as Thêm

$41 AUD / giờ
(16 Nhận xét)
5.4
londonlance

We are a London (Shoreditch) based rails studio. Please get in touch to discuss further. Following are some of our recent projects; [url removed, login to view] a freelancing marketplace built using Rails 4.2 a Thêm

$36 AUD / giờ
(1 Nhận xét)
5.6
kchg

As the TOP 5th freelancer of this site, I can deliver the good result in time. Developed some kind of data scraping using Ruby on Rails and PHP. With the wonderful experience and enough knowledge, I can make the su Thêm

$35 AUD / giờ
(11 Nhận xét)
6.0
zixy

We have worked on the large scale scrapers which work in the same way as you described in the project requirements. Are you available to discuss the details now? Talk soon, Zoran

$40 AUD / giờ
(2 Nhận xét)
5.3
arvindkushwah9

Dear, I have 5+ years experienced in Web Development. I'm full stack Ruby On Rails developer, I have experience in developing the application like you want with, i can do this job very well. Specialties: Ruby on Thêm

$27 AUD / giờ
(35 Nhận xét)
5.6
andriphone0

Hello, I am scraping expert, I have did too many similar projects, Please check my feedback then you will know. Can you tell me more details? Then I will provide demo data for you.

$25 AUD / giờ
(48 Nhận xét)
5.3
TenStar718

I have good experience for web scrap app development. I can setup an app that scrapes at leat 5 large online retail websites. Let's discuss more in PM.

$39 AUD / giờ
(4 Nhận xét)
4.6
abhilashtv

Hi, ➲ 7+ years of full-time experience in Ruby and Python with 50,000+ Upwork hours billed and 50+ successful Python projects ➲ Upwork Top 10 Certification for Ruby and Pythn ➲ Guaranteed Results Policy: Pay only if Thêm

$27 AUD / giờ
(11 Nhận xét)
4.8
BizVenture

I've special skills in web scraping using Python and Ruby. Generally use direct HTTP requests, but if website asks javascript running on client, no problem by using selenium and webdriver. Have scraped on-demand, soc Thêm

$44 AUD / giờ
(2 Nhận xét)
4.1
DarinX

Web design&development project Relevant Skills and Experience Mobile: Java, Swift, Objective-C Design: graphic design, web design, UI/UX design (Photoshop, Sketch, Illustrator, Axure, Balsamiq) Web: (front end) Angu Thêm

$30 AUD / giờ
(2 Nhận xét)
3.1
leadconcept

Hi there, I have read everything and our team is very experienced in Ruby on Rails, so please share complete requirements or let me know from which websites, we have to scrap the data? Do you want to display the dat Thêm

$38 AUD / giờ
(1 Nhận xét)
3.0
mike199

Hi, I’m a Web Designer/Developer from the UK. My name is Mike. Your project description sounds interesting to me and I do have skills & experience that are required to complete this project. Let's have a quick chat whe Thêm

$41 AUD / giờ
(1 Nhận xét)
3.2
zdesign77

Hey, how is it going today? My name is Michael, I'm a web designer/developer from Boston. I've had a look at your project description and feel that my skills match your requirements perfectly. In fact, I did a very sim Thêm

$41 AUD / giờ
(0 Nhận xét)
0.0
AzaLess

Hi, We are focusing on Data analytics and Rails development as well. So it should be a perfect fit. Here are the case studies about the proects where we worked with Text Processing and scraped a lot of data: Thêm

$41 AUD / giờ
(0 Nhận xét)
0.0
paultitov

Thank you for your job posting. I am a talented python developer with rich experience of data scrapping and web programming(django) for 8+ years. About your task: After reading your job posting, I feel I am good Thêm

$41 AUD / giờ
(0 Nhận xét)
0.0
ftp2012

Hello!, do you need a script or my work? I have scraped Amazon 150000 URL (Title, Price, Bullets, Pictures). look please at my portfolio.

$41 AUD / giờ
(0 Nhận xét)
0.0
suhaa18

Dear Sir! Your project is very interesting to me. I want to discuss with you about your project and participate in your project. I have 5+ years of Python and Ruby development experience and have a good knowledge of th Thêm

$41 AUD / giờ
(0 Nhận xét)
0.0
becritical

Hey there, I will develop a web scraper for online retailers using Ruby in combination with Watir. I will implement advance proxy rotation in order to achieve maximum performance. I have a very complex scraping Thêm

$37 AUD / giờ
(0 Nhận xét)
0.0
$37 AUD / giờ
(0 Nhận xét)
0.0