This is a relatively complex and involved Python web scraper project which will require your 100% dedication for about 1 month to develop. The web scraper will be developed as an API of an internet based application developed for several platforms: HTML5, android, android Honeycomb, iOS5 and possibly BlackBerry. The other language directly interacting with the Python API will be OpenCV.
-Expertise in Python. You must code the project in Python (and/or a python variant like stackless python, pypy or psyco).
-Expertise in screen scraping EXTREMELY LARGE AMOUNTS OF DATA from e-commerce databases FREQUENTLY using multiple IP addresses on many computers and routing this data to a central data center. Data will be stored on Amazon EC2 Elastic Cloud.
-Ability to extract large amounts of data frequently without getting IP's blocked.
-Expertise in C++, MySQL, HTML
-You must be able to provide daily email updates and allow me each day to see what code you have developed so far if requested.
-Familiarity with grid computing or distributed computing concepts. Ability to advise me on how to go about extracting data using many geographically dispersed computers.
-Availability to chat/email within 24 hours on business days.
Qualified contractors will have to sign a Non-Disclosure Agreement to receive a complete brief.
This project does not entail creating an e-commerce website, but rather, it is a cloud server-side application with a GUI that has a client interface, but has to be primarily an automated process. I will provide a more detailed description of each process that needs automated.
In addition to designing this program, I will require ongoing support and maintenance of the code, and am willing to pay the fees to provide this service. I anticipate that this project will need to utilize distributed or grid computing to run on multiple virtual amazon EC2 instances, thus horizontally scalable and self-replicating, and auto-healing back-end architecture.
Would like to get a talented company working on this project as soon as possible. This is a LARGE project and I will require the winner to be able to answer emails/Skype PROMPTLY (within no more than 24 hours) 7 days a week during development and afterwards for emergency support. If you cannot be available like this, please do not bid.