Competative Pricing Crawler = data mining

Competative Price website data mining We would like a coder to build a web crawler that will work its way through a website downloading product and competative pricing information that can then be downloaded, collated and displayed. There are similar products like Web Reaper and Web Scraper but I would like to know if someone can build me a specific price crawler for my industry. An example of a website we would like to gather this information from is [login to view URL] or www.dreamtimebaby.com....I have more that I will supply. The crawler will need to be able to search for specific so that a range of keywords can be entered into it offline and then the crawler initiated to seek the pricing information related to those keywords. The crawler should also be able to be programmed to seek website by either URL, Domain level or a full web search. The crawler should also be able to be programmed to automate login/password challenges where required. The program will then download the pricing information relating to the product(s) from at that URL, parsing the HTML as it goes, looking for links to other pages and objects that may relate to price. It will then extract this list of sub-links and download any pricing data from them. This process should continue recursively until there are no more pricing links to fulfil the filter criteria. The program should be fully configurable ??" with custom hierarchical filters that can be constructed from 12 different filter types to allow targeted downloads. Simple filters should be able to be built using a filter wizard, or more complex ones can be hand built using simple syntax. ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Deliverables must be in ready-to-run condition, as follows (depending on the nature of the deliverables): a) For web sites or other server-side deliverables intended to only ever exist in one place in the Buyer's environment--Deliverables must be installed by the Seller in ready-to-run condition in the Buyer's environment. b) For all others including desktop software or software the buyer intends to distribute: A software installation package that will install the software in ready-to-run condition on the platform(s) specified in this bid request. 3) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform windows xp, firefox, IE