Find Jobs
Hire Freelancers

Scrapy Scraping and Web Crawling parallel execution

$250-750 USD

Đã hoàn thành
Đã đăng vào hơn 1 năm trước

$250-750 USD

Thanh toán khi bàn giao
Hey, I need a web crawler for a certain website that discovers links by itself and manages to intelligent scrape some content from it. Requirements: - Can run multiple instances at the same time (so 2/3 crawler for the same website at the same time) - Prevents the crawling of a link twice within a certain period (crawl a link only every 2 weeks) - Crawl (find new links) - Extract around 10 data points - Save to database - Execute JavaScript - Use Proxies (provided by me) - Has restriction of crawls per minute - Uses state-of-the-art methods for hiding that it is not a person (browser fingerprint etc.) Please state your experience and also how you think you can solve the need to manage the multiple instances and prevent crawling twice. Thnaks.
Mã dự án: 34951087

Về dự án

58 đề xuất
Dự án từ xa
Hoạt động 1 năm trước

Bạn muốn kiếm tiền?

Lợi ích khi chào giá trên Freelancer

Thiết lập ngân sách và thời gian
Nhận thanh toán cho công việc
Phác thảo đề xuất của bạn
Miễn phí đăng ký và cháo giá cho công việc
Đã trao cho:
Avatar người dùng
$0 USD trong 15 ngày
5,0 (24 nhận xét)
5,3
5,3
58 freelancer chào giá trung bình $513 USD cho công việc này
Avatar người dùng
Hi sir, This is Lin and I am scraping expert, I scraped 600+ websites, please check my reviews then you will know. I will create scraping tool with PHP & CURL that extracts all requested data, And provides data in the required format (JSON, CSV, EXCEL, SQL...). Can we discuss more details about this project? Then I can provide sample data/script, we can start project after you checked sample. Thanks, Lin
$600 USD trong 7 ngày
5,0 (428 nhận xét)
8,2
8,2
Avatar người dùng
Hi. We already have a tool that does all that you mentioned, all we need todo is implement specific website you have and run it. What is the website ? Thanks,.
$750 USD trong 7 ngày
5,0 (142 nhận xét)
7,9
7,9
Avatar người dùng
Hello, I have 12+ years of experience in Python and have worked on a lot of Django, Flask, AI/ML, ... projects. I can share some demos if you are interested. Also, I work as a fulltime freelancer and have enough time to assign to your project. Thanks, Helmot
$500 USD trong 7 ngày
4,8 (151 nhận xét)
7,8
7,8
Avatar người dùng
Hi Good afternoon , How are you? I just saw your job posting . I see you have been looking for someone experience with these technologies Scrapy, Web Scraping, Data Scraping, Web Crawling and Python. I believe this is some thing I can help you with because I have 9 years of experience working with these tech stacks and 8 years of experience working on freelancer.com which you can confirm from my profile: https://www.freelancer.com/u/AwaisChaudhry I have few questions: 1. These are all the requirements of your job or do you have more? If yes, Please provide detailed requirements. 2. Do you currently have anything done like Wireframes, UI/UX? Or this job has to be done from scratch? 3. What is the timeline to get this job done? Why Choose Me? 1. I have done more than 250 major projects only on freelancer.com. 2. I have not received a single bad feedback since last 5-6 years. 3. You will find 5 star feedback on last 100+ major projects which shows my clients are happy with my work. Portfolio: https://www.freelancer.com/u/AwaisChaudhry Timings: 9am - 9pm Eastern Time (I work as a full time freelancer) Lets start the chat and discuss your job in detail and we will continue from there. Regards, Awais
$750 USD trong 22 ngày
5,0 (9 nhận xét)
6,7
6,7
Avatar người dùng
Hello, there. I am very interested in your proposal. I can instantly help you with your starting project with a successful completion. As a professional software developer, I ensure for a perfect ongoing project. You will never be disappointed in me for sure. Looking forward to meeting you on chat. Thank you. Darrel.
$500 USD trong 7 ngày
5,0 (18 nhận xét)
6,4
6,4
Avatar người dùng
Hi there,I'm biddin on your project "Scrapy Scraping and Web Crawling parallel execution" I have read your project description and i'm an expert in Python and machine learning therefore i can do this project for you perfectly.I still have a few questions. please leave a message on my chat so we can discuss the budget and deadline of the project. Thanks. .. .
$750 USD trong 3 ngày
4,9 (25 nhận xét)
6,5
6,5
Avatar người dùng
Hello, this is Koki from Japan who has been working with Python and Web Scraping development for over 8 years now. I have checked your project description carefully and I think that I can help you to complete this project fully 100% sure to satisfy your requirement. I'd like to have a brief chat or call to discuss further details about your project soon. Looking forward to working with you together on this project. Thanks. Koki.
$1.000 USD trong 5 ngày
4,7 (15 nhận xét)
6,9
6,9
Avatar người dùng
Hello, this is Luka from Serbia who has been working with pyhon development for over 7 years now. I have checked your project description carefully and I think that I can help you to complete this project Looking forward to working with you together on this project. Thanks. Luka
$749,99 USD trong 5 ngày
5,0 (6 nhận xét)
5,5
5,5
Avatar người dùng
Hi, I am senior web scraping deveoper. Python webScraping is my top skill. I have already developed many( more than 100) web scraping tools using python. So I will provide you awesome result . Please contact me for our project. Really want to work for you.
$500 USD trong 7 ngày
5,0 (16 nhận xét)
5,6
5,6
Avatar người dùng
Hello @findoo. I'm interested in your project because I'm experienced web scraper I have been working on web scraping using python for 7 years Especially I'm familiar with scrapy, selenium, BeutifulSoup, request python package. My strong skill is fast scraping. I hope we discuss your project briefly. I can start working right now. Best & Regards Serhii
$577 USD trong 7 ngày
5,0 (6 nhận xét)
5,7
5,7
Avatar người dùng
✓ Hi, How are you ? I have more than 5 years experience in ✔️Python, web scraping, selenium, expert CSV, pandas..✔️. I can start the work right now. Looking forward to hear more about the project via chat. Best regards!
$500 USD trong 7 ngày
5,0 (9 nhận xét)
5,2
5,2
Avatar người dùng
Hi I have 4 years of expirence in creating scraping and webcrwaling programs and can do this task for you I understand your requirements completely and I know methods to run multiple instance at the same time without crawling same link twice and have good experience in avoiding bot detection program I will make sure the quality work under deciesed timeline and budget Thanks.
$500 USD trong 20 ngày
5,0 (4 nhận xét)
5,4
5,4
Avatar người dùng
Dear client I have read your description carefully and I am confident about your work. As a web scraping expert, I have over 7 years of experience in web scraping. I have collected information from social sites, e-commerce sites and booking sites and analyze information. Also I have scraped the betting site and predicted the result. With my extensive experience, I can get solid knowledge in web scraping, automation and bots. I used python(selenium, beautifulsoup, requests), and node.js(axios, cheerio, puppeteer) for scraping. I am sure I can fulfill your requirements. I am confident that you will find my hands-on experience. Thank you for your time and consideration. Please let me know if there is anything further you need from me. Thanks.
$500 USD trong 7 ngày
4,8 (13 nhận xét)
5,2
5,2
Avatar người dùng
Hi there, ★★★ Scrapping / Python / Selenium Expert ★★★ 6+ Years of Experience ★★★ I've read requirements and ready to write script to scrape data from website. Some major works we do: ✔️ Product Websites Scraping: eCommerce (Shopify, eBay, Amazon, AliExpress. etc) ✔️ Advertising or Ads posting sites (gumtree, Rightmove, Asian metal. etc) ✔️ Login required websites ✔️ Process Automated Tasks (Automatic download file from any website pdf, CSV, etc) ✔️ Software and Scripts Development ✔️ With Browser load and Without Browser load ✔️ Web research and Data Entry work as well, ✔️ Scrapped Datastore into multiple form SQL server, CSV, Excel any many more like JSON. We use different tools and techniques to scrape the websites depending on the kind of task but mainly Python, C# Desktop Application and Selenium and Html Agility. Note: Bid is tentative. To provide you the best quote, i'll be glad to discuss requirements so let's have a chat. Best Regards, TechPlus Team
$750 USD trong 7 ngày
4,3 (21 nhận xét)
5,8
5,8
Avatar người dùng
Hello client, First, receive warm greetings, and hoping you are good, I welcome you to the home of Amazing services and quality. I am an expert in named skills,over 9 years of experience, After keenly reading your description and being in possession of all clearly stated required skills as this is my area of professional specialization having the above qualifications, I hereby humbly request you to consider my bid for quality and professional work. It is my duty to ensure 100% satisfaction to you as my client. No timeline and quality excuses. Unlimited revisions until all client requirements are met as required. affordable, professional & quality services are guaranteed always. Message me we discuss more the project. thank you.
$750 USD trong 10 ngày
4,8 (3 nhận xét)
5,4
5,4
Avatar người dùng
Hi I'm Python/Web Scraping/Automation Expert. I’ve gone through your requirements, it perfectly matches my skills. I am interested to work with you on this project. I've been working on scraping e-commerce websites, social media posts from(Facebook, twitter etc..), Google searches.. I can do it using Python(selenium/requests/bs4 libraries depending on the website), and the data can be extracted to csv as you provided. Ready to assist you and start working right away. Best Regards
$500 USD trong 7 ngày
5,0 (13 nhận xét)
4,9
4,9
Avatar người dùng
Dear Sir I just read your description more detail and it sounds like you want scraper. I am very motivated in this project. I have much experiences of scraping software development and have been working for 7 years. I already have developed many odds scraper for sports betting bookies. I am particular proud of my skills. This job is an ideal match for my skills and experiences. I will pull my weight to this project. I would like to discuss about your project more specifics after contacting. Best Regard.
$500 USD trong 7 ngày
5,0 (4 nhận xét)
4,4
4,4
Avatar người dùng
Hi, I am Web scrapping expert. I will use beautiful Soup and selenium for scrapping website. I have scrapped 400+website. Please, check my skills. I hope to discuss about it with you. Thanks.
$250 USD trong 2 ngày
4,8 (11 nhận xét)
4,1
4,1
Avatar người dùng
Hi, How are you? Upon reading brief details I believe it's a doable job I have great experience doing projects with Web Crawling, Scrapy, Python, Web Scraping and Data Scraping. I am confident about getting this done. While I believe we need to have a detailed discussion about it. I work as per Eastern time daily so that would be good for us to catch up on this job, Let's discuss this further and get started for you as soon as possible. Thanks! Muhammad
$250 USD trong 1 ngày
5,0 (4 nhận xét)
4,0
4,0
Avatar người dùng
Hello: It is a pleasure to communicate with you. My name is Anthony Muñoz, I am the lead engineer for DSPro IT agency and I would like to offer you my professional services. I have more than 10 years of working as a Backend and Software developer, I have successfully completed numerous jobs similar to yours on this and other Freelance platforms therefore, and after carefully reading the requirements of your project, I consider this job to be suitable to my area of knowledge and skills. I would love to work together to make this project a reality. I greatly appreciate the time provided and I remain pending for any questions or comments. Feel free to contact me. Greetings
$508 USD trong 7 ngày
5,0 (1 nhận xét)
3,8
3,8

Về khách hàng

Cờ của GERMANY
Essen, Germany
5,0
24
Phương thức thanh toán đã xác thực
Thành viên từ thg 1 12, 2010

Xác thực khách hàng

Cảm ơn bạn! Chúng tôi đã gửi email chứa đường link để bạn lấy tín dụng miễn phí.
Đã xảy ra lỗi trong khi gửi email của bạn. Hãy thử lại.
Người Dùng Đã Đăng Ký Tổng Số Việc Đã Đăng
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Đang tải xem trước
Đã cấp quyền truy cập vị trí.
Phiên đăng nhập của bạn đã hết hạn và bạn đã bị đăng xuất. Hãy đăng nhập lại.