Hey,
I am a web scraping expert and mostly use Scrapy to get the job done.
I use Scrapy for all my projects and scrapy-selenium when I need to emulate browser.
Please open a chat with me, I can show you the source code of my past projects
Waiting for your respone,
Biswajit