I propose a preliminary scrape of (A) the social network URLs only, plus (B) download of logo images to supplement the look of your Website mtechland.com.
If you supply me with the list of your page URLs, I can start from there. Or if you are OK, I can scrape from your Website (just one request per item, pausing about 10 seconds for each one.) There are about 930 items?
Then, I can scrape each one's social network page URLs: facebook, twitter, googleplus, linkedin, bloggerprofile, instagram, pinterest, yelp, youtube, foursquare, feedburner, tumblr. This can be output as, eg, tab-separated text file or CSV file.
At this stage, I do not build any auto scraper for you to get the database output of updated social statistics.
I will only get those SN links if they are available at the Homepage, Contact page, or About pages. There are indeed a few that do NOT have any social network links.
But you can put the SN links to these uses:
1. Include these SN links/logos on your item pages to enrich the pages. But I am not sure if it is a good idea in terms of SEO/marketing, having all these links pointing out of your site. You will be the judge of this, being the marketing expert.
2. Use these links to find useful logos. Currently, you do not have logo pictures on your listing and item pages. Wouldn't it brighten up the pages a lot if you have logos for most item? This I can do for you - download the logos, name them according to your needs (I suggest by the item number). I suppose there should be no copyright problem because you would be using them (large thumbnail size) to point to their Websites?
Generally, the best logos will come from their FB or Twitter pages, being square and large thumbnails.
(3) In the next stage, either I or another freelancer can then build the automatic scraper program based on these SN links.
I propose a project fee of US$100. It involves scraping the SN links (where available) and downloading probably 2 sets of alternative logos for each item (where available). I will also manually look for the logos at the Websites if no social networks are available.
Please let me know what you think of this.
Hi Jason, I think this looks great Jason. Its perfect first step. My request is to use the actual entries on mtechland instead of fixed list. If needed ONLY if needed, I can actually supply a list of their websites, but the format kinda sucks. This is because the website is built into an array variable in two different array types. Pulling directly from site would make it most efficient, as we continue only the new sites added to mtechland would need to be updated with logos and website/SN url. I expect to grow this site to about 3,000 entries over next year. That covers you for number 1 and 2 in the proposal. Then we can talk about #3 and if it makes sense. I have an idea on what I want to do there and It involves a huge API pull that would allow for some cool metrics to be derived from social media activity data. For that it would need data to be pulled into the mysql DB via API. For 1 & 2 for now:
0. Scrape diretly from [url removed, login to view] using 10sec interval
1. Item name (as ID)
2. Website url
3. SN link for facebook, twitter, googleplus, linkedin, bloggerprofile, instagram, pinterest, yelp, youtube, foursquare, feedburner, tumblr
4. At least 2 logos (square preferred)