I need a bot that will crawl websites (e.g huffington post, gizmodo, ect) for new articles:
- Then it would break the article up into the most commonly used 10 words
- Then it would run that data against a mysql field contained with keywords associated with a username
- Then it will compile a list of usernames that have more then 3 keywords associated with that article
- Then it will create a mysql entry posting that the usernames have been mentioned in the artical URL, which will have to be collected.
Can be done in any programming language, but it has to run every 10 minutes or so (If in PHP i'll use a cron job)