We are looking for a basic web crawler/spider that will continuously monitor MySpace profiles, cache a local copy of the profile code (txt/html only, no images etc) and store key demographic data from the profile in an integrated SQL database.
Script will run continuously updating all profile accounts on a regular basis. Script can be configured to begin with user: [url removed, login to view] and proceeding through [url removed, login to view] or whatever the "maximum" is determined to be. Admin will have the ability to update the "maximum profile value" as needed to allow for the continuously increasing number of profiles.
If you are familiar with MySpace coding (or if you aren't, view the source file for any user account) you will see the types of information that is contained in the page code. We'd like to store the standard category values (age, drinker/smoker, physical characteristics etc) as well as capture key words contained within the profile. The user ID number ([url removed, login to view]) will be used as the primary key and record identifier.
Once this portion of the project is complete, we will also need someone to create a front-end search page for the SQL database that can automatically generate lists of user IDs matching a particular set of search criteria. This will be listed as a SEPARATE project, but we will give first priority to the original coder of the rest of the project.
If you have any other questions, please feel free to PM me.