I need a backlinkchecker that looks after a list of backlink.
Available Source Data:
Backlink-URL, Anchor, Link destination, Follow or Nofollow, Checking Interval (Daily, Monthly, Weekly), Backlink ID
Here is want I want to check.
Checking is in 3 Steps.
1. Domain Check
- Checking the domain is alive via DNS and http status code
- Retrieving the IP of the hosting and the land of the IP.
- Retrieve the PR of the domain.
2. Checking the website where the backlink is.
- Look up the robots.txt. Backlink should not blocked via Disallow or Noindex in the robots,txt
- retrieve the webserver header response and checking for a X-Robots tag. Noindex and Nofollow should not be set. Also the http status code should be checked for the website. Must be Http Status Code 200.
- Now some checks on the website HTML. Lookup for Meta Tag Robots, must be Index and Follow. Check for a canonical url and if there is a canonical url it must be the same as the backlink url.
- Get the website Title and save it.
- Get the pagerank of the website
- count all links on the website
3. Checking the Link
- Check the Anchor is the same as in the source data.
- Checking the link destination is the same as in the source data
- checking the rel attribute. must be the same as provided in the sourcce data.
also we want a function that checks the position of the link and the sourrounded text.
I open to suggestions.
All data must be checked inthe intervall (dailiy, monthly, weekly) as provided in the source data.
Every error should be logged and on every check, it must be matched with the last saved data.
Following data should be written in a mysql table like this
Backlink ID, Check Status (True, False), Error (list of errors if check status is false), Domain PR, domain, website PR, all link on the website, IP-adress, last checked date,
I have no favorite programming language for this project, but it should be fast.
I am working on a decision diagramm, that will help to understand the process.
I hope, I have nothing forgotten.