Thursday, January 17, 2008

Robots Exclusion Protocol 101

The Robots Exclusion Protocol (REP) is a conglomerate of standards that regulate Web robot behavior and search engine indexing. Despite the "Exclusion" in its name, the REP covers mechanisms for inclusion too.
Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)