org.archive.modules.net

Class IgnoreRobotsPolicy

java.lang.Object

org.archive.modules.net.RobotsPolicy

org.archive.modules.net.IgnoreRobotsPolicy

public class IgnoreRobotsPolicy
extends RobotsPolicy

Policy to ignore robots.

Contributor:

gojomo

Field Summary

Fields

Modifier and Type	Field and Description
static RobotsPolicy	INSTANCE

Fields inherited from class org.archive.modules.net.RobotsPolicy

STANDARD_POLICIES

Constructor Summary

Constructors

Constructor and Description
IgnoreRobotsPolicy()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	allows(String userAgent, CrawlURI curi, Robotstxt robotstxt)
boolean	obeyMetaRobotsNofollow()

Methods inherited from class org.archive.modules.net.RobotsPolicy

getPathQuery

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

INSTANCE

public static RobotsPolicy INSTANCE

Constructor Detail

IgnoreRobotsPolicy

public IgnoreRobotsPolicy()

Method Detail

allows

public boolean allows(String userAgent,
             CrawlURI curi,
             Robotstxt robotstxt)

Specified by:

allows in class RobotsPolicy

obeyMetaRobotsNofollow

public boolean obeyMetaRobotsNofollow()

Specified by:

obeyMetaRobotsNofollow in class RobotsPolicy