Package | Description |
---|---|
org.archive.crawler.frontier |
Modifier and Type | Class and Description |
---|---|
class |
AssignmentLevelSurtQueueAssignmentPolicy
Create a queueKey based on the SURT authority, reduced to the
public-suffix-plus-one domain (topmost assignable domain).
|
class |
HostnameQueueAssignmentPolicy
QueueAssignmentPolicy based on the hostname:port evident in the given
CrawlURI.
|
class |
IPQueueAssignmentPolicy
Uses target IP as basis for queue-assignment, unless it is unavailable,
in which case it behaves as HostnameQueueAssignmentPolicy.
|
class |
SurtAuthorityQueueAssignmentPolicy
SurtAuthorityQueueAssignmentPolicy based on the surt form of hostname.
|
Copyright © 2003-2014 Internet Archive. All Rights Reserved.