org.archive.crawler.frontier

Class BucketQueueAssignmentPolicy

java.lang.Object

org.archive.crawler.frontier.QueueAssignmentPolicy

org.archive.crawler.frontier.BucketQueueAssignmentPolicy

All Implemented Interfaces:

Serializable, HasKeyedProperties

public class BucketQueueAssignmentPolicy
extends QueueAssignmentPolicy

Uses the target IPs as basis for queue-assignment, distributing them over a fixed number of sub-queues.

Author:

Christian Kohlschuetter

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected ServerCache	serverCache

Fields inherited from class org.archive.crawler.frontier.QueueAssignmentPolicy

kp

Constructor Summary

Constructors

Constructor and Description
BucketQueueAssignmentPolicy()

Method Summary

Methods

Modifier and Type	Method and Description
String	getClassKey(CrawlURI curi) Get the String key (name) of the queue to which the CrawlURI should be assigned.
ServerCache	getServerCache()
int	maximumNumberOfKeys() Returns the maximum number of different keys this policy can create.
void	setServerCache(ServerCache serverCache)

Methods inherited from class org.archive.crawler.frontier.QueueAssignmentPolicy

getForceQueueAssignment, getKeyedProperties, setForceQueueAssignment

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

serverCache

protected ServerCache serverCache

Constructor Detail

BucketQueueAssignmentPolicy

public BucketQueueAssignmentPolicy()

Method Detail

getServerCache

public ServerCache getServerCache()

setServerCache

public void setServerCache(ServerCache serverCache)

getClassKey

public String getClassKey(CrawlURI curi)

Description copied from class: QueueAssignmentPolicy

Get the String key (name) of the queue to which the CrawlURI should be assigned. Note that changes to the CrawlURI, or its associated components (such as CrawlServer), may change its queue assignment.

Specified by:

getClassKey in class QueueAssignmentPolicy

Parameters:

curi - CandidateURI to calculate class key for.

Returns:

the String key of the queue to assign the CrawlURI

maximumNumberOfKeys

public int maximumNumberOfKeys()

Description copied from class: QueueAssignmentPolicy

Returns the maximum number of different keys this policy can create. If there is no maximum, -1 is returned (default).

Overrides:

maximumNumberOfKeys in class QueueAssignmentPolicy

Returns:

Maximum number of different keys, or -1 if unbounded.