Package | Description |
---|---|
org.archive.crawler.frontier | |
org.archive.crawler.frontier.precedence |
Modifier and Type | Class and Description |
---|---|
class |
BdbWorkQueue
One independent queue of items with the same 'classKey' (eg host).
|
Modifier and Type | Field and Description |
---|---|
protected ObjectIdentityCache<WorkQueue> |
WorkQueueFrontier.allQueues
All known queues.
|
protected Set<WorkQueue> |
WorkQueueFrontier.inProcessQueues
all per-class queues from whom a URI is outstanding
|
Modifier and Type | Method and Description |
---|---|
protected WorkQueue |
BdbFrontier.getQueueFor(String classKey)
Return the work queue for the given classKey, or null
if no such queue exists.
|
protected abstract WorkQueue |
WorkQueueFrontier.getQueueFor(String classKey)
Return the work queue for the given classKey, or null
if no such queue exists.
|
Modifier and Type | Method and Description |
---|---|
protected void |
WorkQueueFrontier.deactivateQueue(WorkQueue wq)
Put the given queue on the inactiveQueues queue
|
protected void |
WorkQueueFrontier.handleQueue(WorkQueue wq,
boolean forceRetire,
long now,
long delay_ms)
Send an active queue to its next state, based on the supplied
parameters.
|
protected void |
AbstractFrontier.noteAboutToEmit(CrawlURI curi,
WorkQueue q)
Perform fixups on a CrawlURI about to be returned via next().
|
protected void |
WorkQueueFrontier.readyQueue(WorkQueue wq)
Put the given queue on the readyClassQueues queue
|
protected void |
WorkQueueFrontier.reenqueueQueue(WorkQueue wq)
Enqueue the given queue to either readyClassQueues or inactiveQueues,
as appropriate.
|
protected void |
WorkQueueFrontier.retireQueue(WorkQueue wq)
Put the given queue on the retiredQueues queue
|
Modifier and Type | Method and Description |
---|---|
protected int |
SuccessCountsQueuePrecedencePolicy.calculatePrecedence(WorkQueue wq) |
protected int |
BaseQueuePrecedencePolicy.calculatePrecedence(WorkQueue wq)
Calculate the precedence value for the given queue.
|
protected void |
BaseQueuePrecedencePolicy.installProvider(WorkQueue wq)
Install the appropriate provider helper object into the WorkQueue,
if necessary.
|
protected void |
HighestUriQueuePrecedencePolicy.installProvider(WorkQueue wq) |
abstract void |
QueuePrecedencePolicy.queueCreated(WorkQueue wq)
Set an appropriate initial precedence value on the given
newly-created WorkQueue.
|
void |
BaseQueuePrecedencePolicy.queueCreated(WorkQueue wq) |
abstract void |
QueuePrecedencePolicy.queueReevaluate(WorkQueue wq)
Update an appropriate initial precedence value on the given
already-existing WorkQueue.
|
void |
BaseQueuePrecedencePolicy.queueReevaluate(WorkQueue wq) |
Copyright © 2003-2014 Internet Archive. All Rights Reserved.