Package | Description |
---|---|
org.archive.crawler.frontier | |
org.archive.crawler.prefetch |
Modifier and Type | Class and Description |
---|---|
class |
AntiCalendarCostAssignmentPolicy
CostAssignmentPolicy that further penalizes URIs with
calendar-suggestive strings in them, with an extra unit
of cost.
|
class |
UnitCostAssignmentPolicy
A CostAssignment policy that uses a constant value of 1 for all CrawlURIs.
|
class |
WagCostAssignmentPolicy
A CostAssignmentPolicy based on some wild guesses of kinds of URIs
that should be deferred into the (potentially never-crawled) future.
|
class |
ZeroCostAssignmentPolicy
CostAssignmentPolicy considering all URIs costless -- essentially
disabling budgetting features.
|
Modifier and Type | Method and Description |
---|---|
CostAssignmentPolicy |
FrontierPreparer.getCostAssignmentPolicy() |
Modifier and Type | Method and Description |
---|---|
void |
FrontierPreparer.setCostAssignmentPolicy(CostAssignmentPolicy policy) |
Copyright © 2003-2014 Internet Archive. All Rights Reserved.