Package | Description |
---|---|
org.archive.crawler.postprocessor | |
org.archive.crawler.prefetch |
Modifier and Type | Class and Description |
---|---|
class |
LinksScoper
Deprecated.
Use CandidatesProcessor and CandidateChain/CandidateScoper instead
|
class |
SupplementaryLinksScoper
Run CrawlURI links carried in the passed CrawlURI through a filter
and 'handle' rejections.
|
Modifier and Type | Class and Description |
---|---|
class |
CandidateScoper
Simple single-URI scoper, considers passed-in URI as candidate; sets
fetchstatus negative and skips to end of processing if out-of-scope.
|
class |
FrontierPreparer
Processor to preload URI with as much precalculated policy-based
info as possible before it reaches frontier criticial sections.
|
class |
Preselector
If set to recheck the crawl's scope, gives a yes/no on whether
a CrawlURI should be processed at all.
|
Copyright © 2003-2014 Internet Archive. All Rights Reserved.