Package | Description |
---|---|
org.archive.crawler.postprocessor | |
org.archive.crawler.prefetch | |
org.archive.crawler.processor | |
org.archive.modules |
The beginnings of a refactored settings framework.
|
org.archive.modules.fetcher | |
org.archive.modules.writer |
Modifier and Type | Method and Description |
---|---|
protected ProcessResult |
LowDiskPauseProcessor.innerProcessResult(CrawlURI curi)
Deprecated.
Notes a CrawlURI's content size in its running tally.
|
Modifier and Type | Method and Description |
---|---|
protected ProcessResult |
RuntimeLimitEnforcer.innerProcessResult(CrawlURI curi) |
protected ProcessResult |
Preselector.innerProcessResult(CrawlURI puri) |
protected ProcessResult |
QuotaEnforcer.innerProcessResult(CrawlURI puri) |
protected ProcessResult |
PreconditionEnforcer.innerProcessResult(CrawlURI puri) |
protected ProcessResult |
CandidateScoper.innerProcessResult(CrawlURI curi) |
Modifier and Type | Method and Description |
---|---|
protected ProcessResult |
CrawlMapper.innerProcessResult(CrawlURI puri) |
Modifier and Type | Field and Description |
---|---|
static ProcessResult |
ProcessResult.FINISH |
static ProcessResult |
ProcessResult.PROCEED |
Modifier and Type | Method and Description |
---|---|
protected ProcessResult |
Processor.innerProcessResult(CrawlURI uri) |
static ProcessResult |
ProcessResult.jump(String jumpTarget) |
ProcessResult |
Processor.process(CrawlURI uri)
Processes the given URI.
|
Modifier and Type | Method and Description |
---|---|
protected ProcessResult |
FetchWhois.deferOrFinishGeneric(CrawlURI curi,
String domainOrIp) |
protected ProcessResult |
FetchWhois.innerProcessResult(CrawlURI curi) |
ProcessResult |
FetchHTTP.process(CrawlURI uri) |
Modifier and Type | Method and Description |
---|---|
protected ProcessResult |
WriterPoolProcessor.checkBytesWritten() |
protected abstract ProcessResult |
WriterPoolProcessor.innerProcessResult(CrawlURI uri) |
protected ProcessResult |
WARCWriterProcessor.innerProcessResult(CrawlURI puri)
Writes a CrawlURI and its associated data to store file.
|
protected ProcessResult |
ARCWriterProcessor.innerProcessResult(CrawlURI puri)
Writes a CrawlURI and its associated data to store file.
|
protected ProcessResult |
ARCWriterProcessor.write(CrawlURI curi,
long recordLength,
InputStream in,
String ip) |
protected ProcessResult |
WARCWriterProcessor.write(String lowerCaseScheme,
CrawlURI curi) |
Copyright © 2003-2014 Internet Archive. All Rights Reserved.