public class FetchHistoryProcessor extends Processor
Modifier and Type | Field and Description |
---|---|
protected int |
historyLength
Desired history array length.
|
Constructor and Description |
---|
FetchHistoryProcessor() |
Modifier and Type | Method and Description |
---|---|
int |
getHistoryLength() |
protected void |
innerProcess(CrawlURI puri)
Actually performs the process.
|
protected void |
saveHeader(String name,
org.apache.commons.httpclient.HttpMethod method,
HashMap<String,Object> latestFetch)
Save a header from the given HTTP operation into the AList.
|
void |
setHistoryLength(int length) |
protected boolean |
shouldProcess(CrawlURI curi)
Determines whether the given uri should be processed by this
processor.
|
doCheckpoint, finishCheckpoint, flattenVia, fromCheckpointJson, getBeanName, getEnabled, getKeyedProperties, getRecordedSize, getShouldProcessRule, getURICount, hasHttpAuthenticationCredential, innerProcessResult, innerRejectProcess, isRunning, isSuccess, process, report, setBeanName, setEnabled, setRecoveryCheckpoint, setShouldProcessRule, start, startCheckpoint, stop, toCheckpointJson
public int getHistoryLength()
public void setHistoryLength(int length)
protected void innerProcess(CrawlURI puri) throws InterruptedException
Processor
#ENABLED
, the
#DECIDE_RULES
and the #shouldProcess(ProcessorURI)
tests.innerProcess
in class Processor
puri
- the URI to processInterruptedException
- if the thread is interruptedprotected void saveHeader(String name, org.apache.commons.httpclient.HttpMethod method, HashMap<String,Object> latestFetch)
name
- header name to save into history AListmethod
- http operation containing headerslatestFetch
- AList to get headerprotected boolean shouldProcess(CrawlURI curi)
Processor
shouldProcess
in class Processor
curi
- the URI to testCopyright © 2003-2014 Internet Archive. All Rights Reserved.