public class SupplementaryLinksScoper extends Scoper
fileLogger, isRunning, loggerModule, scope
beanName, kp, recoveryCheckpoint, uriCount
Constructor and Description |
---|
SupplementaryLinksScoper() |
Modifier and Type | Method and Description |
---|---|
DecideRule |
getSupplementaryRule() |
protected void |
innerProcess(CrawlURI puri)
Actually performs the process.
|
protected boolean |
isInScope(CrawlURI caUri)
Schedule the given
CrawlURI with the Frontier. |
protected void |
outOfScope(CrawlURI caUri)
Called when a CrawlURI is ruled out of scope.
|
void |
setSupplementaryRule(DecideRule rule) |
protected boolean |
shouldProcess(CrawlURI puri)
Determines whether the given uri should be processed by this
processor.
|
getLoggerModule, getLogToFile, getScope, isRunning, setLoggerModule, setLogToFile, setScope, start, stop
doCheckpoint, finishCheckpoint, flattenVia, fromCheckpointJson, getBeanName, getEnabled, getKeyedProperties, getRecordedSize, getShouldProcessRule, getURICount, hasHttpAuthenticationCredential, innerProcessResult, innerRejectProcess, isSuccess, process, report, setBeanName, setEnabled, setRecoveryCheckpoint, setShouldProcessRule, startCheckpoint, toCheckpointJson
public SupplementaryLinksScoper()
name
- Name of this filter.public DecideRule getSupplementaryRule()
public void setSupplementaryRule(DecideRule rule)
protected boolean shouldProcess(CrawlURI puri)
Processor
shouldProcess
in class Processor
puri
- the URI to testprotected void innerProcess(CrawlURI puri)
Processor
#ENABLED
, the
#DECIDE_RULES
and the #shouldProcess(ProcessorURI)
tests.innerProcess
in class Processor
puri
- the URI to processprotected boolean isInScope(CrawlURI caUri)
Scoper
CrawlURI
with the Frontier.protected void outOfScope(CrawlURI caUri)
outOfScope
in class Scoper
caUri
- CrawlURI that is out of scope.Copyright © 2003-2014 Internet Archive. All Rights Reserved.