public class RecoveryLogMapper extends Object
Constructor and Description |
---|
RecoveryLogMapper(String recoverLogFileName)
Normal constructor - if encounter not-found seeds while loading
recoverLogFileName, will throw throw SeedUrlNotFoundException.
|
RecoveryLogMapper(String recoverLogFileName,
String seedNotFoundLogFileName)
Constructor to use if you want to allow not-found seeds, logging them to
seedNotFoundLogFileName.
|
Modifier and Type | Method and Description |
---|---|
Iterator<String> |
getIteratorOfURLsSuccessfullyCrawledFromSeedUrl(String seedUrlString) |
static Logger |
getLogger() |
Collection<String> |
getSeedCollection() |
String |
getSeedForUrl(String urlString)
Returns seed for urlString (null if seed not found).
|
Map<String,Set<String>> |
getSeedUrlToDiscoveredUrlsMap() |
Set<String> |
getSuccessfullyCrawledUrls() |
protected void |
load(String recoverLogFileName) |
static void |
main(String[] args) |
public RecoveryLogMapper(String recoverLogFileName) throws FileNotFoundException, IOException, SeedUrlNotFoundException
RecoveryLogMapper(String)
if you want to just log such cases and
keep going. (Those should not happen if the recover log is written
correctly, but we see them in pratice.)recoverLogFileName
- FileNotFoundException
IOException
SeedUrlNotFoundException
public RecoveryLogMapper(String recoverLogFileName, String seedNotFoundLogFileName) throws FileNotFoundException, IOException, SeedUrlNotFoundException
RecoveryLogMapper(String)
will throw SeedUrlNotFoundException when a seed isn't found.recoverLogFileName
- seedNotFoundLogFileName
- FileNotFoundException
IOException
SeedUrlNotFoundException
protected void load(String recoverLogFileName) throws FileNotFoundException, IOException, SeedUrlNotFoundException
public String getSeedForUrl(String urlString)
urlString
- public Map<String,Set<String>> getSeedUrlToDiscoveredUrlsMap()
public Set<String> getSuccessfullyCrawledUrls()
public static Logger getLogger()
public Iterator<String> getIteratorOfURLsSuccessfullyCrawledFromSeedUrl(String seedUrlString) throws SeedUrlNotFoundException
SeedUrlNotFoundException
public Collection<String> getSeedCollection()
public static void main(String[] args)
Copyright © 2003-2014 Internet Archive. All Rights Reserved.