public abstract class PersistProcessor extends AbstractPersistProcessor
Modifier and Type | Field and Description |
---|---|
static BdbModule.BdbConfig |
HISTORY_DB_CONFIG |
static String |
URI_HISTORY_DBNAME
name of history Database
|
onlyStoreIfWriteTagPresent
Constructor and Description |
---|
PersistProcessor() |
Modifier and Type | Method and Description |
---|---|
static int |
copyPersistSourceToHistoryMap(File sourceFile,
com.sleepycat.collections.StoredSortedMap<String,Map> historyMap)
Populates a given StoredSortedMap (history map) from an old
environment db or a persist log.
|
static int |
copyPersistSourceToHistoryMap(URL sourceUrl,
com.sleepycat.collections.StoredSortedMap<String,Map> historyMap)
Populates a given StoredSortedMap (history map) from an old persist log.
|
static void |
main(String[] args)
Utility main for importing a log into a BDB-JE environment or moving a
database between environments (2 arguments), or simply dumping a log
to stderr in a more readable format (1 argument).
|
static String |
persistKeyFor(CrawlURI curi)
Return a preferred String key for persisting the given CrawlURI's
AList state.
|
static String |
persistKeyFor(String uri) |
static int |
populatePersistEnv(String sourcePath,
File envFile)
Populates a new environment db from an old environment db or a persist
log.
|
static EnhancedEnvironment |
setupCopyEnvironment(File env) |
static EnhancedEnvironment |
setupCopyEnvironment(File env,
boolean readOnly) |
getOnlyStoreIfWriteTagPresent, hasWriteTag, setOnlyStoreIfWriteTagPresent, shouldLoad, shouldStore
doCheckpoint, finishCheckpoint, flattenVia, fromCheckpointJson, getBeanName, getEnabled, getKeyedProperties, getRecordedSize, getShouldProcessRule, getURICount, hasHttpAuthenticationCredential, innerProcess, innerProcessResult, innerRejectProcess, isRunning, isSuccess, process, report, setBeanName, setEnabled, setRecoveryCheckpoint, setShouldProcessRule, shouldProcess, start, startCheckpoint, stop, toCheckpointJson
public static final String URI_HISTORY_DBNAME
public static final BdbModule.BdbConfig HISTORY_DB_CONFIG
public static String persistKeyFor(CrawlURI curi)
curi
- CrawlURIpublic static int populatePersistEnv(String sourcePath, File envFile) throws IOException
sourcePath
- source of old entries: can be a path to an existing
environment db, or a URL or path to a persist logenvFile
- path to new environment db (or null for a dry run)com.sleepycat.je.DatabaseException
IOException
public static int copyPersistSourceToHistoryMap(File sourceFile, com.sleepycat.collections.StoredSortedMap<String,Map> historyMap) throws com.sleepycat.je.DatabaseException, IOException
sourceFile
- source of old entries: can be a path to an existing
environment db or persist loghistoryMap
- map to populate (or null for a dry run)com.sleepycat.je.DatabaseException
IOException
public static int copyPersistSourceToHistoryMap(URL sourceUrl, com.sleepycat.collections.StoredSortedMap<String,Map> historyMap) throws com.sleepycat.je.DatabaseException, IOException
sourceUrl
- url of source persist loghistoryMap
- map to populate (or null for a dry run)com.sleepycat.je.DatabaseException
IOException
public static void main(String[] args) throws com.sleepycat.je.DatabaseException, IOException
args
- command-line argumentscom.sleepycat.je.DatabaseException
IOException
public static EnhancedEnvironment setupCopyEnvironment(File env) throws com.sleepycat.je.DatabaseException
com.sleepycat.je.DatabaseException
public static EnhancedEnvironment setupCopyEnvironment(File env, boolean readOnly) throws com.sleepycat.je.DatabaseException
com.sleepycat.je.DatabaseException
Copyright © 2003-2014 Internet Archive. All Rights Reserved.