public class CrawlerJournal extends Object implements Closeable
Modifier and Type | Field and Description |
---|---|
protected File |
gzipFile
File we're writing journal to.
|
protected long |
lines
line count
|
static String |
LOG_ERROR
prefix for error lines
|
static String |
LOG_TIMESTAMP
prefix for timestamp lines
|
protected Writer |
out
Stream on which we record frontier events.
|
protected int |
timestamp_interval
number of lines between timestamps
|
Constructor and Description |
---|
CrawlerJournal(File file)
Create a new crawler journal at the given location
|
CrawlerJournal(String path,
String filename)
Create a new crawler journal at the given location
|
Modifier and Type | Method and Description |
---|---|
void |
close()
Flush and close the underlying IO objects.
|
protected void |
considerTimestamp()
Write a timestamp line if appropriate
|
protected Writer |
initialize(File f) |
protected void |
noteLine()
Count and note a line
|
void |
rotateForCheckpoint(Checkpoint checkpointInProgress)
Handle a checkpoint by rotating the current log to a checkpoint-named
file and starting a new log.
|
void |
seriousError(String err)
Note a serious error vioa a special log line
|
void |
writeLine(it.unimi.dsi.mg4j.util.MutableString mstring)
Write a line.
|
void |
writeLine(String... strs)
Write a line
|
public static final String LOG_ERROR
public static final String LOG_TIMESTAMP
protected Writer out
protected long lines
protected int timestamp_interval
protected File gzipFile
public CrawlerJournal(String path, String filename) throws IOException
path
- Directory to make thejournal in.filename
- Name to use for journal file.IOException
public CrawlerJournal(File file) throws IOException
file
- path at which to make journalIOException
protected Writer initialize(File f) throws FileNotFoundException, IOException
FileNotFoundException
IOException
public void writeLine(String... strs)
string
- Stringpublic void writeLine(it.unimi.dsi.mg4j.util.MutableString mstring)
mstring
- MutableString to writeprotected void noteLine() throws IOException
IOException
protected void considerTimestamp() throws IOException
IOException
public void close()
close
in interface Closeable
close
in interface AutoCloseable
public void seriousError(String err)
err
- public void rotateForCheckpoint(Checkpoint checkpointInProgress)
checkpointDir
- IOException
Copyright © 2003-2014 Internet Archive. All Rights Reserved.