org.archive.crawler.util

Class FPUriUniqFilter

java.lang.Object

org.archive.crawler.util.SetBasedUriUniqFilter

org.archive.crawler.util.FPUriUniqFilter

All Implemented Interfaces:

Serializable, UriUniqFilter

public class FPUriUniqFilter
extends SetBasedUriUniqFilter
implements Serializable

UriUniqFilter storing 64-bit UURI fingerprints, using an internal LongFPSet instance. The passed LongFPSet internal instance may be disk or memory based. Accesses to the underlying LongFPSet are synchronized.

Author:

gojomo

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.archive.crawler.datamodel.UriUniqFilter

UriUniqFilter.CrawlUriReceiver

Field Summary

Fields

Modifier and Type	Field and Description
protected LongFPSet	fpset

Fields inherited from class org.archive.crawler.util.SetBasedUriUniqFilter

duplicateCount, duplicatesAtLastSample, profileLog, receiver

Constructor Summary

Constructors

Constructor and Description
FPUriUniqFilter()
FPUriUniqFilter(LongFPSet fpset) Create FPUriUniqFilter wrapping given long set

Method Summary

Methods

Modifier and Type	Method and Description
LongFPSet	getFpset()
protected boolean	setAdd(CharSequence uri)
protected long	setCount()
void	setFpset(LongFPSet fpset)
protected boolean	setRemove(CharSequence uri)

Methods inherited from class org.archive.crawler.util.SetBasedUriUniqFilter

add, addForce, addNow, close, count, forget, note, pending, profileLog, requestFlush, setDestination, setProfileLog

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fpset

protected LongFPSet fpset

Constructor Detail

FPUriUniqFilter

public FPUriUniqFilter(LongFPSet fpset)

Create FPUriUniqFilter wrapping given long set

Parameters:

fpset -

FPUriUniqFilter

public FPUriUniqFilter()

Method Detail

getFpset

public LongFPSet getFpset()

setFpset

public void setFpset(LongFPSet fpset)

setAdd

protected boolean setAdd(CharSequence uri)

Specified by:

setAdd in class SetBasedUriUniqFilter

setCount

protected long setCount()

Specified by:

setCount in class SetBasedUriUniqFilter

setRemove

protected boolean setRemove(CharSequence uri)

Specified by:

setRemove in class SetBasedUriUniqFilter