org.archive.crawler.util

Class TopNSet

java.lang.Object

org.archive.crawler.util.TopNSet

All Implemented Interfaces:

Serializable

public class TopNSet
extends Object
implements Serializable

Counting Set which only remembers the 'top N' of all String values reported (with counts) to it. Precise if counts reported for a certain String key only ever increase. Otherwise, the current top N might decrement to be smaller than other tallies, which won't be noticed until those other, previously ignored tallies, are re-reported. TODO: Histotable and TopNSet that they could possibly have a closer relationship and share some code (even though Histotable tracks 'all' keys, and handles increments, while TopNSet only remembers a small subset of keys, and requires a fresh full value on each update.)

See Also:

Serialized Form

Contributor:

gojomo

Field Summary

Fields

Modifier and Type	Field and Description
protected String	largestKnownKey
protected long	largestKnownValue
protected int	maxsize
protected ConcurrentMap<String,Long>	set
protected String	smallestKnownKey
protected long	smallestKnownValue

Constructor Summary

Constructors

Constructor and Description
TopNSet(int size)

Method Summary

Methods

Modifier and Type	Method and Description
SortedSet<Map.Entry<?,Long>>	getEntriesDescending() Get descending ordered list of key,count Entries.
String	getLargest()
int	getMaxSize()
String	getSmallest()
ConcurrentMap<String,Long>	getTopSet() Make internal map available (for checkpoint/restore purposes).
void	setMaxSize(int max)
int	size()
void	update(String key, long value) Update the given String key with a new total value, perhaps displacing an existing top-valued entry, and updating the fields recording max/min keys in any case.
void	updateBounds() After an operation invalidating the previous largest/smallest entry, find the new largest/smallest.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

maxsize

protected int maxsize

set

protected ConcurrentMap<String,Long> set

smallestKnownValue

protected volatile long smallestKnownValue

smallestKnownKey

protected volatile String smallestKnownKey

largestKnownValue

protected volatile long largestKnownValue

largestKnownKey

protected volatile String largestKnownKey

Constructor Detail

TopNSet

public TopNSet(int size)

Method Detail

update

public void update(String key,
          long value)

Update the given String key with a new total value, perhaps displacing an existing top-valued entry, and updating the fields recording max/min keys in any case.

Parameters:

key - String key to update

value - long new total value (*not* increment/decrement)

getLargest

public String getLargest()

getSmallest

public String getSmallest()

updateBounds

public void updateBounds()

After an operation invalidating the previous largest/smallest entry, find the new largest/smallest.

getTopSet

public ConcurrentMap<String,Long> getTopSet()

Make internal map available (for checkpoint/restore purposes).

Returns:

HashMap

size

public int size()

getEntriesDescending

public SortedSet<Map.Entry<?,Long>> getEntriesDescending()

Get descending ordered list of key,count Entries.

Returns:

SortedSet of Entry descending-frequency

getMaxSize

public int getMaxSize()

setMaxSize

public void setMaxSize(int max)