org.archive.modules.net

Class CrawlHost

java.lang.Object

org.archive.modules.net.CrawlHost

All Implemented Interfaces:

Serializable, FetchStats.HasFetchStats, IdentityCacheable

public class CrawlHost
extends Object
implements Serializable, FetchStats.HasFetchStats, IdentityCacheable

Represents a single remote "host". An host is a name for which there is a dns record or an IP-address. This might be a machine or a virtual host.

Author:

gojomo

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static long	IP_NEVER_EXPIRES Flag value indicating always-valid IP
static long	IP_NEVER_LOOKED_UP Flag value indicating an IP has not yet been looked up
protected FetchStats	substats

Constructor Summary

Constructors

Constructor and Description
CrawlHost(String hostname) Create a new CrawlHost object.
CrawlHost(String hostname, String countryCode) Create a new CrawlHost object.

Method Summary

Methods

Modifier and Type	Method and Description
static void	autoregisterTo(AutoKryo kryo)
boolean	equals(Object obj)
String	getCountryCode() Get country code of this host
long	getEarliestNextURIEmitTime() Get the earliest time a URI for this host could be emitted.
String	getHostName() Get the host name.
InetAddress	getIP() Get the IP address for this host.
long	getIpFetched() Get the time when the IP address for this host was last looked up.
long	getIpTTL() Get the TTL value from the dns record for this host.
String	getKey()
FetchStats	getSubstats()
boolean	hasBeenLookedUp() Return true if the IP for this host has been looked up.
int	hashCode()
void	makeDirty()
void	setCountryCode(String countryCode) Set country code for this hos
void	setEarliestNextURIEmitTime(long earliestNextURIEmitTime) Set the earliest time a URI for this host could be emitted.
void	setIdentityCache(ObjectIdentityCache<?> cache)
void	setIP(InetAddress address, long ttl) Set the IP address for this host.
String	toString()

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

IP_NEVER_EXPIRES

public static final long IP_NEVER_EXPIRES

Flag value indicating always-valid IP

See Also:

Constant Field Values

IP_NEVER_LOOKED_UP

public static final long IP_NEVER_LOOKED_UP

Flag value indicating an IP has not yet been looked up

See Also:

Constant Field Values

substats

protected FetchStats substats

Constructor Detail

CrawlHost

public CrawlHost(String hostname)

Create a new CrawlHost object.

Parameters:

hostname - the host name for this host.

CrawlHost

public CrawlHost(String hostname,
         String countryCode)

Create a new CrawlHost object.

Parameters:

hostname - the host name for this host.

countryCode - the country code for this host.

Method Detail

hasBeenLookedUp

public boolean hasBeenLookedUp()

Return true if the IP for this host has been looked up. Returns true even if the lookup failed.

Returns:

true if the IP for this host has been looked up.

setIP

public void setIP(InetAddress address,
         long ttl)

Set the IP address for this host.

Parameters:

address -

ttl - the TTL from the dns record in seconds or -1 if it should live forever (is a numeric IP).

getIP

public InetAddress getIP()

Get the IP address for this host.

Returns:

the IP address for this host.

getIpFetched

public long getIpFetched()

Get the time when the IP address for this host was last looked up.

Returns:

the time when the IP address for this host was last looked up.

getIpTTL

public long getIpTTL()

Get the TTL value from the dns record for this host.

Returns:

the TTL value from the dns record for this host -- in seconds -- or -1 if this lookup should be valid forever (numeric ip).

toString

public String toString()

Overrides:

toString in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object

getHostName

public String getHostName()

Get the host name.

Returns:

Returns the host name.

getEarliestNextURIEmitTime

public long getEarliestNextURIEmitTime()

Get the earliest time a URI for this host could be emitted. This only has effect if constraints on bandwidth per host is set.

Returns:

Returns the earliestNextURIEmitTime.

setEarliestNextURIEmitTime

public void setEarliestNextURIEmitTime(long earliestNextURIEmitTime)

Set the earliest time a URI for this host could be emitted. This only has effect if constraints on bandwidth per host is set.

Parameters:

earliestNextURIEmitTime - The earliestNextURIEmitTime to set.

getCountryCode

public String getCountryCode()

Get country code of this host

Returns:

Retruns country code or null if not availabe

setCountryCode

public void setCountryCode(String countryCode)

Set country code for this hos

Parameters:

countryCode - The country code of this host

getSubstats

public FetchStats getSubstats()

Specified by:

getSubstats in interface FetchStats.HasFetchStats

autoregisterTo

public static void autoregisterTo(AutoKryo kryo)

getKey

public String getKey()

Specified by:

getKey in interface IdentityCacheable

makeDirty

public void makeDirty()

Specified by:

makeDirty in interface IdentityCacheable

setIdentityCache

public void setIdentityCache(ObjectIdentityCache<?> cache)

Specified by:

setIdentityCache in interface IdentityCacheable