org.archive.crawler

Class Heritrix

java.lang.Object

org.archive.crawler.Heritrix

public class Heritrix
extends Object

Main class for Heritrix crawler. Heritrix is usually launched by a shell script that backgrounds heritrix that redirects all stdout and stderr emitted by heritrix to a log file. So that startup messages emitted subsequent to the redirection of stdout and stderr show on the console, this class prints usage or startup output such as where the web UI can be found, etc., to a STARTLOG that the shell script is waiting on. As soon as the shell script sees output in this file, it prints its content and breaks out of its wait. See ${HERITRIX_HOME}/bin/heritrix.

Heritrix can also be embedded or launched by webapp initialization or by JMX bootstrapping. So far I count 4 methods of instantiation:

1. From this classes main -- the method usually used;
2. From the Heritrix UI (The local-instances.jsp) page;
3. A creation by a JMX agent at the behest of a remote JMX client; and
4. A container such as tomcat or jboss.

Author:

gojomo, Kristinn Sigurdsson, Stack

Field Summary

Fields

Modifier and Type	Field and Description
protected org.restlet.Component	component
protected Engine	engine

Constructor Summary

Constructors

Constructor and Description
Heritrix()

Method Summary

Methods

Modifier and Type	Method and Description
org.restlet.Component	getComponent()
Engine	getEngine()
protected static File	getHeritrixHome() Exploit -Dheritrix.home if available to us.
void	instanceMain(String[] args)
static void	main(String[] args) Launches a local Engine and restfgul web interface given the command-line options or defaults.
protected void	setupGlobalProperties(int port) Setup global system properties that may be of use elsewhere.
protected void	setupServer(int port, String address, String keystorePath, String keystorePassword, String keyPassword) Create an HTTPS restlet Server instance matching the given parameters.
protected void	useAdhocKeystore(PrintStream startupOut) Perform preparation to use an ad-hoc, created-as-necessary certificate/keystore for HTTPS access.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

engine

protected Engine engine

component

protected org.restlet.Component component

Constructor Detail

Heritrix

public Heritrix()

Method Detail

main

public static void main(String[] args)
                 throws Exception

Launches a local Engine and restfgul web interface given the command-line options or defaults.

Parameters:

args - Command line arguments.

Throws:

Exception

instanceMain

public void instanceMain(String[] args)
                  throws Exception

Throws:

Exception

setupGlobalProperties

protected void setupGlobalProperties(int port)

Setup global system properties that may be of use elsewhere.

Parameters:

port -

useAdhocKeystore

protected void useAdhocKeystore(PrintStream startupOut)

Perform preparation to use an ad-hoc, created-as-necessary certificate/keystore for HTTPS access. A keystore with new cert is created if necessary, as adhoc.keystore in the working directory. Otherwise, a preexisting adhoc.keystore is read and the certificate fingerprint shown to assist in operator browser-side verification.

Parameters:

startupOut - where to report fingerprint

setupServer

protected void setupServer(int port,
               String address,
               String keystorePath,
               String keystorePassword,
               String keyPassword)

Create an HTTPS restlet Server instance matching the given parameters.

Parameters:

port -

address -

keystorePath -

keystorePassword -

keyPassword -

getHeritrixHome

protected static File getHeritrixHome()

Exploit -Dheritrix.home if available to us. Is current working dir if no heritrix.home property supplied.

Returns:

Heritrix home directory.

Throws:

IOException

getEngine

public Engine getEngine()

getComponent

public org.restlet.Component getComponent()