org.archive.crawler.migrate

Class MigrateH1to3Tool

java.lang.Object

org.archive.crawler.migrate.MigrateH1to3Tool

public class MigrateH1to3Tool
extends Object

Utility class which takes a H1 order.xml and creates a similar H3 job directory, with as many simple settings converted over (as top-of-crawler-beans overrides) as possible at this time. (Future versions will handle more complicated H1 settings customizations, such as per-host overrides or choices of alternate implementing classes for Scope, Processors, etc.)

Contributor:

siznax, gojomo

Field Summary

Fields

Modifier and Type	Field and Description
protected static DocumentBuilder	DOCUMENT_BUILDER
protected Document	sourceOrderXmlDom

Constructor Summary

Constructors

Constructor and Description
MigrateH1to3Tool()

Method Summary

Methods

Modifier and Type	Method and Description
static Map<String,String>	flattenH1Order(Document h1order) Given a Document, return a Map of all non-blank simple text nodes, keyed by the pseudo-XPath to their parent element.
protected Map<String,String>	getMigrateMap()
protected static String	getPseudoXpath(Node node) Given a node, give back an XPath-like string that addresses it.
void	instanceMain(String[] args)
protected void	listProblems(List<String> problems)
static void	main(String[] args)
protected void	printHelp()
protected void	splitH1userAgent(String userAgent, StringBuilder sb)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sourceOrderXmlDom

protected Document sourceOrderXmlDom

DOCUMENT_BUILDER

protected static DocumentBuilder DOCUMENT_BUILDER

Constructor Detail

MigrateH1to3Tool

public MigrateH1to3Tool()

Method Detail

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception

instanceMain

public void instanceMain(String[] args)
                  throws Exception

Throws:

Exception

listProblems

protected void listProblems(List<String> problems)

printHelp

protected void printHelp()

splitH1userAgent

protected void splitH1userAgent(String userAgent,
                    StringBuilder sb)

getMigrateMap

protected Map<String,String> getMigrateMap()
                                    throws IOException

Throws:

IOException

flattenH1Order

public static Map<String,String> flattenH1Order(Document h1order)
                                         throws XPathExpressionException

Given a Document, return a Map of all non-blank simple text nodes, keyed by the pseudo-XPath to their parent element.

Parameters:

h1order - Document to extract Map

Returns:

Map Xpath-like-String -> non-blank text content

Throws:

XPathExpressionException

getPseudoXpath

protected static String getPseudoXpath(Node node)

Given a node, give back an XPath-like string that addresses it. (For our constrained order.xml files, it is a valid and unique XPath, but the simple approach used here might not generate unique XPaths on all XML.

Parameters:

node - node to get pseudo-XPath

Returns:

String pseudo-XPath