ActionDirectory (Heritrix 3 3.2.0 API)

java.lang.Object
- org.archive.crawler.framework.ActionDirectory

All Implemented Interfaces:

Runnable, org.springframework.context.ApplicationContextAware, org.springframework.context.Lifecycle
```
public class ActionDirectory
extends Object
implements org.springframework.context.ApplicationContextAware, org.springframework.context.Lifecycle, Runnable
```
Directory watched for new files. Depending on their extension, will process with regard to current crawl, and rename with a datestamp into the 'done' directory. Currently supports: - .seeds(.gz) add each URI found in file as a new seed (to be crawled if not already; to affect scope if appropriate). - (.s).recover(.gz) treat as traditional recovery log: consider all 'Fs'-tagged lines included, then try-rescheduling all 'F+'-tagged lines. (If ".s." present, try scoping URIs before including/scheduling.) - (.s).include(.gz) add each URI found in a recover-log like file (regardless of its tagging) to the frontier's alreadyIncluded filter, preventing them from being recrawled. ('.s.' indicates to apply scoping.) - (.s).schedule(.gz) add each URI found in a recover-log like file (regardless of its tagging) to the frontier's queues. ('.s.' indicates to apply scoping.) Future support planned: - .robots: invalidate robots ASAP - (?) .block: block-all on named site(s) - .overlay: add new overlay settings - .js .rb .bsh .rb etc - execute arbitrary script (a la ScriptedProcessor)

Contributor:

gojomo

Field Summary

Fields
Modifier and Type	Field and Description
`protected ConfigPath`	`actionDir`
`protected org.springframework.context.ApplicationContext`	`appCtx`
`protected int`	`delaySeconds` delay between scans of actionDirectory for new files
`protected ConfigPath`	`doneDir`
`protected ScheduledExecutorService`	`executor`
`protected Frontier`	`frontier` autowired frontier for actions
`protected int`	`initialDelaySeconds` how long after crawl start to first scan action directory
`protected static ScriptEngineManager`	`MANAGER` shared ScriptEngineManager
`protected SeedModule`	`seeds`

Constructor Summary

Constructors
Constructor and Description

ActionDirectory()

Constructors
Constructor and Description
`ActionDirectory()`

Method Summary

Methods
Modifier and Type	Method and Description
`protected void`	`actOn(File actionFile)` Process an individual action file found
`ConfigPath`	`getActionDir()`
`int`	`getDelaySeconds()`
`ConfigPath`	`getDoneDir()`
`Frontier`	`getFrontier()`
`int`	`getInitialDelaySeconds()`
`SeedModule`	`getSeeds()`
`boolean`	`isRunning()`
`void`	`run()` Action taken at scheduled intervals
`protected void`	`scanActionDirectory()` Find any new files in the 'action' directory; process each in order.
`void`	`setActionDir(ConfigPath actionDir)`
`void`	`setApplicationContext(org.springframework.context.ApplicationContext applicationContext)`
`void`	`setDelaySeconds(int delay)`
`void`	`setDoneDir(ConfigPath doneDir)`
`void`	`setFrontier(Frontier frontier)`
`void`	`setInitialDelaySeconds(int initialDelay)`
`void`	`setSeeds(SeedModule seeds)`
`void`	`start()`
`void`	`stop()`
`protected boolean`	`tryAsScript(File actionFile, String timestamp)` Try the actionFile as a script, deducing the proper scripting language from its file extension.

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - executor
```
protected ScheduledExecutorService executor
```
  - initialDelaySeconds
```
protected int initialDelaySeconds
```
    how long after crawl start to first scan action directory
  - delaySeconds
```
protected int delaySeconds
```
    delay between scans of actionDirectory for new files
  - actionDir
```
protected ConfigPath actionDir
```
  - doneDir
```
protected ConfigPath doneDir
```
  - appCtx
```
protected org.springframework.context.ApplicationContext appCtx
```
  - seeds
```
protected SeedModule seeds
```
  - frontier
```
protected Frontier frontier
```
    autowired frontier for actions
  - MANAGER
```
protected static ScriptEngineManager MANAGER
```
    shared ScriptEngineManager
- Constructor Detail
  - ActionDirectory
```
public ActionDirectory()
```
- Method Detail
  - getInitialDelaySeconds
```
public int getInitialDelaySeconds()
```
  - setInitialDelaySeconds
```
public void setInitialDelaySeconds(int initialDelay)
```
  - getDelaySeconds
```
public int getDelaySeconds()
```
  - setDelaySeconds
```
public void setDelaySeconds(int delay)
```
  - getActionDir
```
public ConfigPath getActionDir()
```
  - setActionDir
```
public void setActionDir(ConfigPath actionDir)
```
  - getDoneDir
```
public ConfigPath getDoneDir()
```
  - setDoneDir
```
public void setDoneDir(ConfigPath doneDir)
```
  - setApplicationContext
```
public void setApplicationContext(org.springframework.context.ApplicationContext applicationContext)
                           throws org.springframework.beans.BeansException
```
    Specified by:
    
    setApplicationContext in interface org.springframework.context.ApplicationContextAware
    
    Throws:
    
    org.springframework.beans.BeansException
  - getSeeds
```
public SeedModule getSeeds()
```
  - setSeeds
```
public void setSeeds(SeedModule seeds)
```
  - getFrontier
```
public Frontier getFrontier()
```
  - setFrontier
```
public void setFrontier(Frontier frontier)
```
  - isRunning
```
public boolean isRunning()
```
    Specified by:
    
    isRunning in interface org.springframework.context.Lifecycle
  - start
```
public void start()
```
    Specified by:
    
    start in interface org.springframework.context.Lifecycle
  - stop
```
public void stop()
```
    Specified by:
    
    stop in interface org.springframework.context.Lifecycle
  - run
```
public void run()
```
    Action taken at scheduled intervals
    
    Specified by:
    
    run in interface Runnable
    
    See Also:
    Runnable.run()
  - scanActionDirectory
```
protected void scanActionDirectory()
```
    Find any new files in the 'action' directory; process each in order.
  - actOn
```
protected void actOn(File actionFile)
```
    Process an individual action file found
    
    Parameters:
    actionFile - File to process
  - tryAsScript
```
protected boolean tryAsScript(File actionFile,
                  String timestamp)
```
    Try the actionFile as a script, deducing the proper scripting language from its file extension. Return true if evaluation was tried with a known script engine. Provides 'appCtx' and 'rawOut' to script for accessing crawl and outputting text to a '.out' file paired with the 'done/' action file. If an exception occurs, it will be logged to an '.ex' file alongside the script file in 'done/'.
    
    Parameters:
    actionFile - file to try
    timestamp - timestamp correlating out/ex files with done script
    
    Returns:
    true if engine evaluation began (even if an error occurred)

Class ActionDirectory

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

executor

initialDelaySeconds

delaySeconds

actionDir

doneDir

appCtx

seeds

frontier

MANAGER

Constructor Detail

ActionDirectory

Method Detail

getInitialDelaySeconds

setInitialDelaySeconds

getDelaySeconds

setDelaySeconds

getActionDir

setActionDir

getDoneDir

setDoneDir

setApplicationContext

getSeeds

setSeeds

getFrontier

setFrontier

isRunning

start

stop

run

scanActionDirectory

actOn

tryAsScript