org.apache.nutch.tools.compat
Class CrawlDbConverter
java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.nutch.tools.compat.CrawlDbConverter
- All Implemented Interfaces:
- Closeable, org.apache.hadoop.conf.Configurable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.WritableComparable,CrawlDatum,org.apache.hadoop.io.Text,CrawlDatum>, org.apache.hadoop.util.Tool
public class CrawlDbConverter
- extends org.apache.hadoop.conf.Configured
- implements org.apache.hadoop.util.Tool, org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.WritableComparable,CrawlDatum,org.apache.hadoop.io.Text,CrawlDatum>
This tool converts CrawlDb created in old <UTF8, CrawlDatum> format
(Nutch versions < 0.9.0) to the new <Text, CrawlDatum> format.
Optionally CrawlDatum.metaData
can be converted
too from using UTF8 keys to using Text keys.
- Author:
- Andrzej Bialecki
Method Summary |
void |
close()
|
void |
configure(org.apache.hadoop.mapred.JobConf job)
|
static void |
main(String[] args)
|
void |
map(org.apache.hadoop.io.WritableComparable key,
CrawlDatum value,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,CrawlDatum> output,
org.apache.hadoop.mapred.Reporter reporter)
|
int |
run(String[] args)
|
Methods inherited from class org.apache.hadoop.conf.Configured |
getConf, setConf |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface org.apache.hadoop.conf.Configurable |
getConf, setConf |
CrawlDbConverter
public CrawlDbConverter()
configure
public void configure(org.apache.hadoop.mapred.JobConf job)
- Specified by:
configure
in interface org.apache.hadoop.mapred.JobConfigurable
map
public void map(org.apache.hadoop.io.WritableComparable key,
CrawlDatum value,
org.apache.hadoop.mapred.OutputCollector<org.apache.hadoop.io.Text,CrawlDatum> output,
org.apache.hadoop.mapred.Reporter reporter)
throws IOException
- Specified by:
map
in interface org.apache.hadoop.mapred.Mapper<org.apache.hadoop.io.WritableComparable,CrawlDatum,org.apache.hadoop.io.Text,CrawlDatum>
- Throws:
IOException
close
public void close()
throws IOException
- Specified by:
close
in interface Closeable
- Throws:
IOException
main
public static void main(String[] args)
throws Exception
- Parameters:
args
-
- Throws:
Exception
run
public int run(String[] args)
throws Exception
- Specified by:
run
in interface org.apache.hadoop.util.Tool
- Throws:
Exception
Copyright © 2006 The Apache Software Foundation