1. through commands and viewing log files to hadoop startup and operation
In NameNode side, you can
tail -100 /var/log/hadoop/hadoop/hadoop-hadoop-namenode-hadoop-namenode.log
View NameNode running log
can also end in DataNode
cat /var/log/hadoop/hadoop/hadoop-hadoop-datanode-hadoop-datanode1.log
View DataNode running log.
through jps commands are run in the datanode and namenode side, view your active service.
2. NameNode not start:
Cannot lock storage ...... tmp / dfs / name. The directory is already locked.
perhaps because run hadoop account for this folder tmp / dfs / name without permission. You can use the following command to solve
chown -R hadoop:hadoop /usr/hadoop
3. DataNode not start :
in the client log shows namenode namespaceID = 1713611278; datanode namespaceID = 596511341
This problem is basically because in the end times namenode run hadoop namenode-format caused. The core-site.xml in hadoop file (different hadoop versions have different names) found
[hadoop@hadoop-datanode1 hadoop]$ cat core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!--global properties -->
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/hadoop/tmp</value>
</property>
Empty
[hadoop@hadoop-datanode1 tmp]$ rm -rf /usr/hadoop/tmp/*
then restart hadoop, in datanode side with jps see if datanode already started.
4. wordcount program run fs can not find the folder:
Input path does not exist: hdfs :/ / localhost: 9000/user/input
in a clustered environment, the files are processed in hdfs, so must be processed hadoop files to a folder. The following example, a new folder in the fs, well in advance of the band hdfs wordcount file copy, and finally run the program.
[hadoop@hadoop-namenode ~]$ hadoop fs -mkdir /tmp/wordcount/input
[hadoop@hadoop-namenode ~]$ hadoop fs -put /home/hadoop/wordcount/input /tmp/wordcount/input
[hadoop@hadoop-namenode ~]$ hadoop fs -ls /tmp/wordcount/input
hadoop jar /home/hadoop/hadoop-examples-1.1.2.jar wordcount /tmp/wordcount/input/input /tmp/wordcount/output
View Results
hadoop fs -cat /tmp/wordcount/output/part-r-00000
This was helpful.. hadoop
回复删除