hadoop怎么安装 centos

发布网友发布时间：2022-04-21 17:41

共5个回答

热心网友时间：2022-04-12 01:31

1、新建Hadoop用户配置免密码登陆

[root@ipython ~]# groupadd hadoop
[root@ipython ~]# useradd hadoop -g hadoop
[root@ipython ~]# passwd hadoop
[root@ipython ~]# mkdir /tools
[root@ipython ~]# chown hadoop:hadoop /tools/

##Ssh##
[root@ipython ~]# su - hadoop
[hadoop@ipython ~]$ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
+--[ DSA 1024]----+
|BE* |
|.*.= |
|+ o o . . |
|. o . o + |
| . . S o . |
| = o . |
| o o |
| . |
| |
+-----------------+
[hadoop@ipython ~]$ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys
[hadoop@ipython ~]$ chmod 0600 ~/.ssh/authorized_keys

2、JAVA.JDK 已安装（请参考:【CentOS JDK-1.8安装】）

[hadoop@ipython ~]$ java -version
java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

3、下载并解包Hadoop

[hadoop@ipython ~]$ cd /tools/
[hadoop@ipython source]$ wget https://archive.apache.org/dist/hadoop/core/hadoop-2.2.0/hadoop-2.2.0.tar.gz
[hadoop@ipython source]$ tar zxf hadoop-2.2.0.tar.gz
[hadoop@ipython source]$ ln -s /tools/source/hadoop-2.2.0 /tools/hadoop

4、添加Hadoop环境变量

[hadoop@ipython source]$ cat >> ~/.bashrc <<EOF
export HADOOP_PREFIX="/tools/hadoop"
export PATH=\$PATH:\$HADOOP_PREFIX/bin
export PATH=\$PATH:\$HADOOP_PREFIX/sbin
export HADOOP_MAPRED_HOME=\${HADOOP_PREFIX}
export HADOOP_COMMON_HOME=\${HADOOP_PREFIX}
export HADOOP_HDFS_HOME=\${HADOOP_PREFIX}
export YARN_HOME=\${HADOOP_PREFIX}
####hadoop-env####
export JAVA_HOME="/tools/java"
export HADOOP_COMMON_LIB_NATIVE_DIR=\${HADOOP_PREFIX}/lib/native
export HADOOP_OPTS="-Djava.library.path=\$HADOOP_PREFIX/lib"
####yarn-env####
export HADOOP_COMMON_LIB_NATIVE_DIR=\${HADOOP_PREFIX}/lib/native
export HADOOP_OPTS="-Djava.library.path=\$HADOOP_PREFIX/lib"
EOF
[hadoop@ipython source]$ source ~/.bashrc

5、变更配置文件

[hadoop@ipython source]$ cd $HADOOP_PREFIX/etc/hadoop
[hadoop@ipython hadoop]$ vi core-site.xml
#-------------------------------------------------------#
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://10.211.55.40:9000</value>
<final>true</final>
</property>
</configuration>
#-------------------------------------------------------#

[hadoop@ipython hadoop]$ vi hdfs-site.xml
#-------------------------------------------------------#
<configuration>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/hadoop/dfs/name</value>
<final>true</final>
</property>

<property>
<name>dfs.datanode.data.dir</name>
<value>file:/hadoop/dfs/data</value>
<final>true</final>
</property>

<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>
#-------------------------------------------------------#

[hadoop@ipython hadoop]$ cp mapred-site.xml.template mapred-site.xml
[hadoop@ipython hadoop]$ vi mapred-site.xml
#-------------------------------------------------------#
<configuration>
<property>
<name>maprece.framework.name</name>
<value>yarn</value>
</property>

<property>
<name>mapred.system.dir</name>
<value>file:/hadoop/mapred/system</value>
<final>true</final>
</property>

<property>
<name>mapred.local.dir</name>
<value>file:/hadoop/mapred/local</value>
<final>true</final>
</property>
</configuration>
#-------------------------------------------------------#

[hadoop@ipython hadoop]$ vi yarn-site.xml
#-------------------------------------------------------#
<configuration>

<property>
<name>yarn.nodemanager.aux-services</name>
<value>maprece_shuffle</value>
</property>

<property>
<name>yarn.nodemanager.aux-services.maprece.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
#-------------------------------------------------------#

6、启动HDFS
[hadoop@ipython hadoop]$ hdfs namenode -format
15/01/23 23:55:40 INFO namenode.FSImage: Saving image file /tools/hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression
15/01/23 23:55:40 INFO namenode.FSImage: Image file /tools/hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 198 bytes saved in 0 seconds.
15/01/23 23:55:40 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
15/01/23 23:55:40 INFO util.ExitUtil: Exiting with status 0
15/01/23 23:55:40 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ipython.me/10.211.55.40
************************************************************/
##Start All(namenode,datanode,yarn)###
[hadoop@ipython hadoop]$ cd $HADOOP_PREIFX/sbin
[hadoop@ipython sbin]$ start-all.sh
##Jps##
[hadoop@ipython sbin]$ jps
2656 Jps
2000 DataNode
2275 ResourceManager
1892 NameNode
2374 NodeManager
2141 SecondaryNameNode
访问HDFS NameNode Web

hadoop-hadoop-cluster

访问NameNode web UI

hadoop-namenode-info

访问 Resource Manager 接口

hadoop-node-manager

测试Hadoop

[hadoop@ipython hadoop]$ hdfs dfs -mkdir /user
[hadoop@ipython hadoop]$ hdfs dfs -put /tmp /test/logs
hadoop_test

热心网友时间：2022-04-12 02:49

CentOs安装hadoop

今年五月份学习hadoop时,看到一篇关于hadoop集群安装,并附带相关测试的博客,写得特别好,也很具体,本人亲测,运行顺畅,学习一个月之后,由于种种原因
,中间间隔些时间,这下又要搞了,赶紧翻开这篇博客:
http://www.cnblogs.com/xia520pi/archive/2012/05/16/2503949.html
很具有实用价值,这下要去操作服务器集群,或许需要更多的资料,和经验。

热心网友时间：2022-04-12 04:24

Hadoop的优缺点介绍：（一）优点：（一）高可靠性。Hadoop按位存储和处理数据的能力值得人们信赖;（二）高扩展性。Hadoop是在可用的计算机集簇间分配数据并完成计算任务的，这些集簇可以方便地扩展到数以千计的节点中。（三）高效性。Hadoop能够在节点之间动态地移动数据，并保证各个节点的动态平衡，因此处理速度非常快。（四）高容错性。Hadoop能够自动保存数据的多个副本，并且能够自动将失败的任务重新分配。（二）缺点：（一）不适合低延迟数据访问。（二）无法高效存储大量小文件。（三）不支持多用户写入及任意修改文件。了解开源相关，去LUPA社区看看吧。

热心网友时间：2022-04-12 06:15

不是 hadoop 怎么安装
centos

centos 是系统应该是
centos 系统里面如何安装 hadoop

热心网友时间：2022-04-12 08:23

CentOs安装hadoop_百度经验：
https://jingyan.baidu.com/article/219f4bf726acb1de442d38fd.html