storm安装详解

单机版 

yum  install gcc  g++  make automake uuid-devel libtool
 

安装zookeeper

wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.7/zookeeper-3.4.7.tar.gz
tar -zxvf zookeeper-3.4.7.tar.gz
cd zookeeper-3.4.7
安装storm
wget http://www.apache.org/dyn/closer.lua/storm/apache-storm-0.10.0/apache-storm-0.10.0.tar.gz
tar -zxvf apache-storm-0.10.0.tar.gz
cd apache-storm-0.10.0
vim apache-storm-0.10.0/conf
修改配置文件
########### These MUST be filled in for a storm configuration
 storm.zookeeper.servers:
     - "localhost"
#     - "server2"
#
 nimbus.host: "192.168.65.128"
接着启动zkServer  nimbus,supervisor和ui几个服务:
 
启动
/opt/zookeeper-3.4.7/bin/zkServer.sh start
/opt/zookeeper-3.4.7/bin/zkServer.sh status  #查看启动状态
./bin/storm nimbus&
./bin/storm supervisor&
./bin/storm ui&

通过 http://ip:8080Storm Web UI看看Storm集群的状态

  在命令窗口输入: jps  可以查看各个进程的运行状态

[root@localhost ~]# jps
15076 core
15545 QuorumPeerMain
16026 Jps
15659 supervisor
15887 nimbus

stormjps

stormjps

上图中: 15076 core   对应的进程是Storm UI 16026 Jps   对应的进程是Java jps 15022 nimbus 对应的进程是Storm nimbus 15659 supervisor对应的进程是Storm supervisor 27380 worker     对应的线程是Storm main函数里面设置的 conf.setNumWorkers(15) ,所以本例中有15个worker。 15545 QuorumPeerMain对应的进程是zkServer.sh 其中在nimbus上nimbus、 core和 QuorumPeerMain 这三个进程必须启动,如果启动失败,需要研究一下日志中失败的原因,按照上述的顺序再启动一次。否则 会导致下一步的topology提交失败,切记。

安装示例

git clone git://github.com/apache/storm.git && cd storm/examples/storm-starter

安装pyleus

$ git clone https://github.com/Yelp/pyleus.git
$ pyleus build pyleus/examples/exclamation_topology/pyleus_topology.yaml
$ pyleus local exclamation_topology.jar
 

可能需要安装的组件

wget http://download.zeromq.org/zeromq-4.1.3.tar.gz

wget https://download.libsodium.org/libsodium/releases/libsodium-1.0.7.tar.gz

wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz


碰到问题

在执行 pyleus build pyleus/examples/exclamation_topology/pyleus_topology.yaml时报错:

pyleus build: error: [JarError] Base jar not found

这是由于是非pip安装,造成缺少pyleus-base.jar,在pyleus根目录执行make all,会生成该文件

然后执行

 python -c 'import pyleus; print pyleus.BASE_JAR_PATH'


查找python的依赖目录

将该文件复制到该目录

cp /opt/pyleus/pyleus/pyleus-base.jar /usr/lib/python2.6/site-packages/pyleus-0.3.0-py2.6.egg/pyleus
参考

http://blog.sina.com.cn/s/blog_5ca749810101c2dy.html

storm ui

storm ui

本类最新
热门标签