最具影响力的数字化技术在线社区

merryluo 发表于 2015-9-7 19:45:17

用spark-submit运行程序是出错,求指导

我用的是王家林《大数据Sparkq企业级实战》,照着该书写程序到第四章最后时出了问题,不晓得是什么原因~
这是第四章156页,使用spark-submit运行编译后的.jar文件的的一个例子:root@Ubuntu:/usr/local/spark/spark-1.0.0-bin-hadoop1/bin# ./spark-submit --master spark://Ubuntu:7077 --class com.spark.firstApp.HelloSpark --executor memory 1g ~/IdeaProjects/FirstSparkApp/out/artifacts/FirstSparkAppJar/FirstSparkApp.jarhdfs://SparkMaster:9000/data/SogouQ.mini hdfs://SparkMaster:9000/data/SogouQResult
运行后的错误15/09/06 23:45:11 WARN client.AppClient$ClientActor: Could not connect to akka.tcp://sparkMaster@Ubuntu:7077: akka.remote.EndpointAssociationException: Association failed with 15/09/06 23:45:11 INFO storage.MemoryStore: ensureFreeSpace(31527) called with curMem=0, maxMem=311387750
15/09/06 23:45:11 INFO storage.MemoryStore: Block broadcast_0 stored as values to memory (estimated size 30.8 KB, free 296.9 MB)
15/09/06 23:45:12 INFO ipc.Client: Retrying connect to server: Ubuntu/192.168.64.139:9000. Already tried 0 time(s).
15/09/06 23:45:13 INFO ipc.Client: Retrying connect to server: Ubuntu/192.168.64.139:9000. Already tried 1 time(s).
Caused by: java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)


作为一个新生,只能看出是链接不上 akka.tcp://sparkMaster@Ubuntu:7077 具体什么原因搞不清楚,希望大家多提意见!非常感谢!!


乔帮主 发表于 2015-9-8 08:06:44

王家林老师 回复:请重启下Spark集群
页: [1]
查看完整版本: 用spark-submit运行程序是出错,求指导