remote access - Failed to remotely execute R script which loads library "rhdfs" -
i'm working on project using r-hadoop, , got problem.
i'm using jsch in java ssh remote hadoop pseudo-cluster, , here part of java code create connection.
/* create connection instance */ connection conn = new connection(hostname); /* connect */ conn.connect(); /* authenticate */ boolean isauthenticated = conn.authenticatewithpassword(username, password); if (isauthenticated == false) throw new ioexception("authentication failed."); /* create session */ session sess = conn.opensession(); //sess.execcommand("uname -a && date && uptime && who"); sess.execcommand("rscript -e 'args1 <- \"dell\"; args2 <- 1; source(\"/usr/local/r/mytest.r\")'"); //sess.execcommand("ls"); sess.waitforcondition(channelcondition.timeout, 50);
i tried several simple r scripts, , codes worked fine. when comes r-hadoop, r script stop running. if run rscript -e 'args1 <- "dell"; args2 <- 1; source("/usr/local/r/mytest.r")'
directly in remote server, works fine.
here got after taking hong ooi's suggestion: instead of using rscript, used following command:
sess.execcommand("r cmd batch --no-save --no-restore '--args args1=\"dell\" args2=1' /usr/local/r/mytest.r /usr/local/r/whathappened.txt");
and in whathappened.txt, got following error:
> args=(commandargs(true)) > for(i in 1:length(args)){ + eval(parse(text=args[[i]])) + } > source("/usr/local/r/main.r") > main(args1,args2) loading required package: rjava error : .onload failed in loadnamespace() 'rhdfs', details: call: fun(libname, pkgname) error: environment variable hadoop_cmd must set before loading package rhdfs error: package/namespace load failed 鈥榬hdfs鈥? execution halted
well, problem clearer. unfortunately, i'm pretty new linux, , have no idea how solve this.
well, solved problem this:
sess.execcommand("source /etc/profile; r cmd batch --no-save --no-restore '--args args1=\"dell\" args2=1' /usr/local/r/mytest.r /usr/local/r/whathappened.txt");
the problem caused environment. ssh remote hadoop cluster uses different environment, variables $hadoop_cmd not discovered. there multiple ways let ssh session know how pick environment variables.
in method, "source /etc/profile"
can tell sshed environment find environment virables.
Comments
Post a Comment