remote access - Failed to remotely execute R script which loads library "rhdfs" -


i'm working on project using r-hadoop, , got problem.

i'm using jsch in java ssh remote hadoop pseudo-cluster, , here part of java code create connection.

/* create connection instance */ connection conn = new connection(hostname); /* connect */ conn.connect(); /* authenticate */ boolean isauthenticated = conn.authenticatewithpassword(username, password); if (isauthenticated == false) throw new ioexception("authentication failed."); /* create session */ session sess = conn.opensession(); //sess.execcommand("uname -a && date && uptime && who"); sess.execcommand("rscript -e 'args1 <- \"dell\"; args2 <- 1; source(\"/usr/local/r/mytest.r\")'"); //sess.execcommand("ls"); sess.waitforcondition(channelcondition.timeout, 50); 

i tried several simple r scripts, , codes worked fine. when comes r-hadoop, r script stop running. if run rscript -e 'args1 <- "dell"; args2 <- 1; source("/usr/local/r/mytest.r")' directly in remote server, works fine.

here got after taking hong ooi's suggestion: instead of using rscript, used following command:

sess.execcommand("r cmd batch --no-save --no-restore '--args args1=\"dell\" args2=1' /usr/local/r/mytest.r /usr/local/r/whathappened.txt"); 

and in whathappened.txt, got following error:

> args=(commandargs(true)) > for(i in 1:length(args)){ +      eval(parse(text=args[[i]])) + } > source("/usr/local/r/main.r") > main(args1,args2) loading required package: rjava error : .onload failed in loadnamespace() 'rhdfs', details:   call: fun(libname, pkgname)   error: environment variable hadoop_cmd must set before loading package rhdfs error: package/namespace load failed 鈥榬hdfs鈥? execution halted 

well, problem clearer. unfortunately, i'm pretty new linux, , have no idea how solve this.

well, solved problem this:

sess.execcommand("source /etc/profile; r cmd batch --no-save --no-restore '--args args1=\"dell\" args2=1' /usr/local/r/mytest.r /usr/local/r/whathappened.txt"); 

the problem caused environment. ssh remote hadoop cluster uses different environment, variables $hadoop_cmd not discovered. there multiple ways let ssh session know how pick environment variables.

in method, "source /etc/profile" can tell sshed environment find environment virables.


Comments

Popular posts from this blog

javascript - Count length of each class -

What design pattern is this code in Javascript? -

hadoop - Restrict secondarynamenode to be installed and run on any other node in the cluster -