Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
utile:hadoop_hbase [2015/09/25 12:40] Andrei Roșu-Cojocaru |
utile:hadoop_hbase [2015/09/28 02:09] (current) Andrei Roșu-Cojocaru [Plugin Hadoop pentru Eclipse] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== Instalare Hadoop 2.6.0 ====== | + | ====== Instalare Hadoop 2.6.1 ====== |
| Pentru ca framework-ul Hadoop să poată funcționa, este necesar ca în prealabil să se instaleze: | Pentru ca framework-ul Hadoop să poată funcționa, este necesar ca în prealabil să se instaleze: | ||
| Line 21: | Line 21: | ||
| The key fingerprint is: | The key fingerprint is: | ||
| 36:56:1b:da:48:03:69:66:d3:36:f5:e6:9c:f7:a7:9c student@aipi2015 | 36:56:1b:da:48:03:69:66:d3:36:f5:e6:9c:f7:a7:9c student@aipi2015 | ||
| - | </code>\\ Deși nu este recomandată folosirea de parole vide, adoptarea unui astfel de comportament în acest caz este dictată de faptul că, în caz contrat, această informație va fi solicitată de fiecare dată când se realizează o conexiune între framework-ul Hadoop și nodurile pe care le gestionează. | + | </code> Deși nu este recomandată folosirea de parole vide, adoptarea unui astfel de comportament în acest caz este dictată de faptul că, în caz contrar, această informație va fi solicitată de fiecare dată când se realizează o conexiune între framework-ul Hadoop și nodurile pe care le gestionează. |
| - se indică permisiunile de acces SSH pe mașina locală folosind cheia generată\\ <code> | - se indică permisiunile de acces SSH pe mașina locală folosind cheia generată\\ <code> | ||
| - | aipi2014@ubuntu:~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys | + | student@aipi2015:~$ cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys |
| </code> | </code> | ||
| - se testează comunicația folosind SSH printr-o conexiune la mașina locală cu utilizatorul curent; astfel, se stochează și cheia mașinii locale în fișierul ''known_host'' a utilizatorului curent\\ <code> | - se testează comunicația folosind SSH printr-o conexiune la mașina locală cu utilizatorul curent; astfel, se stochează și cheia mașinii locale în fișierul ''known_host'' a utilizatorului curent\\ <code> | ||
| - | aipi2014@ubuntu:~$ ssh localhost | + | student@aipi2015:~$ ssh localhost |
| The authenticity of host 'localhost (127.0.0.1)' can't be established. | The authenticity of host 'localhost (127.0.0.1)' can't be established. | ||
| - | ECDSA key fingerprint is bb:40:19:2c:64:82:15:d6:0f:a3:0e:a2:b2:a7:ae:5a. | + | ECDSA key fingerprint is 7a:0d:10:b0:d2:54:09:82:85:57:37:bd:7a:11:3a:d4. |
| Are you sure you want to continue connecting (yes/no)? yes | Are you sure you want to continue connecting (yes/no)? yes | ||
| Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts. | Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts. | ||
| - | Welcome to Ubuntu 14.10 (GNU/Linux 3.16.0-28-generic i686) | + | Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.19.0-25-generic x86_64) |
| * Documentation: https://help.ubuntu.com/ | * Documentation: https://help.ubuntu.com/ | ||
| Line 49: | Line 49: | ||
| Încărcarea unei astfel de configurații (în situația în care au fost realizate modificări) se face prin comanda\\ <code> | Încărcarea unei astfel de configurații (în situația în care au fost realizate modificări) se face prin comanda\\ <code> | ||
| - | aipi2014@ubuntu:~$ sudo /etc/init.d/ssh reload | + | student@aipi2015:~$ sudo /etc/init.d/ssh reload |
| </code> | </code> | ||
| Line 61: | Line 61: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ sudo nano /etc/sysctl.conf | + | student@aipi2015:~$ sudo nano /etc/sysctl.conf |
| </code> | </code> | ||
| <file bash sysctl.conf> | <file bash sysctl.conf> | ||
| Line 75: | Line 75: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ cat /proc/sys/net/ipv6/conf/all/disable_ipv6 | + | student@aipi2015:~$ cat /proc/sys/net/ipv6/conf/all/disable_ipv6 |
| </code> | </code> | ||
| Line 91: | Line 91: | ||
| * se descarcă cea mai recentă versiune stabilă (lista poate fi consultată pe [[http://hadoop.apache.org/releases.html#Download|Hadoop Releases]])\\ <code> | * se descarcă cea mai recentă versiune stabilă (lista poate fi consultată pe [[http://hadoop.apache.org/releases.html#Download|Hadoop Releases]])\\ <code> | ||
| - | aipi2014@ubuntu:~$ cd /tmp | + | student@aipi2015:~$ cd /tmp |
| - | aipi2014@ubuntu:/tmp$ wget http://mirrors.hostingromania.ro/apache.org/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz | + | student@aipi2015:/tmp$ wget http://mirrors.hostingromania.ro/apache.org/hadoop/common/hadoop-2.6.1/hadoop-2.6.1.tar.gz |
| </code> | </code> | ||
| * se copiază în /usr/local, creându-se și o legătură simbolică spre aceasta\\ <code> | * se copiază în /usr/local, creându-se și o legătură simbolică spre aceasta\\ <code> | ||
| - | aipi2014@ubuntu:/tmp$ tar xzf hadoop-2.6.0.tar.gz | + | student@aipi2015:/tmp$ tar xzf hadoop-2.6.1.tar.gz |
| - | aipi2014@ubuntu:/tmp$ sudo mv hadoop-2.6.0 /usr/local/ | + | student@aipi2015:/tmp$ sudo mv hadoop-2.6.1 /usr/local/ |
| - | aipi2014@ubuntu:/tmp$ sudo ln -s /usr/local/hadoop-2.6.0/ /usr/local/hadoop | + | student@aipi2015:/tmp$ sudo ln -s /usr/local/hadoop-2.6.1/ /usr/local/hadoop |
| - | aipi2014@ubuntu:/tmp$ rm -f hadoop-2.6.0.tar.gz | + | student@aipi2015:/tmp$ rm -f hadoop-2.6.1.tar.gz |
| </code> | </code> | ||
| * se specifică utilizatorul curent drept proprietarul acestui director\\ <code> | * se specifică utilizatorul curent drept proprietarul acestui director\\ <code> | ||
| - | aipi2014@ubuntu:~$ sudo chown -R aipi2014 /usr/local/hadoop | + | student@aipi2015:~$ sudo chown -R student /usr/local/hadoop |
| </code> | </code> | ||
| Line 109: | Line 109: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ mkdir $HOME/hdfs | + | student@aipi2015:~$ mkdir $HOME/hdfs |
| - | aipi2014@ubuntu:~$ mkdir $HOME/hdfs/name | + | student@aipi2015:~$ mkdir $HOME/hdfs/name |
| - | aipi2014@ubuntu:~$ mkdir $HOME/hdfs/data | + | student@aipi2015:~$ mkdir $HOME/hdfs/data |
| - | aipi2014@ubuntu:~$ mkdir $HOME/hdfs/tmp | + | student@aipi2015:~$ mkdir $HOME/hdfs/tmp |
| - | aipi2014@ubuntu:~$ sudo chmod -R 755 $HOME/hdfs | + | student@aipi2015:~$ sudo chmod -R 755 $HOME/hdfs |
| </code> | </code> | ||
| Line 121: | Line 121: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ sudo nano ~/.bashrc | + | student@aipi2015:~$ sudo nano ~/.bashrc |
| </code> | </code> | ||
| <file bash .bashrc> | <file bash .bashrc> | ||
| ... | ... | ||
| - | export JAVA_HOME=/usr/local/java/jdk1.8.0_25 | + | export JAVA_HOME=/usr/local/java/jdk1.8.0_60 |
| export HADOOP_INSTALL=/usr/local/hadoop | export HADOOP_INSTALL=/usr/local/hadoop | ||
| export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin | export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin | ||
| Line 138: | Line 138: | ||
| <note tip>Încărcarea acestor valori specificate pentru variabilele de sistem se face prin comanda:\\ <code> | <note tip>Încărcarea acestor valori specificate pentru variabilele de sistem se face prin comanda:\\ <code> | ||
| - | aipi2014@ubuntu:~$ source ~/.bashrc | + | student@aipi2015:~$ source ~/.bashrc |
| </code></note> | </code></note> | ||
| Line 146: | Line 146: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano hadoop-env.sh | + | student@aipi2015:/usr/local/hadoop/etc/hadoop$ sudo nano hadoop-env.sh |
| </code> | </code> | ||
| <file bash hadoop-env.sh> | <file bash hadoop-env.sh> | ||
| ... | ... | ||
| - | export JAVA_HOME=/usr/local/java/jdk1.8.0_25 | + | export JAVA_HOME=/usr/local/java/jdk1.8.0_60 |
| </file> | </file> | ||
| Line 160: | Line 160: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano core-site.xml | + | student@aipi2015:/usr/local/hadoop/etc/hadoop$ sudo nano core-site.xml |
| </code> | </code> | ||
| <file xml core-site.xml> | <file xml core-site.xml> | ||
| Line 170: | Line 170: | ||
| <property> | <property> | ||
| <name>hadoop.tmp.dir</name> | <name>hadoop.tmp.dir</name> | ||
| - | <value>/home/aipi2014/hdfs/tmp</value> | + | <value>/home/student/hdfs/tmp</value> |
| </property> | </property> | ||
| </configuration> | </configuration> | ||
| Line 180: | Line 180: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano yarn-site.xml | + | student@aipi2015:/usr/local/hadoop/etc/hadoop$ sudo nano yarn-site.xml |
| </code> | </code> | ||
| <file xml yarn-site.xml> | <file xml yarn-site.xml> | ||
| Line 200: | Line 200: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ cp mapred-site.xml.template mapred-site.xml | + | student@aipi2015:/usr/local/hadoop/etc/hadoop$ cp mapred-site.xml.template mapred-site.xml |
| - | aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ nano mapred-site.xml | + | student@aipi2015:/usr/local/hadoop/etc/hadoop$ nano mapred-site.xml |
| </code> | </code> | ||
| <file xml mapred-site.xml> | <file xml mapred-site.xml> | ||
| Line 220: | Line 220: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano hdfs-site.xml | + | student@aipi2015:/usr/local/hadoop/etc/hadoop$ sudo nano hdfs-site.xml |
| </code> | </code> | ||
| <file xml hdfs-site.xml> | <file xml hdfs-site.xml> | ||
| Line 226: | Line 226: | ||
| <property> | <property> | ||
| <name>dfs.name.dir</name> | <name>dfs.name.dir</name> | ||
| - | <value>/home/aipi2014/hdfs/name</value> | + | <value>/home/student/hdfs/name</value> |
| </property> | </property> | ||
| <property> | <property> | ||
| <name>dfs.data.dir</name> | <name>dfs.data.dir</name> | ||
| - | <value>/home/aipi2014/hdfs/data</value> | + | <value>/home/student/hdfs/data</value> |
| </property> | </property> | ||
| <property> | <property> | ||
| Line 244: | Line 244: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ hdfs namenode -format | + | student@aipi2015:~$ hdfs namenode -format |
| - | 14/12/17 14:15:34 INFO namenode.NameNode: STARTUP_MSG: | + | 15/09/26 04:54:19 INFO namenode.NameNode: STARTUP_MSG: |
| /************************************************************ | /************************************************************ | ||
| STARTUP_MSG: Starting NameNode | STARTUP_MSG: Starting NameNode | ||
| - | STARTUP_MSG: host = ubuntu/127.0.1.1 | + | STARTUP_MSG: host = aipi2015/127.0.1.1 |
| STARTUP_MSG: args = [-format] | STARTUP_MSG: args = [-format] | ||
| - | STARTUP_MSG: version = 2.6.0 | + | STARTUP_MSG: version = 2.6.1 |
| - | STARTUP_MSG: classpath = /usr/local/hadoop-2.6.0/etc/hadoop:/usr/local/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/hadoop/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/common/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/usr/local/hadoop/share/hadoop/common/hadoop-nfs-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.0.jar:/usr/local/hadoop/share/hadoop/hdfs:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/contrib/capacity-scheduler/*.jar | + | STARTUP_MSG: classpath = /usr/local/hadoop-2.6.1/etc/hadoop:/usr/local/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-annotations-2.6.1.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-auth-2.6.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.1.jar:/usr/local/hadoop/share/hadoop/common/hadoop-nfs-2.6.1.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.1-tests.jar:/usr/local/hadoop/share/hadoop/hdfs:/usr/local/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.1-tests.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/guice-3.0.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/asm-3.2.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/xz-1.0.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-registry-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-client-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-api-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-server-common-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-common-2.6.1.jar:/usr/local/hadoop-2.6.1/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.1-tests.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.1.jar:/contrib/capacity-scheduler/*.jar |
| - | STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z | + | STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r b4d876d837b830405ccdb6af94742f99d49f9c04; compiled by 'jenkins' on 2015-09-16T21:07Z |
| - | STARTUP_MSG: java = 1.8.0_25 | + | STARTUP_MSG: java = 1.8.0_60 |
| ************************************************************/ | ************************************************************/ | ||
| - | 14/12/17 14:15:35 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] | + | 15/09/26 04:54:19 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] |
| - | 14/12/17 14:15:35 INFO namenode.NameNode: createNameNode [-format] | + | 15/09/26 04:54:19 INFO namenode.NameNode: createNameNode [-format] |
| - | 14/12/17 14:15:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | + | 15/09/26 04:54:21 WARN common.Util: Path /home/student/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration. |
| - | 14/12/17 14:15:45 WARN common.Util: Path /home/aipi2014/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration. | + | 15/09/26 04:54:21 WARN common.Util: Path /home/student/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration. |
| - | 14/12/17 14:15:45 WARN common.Util: Path /home/aipi2014/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration. | + | Formatting using clusterid: CID-9fa0722d-3fbc-452a-b3a0-b4784d27a95b |
| - | Formatting using clusterid: CID-43be76c0-81e9-42b7-bae9-b210dbd132f6 | + | 15/09/26 04:54:21 INFO namenode.FSNamesystem: No KeyProvider found. |
| - | 14/12/17 14:15:45 INFO namenode.FSNamesystem: No KeyProvider found. | + | 15/09/26 04:54:21 INFO namenode.FSNamesystem: fsLock is fair:true |
| - | 14/12/17 14:15:46 INFO namenode.FSNamesystem: fsLock is fair:true | + | 15/09/26 04:54:21 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000 |
| - | 14/12/17 14:15:47 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000 | + | 15/09/26 04:54:21 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true |
| - | 14/12/17 14:15:47 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000 |
| - | 14/12/17 14:15:47 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000 | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: The block deletion will start around 2015 Sep 26 04:54:21 |
| - | 14/12/17 14:15:47 INFO blockmanagement.BlockManager: The block deletion will start around 2014 Dec 17 14:15:47 | + | 15/09/26 04:54:21 INFO util.GSet: Computing capacity for map BlocksMap |
| - | 14/12/17 14:15:47 INFO util.GSet: Computing capacity for map BlocksMap | + | 15/09/26 04:54:21 INFO util.GSet: VM type = 64-bit |
| - | 14/12/17 14:15:47 INFO util.GSet: VM type = 32-bit | + | 15/09/26 04:54:21 INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB |
| - | 14/12/17 14:15:47 INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB | + | 15/09/26 04:54:21 INFO util.GSet: capacity = 2^21 = 2097152 entries |
| - | 14/12/17 14:15:47 INFO util.GSet: capacity = 2^22 = 4194304 entries | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: defaultReplication = 1 |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: defaultReplication = 1 | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: maxReplication = 512 |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: maxReplication = 512 | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: minReplication = 1 |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: minReplication = 1 | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: maxReplicationStreams = 2 |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: maxReplicationStreams = 2 | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000 |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000 | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: encryptDataTransfer = false |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: encryptDataTransfer = false | + | 15/09/26 04:54:21 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000 |
| - | 14/12/17 14:15:50 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000 | + | 15/09/26 04:54:21 INFO namenode.FSNamesystem: fsOwner = student (auth:SIMPLE) |
| - | 14/12/17 14:15:51 INFO namenode.FSNamesystem: fsOwner = aipi2014 (auth:SIMPLE) | + | 15/09/26 04:54:21 INFO namenode.FSNamesystem: supergroup = supergroup |
| - | 14/12/17 14:15:51 INFO namenode.FSNamesystem: supergroup = supergroup | + | 15/09/26 04:54:21 INFO namenode.FSNamesystem: isPermissionEnabled = true |
| - | 14/12/17 14:15:51 INFO namenode.FSNamesystem: isPermissionEnabled = true | + | 15/09/26 04:54:21 INFO namenode.FSNamesystem: HA Enabled: false |
| - | 14/12/17 14:15:51 INFO namenode.FSNamesystem: HA Enabled: false | + | 15/09/26 04:54:21 INFO namenode.FSNamesystem: Append Enabled: true |
| - | 14/12/17 14:15:51 INFO namenode.FSNamesystem: Append Enabled: true | + | 15/09/26 04:54:22 INFO util.GSet: Computing capacity for map INodeMap |
| - | 14/12/17 14:15:56 INFO util.GSet: Computing capacity for map INodeMap | + | 15/09/26 04:54:22 INFO util.GSet: VM type = 64-bit |
| - | 14/12/17 14:15:56 INFO util.GSet: VM type = 32-bit | + | 15/09/26 04:54:22 INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB |
| - | 14/12/17 14:15:56 INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB | + | 15/09/26 04:54:22 INFO util.GSet: capacity = 2^20 = 1048576 entries |
| - | 14/12/17 14:15:56 INFO util.GSet: capacity = 2^21 = 2097152 entries | + | 15/09/26 04:54:22 INFO namenode.NameNode: Caching file names occuring more than 10 times |
| - | 14/12/17 14:16:00 INFO namenode.NameNode: Caching file names occuring more than 10 times | + | 15/09/26 04:54:22 INFO util.GSet: Computing capacity for map cachedBlocks |
| - | 14/12/17 14:16:00 INFO util.GSet: Computing capacity for map cachedBlocks | + | 15/09/26 04:54:22 INFO util.GSet: VM type = 64-bit |
| - | 14/12/17 14:16:00 INFO util.GSet: VM type = 32-bit | + | 15/09/26 04:54:22 INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB |
| - | 14/12/17 14:16:00 INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB | + | 15/09/26 04:54:22 INFO util.GSet: capacity = 2^18 = 262144 entries |
| - | 14/12/17 14:16:00 INFO util.GSet: capacity = 2^19 = 524288 entries | + | 15/09/26 04:54:22 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 |
| - | 14/12/17 14:16:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 | + | 15/09/26 04:54:22 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0 |
| - | 14/12/17 14:16:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0 | + | 15/09/26 04:54:22 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000 |
| - | 14/12/17 14:16:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000 | + | 15/09/26 04:54:22 INFO namenode.FSNamesystem: Retry cache on namenode is enabled |
| - | 14/12/17 14:16:01 INFO namenode.FSNamesystem: Retry cache on namenode is enabled | + | 15/09/26 04:54:22 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis |
| - | 14/12/17 14:16:01 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis | + | 15/09/26 04:54:22 INFO util.GSet: Computing capacity for map NameNodeRetryCache |
| - | 14/12/17 14:16:01 INFO util.GSet: Computing capacity for map NameNodeRetryCache | + | 15/09/26 04:54:22 INFO util.GSet: VM type = 64-bit |
| - | 14/12/17 14:16:01 INFO util.GSet: VM type = 32-bit | + | 15/09/26 04:54:22 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB |
| - | 14/12/17 14:16:01 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB | + | 15/09/26 04:54:22 INFO util.GSet: capacity = 2^15 = 32768 entries |
| - | 14/12/17 14:16:01 INFO util.GSet: capacity = 2^16 = 65536 entries | + | 15/09/26 04:54:22 INFO namenode.NNConf: ACLs enabled? false |
| - | 14/12/17 14:16:02 INFO namenode.NNConf: ACLs enabled? false | + | 15/09/26 04:54:22 INFO namenode.NNConf: XAttrs enabled? true |
| - | 14/12/17 14:16:02 INFO namenode.NNConf: XAttrs enabled? true | + | 15/09/26 04:54:22 INFO namenode.NNConf: Maximum size of an xattr: 16384 |
| - | 14/12/17 14:16:02 INFO namenode.NNConf: Maximum size of an xattr: 16384 | + | 15/09/26 04:54:22 INFO namenode.FSImage: Allocated new BlockPoolId: BP-819976783-127.0.1.1-1443232462466 |
| - | 14/12/17 14:16:07 INFO namenode.FSImage: Allocated new BlockPoolId: BP-756723125-127.0.1.1-1418818563112 | + | 15/09/26 04:54:22 INFO common.Storage: Storage directory /home/student/hdfs/name has been successfully formatted. |
| - | 14/12/17 14:16:07 INFO common.Storage: Storage directory /home/aipi2014/hdfs/name has been successfully formatted. | + | 15/09/26 04:54:22 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0 |
| - | 14/12/17 14:16:08 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0 | + | 15/09/26 04:54:22 INFO util.ExitUtil: Exiting with status 0 |
| - | 14/12/17 14:16:08 INFO util.ExitUtil: Exiting with status 0 | + | 15/09/26 04:54:22 INFO namenode.NameNode: SHUTDOWN_MSG: |
| - | 14/12/17 14:16:08 INFO namenode.NameNode: SHUTDOWN_MSG: | + | |
| /************************************************************ | /************************************************************ | ||
| - | SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1 | + | SHUTDOWN_MSG: Shutting down NameNode at aipi2015/127.0.1.1 |
| ************************************************************/ | ************************************************************/ | ||
| </code> | </code> | ||
| Line 320: | Line 319: | ||
| Script-urile pentru operațiile de pornire / oprire a proceselor Hadoop sunt plasate în ''$HADOOP_INSTALL/sbin'' și se referă la: | Script-urile pentru operațiile de pornire / oprire a proceselor Hadoop sunt plasate în ''$HADOOP_INSTALL/sbin'' și se referă la: | ||
| * sistemul de fișiere distribuit HDFS care gestionează procesele demon NameNode, DataNode și SecondaryDataNode\\ <code> | * sistemul de fișiere distribuit HDFS care gestionează procesele demon NameNode, DataNode și SecondaryDataNode\\ <code> | ||
| - | aipi2014@ubuntu:~$ start-dfs.sh | + | student@aipi2015:~$ start-dfs.sh |
| Starting namenodes on [localhost] | Starting namenodes on [localhost] | ||
| - | localhost: starting namenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-aipi2014-namenode-ubuntu.out | + | Starting namenodes on [localhost] |
| - | localhost: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-aipi2014-datanode-ubuntu.out | + | localhost: starting namenode, logging to /usr/local/hadoop-2.6.1/logs/hadoop-student-namenode-aipi2015.out |
| + | localhost: starting datanode, logging to /usr/local/hadoop-2.6.1/logs/hadoop-student-datanode-aipi2015.out | ||
| Starting secondary namenodes [0.0.0.0] | Starting secondary namenodes [0.0.0.0] | ||
| The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established. | The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established. | ||
| - | ECDSA key fingerprint is 36:dc:33:57:17:ce:fc:3d:d6:11:52:5e:b5:7f:24:7c. | + | ECDSA key fingerprint is 7a:0d:10:b0:d2:54:09:82:85:57:37:bd:7a:11:3a:d4. |
| Are you sure you want to continue connecting (yes/no)? yes | Are you sure you want to continue connecting (yes/no)? yes | ||
| 0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts. | 0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts. | ||
| - | 0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-aipi2014-secondarynamenode-ubuntu.out | + | 0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.6.1/logs/hadoop-student-secondarynamenode-aipi2015.out |
| </code> | </code> | ||
| * pornire: ''start-dfs.sh'' | * pornire: ''start-dfs.sh'' | ||
| * oprire: ''stop-dfs.sh'' | * oprire: ''stop-dfs.sh'' | ||
| * procesul de gestiune a resurselor YARN care gestionează procesele demon ResourceManager și NodeManager\\ <code> | * procesul de gestiune a resurselor YARN care gestionează procesele demon ResourceManager și NodeManager\\ <code> | ||
| - | aipi2014@ubuntu:~$ start-yarn.sh | + | student@aipi2015:~$ start-yarn.sh |
| starting yarn daemons | starting yarn daemons | ||
| - | starting resourcemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-aipi2014-resourcemanager-ubuntu.out | + | starting resourcemanager, logging to /usr/local/hadoop-2.6.1/logs/yarn-student-resourcemanager-aipi2015.out |
| - | localhost: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-aipi2014-nodemanager-ubuntu.out | + | localhost: starting nodemanager, logging to /usr/local/hadoop-2.6.1/logs/yarn-student-nodemanager-aipi2015.out |
| </code> | </code> | ||
| * pornire: ''start-yarn.sh'' | * pornire: ''start-yarn.sh'' | ||
| Line 345: | Line 345: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ jps | + | student@aipi2015:~$ jps |
| - | 4883 ResourceManager | + | 4272 SecondaryNameNode |
| - | 4566 DataNode | + | 4821 Jps |
| - | 5223 Jps | + | 3959 NameNode |
| - | 5193 NodeManager | + | 4073 DataNode |
| - | 4427 NameNode | + | 4410 ResourceManager |
| - | 4749 SecondaryNameNode | + | 4526 NodeManager |
| </code> | </code> | ||
| Line 357: | Line 357: | ||
| Framework-ul Hadoop pune la dispoziția utilizatorilor interfețe web ce permit monitorizarea cluster-ului: | Framework-ul Hadoop pune la dispoziția utilizatorilor interfețe web ce permit monitorizarea cluster-ului: | ||
| - | * procesul NameNode ([[http://localhost:50070|]]) oferă informații cu privire la sistemul de fișiere distribuit HDFS, putând fi consultată inclusiv structura acestuia și descărcate resurse din cadrul său;\\ {{ :utile:hadoop_namenode.png?nolink&700 }} | + | * procesul NameNode ([[http://localhost:50070|]]) oferă informații cu privire la sistemul de fișiere distribuit HDFS, putând fi consultată inclusiv structura acestuia și descărcate resurse din cadrul său;\\ {{ :utile:hadoop_hbase:hadoop_namenode.png?nolink }} |
| - | * procesul ResourceManager ([[http://localhost:8088|]]) conține date referitoare la sarcinile care au fost rulate, starea acestora, putând fi consultate și statistici cu privire la aplicațiile executate.\\ {{ :utile:hadoop_resourcemanager.png?nolink&700 }} | + | * procesul ResourceManager ([[http://localhost:8088|]]) conține date referitoare la sarcinile care au fost rulate, starea acestora, putând fi consultate și statistici cu privire la aplicațiile executate.\\ {{ :utile:hadoop_hbase:hadoop_resourcemanager.png?nolink&800 }} |
| - | ====== Instalare HBase 0.98.8 ====== | + | ====== Instalare HBase 1.1.2 ====== |
| ===== Obținerea celei mai recente versiuni ===== | ===== Obținerea celei mai recente versiuni ===== | ||
| * se descarcă cea mai recentă versiune stabilă (lista poate fi consultată pe [[http://www.apache.org/dyn/closer.cgi/hbase/|HBase Downloads]])\\ <code> | * se descarcă cea mai recentă versiune stabilă (lista poate fi consultată pe [[http://www.apache.org/dyn/closer.cgi/hbase/|HBase Downloads]])\\ <code> | ||
| - | aipi2014@ubuntu:~$ cd /tmp | + | student@aipi2015:~$ cd /tmp |
| - | aipi2014@ubuntu:/tmp$ wget http://mirrors.hostingromania.ro/apache.org/hbase/stable/hbase-0.98.8-hadoop2-bin.tar.gz | + | student@aipi2015:/tmp$ wget http://mirrors.hostingromania.ro/apache.org/hbase/1.1.2/hbase-1.1.2-bin.tar.gz |
| </code> | </code> | ||
| * se copiază în /usr/local, creându-se și o legătură simbolică spre aceasta\\ <code> | * se copiază în /usr/local, creându-se și o legătură simbolică spre aceasta\\ <code> | ||
| - | aipi2014@ubuntu:/tmp$ tar xzf hbase-0.98.8-hadoop2-bin.tar.gz | + | student@aipi2015:/tmp$ tar xzf hbase-1.1.2-bin.tar.gz |
| - | aipi2014@ubuntu:/tmp$ sudo mv hbase-0.98.8-hadoop2 /usr/local/ | + | student@aipi2015:/tmp$ sudo mv hbase-1.1.2 /usr/local/ |
| - | aipi2014@ubuntu:/tmp$ sudo ln -s /usr/local/hbase-0.98.8-hadoop2/ /usr/local/hbase | + | student@aipi2015:/tmp$ sudo ln -s /usr/local/hbase-1.1.2/ /usr/local/hbase |
| - | aipi2014@ubuntu:/tmp$ rm -f hbase-0.98.8-hadoop2-bin.tar.gz | + | student@aipi2015:/tmp$ rm -f hbase-1.1.2-bin.tar.gz |
| </code> | </code> | ||
| * se specifică utilizatorul curent drept proprietarul acestui director\\ <code> | * se specifică utilizatorul curent drept proprietarul acestui director\\ <code> | ||
| - | aipi2014@ubuntu:~$ sudo chown -R aipi2014 /usr/local/hbase | + | student@aipi2015:~$ sudo chown -R student /usr/local/hbase |
| </code> | </code> | ||
| Line 383: | Line 383: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/common/*.jar ./ | + | student@aipi2015:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/common/*.jar ./ |
| - | aipi2014@ubuntu:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/common/lib/*.jar ./ | + | student@aipi2015:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/common/lib/*.jar ./ |
| - | aipi2014@ubuntu:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/hdfs/*.jar ./ | + | student@aipi2015:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/hdfs/*.jar ./ |
| - | aipi2014@ubuntu:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/mapreduce/*.jar ./ | + | student@aipi2015:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/mapreduce/*.jar ./ |
| - | aipi2014@ubuntu:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/yarn/*.jar ./ | + | student@aipi2015:/usr/local/hbase/lib$ cp /usr/local/hadoop/share/hadoop/yarn/*.jar ./ |
| - | aipi2014@ubuntu:/usr/local/hbase/lib$ rm -f hadoop*2.2.0* | + | student@aipi2015:/usr/local/hbase/lib$ rm -f hadoop*2.5.1* |
| </code> | </code> | ||
| Line 400: | Line 400: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hbase/conf$ sudo nano hbase-env.sh | + | student@aipi2015:/usr/local/hbase/conf$ sudo nano hbase-env.sh |
| </code> | </code> | ||
| <file sh hbase-env.sh> | <file sh hbase-env.sh> | ||
| ... | ... | ||
| - | export JAVA_HOME=/usr/local/java/jdk1.8.0_25 | + | export JAVA_HOME=/usr/local/java/jdk1.8.0_60 |
| export HBASE_REGIONSERVERS=/usr/local/hbase/conf/regionservers | export HBASE_REGIONSERVERS=/usr/local/hbase/conf/regionservers | ||
| export HBASE_MANAGES_ZK=true | export HBASE_MANAGES_ZK=true | ||
| Line 423: | Line 423: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:/usr/local/hbase/conf$ nano hbase-site.xml | + | student@aipi2015:/usr/local/hbase/conf$ nano hbase-site.xml |
| </code> | </code> | ||
| <file xml hbase-site.xml> | <file xml hbase-site.xml> | ||
| Line 445: | Line 445: | ||
| <property> | <property> | ||
| <name>hbase.zookeeper.property.dataDir</name> | <name>hbase.zookeeper.property.dataDir</name> | ||
| - | <value>/home/aipi2014/hbase/zookeeper</value> | + | <value>/home/student/hbase/zookeeper</value> |
| </property> | </property> | ||
| </configuration> | </configuration> | ||
| Line 454: | Line 454: | ||
| Script-urile pentru operațiile de pornire / oprire a proceselor Hadoop sunt plasate în ''$HBASE_INSTALL/bin'' și gestionează procesele HMaster, HRegionServer ale sistemului de baze de date distribuite HBase și HQuorumPeer al procesului ZooKeeper. | Script-urile pentru operațiile de pornire / oprire a proceselor Hadoop sunt plasate în ''$HBASE_INSTALL/bin'' și gestionează procesele HMaster, HRegionServer ale sistemului de baze de date distribuite HBase și HQuorumPeer al procesului ZooKeeper. | ||
| * pornire\\ <code> | * pornire\\ <code> | ||
| - | aipi2014@ubuntu:~$ start-hbase.sh | + | student@aipi2015:~$ start-hbase.sh |
| - | localhost: starting zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-aipi2014-zookeeper-ubuntu.out | + | localhost: starting zookeeper, logging to /usr/local/hbase/bin/../logs/hbase-student-zookeeper-aipi2015.out |
| - | starting master, logging to /usr/local/hbase/bin/../logs/hbase-aipi2014-master-ubuntu.out | + | starting master, logging to /usr/local/hbase/bin/../logs/hbase-student-master-aipi2015.out |
| - | localhost: starting regionserver, logging to /usr/local/hbase/bin/../logs/hbase-aipi2014-regionserver-ubuntu.out | + | starting regionserver, logging to /usr/local/hbase/bin/../logs/hbase-student-1-regionserver-aipi2015.out |
| </code> | </code> | ||
| * oprire\\ <code> | * oprire\\ <code> | ||
| - | aipi2014@ubuntu:~$ stop-hbase.sh | + | student@aipi2015:~$ stop-hbase.sh |
| </code> | </code> | ||
| Line 466: | Line 466: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ jps | + | student@aipi2015:~$ jps |
| - | 4282 ResourceManager | + | 7089 HMaster |
| - | 7277 HQuorumPeer | + | 6167 SecondaryNameNode |
| - | 4489 NodeManager | + | 5976 DataNode |
| - | 3853 DataNode | + | 6360 ResourceManager |
| - | 7334 HMaster | + | 7032 HQuorumPeer |
| - | 7685 Jps | + | 5849 NameNode |
| - | 7541 HRegionServer | + | 6489 NodeManager |
| - | 4109 SecondaryNameNode | + | 7593 Jps |
| - | 3653 NameNode | + | 7180 HRegionServer |
| </code> | </code> | ||
| Line 482: | Line 482: | ||
| Sistemul de baze de date distribuit HBase permite gestiunea informațiilor stocate: | Sistemul de baze de date distribuit HBase permite gestiunea informațiilor stocate: | ||
| * în mod text, prin intermediul unei console (shell); | * în mod text, prin intermediul unei console (shell); | ||
| - | * vizual, prin intermediul unei interfețe web. | + | * vizual, prin intermediul unei interfețe web (neimplementată pentru versiunea 1.1.2). |
| ==== Consola (Shell) ==== | ==== Consola (Shell) ==== | ||
| Line 489: | Line 489: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~$ hbase shell | + | student@aipi2015:~$ hbase shell |
| - | 2014-10-10 23:07:59,084 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available | + | |
| HBase Shell; enter 'help<RETURN>' for list of supported commands. | HBase Shell; enter 'help<RETURN>' for list of supported commands. | ||
| Type "exit<RETURN>" to leave the HBase Shell | Type "exit<RETURN>" to leave the HBase Shell | ||
| - | 0.98.8-hadoop2, r6cfc8d064754251365e070a10a82eb169956d5fe, Fri Nov 14 18:26:29 PST 2014 | + | Version 1.1.2, rcc2b70cf03e3378800661ec5cab11eb43fafe0fc, Wed Aug 26 20:11:27 PDT 2015 |
| hbase(main):001:0> list | hbase(main):001:0> list | ||
| TABLE | TABLE | ||
| - | 0 row(s) in 0.1250 seconds | + | 0 row(s) in 0.4310 seconds |
| => [] | => [] | ||
| Line 506: | Line 505: | ||
| Prin intermediul interfeței web, disponibilă la [[http://localhost:60010|]], utilizatorul are la dispoziție doar operații de consultare a stării bazei de date sau a vizualizării tabelelor. | Prin intermediul interfeței web, disponibilă la [[http://localhost:60010|]], utilizatorul are la dispoziție doar operații de consultare a stării bazei de date sau a vizualizării tabelelor. | ||
| - | |||
| - | {{ :utile:hadoop_hbase.png?nolink&700 }} | ||
| ====== Plugin Hadoop pentru Eclipse ===== | ====== Plugin Hadoop pentru Eclipse ===== | ||
| - | Dezvoltarea aplicațiilor Hadoop / HDFS în Eclipse poate fi facilitată de folosirea unui plugin: | + | Dezvoltarea aplicațiilor Hadoop / HDFS în Eclipse poate fi facilitată de folosirea [[http://aipi2015.andreirosucojocaru.ro/hadoop-eclipse-plugin-2.6.1.jar|unui plugin]]. |
| - | + | ||
| - | * [[http://aipi2014.andreirosucojocaru.ro/hadoop-eclipse-kepler-plugin-2.2.0.jar|Hadoop 2.2.0]] | + | |
| - | * [[http://aipi2014.andreirosucojocaru.ro/hadoop-eclipse-kepler-plugin-2.6.0.jar|Hadoop 2.6.0]] | + | |
| Arhiva ''.jar'' corespunzătoare acestuia trebuie copiată în subdirectorul ''plugins'' al mediului de dezvoltare Eclipse pentru a putea fi folosit corespunzător. | Arhiva ''.jar'' corespunzătoare acestuia trebuie copiată în subdirectorul ''plugins'' al mediului de dezvoltare Eclipse pentru a putea fi folosit corespunzător. | ||
| Line 520: | Line 514: | ||
| <note important>Există posibilitatea de a exista unele probleme de compatibilitate cu anumite versiuni Hadoop / Eclipse. | <note important>Există posibilitatea de a exista unele probleme de compatibilitate cu anumite versiuni Hadoop / Eclipse. | ||
| - | Pentru remedierea acestuia, plugin-ul trebuie compilat din surse. | + | Pentru remedierea acestora, plugin-ul trebuie compilat din surse. |
| **1.** Se indică versiunile curente ale bibliotecilor Hadoop existente în versiunea instalată pe mașina locală, în fișierele | **1.** Se indică versiunile curente ale bibliotecilor Hadoop existente în versiunea instalată pe mașina locală, în fișierele | ||
| Line 531: | Line 525: | ||
| <code> | <code> | ||
| - | aipi2014@ubuntu:~/hadoop2x-eclipse-plugin$ cd src/contrib/eclipse-plugin | + | student@aipi2015:~/hadoop2x-eclipse-plugin$ cd src/contrib/eclipse-plugin |
| - | aipi2014@ubuntu:~/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin$ ant jar -Dversion=2.6.0 -Declipse.home=/opt/eclipse -Dhadoop.home=/usr/local/hadoop | + | student@aipi2015:~/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin$ ant jar -Dversion=2.6.1 -Declipse.home=/opt/eclipse -Dhadoop.home=/usr/local/hadoop |
| </code> | </code> | ||
| Arhiva ''.jar'' corespunzătoare plugin-ului obținut va fi plasată în ''~/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin''. | Arhiva ''.jar'' corespunzătoare plugin-ului obținut va fi plasată în ''~/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin''. | ||
| </note> | </note> | ||
| - | Se selectează ''Window'' → ''Open Perspective'' → ''Other...'' și din cadrul listei de perspective disponibile se selectează opțiunea Map/Reduce. | + | Se selectează //Window// → //Perspective// → //Open Perspective// → //Other...// și din cadrul listei de perspective disponibile se selectează opțiunea //Map/Reduce//. |
| - | {{ :utile:hadoop_plugin01.jpg?nolink }} | + | {{ :utile:hadoop_hbase:hadoop_plugin01.png?nolink }} |
| - | În bara de perspective va apărea secțiunea ''Map/Reduce Locations'' în care vor trebui configurați anumiți parametrii legați de localizarea framework-ului Hadoop, prin: | + | În bara de perspective va apărea secțiunea //Map/Reduce Locations// în care vor trebui configurați anumiți parametrii legați de localizarea framework-ului Hadoop, prin: |
| - click dreapta în zona de lucru a perspectivei; sau | - click dreapta în zona de lucru a perspectivei; sau | ||
| - | - accesarea pictogramei reprezentând un elefant și semnul + ({{:utile:hadoop_plugin_pictograma.jpg?nolink}}). | + | - accesarea pictogramei reprezentând un elefant și semnul + ({{:utile:hadoop_hbase:hadoop_icon.png?nolink}}). |
| - | {{ :utile:hadoop_plugin03a.jpg?nolink&750 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin02.png?nolink&750 }} |
| - | {{ :utile:hadoop_plugin03b.jpg?nolink&750 }} | + | |
| Definirea unei locații Hadoop presupune: | Definirea unei locații Hadoop presupune: | ||
| Line 554: | Line 547: | ||
| * indicarea utilizatorului care va dezvolta aplicații de tip Map/Reduce. | * indicarea utilizatorului care va dezvolta aplicații de tip Map/Reduce. | ||
| - | {{ :utile:hadoop_plugin02.jpg?nolink }} | + | {{ :utile:hadoop_hbase:hadoop_plugin03.png?nolink }} |
| + | |||
| + | În urma realizării acestor configurații, locația respectivă va fi disponibilă pentru a fi accesată în secțiunea //Map/Reduce Locations//: | ||
| + | |||
| + | {{ :utile:hadoop_hbase:hadoop_plugin04.png?nolink&750 }} | ||
| + | |||
| + | De asemenea, în secțiunea //Project Explorer// va fi disponibil un navigator pentru sistemul de fișiere HDFS (DFS Locations) care oferă posibilitatea interacțiunii cu sistemul de fișiere local prin operații de tip încărcare / descărcare de diferite resurse. | ||
| - | Astfel, în secțiunea Project Explorer va fi disponibil un navigator pentru sistemul de fișiere HDFS (DFS Locations) care oferă posibilitatea interacțiunii cu sistemul de fișiere local prin operații de tip încărcare / descărcare de diferite resurse. | + | {{ :utile:hadoop_hbase:hadoop_plugin05.png?nolink }} |
| - | {{ :utile:hadoop_plugin04.jpg?nolink }} | + | Configurarea unui proiect Map/Reduce se face prin //New// → //Project// → //Map/Reduce// → //Map/Reduce Project//, specificându-se și locația unde a fost instalat framework-ul Hadoop pe discul local (//Use Default Hadoop// → //Configure Hadoop install directory...//) |
| - | Configurarea unui proiect Map/Reduce se face prin ''New'' → ''Project'' → ''Map/Reduce'' → ''Map/Reduce Project'', specificându-se și locația unde a fost instalat framework-ul Hadoop pe discul local (''Use Default Hadoop'' → ''Configure Hadoop install directory...'') | + | {{ :utile:hadoop_hbase:hadoop_plugin06.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin05.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin07.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin06.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin08.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin07.jpg?nolink&600 }} | + | Ulterior, clasele specifice acestui tip de aplicaţii (clasa principală, respectiv clasele Mapper şi Reducer) se vor genera automat (//New// → //Other// → //Map/Reduce// de unde se selectează tipurile de clasă respective: //MapReduce Driver//, //Mapper// şi //Reducer//), utilizatorul completând numai conţinutul metodelor responsabile cu sarcinile executate pe fiecare cluster în parte şi cu prelucrarea rezultatelor finale pe baza informaţiilor provenite de la proces care a rulat o anumită parte a sarcinii. |
| - | Ulterior, clasele specifice acestui tip de aplicaţii (clasa principală, respectiv clasele Mapper şi Reducer) se vor genera automat (''New'' → ''Other'' → ''Map/Reduce'' de unde se selectează tipurile de clasă respective: ''MapReduce Driver'', ''Mapper'' şi ''Reducer''), utilizatorul completând numai conţinutul metodelor responsabile cu sarcinile executate pe fiecare cluster în parte şi cu prelucrarea rezultatelor finale pe baza informaţiilor provenite de la proces care a rulat o anumită parte a sarcinii. | + | {{ :utile:hadoop_hbase:hadoop_plugin09.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin08.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin10.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin09.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin11.png?nolink&600 }} |
| De asemenea, pentru clasele de tip ''Mapper'' / ''Reducer'', vor trebui specificate tipul de intrare şi tipul de ieşire prin indicarea clasei de bază parametrizate pe care o vor moșteni. | De asemenea, pentru clasele de tip ''Mapper'' / ''Reducer'', vor trebui specificate tipul de intrare şi tipul de ieşire prin indicarea clasei de bază parametrizate pe care o vor moșteni. | ||
| - | {{ :utile:hadoop_plugin10.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin12.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin11.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin13.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin12.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin14.png?nolink&600 }} |
| - | {{ :utile:hadoop_plugin13.jpg?nolink&600 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin15.png?nolink&600 }} |
| Întrucât se doreşte ca rezultatele procesărilor distribuite să fie afişate în cadrul consolei Eclipse, se va crea un fişier ''log4j.properties'' care va fi plasat în subdirectorul ''bin'' al proiectului respectiv: | Întrucât se doreşte ca rezultatele procesărilor distribuite să fie afişate în cadrul consolei Eclipse, se va crea un fişier ''log4j.properties'' care va fi plasat în subdirectorul ''bin'' al proiectului respectiv: | ||
| Line 594: | Line 593: | ||
| </file> | </file> | ||
| - | Rularea proiectului va trebui realizată în contexul Hadoop, astfel încât să fie invocate procesele care sunt responsabile cu distribuirea sarcinii, respectiv cu preluarea rezultatelor obţinute în urma prelucrărilor pe diferite sarcini. Astfel, din meniul contextual al proiectului va trebui să se specifice ''Run As'' → ''Run On Hadoop''. | + | Rularea proiectului va trebui realizată în contexul Hadoop, astfel încât să fie invocate procesele care sunt responsabile cu distribuirea sarcinii, respectiv cu preluarea rezultatelor obţinute în urma prelucrărilor pe diferite sarcini. Astfel, din meniul contextual al proiectului va trebui să se specifice //Run As// → //Run On Hadoop//. |
| - | {{ :utile:hadoop_plugin14.jpg?nolink&500 }} | + | {{ :utile:hadoop_hbase:hadoop_plugin16.png?nolink }} |
| Pe măsură ce este rulată aplicaţia, în consolă vor fi afişate actualizările privind sarcinile de tip map şi sarcinile de tip reduce, exprimate procentual. | Pe măsură ce este rulată aplicaţia, în consolă vor fi afişate actualizările privind sarcinile de tip map şi sarcinile de tip reduce, exprimate procentual. | ||
