fs command in Hadoop

This is for Hadoop eco system like HDFS, Map reduce, Hive, Hbase, Pig, sqoop,sqoop2, Avro, solr, hcatalog, impala, Oozie, Zoo Keeper and Hadoop distribution like Cloudera, Hortonwork etc.
Posts: 162
Joined: Sat Sep 20, 2014 11:29 pm

fs command in Hadoop

Postby mohit123 » Wed Sep 24, 2014 12:45 am

What is the use of fs command in Hadoop?


Re: fs command in Hadoop

Postby Guest » Fri Jun 19, 2015 9:19 pm

The File System (FS) shell includes various shell-like commands that directly interact with the Hadoop Distributed File System (HDFS) as well as other file systems that Hadoop supports, such as Local FS, HFTP FS, S3 FS, and others. The FS shell is invoked by:
bin/hdfs dfs <args>

All FS shell commands take path URIs as arguments. The URI format is scheme://autority/path. For HDFS the scheme is hdfs, and for the Local FS the scheme is file. The scheme and authority are optional. If not specified, the default scheme specified in the configuration is used. An HDFS file or directory such as /parent/child can be specified as hdfs://namenodehost/parent/child or simply as /parent/child (given that your configuration is set to point to hdfs://namenodehost).

Most of the commands in FS shell behave like corresponding Unix commands. Differences are described with each of the commands. Error information is sent to stderr and the output is sent to stdout.


Usage: hdfs dfs -cat URI [URI …]

Copies source paths to stdout.

• hdfs dfs -cat hdfs://nn1.example.com/file1 hdfs://nn2.example.com/file2
• hdfs dfs -cat file:///file3 /user/hadoop/file4

Exit Code:
Returns 0 on success and -1 on error.


Usage: hdfs dfs -chgrp [-R] GROUP URI [URI …]

Change group association of files. With -R, make the change recursively through the directory structure. The user must be the owner of files, or else a super-user. Additional information is in the Permissions Guide.

Posts: 61
Joined: Sun Aug 30, 2015 8:02 am

Re: fs command in Hadoop

Postby snehalshah » Wed Sep 23, 2015 5:07 pm

fs will access hdfs file system. so use it before any file system command like copy, merge, ls etc...

hadoop fs -put a.txt a.txt

Return to “Hadoop and Big Data”

Who is online

Users browsing this forum: No registered users and 1 guest