How to create user in hadoop?

[DataFlair Team]

I want to share my cluster with other team, I want to give them limited permissions they should not perform any admin task..
How to create user in hadoop with limited access?

[DataFlair Team]

[1] Create a New User

$ sudo adduser –ingroup

[2] Now change the permission of Apache hadoop temp directory –
Change the permission of a directory in HDFS (Hadoop Distributed FileSystem) where hadoop stores its temporary data.

Do the following steps from the admin account

$ hadoop dfs –chmod -R 1777 /tmp

[3] In user machine, give write access to hadoop temp directory-
Now, give write permission to the user group on hadoop.tmp.dir.To get the path for hadoop.tmp.dir open core-site.xml ., And, this should be done only in the node (Machine), where the new user is added.

$ chmod 777 /path/to/hadoop/tmp/directory

[4] Now Create user home directory in HDFS-
For New user create a directory structure in HDFS.

$ hadoop dfs –mkdir /user/username/

[5] In HDFS,change the ownership of user home directory 
Superuser has the ownership of newly created directory structure.But the new user will not be able to run MapReduce programs with this. So, to achieve this, change the ownership of newly created directory in HDFS to the new user.

$ hadoop dfs –chown –R username:groupname /user/username/

For more details, please follow: Hadoop Tutorial

[DataFlair Team]

Often while starting up the hadoop cluster admin might want to create new users to allow then access to HDFS and execute apps.

This can be done at the OS level and by assigning proper permissions to the users.- if Hadoop cluster is following the simple user account policy.

1) create an operating system group – OR use the existing default one if there is already a group.

groupadd hadoopusers

2) Create an OS user and associate it with the group name needed.
useradd -g hadoopusers hsree

3) If wanted set password for the created user
passwd hsree

4) Set the value of hadoop.tmp.dir in core-site.xml – which is the base directory for the hdfs data nodes and set the permissions to 777 – to make sure all users access it.

5) And create the directory structure ( home directory for the new user )

sudo -u hdfs dfs -mkdir /user/hsree/
sudo -u hdfs dfs -chown -R hsree:hadoopusers /user/hsree

6) You can refresh the user and the group mappings to have the namenode knows about the new user.

sudo -u hdfs hdfs dfsadmin -refreshUserToGroupsMappings

For more details, please follow:
http://data-flair.training/blogs/hadoop-hdfs-commands-tutorial/

[DataFlair Team]

The below answer explains Creating Multiple Users in Hadoop cluster?
What happens if we share the same user among multiple users?

If we share same hadoop user account among multiple users then we suffer from following problems:

• Difficult to trace the jobs and track the tasks/defects done by each user
• Shared account arises security issue.
• If all user have same user account, so all users will have the same privilege and all can
  access everyone’s data, can modify it, can perform the execution, can delete it also.

What are the benefits of creating multiple users?
________________________________________

• The directories/files of one user cannot be modified by other user.
• To a user’s directory other user can not add new files..
• On the user’s files, other users will not have the privilage to perform any job (mapreduce etc).

Steps for setting up multiple User Accounts
I. Create a New User

Ubuntu: $ sudo adduser –ingroup
Redhat: $ useradd -g
All: $ passwd

II. Chanage permission of hadoop temp directory
Change the permission of a directory in HDFS where hadoop stores its temporary data.
Find that directory by looking entry hadoop.tmp.dir into core-site.xml
Then from the superuser account do the following step.
$ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory

III. Give write access to hadoop temp directory in user machine
The next step is to give write permission to our user group on hadoop.tmp.dir. Open core-site.xml to get the path for hadoop.tmp.dir. and, this should be done only in the node where the new user is added.
$ chmod 777 /path/to/hadoop/tmp/dirctory

IV. Create user home directory in HDFS
The next step is to create a directory structure in HDFS for the new user.
For that from the superuser, create a directory structure.
$ hadoop fs –mkdir /user/username/

V. Change the ownership of user home directory in HDFS
The ownership of the newly created directory structure is with superuser. With this new user
will not be able to run mapreduce programs. So change the ownership of newly created directory in HDFS to the new user.
$ hadoop fs –chown –R username:groupname
Eg: hadoop fs –chown –R username:groupname /user/username/

VI. Now run jobs with new user
Login with new user and run jobs

$ su – newuser
$ bin/hadoop jar pathToJar.jar mainClass input/path output/path

[Author]

Posts