Live instructor-led & Self-paced Online Certification Training Courses (Big Data, Hadoop, Spark) › Forums › Hadoop › How to create user in hadoop?
September 20, 2018 at 11:34 am #4660
I want to share my cluster with other team, I want to give them limited permissions they should not perform any admin task..
How to create user in hadoop with limited access?September 20, 2018 at 11:34 am #4661
 Create a New User
$ sudo adduser –ingroup
Do the following steps from the admin account
$ hadoop dfs –chmod -R 1777 /tmp
 In user machine, give write access to hadoop temp directory-
Now, give write permission to the user group on hadoop.tmp.dir.To get the path for hadoop.tmp.dir open core-site.xml ., And, this should be done only in the node (Machine), where the new user is added.
$ chmod 777 /path/to/hadoop/tmp/directory
 Now Create user home directory in HDFS-
For New user create a directory structure in HDFS.
$ hadoop dfs –mkdir /user/username/
 In HDFS,change the ownership of user home directory
Superuser has the ownership of newly created directory structure.But the new user will not be able to run MapReduce programs with this. So, to achieve this, change the ownership of newly created directory in HDFS to the new user.
$ hadoop dfs –chown –R username:groupname /user/username/
For more details, please follow: Hadoop TutorialSeptember 20, 2018 at 11:34 am #4662
Often while starting up the hadoop cluster admin might want to create new users to allow then access to HDFS and execute apps.
This can be done at the OS level and by assigning proper permissions to the users.- if Hadoop cluster is following the simple user account policy.
1) create an operating system group – OR use the existing default one if there is already a group.
2) Create an OS user and associate it with the group name needed.
useradd -g hadoopusers hsree
3) If wanted set password for the created user
4) Set the value of hadoop.tmp.dir in core-site.xml – which is the base directory for the hdfs data nodes and set the permissions to 777 – to make sure all users access it.
5) And create the directory structure ( home directory for the new user )
sudo -u hdfs dfs -mkdir /user/hsree/ sudo -u hdfs dfs -chown -R hsree:hadoopusers /user/hsree
6) You can refresh the user and the group mappings to have the namenode knows about the new user.
sudo -u hdfs hdfs dfsadmin -refreshUserToGroupsMappings
For more details, please follow:
http://data-flair.training/blogs/hadoop-hdfs-commands-tutorial/September 20, 2018 at 11:35 am #4663
The below answer explains Creating Multiple Users in Hadoop cluster?
What happens if we share the same user among multiple users?
If we share same hadoop user account among multiple users then we suffer from following problems:
- Difficult to trace the jobs and track the tasks/defects done by each user
- Shared account arises security issue.
- If all user have same user account, so all users will have the same privilege and all can
access everyone’s data, can modify it, can perform the execution, can delete it also.
What are the benefits of creating multiple users?
- The directories/files of one user cannot be modified by other user.
- To a user’s directory other user can not add new files..
- On the user’s files, other users will not have the privilage to perform any job (mapreduce etc).
Steps for setting up multiple User Accounts
I. Create a New User
Ubuntu: $ sudo adduser –ingroup Redhat: $ useradd -g All: $ passwd
II. Chanage permission of hadoop temp directory
Change the permission of a directory in HDFS where hadoop stores its temporary data.
Find that directory by looking entry hadoop.tmp.dir into core-site.xml
Then from the superuser account do the following step.
$ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory
III. Give write access to hadoop temp directory in user machine
The next step is to give write permission to our user group on hadoop.tmp.dir. Open core-site.xml to get the path for hadoop.tmp.dir. and, this should be done only in the node where the new user is added.
$ chmod 777 /path/to/hadoop/tmp/dirctory
IV. Create user home directory in HDFS
The next step is to create a directory structure in HDFS for the new user.
For that from the superuser, create a directory structure.
$ hadoop fs –mkdir /user/username/
V. Change the ownership of user home directory in HDFS
The ownership of the newly created directory structure is with superuser. With this new user
will not be able to run mapreduce programs. So change the ownership of newly created directory in HDFS to the new user.
$ hadoop fs –chown –R username:groupname
Eg: hadoop fs –chown –R username:groupname /user/username/
VI. Now run jobs with new user
Login with new user and run jobs
$ su – newuser $ bin/hadoop jar pathToJar.jar mainClass input/path output/path
You must be logged in to reply to this topic.