This topic contains 3 replies, has 1 voice, and was last updated by  dfbdteam3 1 year, 6 months ago.

Viewing 4 posts - 1 through 4 (of 4 total)
  • Author
    Posts
  • #4660

    dfbdteam3
    Moderator

    I want to share my cluster with other team, I want to give them limited permissions they should not perform any admin task..
    How to create user in hadoop with limited access?

    #4661

    dfbdteam3
    Moderator

    [1] Create a New User

    $ sudo adduser –ingroup

    [2] Now change the permission of Apache hadoop temp directory –
    Change the permission of a directory in HDFS (Hadoop Distributed FileSystem) where hadoop stores its temporary data.

    Do the following steps from the admin account

    $ hadoop dfs –chmod -R 1777 /tmp

    [3] In user machine, give write access to hadoop temp directory-
    Now, give write permission to the user group on hadoop.tmp.dir.To get the path for hadoop.tmp.dir open core-site.xml ., And, this should be done only in the node (Machine), where the new user is added.

    $ chmod 777 /path/to/hadoop/tmp/directory

    [4] Now Create user home directory in HDFS-
    For New user create a directory structure in HDFS.

    $ hadoop dfs –mkdir /user/username/

    [5] In HDFS,change the ownership of user home directory 
    Superuser has the ownership of newly created directory structure.But the new user will not be able to run MapReduce programs with this. So, to achieve this, change the ownership of newly created directory in HDFS to the new user.

    $ hadoop dfs –chown –R username:groupname /user/username/

    For more details, please follow: Hadoop Tutorial

    #4662

    dfbdteam3
    Moderator

    Often while starting up the hadoop cluster admin might want to create new users to allow then access to HDFS and execute apps.

    This can be done at the OS level and by assigning proper permissions to the users.- if Hadoop cluster is following the simple user account policy.

    1) create an operating system group – OR use the existing default one if there is already a group.

    groupadd hadoopusers

    2) Create an OS user and associate it with the group name needed.
    useradd -g hadoopusers hsree

    3) If wanted set password for the created user
    passwd hsree

    4) Set the value of hadoop.tmp.dir in core-site.xml – which is the base directory for the hdfs data nodes and set the permissions to 777 – to make sure all users access it.

    5) And create the directory structure ( home directory for the new user )

    sudo -u hdfs dfs -mkdir /user/hsree/
    sudo -u hdfs dfs -chown -R hsree:hadoopusers /user/hsree

    6) You can refresh the user and the group mappings to have the namenode knows about the new user.

    sudo -u hdfs hdfs dfsadmin -refreshUserToGroupsMappings

    For more details, please follow:
    http://data-flair.training/blogs/hadoop-hdfs-commands-tutorial/

    #4663

    dfbdteam3
    Moderator

    The below answer explains Creating Multiple Users in Hadoop cluster?
    What happens if we share the same user among multiple users?

    If we share same hadoop user account among multiple users then we suffer from following problems:

    • Difficult to trace the jobs and track the tasks/defects done by each user
    • Shared account arises security issue.
    • If all user have same user account, so all users will have the same privilege and all can
      access everyone’s data, can modify it, can perform the execution, can delete it also.

    What are the benefits of creating multiple users?
    ________________________________________

    • The directories/files of one user cannot be modified by other user.
    • To a user’s directory other user can not add new files..
    • On the user’s files, other users will not have the privilage to perform any job (mapreduce etc).

    Steps for setting up multiple User Accounts
    I. Create a New User

    Ubuntu: $ sudo adduser –ingroup
    Redhat: $ useradd -g
    All: $ passwd

    II. Chanage permission of hadoop temp directory
    Change the permission of a directory in HDFS where hadoop stores its temporary data.
    Find that directory by looking entry hadoop.tmp.dir into core-site.xml
    Then from the superuser account do the following step.
    $ hadoop fs –chmod -R 1777 /path/to/hadoop/tmp/directory

    III. Give write access to hadoop temp directory in user machine
    The next step is to give write permission to our user group on hadoop.tmp.dir. Open core-site.xml to get the path for hadoop.tmp.dir. and, this should be done only in the node where the new user is added.
    $ chmod 777 /path/to/hadoop/tmp/dirctory

    IV. Create user home directory in HDFS
    The next step is to create a directory structure in HDFS for the new user.
    For that from the superuser, create a directory structure.
    $ hadoop fs –mkdir /user/username/

    V. Change the ownership of user home directory in HDFS
    The ownership of the newly created directory structure is with superuser. With this new user
    will not be able to run mapreduce programs. So change the ownership of newly created directory in HDFS to the new user.
    $ hadoop fs –chown –R username:groupname
    Eg: hadoop fs –chown –R username:groupname /user/username/

    VI. Now run jobs with new user
    Login with new user and run jobs

    $ su – newuser
    $ bin/hadoop jar pathToJar.jar mainClass input/path output/path
Viewing 4 posts - 1 through 4 (of 4 total)

You must be logged in to reply to this topic.