Hadoop Administrator

Hadoop Administrator

Angel Broking
Mumbai India
1-3 years
Not Specified

Job Description


Group Company: Angel Broking Limited
Designation: Specialist - Hadoop Administrator (ABL_SS_614)
Position description:
  • Responsible for implementation and ongoing administration of Hadoop infrastructure.
  • Installing and setup of Hadoop clusters and its maintenance
  • Resource management in a cluster environment. This involves new node creation and removal of unused ones
  • Implementing and administering Hadoop infrastructure
  • Installation, configuration, performance tuning, supporting and managing clusters anywhere from 3 to 100+ nodes
  • Installing hardware and software in Hadoop environment. Expanding of existing environments
  • Manage large scale Hadoop environments, handle builds, including design, capacity planning, cluster setup, performance tuning and ongoing monitoring.
  • Evaluate and recommend systems software and hardware for the enterprise system including capacity modeling.
  • Architect our Hadoop infrastructure to meet changing requirements for scaling, reliability, performance and manageability.
  • Work with core production support personnel in IT and Engineering to automate deployment and operations of the infrastructure. Manage, deploy, and configure infrastructure with Ansible or other automation tool sets.
  • Create metrics and measures of utilization and performance.
  • Increase capacity by planning to implement new/upgraded hardware and software including storage infrastructure.
  • Ability to work well with a global team of highly motivated and skilled personnel - interaction and dialogue are requisites in our dynamic environment.
  • Research and recommend innovative solutions, and where possible, automated approaches for system administration tasks. Identify approaches that leverage our resources, provide economies of scale, and simplify remote/global support issues.
  • Performance tuning of Hadoop clusters and Hadoop MapReduce routines.
  • Screen Hadoop cluster job performances and capacity planning
  • Monitor Hadoop cluster connectivity and security
  • General Administration of Linux installation, Users Creation, Group Creation, Permissions and package management using YUM and RPM.
  • Screen Hadoop cluster job performances and Capacity Planning.
  • Troubleshooting, diagnosing, performance tuning and solving the Hadoop issues.
  • Configuring and maintaining Name and Space quotas of users and File System.
  • Manage and review Hadoop log files.File system management and monitoring. HDFS support and maintenance.
Primary Responsibilities:
  • Managing and Monitoring of hadoop cluster
Reporting TeamReporting Designation: Head - Big Data Engineering and Cloud Development (ABL_SS_414)Reporting Department: Application Development (2487)
Required Skills:
  • Deep understanding of all hadoop components and architecture
  • 1+ years of hadoop cluster administration
  • HBase administration or ability to learn
  • Experience with core JAVA or Python or any of the scripting language
  • Highly skilled on Hadoop Cluster Setup and Hadoop Administration.
  • 1+ years’ experience on Hadoop and its eco-system components HDFS, MapReduce, YARN, Sqoop, Flume, Pig, Oozie , Strom, Ranger, Kerberos, Hive, HBase, and ZooKeeper
  • Worked in 24x7 environment for production support in an on-call rotation
  • Experience in designing and building scalable infrastructure and platforms to collect and process very large amounts of structured and unstructured data.
  • Experience in adding and removing nodes, monitoring critical alerts, configuring high availability, configuring data backups and data purging.
  • Knowledge on configuring Kerberos for the authentication of users and Hadoop daemons.

Similar Jobs

Career Advice to Find Better