Send me more jobs like this

Scala Architect

Keywords / Skills : Spark, SCALA, Python, Java, ETL, Hive, Tableau, NoSQL, Java, Shell Scripting, Apache spark, Hbase, SCALA, Apache NIFI, hortonwork hadoop distribution, NiFi, Spark SQL, SCALA, HDFS, Ranger

10 - 15 years
Posted: 2019-06-24

Industry
IT/Computers - Software
Function
IT
Role
Software Engineer/ Programmer
Posted On
24th Jun 2019
Job Description
Job Description

Project Platform - Hortonworks Hadoop distribution, NiFi, Spark and Spark SQL, Scala, Java, Python, SQL, Shell scripting, Hive, HBase, HDFS, Ranger, Tableau

· Looking for an extensive IT industry experience for about 10 to 15 years.

· Should have good experience in Team handling and will be directly reporting to our country head.

· Should have good exposure in client handling & should be an individual contributor in delivering the projects within the stipulated timeline.

· Create data integration pipelines to extract, cleanse, and integrate data from a variety of sources and formats for analysis and use across use cases.

· Perform data profiling, discovery, and analysis to identify/determine location, suitability and coverage of data, and identify the various data types, formats, and data quality which exist within a given data source.

· Experience in large-scale, distributed system design and development with low latency.

· Create re-usable data extraction/ingestion pipelines and templates to demonstrate the logical flow and manipulation of data required to move data from customer source systems into the target data lake, warehouse, and/or sandbox.

· Perform hands-on data development to build the data pipeline using Nifi to extract the data from RDBMS and uploaded to hdfs and hive.

· Automate the incremental data extraction & storage process into hdfs using Nifi.

· Should be conversant with Apache Spark architecture, RDDs, various transformations and actions, spark configuration and tuning techniques.

· Should have an excellent experience using ScalaSpark with Spark RDDs, Spark SQL DataFrames.

· Good understanding of the map-reduce paradigm and able to develop Spark applications/ map reduce jobs.

· Management of Hadoop cluster, with all included services.

· Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala.

· Experience with NoSQL databases, such as HBase, Cassandra, MongoDB.

· Provide mentoring of customer resources and other consultants.

About Company

we are a recruitment agency.
Similar Jobs
View All Similar Jobs


Walkin for you