Send me more jobs like this

Sorry, This job is expired.

Lead Data Engineer

Keywords / Skills : Spark, Kafka, , AZURE, Nifi, Apache, RESTful, Sqoop, NoSQL, Kylo, Confluent, Elastic Search, Lambda, Druid, Bigquery

4 - 8 years
Posted: 2019-03-29

Job Description
Key skills

Apache NiFi, Kafka Connect (Confluent), Sqoop, Kylo, Spark, Druid, Presto, RESTful services, Lambda / Kappa architectures


- Build a scalable, reliable, operable and performant big data platform for both streaming and batch analytics

- Design and implement data aggregation, cleansing and transformation layers


- 4+ years of hands-on experience designing and operating large data platforms

- Experience in Big data Ingestion, Transformation and stream / batch processing technologies using Apache NiFi, Apache Kafka, Kafka Connect (Confluent), Sqoop, Spark, Storm, Hive etc;

- Experience in designing and building streaming data platforms in Lambda, Kappa architectures

- Should have working experience in one of NoSQL, OLAP data stores like Druid, Cassandra, Elasticsearch, Pinot etc;

- Experience in one of data warehousing tools like RedShift, BigQuery, Azure SQL Data Warehouse

- Exposure to other Data Ingestion, Data Lake and querying frameworks like Marmaray, Kylo, Drill, Presto

- Experience in designing and consuming microservices

- Exposure to security and governance tools like Apache Ranger, Apache Atlas

- Any contributions to open source projects a plus

- Experience in performance benchmarks will be a plus

About Company

CL is an HRTech & Consulting firm based at Gurgaon.
We specialized in domains like Data Analytics, Machine Learning, Artificial Intelligence & IOT.
We focus on E-commerce & Education Industry.
Similar Jobs
View All Similar Jobs
Walkin for you