Data Engineer - Big Data - Chennai

Data Engineer - Big Data - Chennai

2COMS Consulting Private Limited

Job Description

Urgent hiring experienced professionals in a leading brand company


Location
: Chennai

Responsibilities:

  • As a Data Engineer, you will build a variety of big data analytics solutions, including big data lakes. More specifically, you will:
  • Design and build scalable data ingestion pipelines to handle real time streams, CDC events, and batch data
  • Execute high-performance data processing for structured and unstructured data, and data harmonization
  • Schedule, orchestrate, and validate pipelines
  • Design exception handling and log monitoring for debugging
  • Make tech stack and tools related decisions
  • Collaborate with business consultants, data scientists, and application developers to develop analytics solutions

Designation will be commensurate with expertise/experience. Compensation packages are among the best in the industry



Requirements

Hands-on experience with:

  • Hadoop ecosystem - HDFS, Hive, Sqoop, Kafka, ELK Stack etc
  • Spark, Scala, Python and core/advance Java
  • NOSQL databases e.g. Hbase, Cassandra, MongoDB
  • Relevant AWS or Azure components required to build big data solutions
  • Good to know: Databricks, Snowflake
  • Ability to develop and manage scalable Hadoop cluster environments
  • Good understanding of data warehousing concepts, distributed systems, data pipelines, ETL
  • 5-10 years of tech experience with at least 3 years in big data engineering


Similar Jobs

Career Advice to Find Better