Lead Engineer - Solution Architect / Data Science

Lead Engineer - Solution Architect / Data Science

IDC Technologies Solutions India Private Limited
Pune
10 - 12 Years
Not Specified

Job Description


If you desire to be part of something special, to be part of a winning team, to be part of a fun team - winning is fun. We are looking forward to a Lead Engineer - Solution Architect based in Pune, India . In Eaton, making our work exciting, engaging, meaningful; ensuring safety, health, wellness; and being a model of inclusion & diversity are already embedded in who we are - it's in our values, part of our vision, and our clearly defined aspirational goals. This exciting role offers opportunity to:

  • The candidate will demonstrate exceptional impact in delivering projects in terms of architecture, technical deliverables and project delivery throughout the project lifecycle. The candidate is expected to be conversant with Agile methodologies and tools and have a track record of delivering products in a production environment
  • Work with a team of experts in deep learning, machine learning, distributed systems, program management, and product teams, and work on all aspects of design, development and delivery of deep learning enabled end-to-end pipelines and solutions.
  • Lead the development of technical solutions and implement architectures for project and products across data engineering and data science teams
  • Will be able to work in a hands-on fashion with Big Data tools such as Kafka, Cassandra, Hadoop, Hive, Time Series Databases such as InfluxDB, KairosDB
  • Knowledge with microservices, cloud APIS (e.g. AWS and MS Azure)
  • Evaluate business requirements to determine potential solutions
  • Work with your team and others, defining the architecture, design, and management of secure, large-scale, highly-distributed, geo-redundant, multi-tenant data stores.
  • Recommend and set up appropriate performance monitoring solutions; author and implement based on the results
  • Is accountable for end-to end delivery of solutions from requirements gathering to production
  • Author high-quality, highly-performance, unit-tested code to extract and transform data based on business and data science needs
  • Work directly with stakeholders, engineering, and test to create high quality solutions that solve end-user problems.
  • Mentor others in the use of tools and techniques
  • Develop and execute agile work plans for iterative and incremental project delivery, CI and dev/ops
  • Explore and recommend new tools and processes which can be leveraged across the data preparation pipeline for capabilities and efficiencies
  • Collaborate broadly across multiple functions (data science, engineering, product management, IT, etc.) to readily make key data readily available and easily consumable

Required Candidate profileQualifications
Requirement:
  • Master's degree or Ph.D. (preferred) in computer science, software engineering, digital signal processing or related field.
  • Knowledge of ETL data pipelines, Azure DataFactory, AWS DataPipelines, Oozie, etc.
  • Experience in at least one cloud provider (AWS, Azure, Google)
  • Understanding of devops, provisioning and automation of cloud deployments.
  • Experience in Spark data processing, either batch or streaming.
  • Experience in testing in Spark
  • Programming experience in one of Python or Scala
  • 10+ years of progressive experience in delivering technology solutions in a production environment
  • 6+ years of experience in the software industry as a developer, with a proven track record of shipping high quality products
  • 4 years working with customers (internal and external) on developing requirements and working as a solutions architect to deliver end-to-end systems to customers in a production environment
  • Excellent communication (verbal, presentation, documentation) skills, working with teams that are geographically dispersed, to produce solutions that satisfy functional and non-functional requirements
  • Ability to specify and write code that is accessible, secure, and performs in an optimized manner with an ability to output to different types of consumers and systems
  • Strong knowledge of big data query tools to perform ad hoc queries of large datasets
  • Solid understanding of relational and non-relational (NoSQL, time-series) database systems
  • Experience with in-memory, file-based and other data stores
  • Solid understanding of Java and/or Python and associated IDE's (Eclipse, IntelliJ, etc.)
  • Extensive experience with Agile development methodologies and concepts
  • Strong problem solving and software debugging skills
  • Experience building APIs to support data consumption needs of other roles

Excellent verbal andwritten communication skills including the ability to effectively explaintechnical concepts
Abreast of upcomingsoftware development/engineering tools, trends, and methodologies
Good judgment, timemanagement, and decision-making skills
Knowledge of clouddevelopment platforms such as Azure or AWS and their associated data storageoptions
Knowledge ofstreaming technologies like Apache Kafka, AWS Kinesis, Azure EventHubs
Knowledge of designand deployment of a Data Lake
Knowledge of ClouderaHadoop
Knowledge of IoTtechnologies, including cloud processing, like Azure IoT Hub.
Knowledge of dataanalysis tools, like Apache Presto, Hive, Azure Data Lake Analytics, AWSAthena, Zeppelin
Experience in DesignThinking or human-centered methods to identify and creatively solve customerneeds, through a holistic understanding of customer's problem area
Advanced degreeand/or specialization in related descipline (e.g. machine learning)
Knowledgeable inleveraging multiple data transit protocols and technologies (MQTT, Rest API,JDBC, etc)
Knowledge of Hadoopand MapReduce /Spark or related frameworks
Knowledge of MongoDB,Document DB, CosmosDB
Knowledge of Scala

IDC Technologies Solutions India Private Limited

People Also Considered

Career Advice to Find Better