Design and Document the end-to-end architecture Design and Document security, scaling, performance Guide Developers towards delivering the designed solution Standard practices for PoC to investigate new technologies
. Data engineer with strong programming experience in Python . Extensive experience related to processing frameworks such as Spark, Spark Streaming, Airflow, Hive, Sqoop, Kafka etc. . Experience with big data processing within cloud AWS S3.