Data Engineer - Consulting Firm

Data Engineer

Location: Gurgaon

Experience: 3.0 - 5.0 Years

Must-Have:

- Bachelor's degree in a quantitative field (e.g., Engineering, Mathematics, Operations Research, Statistics, etc.) from Tier 1/2 institute.

- Experience in Big data Technologies like Hands-on with Scala, Apache Spark, Hadoop, HDFS, MLlib, Airflow

- Experience in AWS

Desired background/skills:

Background:

- Bachelor degree in a quantitative field (e.g., Engineering, Mathematics, Operations Research, Statistics, etc.) from Tier 1/2 colleges.

- 4 or more years of professional experience in areas like development of data pipelines, implementation of data lakes, setting up of cloud ecosystem, development of ETL tools etc.

- Previous exposure of working with clickstream data coming from Adobe analytics and Google analytics will be given extra consideration.

- Previous experience of working directly with clients in US or Europe is a must

Required Skills:

- Strong AWS Experience. Experience of working on different components of the ecosystem.

- A certification would be helpful.

- Experience with the following aspect of data engineering

- Building and optimizing - big data- data pipelines, architectures and data sets.

- Manipulating, processing, and extracting value from large disconnected datasets

- Development of the data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.

- Knowledge and working experience of Spark/Scala

- Good experience with at least one ETL tool like Talend, Informatica etc.

- Build processes supporting data transformation, data structures, metadata, dependency and workload management

- Expertise in SQL and Relational Databases.

- Expertise in at least one of the NoSQL databases.