Microsoft Hiring - Data Engineer

Qualification:

BS/MS in Computer Science, Engineering, or related technical field
Strong Experience with Python, C#, test-driven development, refactoring and object-oriented development.
Experience with data aggregation platforms based on technologies such as Spark, SQL, Azure Data Lake, Hadoop, Cosmos, etc.
Solid understanding and proven skills in raw and processed stream design,
relational database design and dimensional models
Experience using scalable data ingestion and transformation systems in batch and near real time environments.
Experience with SQL, relational modelling, and big data tools such as Cosmos, Hive, Scope or Spark
Experience with ETL, data modelling, and working with Business Intelligence
systems.
Solid understanding of event processing including publish/subscribe mechanisms.
Demonstrated ability to create and ship high quality code by using engineering best practices.

Responsibilities:

You will work with design and build automated systems with pipelines,
transformations, data quality analytics and control logic.
Contribute to the design and development of data platform components to
validate and improve data quality, detect anomalies, and predict missing values.
Generate, process, log and correlate events across multiple sources.
Work with engineering teams and data scientists to integrate insights with
automation and deliver near term business impact.
Use engineering best practices throughout the software development lifecycle to establish maintainable, reliable, and secure systems
Work in a culture of continuous improvement, adaptation, reflection, and growth
Drive end-to-end features that touch multiple engineering aspects: designing, implementing, integrating, testing, and deploying.

Preferred Qualification:

Ability to understand complex distributed systems and to know your way around them.
Experience with machine learning and predictive analytics is a plus.
Strong scripting skills to perform data/file manipulation – Python experience is a plus.
Demonstrated ability to work efficiently, prioritize workflow, and meet demanding deadlines Proven ability to deal with ambiguity and drive for clarity.

Required Skills: