In addition to creating and maintaining an optimal pipeline architecture, typical duties and responsibilities for a Data Engineer position may include:
Assembling large, complex sets of data that meet non-functional and functional business requirements using the medallion architecture.
Identifying, designing and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes via meta data orchestration.
Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using Microsoft technologies.
(Azure data factory, Fabric, SQL Server, and PowerBI)
Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition.
Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issues.
Working with stakeholders including the Executive, Product, Data and Design teams to support their data infrastructure needs while assisting with data-related technical issues
Key Responsibilities
Skills, Knowledge and Expertise
Bachelor degree in computer science or related field.
Ability to build and optimize data sets, ‘big data’ data pipelines and architectures with ADF.
Ability to perform root cause analysis on external and internal processes and data to identify opportunities for improvement and answer questions
3-5 years experience managing BI or Data team.
Ability to build processes that support data transformation, workload management, data structures, dependency and metadata
Excellent written and verbal communication skills, demonstrated through the preparation of reports and presentations for diverse audiences.
jobapplier