Senior Data Engineer

Summary Responsible for developing, testing and implementing data engineering solutions to generate analytical and reporting solutions.

Responsibilities

• Responsible for analyzing and preparing the data needed for data science based outcomes.
• Also responsible for managing and maintaining metadata data structures besides providing necessary support for post-deployment related activities when needed.
• Design and build automations and incorporate low code-no code solutions to accelerate development cycle time. tex
• Roles and Responsibilities In this role, you will: - Design and build automations and incorporate low code-no code solutions to accelerate development cycle time. - Leverage technical data dictionaries and business glossaries to analyze the datasets. - Perform data profiling and data analysis for any source systems and the target data repositories. - Understand metadata and the underlying data structures needed to standardize the data load processes. - Develop data mapping specifications based on the results of data analysis and functional requirements - Perform a variety of data loads & data transformations using multiple tools and technologies. - Build automated Extract, Transform & Load (ETL) jobs based on data mapping specifications. - Validate the data mapping results and match with the expected results. - Leverage/Implement Data Quality (DQ) rules provided.

Requirements

• Bachelor's Degree in computer science or equivalent with minimum 5+ years of hands on experience.
• Ability to understand logical and physical data models, big data storage architecture, data modeling methodologies, metadata management, master data management & data lineage techniques. - Hands-on experience in AWS services specially
• Redshift, Glue, RDS, Step Functions, S3, Sage maker, Kinesis. - Hands-on experience in programming languages like Python.
• Unix - Hands-on experience in writing SQL scripts for Oracle, MySQL, PostgreSQL. - Experience in handling both Online Transaction Processing (OLTP) and Online Analytical Processing (OLAP) data models. - Experience with data warehousing, Big Data.
• Exposure to handling enterprise ERP and CRM data. - Knowledge of for industrial applications in a commercial/finance/manufacturing settings. - Exposure to finance and accounting data domains.
• Partner with other team members to understand the project objectives and resolve technical issues. - Communicate project status or challenges in a clear and concise manner to the cross team members. - Ability to drive technical solutions to meet business outcomes.

Preferred

• NRT - Kafka. - Exposure to unstructured datasets and ability to handle XML, JSON file formats - Exposure to Extract, Transform & Load (ETL) tools like
• Matillion Or Informatica. -Exposure or experience in Databricks. - Experience using 'Low code, No code' tools like Microsoft/Github co-pilot.