Azure Data Engineer - Huntersville, United States - SMART IT People
Description
Huntersville, NC (Preference for onsite/hybrid but will consider 100% remote for the right candidate)SMART IT People has a contract opportunity for an experienced Azure Data Engineer.
The ideal candidate will have a strong background in data engineering, Azure Data Factory (ADF), Azure Synapse, along with experience ingesting and consolidating data from various data sources including multiple ERP systems.
The Senior Azure Data Engineer will play a crucial role in ensuring the security, reliability, and performance of our data infrastructure while collaborating with cross-functional teams to meet business objectives.
Requirements:
Minimum of Bachelors' degree in Computer Science, Engineering, Mathematics or related fields
5+ years of work experience in designing, developing, and troubleshooting complex SQL queries, on DBMS solutions such as Snowflake, Oracle, MS SQL Server, PostgreSQL, and MySQL
3+ years of hands-on experience in ETL/ELT tools
3+ years of work experience with Data Platforms
1+ year development experience in Python or similar scripting languages for data automation
Proven experience as an Azure Data Engineer or similar role, with a focus on designing and implementing data pipelines and data warehousing solutions using Azure services.
Expertise in Azure Data Factory (ADF), Azure Synapse, and multiple ERP systems.Experience working with On Premise and Cloud systems and data sources
Strong analytical skills related to working with unstructured datasets and designing efficient data processing workflows.
Proficiency in coding and scripting languages such as SQL, Python, or PowerShell.
Implement DevOps and automation practices to automate data pipelines, monitor data processes, and manage deployments using Azure DevOps, Azure Monitor, and Azure Automation.
Excellent communication skills, with the ability to collaborate effectively with cross-functional teams and drive alignment towards common goals.Regularly check and maintain data quality by validating data against predefined standards and data governance policies.
Fix data quality issues and errors in data pipelines to maintain accurate data.
Nice to Have:
Certifications such as Azure Solutions Architect Expert or Azure DevOps Engineer Expert.
Experience with other Azure services and frameworks beyond ADF and Synapse including Azure Databricks and Snowflake.
Knowledge of data modeling and analysis techniques using Azure services like Azure Analysis Services.
Experience with Microsoft Fabric
Experience with Informatica Data Quality and MDM
Responsibilities:
Collaborate with stakeholders to understand data requirements and design and maintain secure and compliant data processing pipelines using Azure Data Factory, Azure Synapse, and other tools.
Ingest data from various sources such as databases, files, APIs, and streaming platforms into Azure, ensuring efficient and reliable data transfer.
Perform data transformation and processing tasks using Azure Synapse to prepare data for analysis and reporting.Select and manage appropriate data storage solutions in Azure, including Azure SQL Database, Azure Data Lake Storage, and Azure Blob Storage, based on data requirements.
Successful data movements and transformations using various file formats such as Delta, Parquet, Avro, and CSV.Implement and optimize data warehousing solutions using Azure Synapse SQL and Azure Synapse Analytics to support large-scale analytics and reporting.
Collaborate with data scientists and analysts to design and implement data models for analytical workloads using Azure Analysis Services or Power BI.
Implement DevOps and automation practices to automate data pipelines, monitor data processes, and manage deployments using Azure DevOps, Azure Monitor, and Azure Automation.
Support and Troubleshoot On Premise gateways#J-18808-Ljbffr