Data Engineer - Chicago, United States - HexawareAvance Consulting

Description

The data engineer considers themselves an expert in data management with cloud technologies.

The data engineer leads the definition and implementation of data pipelines, data storage, and data access/security for a complex Azure hosted application environment.

They are responsible for curating a sustainable and scalable data environment, servicing a mix of business.

A number of operational data sources will require integration, data structuring/cleaning, ETL and data quality reviews on an ongoing basis.

The data engineer works closely with the digital business infrastructure operations (DBIO) lead and analytics lead, and to ensure data quality, performance, and structure requirements scale with the customer needs.

The data engineer will be working within a team setting and will support a range of data requirements, supporting cloud operations/monitoring, mobile application analytics, and user personalization use cases for a global 100 technology leading client.

The environment is fast-paced and dynamic and will provide a range of requirements expected to evolve.


Requirements:
High-level proficiency in data engineeringProficiency in data engineering best practicesDeep experience with Azure data management related services and related data architecturesKnowledge of security standardsfor Azure data environmentsBachelor of Science in Computer Science or other relevant experienceExperience with Java, Python, SQL, and noSQLResponsibilities:Configure data sources for data loading, batch, and continuous data streamingConfigure and implement ETL operationsConfiguring data storage tools / platformsSupport enterprise architect for data architecture strategy and designDevelop data models in support of data architectureLead data lake design and implementation, aligned with data architectureData set curation, cleaning, loadingIntegrate analytics and monitoring tools with consolidated data environmentEstablishing and maintaining consolidated data lake environmentManaging data storage resources, monitoring, data operationsIdentify areas for continuous improvement in data related technology and processesCreate and communicate high quality technical documentationInteract regularly with managers, SMEs, and other technical leadershipDesired Skills and Experience:Minimum 5-10 years consulting / professional services experience Azure Architect certified (Associate or Professional)Azure experience (data pipeline, database, network, security, CI/CD)Strong written/verbal communication skillsAbility to handle ambiguity and deliver outcomes Experience with Snowflake or Databricks (preferred, but not required)#J-18808-Ljbffr