Data Engineer - Madison, United States - M3ins

Description

ESSENTIAL FUNCTIONS AND RESPONSIBILITIES
Design and develop data architectures, including data models, data flows, and data storage solutions
Build and maintain data pipelines to move data to/from source systems to the M3 data warehouse
Develop ETL (Extract, Transform, Load) processes to ensure data is clean, accurate, and ready for analysis and use
Optimize data pipelines and architectures for performance and scalability
Collaborate with business users, data scientists, and other stakeholders to understand their data needs and design solutions to meet those needs
Integrate new data sources that can generate additional analytical value
Implement data governance policies and ensure compliance with data privacy regulations
Develop and maintain documentation for data systems and processes
Continuously monitor data systems to ensure data quality and reliability
Stay up-to-date with the latest developments in data engineering technologies and tools
Continuously Advance Enterprise Data Warehouse and Data Modeling Capabilities
Design, implement and maintain 's data and analytics infrastructure to enable high quality creation of data visualizations, models and other data products that will be delivered through the digital core experience layer
Design, maintain and continuously improve the enterprise data warehouse, optimizing performance, scalability, cost, security and governance based on current and projected needs
Execute the creation and optimization of data pipelines to/from the data warehouse to ensure a complete customer analytic record for use in downstream systems of engagement and to also ensure the systems of record are updated and kept current
Develop, execute and maintain ETL processes to ensure data is clean, accurate, and ready for analysis and use
Optimize data pipelines and architectures for performance and scalability
Create and optimize SDKs and APIs to connect data sources to the data warehouse
Drive business results through data and analytics application
Maintain and update the M3 enterprise data model according to external standards and internal needs
Seek out and connect new data sources to the data warehouse to advance M3's data benchmarking and modeling capabilities in collaboration with internal and external stakeholders and users
Collaborate with business users, data scientists, and other stakeholders to understand their data needs and design solutions to meet those needs
Collaborate directly with internal and other external stakeholders to create data/analytics products that are extensible to other clients and brokers so that M3 can maximize the reach and value of its data products
Implement Data Governance
Implement data governance policies and ensure compliance with data privacy regulations
Develop and maintain documentation for data systems and processes
Stay up-to-date with the latest developments in data engineering technologies and tools
Qualifications

Experience

Preferred

Bachelor's degree in Computer Science, Information Systems, Engineering or equivalent
Familiarity with Kimball, Star Schema, etc.
1-3 years of experience with Cloud based DW such as RDS, Redshift, Snowflake etc.
Experience data modeling best practices
1-3 years experience with orchestration tools such as Dagster, Airflow, Prefect, etc.
Experience with container management frameworks such as Docker, Kubernetes, ECR etc.
Experience working on CI/CD processes such GitHub Actions and source control tools such as GitHub, etc.
Strong experience in coding languages like Python, Scala & Java

#J-18808-Ljbffr