The data engineer role is a technical person who is involved with architecting, building, testing, and maintaining the data platform.
Data engineers will implement infrastructure for data processing, analysis, reporting, integrations, and machine learning model deployment.
RESPONSIBILITIES:
Technical expertise with distributed Spark or other distributed data processing technologies
Work with large, complex data sets and high throughput data pipelines that meet business requirements.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources.
Build data and analytics tools that utilize the data pipeline to provide actionable insights to operational efficiency and other key business performance metrics.
Work with internal and external stakeholders to assist with data-related technical issues and support data infrastructure needs.
Collaborate with data scientists and architects on several projects.
Solve various complex problems.
Designing processes to integrate data from multiple sources to facilitate client centric advanced analytics.
Developing efficient, scalable, and repeatable processes to transform data into insight on regular basis.
QUALIFICATIONS:
Degree in an analytical field such as Data Science, Machine Learning, Analytics, Statistics, Computer Science, or highly quantitative engineering
Previous experience as a data engineer or in a similar role
5+ years of Python development experience is necessary.
Hands-on experience with database technologies (e.g. SQL and NoSQL)
Experience building high throughput data pipelines.
Technical expertise with distributed Spark or other distributed data processing technologies
Experience with machine learning techniques.
Great numerical and analytical skills
Ability to write reusable code components.
Open-minded to the new technologies, frameworks
Thorough business analysis skills
Understanding Blockchain system mechanism
Knowledge of data preparation techniques to aid statistical analysis.
Results-oriented self-starter who is confident in defending his/her critical thinking abilities.
Bigdata Developer - Bigdata/Spark Developer (No C2C)
We have other current jobs related to this field that you can find below
