- 5+ years of experience in a Data Engineer role in a cloud native echo-system.
- Bachelor (Graduate preferred) degree in Computer Science, Mathematics, Informatics, Information Systems or another quantitative field.
- Working experience with the following technologies/tools:
- big data tools: Spark, ElasticSearch, Kafka, Kinesis etc.
- Relational SQL and NoSQL databases, such as MySQL or Postgres and DynamoDB or Cassandra.
- Hands-on Experience with at least 1 Data Warehouse (Snowflake, Synapse, BigQuery, ADW, Teradata, Exadata, Netezza, SQL Server, Oracle)
- Cloud data services.
- Functional and scripting languages: Python, Java, Scala, etc.
- Advanced SQL
- Experience utilizing SCM tools such as GitHub/GitLab/BitBucket
- Experience building and optimizing 'big data' pipelines, architectures and data sets.
- Working knowledge of message queuing, stream processing, and highly scalable 'big data' stores.
- Experience supporting and working with external customers in a dynamic environment.
- Articulate with great communication and presentation skills
- Team player that can train as well as learn from others.
- Ability to design tables in a data warehouse that house historical data.
- Ability to conduct initial and incremental loads.
- Advantage: Experience with various ML models for classification, scoring and more.
- Advantage: Experience with Deep Learning Neural Networks (Convolution, NLP etc.)
- One or more AWS Certification(s) (Cloud Practitioner, SA Associate, Data Analytics Specialty, etc.)
- Hands-on BI basic dashboard creation
- Hands-on Snowflake Experience
- GitFlow Branch Methodology Experience
- Data Vault 2.0 Table Structure and Load Patterns
- Experience Hashing Data Attributes using MD5 or sha-256 in accordance with Data Vault 2.0 Requirements
- PySpark and SparkQL Experience
- Technical Documentation of Solutions
Data Engineer - New York, United States - AllCloud
Description
Data Engineer
Location: US / Canada (Eastern Time) - Home based
Job Type: Full-time, Permanent
About AllCloud
AllCloud is a global professional services company providing organizations with cloud enablement and transformation tools. As an AWS Premier Consulting Partner and audited MSP, a Salesforce Platinum Partner, and a Snowflake Premier Partner, AllCloud helps clients connect their front and back offices by building a new operating model to harness the benefits of cloud technology and data and analytics.
Job Summary
AllCloud is looking for a savvy Data Engineer to join our growing team of data experts. The hire will be responsible for migrating to the cloud and optimizing our customers' databases and data flows and enrich our operational and functional data flows with AI/ML algorithms.
The ideal candidate is not afraid of data in any form or scale, and is experienced with cloud services to ingest, stream, store and manipulate data. The Data Engineer will support new systems designs and migration of existing ones, working closely with solutions architects, project managers and data scientists. The candidate must be self-directed, a fast-learner and comfortable supporting the data needs of multiple teams, systems and products. The right candidate will be excited by the prospect of optimizing or re-designing our customers' data architecture to support their next generation of products and data initiatives and machine learning systems.
Nice to Have Skills:
Why work for us?
Our team inspires progress in each other and in our customers through our relentless pursuit of excellence; you will work with leaders who promote learning and personal development.
AllCloud is an Equal Opportunity Employer and considers applicants for employment without regard to race, color, religion, sex, orientation, national origin, age, disability, genetics or any other basis forbidden under federal, provincial, or local law.