Data Engineer Lead - Seattle, United States - Apex Systems

Description

Key projects:
Exploration of generative AI Technology requirements:
Proficiency in Apache Spark, including Spark Core, Spark SQL and Spark Streaming.
Proficiency in languages such as Python for data processing and scripting.

Degree or certifications required:
Bachelor's degree in computer science, management information systems, or related discipline Years experience: 5+ Required background? Skills?:
Experience in designing and implementing ETL processes usingDatabricksnotebooks for efficient data extraction, transformation, and loading.
In-depth knowledge of theDatabricksUnified Analytics Platform and its features for collaborative big data analytics.
Understanding ofdata modelingconcepts for designing database structures.
Proficiency in working with both relational databases andNoSQLdatabases.
Integration of data from diverse sources, including APIs, databases, streaming data, and external data feeds.
Implementation of processes to ensure data quality, includingdata validation, cleansing, and error handling.
Knowledge of cluster management, optimization, and scaling for efficient data processing.
Optimization of Spark jobs and Databricks clusters for better performance.
Proficiency in cloud platforms such as Azurefor building scalable and flexible data architectures.
Use of tools likeApache Airflow, ADF, or Databricksto orchestrate and schedule data workflows.

Nice-to-Haves :

Knowledge ofdata securitybest practices and the implementation of measures to ensure data privacy and compliance.
Implementation of monitoring and logging solutions to track the health and performance of pipelines.
Familiarity with monitoring platforms likeDataDog and New Relic
Azure

Top 3: 1 Skill : Data Engineering 5+ 2 Skill : Data Bricks 5+ 3 Skill : Sparks/ Dev ops practices 5+ EEO EmployerApex Systems is an equal opportunity employer.

We do not discriminate or allow discrimination on the basis of race, color, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law.

Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law.

If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Employee Services Department at [email protected] or Apex Systems is a world-class IT services company that serves thousands of clients across the globe.

When you join Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package.

Our commitment to excellence is reflected in many awards, including ClearlyRated's Best of Staffing in Talent Satisfaction in the United States and Great Place to Work in the United Kingdom and Mexico.