Sr. Data Engineer - Chantilly, VA, United States - DB Recruitment Group

Description

Job Role and Responsibilities We are on the lookout for a Senior Data Engineer who specializes in ETL, ready to deliver industry-leading capabilities to our clients

In this role, you will:
Architect, develop, and sustain ETL (Extract, Transform, Load) processes for our Master Data Management (MDM) system.

Construct and refine data pipelines for data extraction from diverse sources, transformation into necessary formats, and loading using Databricks and AWS services.

Collaborate with various stakeholders to determine data requirements, understand data origins, and uphold data quality and integrity throughout the ETL process.

Apply data validation, cleansing, and enrichment methods to enhance data precision and completeness.
Monitor and debug ETL processes to promptly identify and resolve issues.
Contribute to a team-oriented environment to design, develop, and maintain a software system currently undergoing modernization.
Participate in migrating the application to the cloud and the introduction of new technologies into the technology stack.
Engage in Agile Scrum SDLC activities and assist in the creation of Agile SDLC phase documentation.
Conduct unit and integration testing of software/systems before user acceptance testing release

Required Qualifications A BS degree with eight (8) years of experience in programming and software development, including analysis, design, development, implementation, testing, maintenance, quality assurance, troubleshooting, and software system upgrades.

Strong proficiency in programming languages such as Scala or Java.
Experience designing and developing ETL workflows using tools like Apache Spark or AWS Glue.
Deep understanding of ETL best practices, data integration techniques, and data quality management.
Familiarity with different data storage technologies and databases, such as Amazon S3 or Amazon Redshift.
Understanding of Data Lakehouse architecture and OpenSearch concepts is a plus.
A current Top Secret security clearance with SCI eligibility and the ability to obtain a polygraph
Desired Qualifications Knowledge of Tika.
Experience with AWS cloud services.
Understanding of data acquisition and ingestion of structured and unstructured data sources, ensuring quality and data integrity.
Experience with open-source technologies like Docker, ElasticSearch, and NoSQL databases.
Experience with AWS Lambda, SQS, or NiFi.