ETL Hadoop Developer - Atlanta, United States - Collabera

    Collabera
    Collabera Atlanta, United States

    4 weeks ago

    Default job background
    Technology / Internet
    Description

    Job Description:

    • As a Senior ETL Developer, you will play a key role in designing, developing, and maintaining ETL processes for our data pipeline.
    • You will work with large volumes of structured and unstructured data, utilizing a variety of tools and technologies to ensure data integrity and seamless integration.
    • This position requires advanced technical skills, strong analytical abilities, and a deep understanding of data architecture.

    Responsibilities:

    • Design, develop, and implement ETL processes using tools such as Informatica, SSIS, etc.
    • Work with Big Data technologies including Hadoop, Spark, Impala, Hive, and Python to build scalable data solutions.
    • Develop applications that process large amounts of structured and unstructured data using MapReduce and other big data frameworks.
    • Collaborate with data architects, data engineers, and business stakeholders to understand data requirements and ensure ETL solutions meet business needs.
    • Create and maintain Autosys JIL scripts for job scheduling and automation.
    • Troubleshoot data-related issues and provide solutions to ensure data quality and integrity.
    • Develop complex SQL queries and scripts to extract, transform, and load data into relational databases.
    • Participate in code reviews and ensure best practices are followed for ETL development.
    • Document ETL processes, data flows, and data mappings for reference and compliance purposes.

    Qualifications:

    • 10+ years of experience in ETL development, with a focus on Big Data.
    • 3+ years of experience developing ETL solutions using tools like Informatica, SSIS, etc.
    • Proficient in Hadoop, Spark, Impala, Hive, and Python.
    • Experience with Cloudera Hadoop Ecosystem and troubleshooting related issues.
    • Strong scripting skills in Unix shell and Perl.
    • Extensive experience with SQL and relational database development.
    • Strong problem-solving skills and attention to detail.
    • Excellent communication skills and the ability to work collaboratively in a team environment.

    Preferred Qualifications:

    • Experience with cloud-based data solutions and technologies.
    • Familiarity with Agile development methodologies.
    • Experience with additional ETL tools or programming languages.