ETL Developer - New York, United States - Creative Data Resources

    Default job background
    Description
    Onsite/remote schedule - 2 days/week onsite

    • Migrate existing SSIS ETL scripts to Python; develop new ETL scripts
    • Support existing SSIS SQL Projects
    • Maintain ETL pipelines in and out of data warehouse using combination of Python and Snowflakes SnowSQL
    • Write SQL queries against Snowflake.
    • Understanding data pipelines and modern ways of automating data pipeline using cloud-based
    • Work closely with existing senior integration staff to flush out design, priority and build.
    • Scaffolding and framework will be needed for staging and transforming datasets.
    • Use existing DevOps pipeline for Python and enhance if necessary.
    • utomate Python scripts on our enterrprise scheduler.
    Strong troubleshooting skills to identify root cause and resolve production issuesSkills:
    Skills

    • least 5 years of full-time development experience using Python.
    • Designing and implementing highly performant data ingestion pipelines from multiple sources using Azure Databricks.
    • Direct experience of building data piplines using Azure Data Factory and (preferably Databricks).
    • Extensive experience in software development and the entire SDLC.
    • Solid understanding of a variety of programming tools and development platforms.
    • Experience in creating high-level product specifications and design documents.
    • Experience in writing Python applications with the benefit of frameworks like Django, Flask, Pyramid, or Tornado.
    • Experience in Python testing and code analysis tools like Pytest and Pylint.
    • Integration experience (ETL, ELT) with Python.
    • Strong SQL skills.
    • Familarity with SSIS would be helpful.
    • General development expertise, use of version control, ticketing, and continuous integration systems.
    • Experience in using an Enterprise Scheduler (Tidal)
    • Experience in an Agile Development environment # On-point communication skills to concisley report status of work/issues and next action, and articulate technical complexity to Businss analysts, Project manager and Business usersKeywords: Education: Bachelors degree in Computer Science or Finance Skills and Experience:

    Required Skills:
    DATABASE DATA MODELING ETL DATABASE MODELING DATA WAREHOUSE