Jobs
>
Rockville

    Lead Big Data Engineer - Rockville, United States - Tential

    Tential
    Tential Rockville, United States

    6 days ago

    Default job background
    Description

    Lead Big Data Developer
    Are you passionate in data? Do you like working in a challenging environment where massive volume of data is ingested and processed every day? Are you a continuous learner who wants to learn new tools and technologies in evolving big data and data science technology? Do you have passion in detecting data anomalies in large datasets? Do you expect the best from yourself and those around you?

    We are looking for lead big data developer for ETL project in our enterprise Transparency Services group. Big data developer will work on designing, ingesting, storing, validating and disseminating, after transforming data in a consumable format, for business intelligence teams and data analysts to get deeper business insight from the data.

    Job Responsibilities

    • Understand complex business requirements
    • Design and develop ETL pipeline for collecting, validating and transforming data according to the specification
    • Develop automated unit tests, functional tests and performance tests.
    • Maintain optimal data pipeline architecture
    • Design ETL jobs for optimal execution in AWS cloud environment
    • Reduce processing time and cost of ETL workloads
    • Lead peer reviews and design/code review meetings
    • Provide support for production support operations team
    • Implement data quality checks.
    • Identify areas where machine learning can be used to identify data anomalies
    Experience & Qualifications
    • 7+ years of experience in programming language Java or Scala
    • 7+ years of experience in ETL projects
    • 5+ years of experience in big data projects
    • 3+ years of experience with API development (REST API's)
    • Believes in Scrum/Agile, and has deep experience delivering software when working on teams that use Scrum/Agile methodology
    • Strong and creative analytical and problem-solving skills
    Required Technical Skills & Knowledge
    • Strong experience in Java or Scala
    • Strong experience in big data technologies like AWS EMR, AWS EKS, Apache Spark
    • Strong experience with serverless technologies like AWS Dynamo DB, AWS Lambda
    • Strong experience in processing with JSON and csv files
    • Must be able to write complex SQL queries
    • Experience in performance tuning and optimization
    • Familiar with columnar storage formats (ORC, Parquet) and various compression techniques
    • Experience in writing Unix shell scripts
    • Unit testing using JUnit or ScalaTest
    • Git/Maven/Gradle
    • Code Reviews
    • Experience with CI/CD pipelines
    • Agile
    The following skills a plus:
    • AWS Cloud
    • BPM/ AWS Step Functions
    • Python scripting
    • Performance testing tools like Gatling or JMeter
    #LI-WB
    #Dice
  • Eclipse Enterprises

    Data Engineer

    1 week ago

    Eclipse Enterprises Cabin John, United States

    Be Yourself - and Develop Your Talent · At INFOMOTION, our focus is on extracting the maximum value from data - and soon for you too, if you are passionate about data and understand your digital craft. Just like us. · With almost two decades of experience, we are one of the leadi ...

  • University of Maryland Medical System

    Data Engineer

    1 week ago

    University of Maryland Medical System Rockville, United States

    Job Description · Job DescriptionCompany Description · The University of Maryland Medical System is a 14-hospital system with academic, community and specialty medical services reaching every part of Maryland and beyond. UMMS is a national and regional referral center for trauma, ...

  • ASCENDING

    Data Engineer

    5 days ago

    ASCENDING Rockville, United States

    Job Description · Job DescriptionFull-time, Remote - Location Requirement: Must be located in the Eastern Time Zone · Available for W-2 or 1099 Individual. · Our client, one of the largest Amazon Web Services (AWS) partners for data services, is looking for a true Mid level Big ...

  • Publicis Media

    Data Engineer

    1 week ago

    Publicis Media Rockville, United States

    Job Description · Does managing data hygiene make you smile? Does architecting and loading a wide set of data sources to a single source of truth warehouse make your heart sing? Do you love animals? Then this is the role for you · This is the perfect opportunity for someone to dr ...

  • Apex Ventures

    Data Engineer

    1 week ago

    Apex Ventures Marlboro, United States

    Data Engineer (m/w/d) Be yourself - and develop your talent with us. Our focus is on extracting the maximum value from data - and soon for you too. Assuming you are passionate about data and understand your digital craft. Just like us. Here you prove your entrepreneurial spirit. ...

  • BizFirst

    Data Engineer

    4 days ago

    BizFirst Alexandria, United States

    Job Title: Data Engineer · Company: Client of BizFirst · Location: Crystal City/Alexandria, VA (Onsite requirement: 3 days per week) · Employment Type: Full-time W-2 Employee · Contract Length: 5 years · Security Clearance: Secret clearance required; TS/SCI clearance preferred · ...

  • Optomi

    Lead Data Engineer

    3 days ago

    Optomi Rockville, United States

    Lead Data Engineer | Remote · Optomi, in partnership with a global leader in the financial services industry, is seeking a Lead Data Engineer to join their team This candidate will be responsible for designing, ingesting, storing, validating and disseminating data in a consumable ...

  • ASCENDING LLC

    Sr. Data Engineer

    1 week ago

    ASCENDING LLC Rockville, United States

    Our client is a leading provider of financial and economic data to the investment banking industry. They are seeking an experienced Senior Data Engineer to join and growing team. · - This role is only available for W2 or individual contracts. Please no C2C. · - Remote 100% · Re ...

  • Financial Industry Regulatory Authority , Inc.

    Director, Data Engineering

    12 hours ago

    Financial Industry Regulatory Authority , Inc. Rockville, United States

    This Director role will lead a team of data engineers supporting data acquisition and pipelining for our Advanced Analytics program, including FINRAs knowledge graph and natural language practices. The Director is generally autonomous in their overs Data Engineer, Director, Engin ...

  • ASCENDING

    Sr. Data Engineer

    1 week ago

    ASCENDING Rockville, United States

    Job Description · Job DescriptionOur client is a leading provider of financial and economic data to the investment banking industry. They are seeking an experienced Senior Data Engineer to join and growing team. · - This role is only available for W2 or individual contracts. Plea ...

  • ConsultNet

    Big Data Engineer

    1 week ago

    ConsultNet Rockville, United States

    Big Data Engineer · Location: Fully remote in the continental US. Work outside of the US not permitted · Contract to Hire, long term contract · Job Description: · Our client is looking for a rock star Big Data Engineer to support an internal business stakeholder with their bi ...

  • fusionSpan

    Senior Data Engineer

    1 week ago

    fusionSpan Rockville, United States

    Job Description · Job DescriptionfusionSpan is a dynamic, fast-paced organization. We are a team of highly committed individuals who are inspired by the role technology plays in society. · This position is 100% remote across the US. Working hours align with EST timezone. · A dat ...

  • RIT Solutions, Inc.

    Sr. Data Engineer

    1 week ago

    RIT Solutions, Inc. Rockville, United States

    · • Promotes and adheres to Client's architectural standards and guidelines and works closely with the Enterprise Architects to develop target architectures and technology standards · • Provides explicit architecture consulting to project teams, across corporate functions, unde ...

  • GovFirst

    Data Automation Engineer

    2 days ago

    GovFirst Herndon, United States

    We are actively seeking a talented and experienced Data Automation Engineer (Cyber Security Systems) to join our team in supporting our customer's Division level Authorization and Accreditation (A&A) projects. The ideal candidate will have a minimum of 16 years of experience and ...

  • MSys Inc

    Big Data Engineer-Hybrid

    2 weeks ago

    MSys Inc Rockville, United States

    Job summary: · Title: · Big Data Engineer-Hybrid · Location: · Rockville, MD, USA · Length and terms: · Long term - W2 or C2C · Position created on 04/29/2024 05:08 pm · Job description: · * Webcam interview Long term project Hybrid-1 Day a week onsite · As a Big Data Developer, ...

  • SSATI

    Data Analytics Software Engineer

    1 week ago

    SSATI Annapolis Junction, United States

    Data Analytics Software Engineer, Clearance Required – TS/SCI w/Polygraph · Please note, this position requires all candidates to currently possess an active Top-Secret SCI Clearance with a Polygraph. This position is not an opportunity to be sponsored or nominated for a ...

  • University of Maryland Medical System

    Senior Data Engineer, Epic Chronicle

    1 week ago

    University of Maryland Medical System Rockville, United States

    Job Description · Job DescriptionCompany Description · Renowned as the academic flagship of the University of Maryland Medical System, our Magnet-designated facility is a nationally recognized, academic medical center with opportunities across the continuum of care. Come join UMM ...

  • Spectrum Dynamics

    Software developer with a focus on data engineering

    1 week ago

    Spectrum Dynamics Barnard, United States

    Position: Associate · Working hours: Full-time · JobID: REF228219A · Company Description: · Our drive and control technologies move machines and projects of all sizes around the world. We inspire our customers with intelligent components, customized system solutions, and services ...

  • Xcelerate Solutions

    Data Engineer

    5 hours ago

    Xcelerate Solutions Bethesda, United States

    Data Engineer - TS/SCI with CI Poly Xcelerate Solutions is supporting the IC Chief Data Office to provide senior oversight and strategic vision for building a data-driven IC enterprise. Our teams are focused on end-to-end data lifecycle management, reducing the time from collecti ...

  • HelioCampus

    Data Engineer

    1 week ago

    HelioCampus Bethesda, United States

    Job Description · Job DescriptionWho We Are: · Born out of higher education, HelioCampus works with institutions throughout North America to optimize financial decision-making and assessment/accreditation processes, and provide a more integrated approach to strategic planning an ...