Big Data/Machine Learning Engineer - McLean, United States - Soft source inc

    Soft source inc
    Soft source inc McLean, United States

    1 month ago

    Default job background
    Description

    Job Description

    Job DescriptionTitle: Big Data/Machine Learning Engineer - Sr
    Duration: 12 months (15th May 2025) Can be extended
    Location : McLean VA or Richmond VA (2 days week)
    Interview : 2 Technical Round Interview for sure .
    LOB:EDML- Data and Machine Learning - Data Space EDT

    Project:
    • Application is doing large Volume Data transformation with in AWS.
    • SDK's were developed in Scala / Spark
    • Lambda were based on python.
    • SQL is for Data Transformation and Scripting
    • Behave Python Automation
    • 80% of work is of Data Engineering and 20% of work is Automation.
    AWS
    • Glue- running the spark applications,
    • S3-where the data moves through,
    • Lambda executed by the step functions
    • Step Functions for orchestrations,
    • Fargate/ECS elastic server where the APIs are hosted.

    Must Haves:
    • Data Engineer
    • Scala
    • Python
    • Data Processing using Spark framework
    • AWS (Glue / Step functions / Lambda / fargate )** Must
    • SQL
    • Automation Testing (Behave for Python and Scala for anything else)