Data Engineer - Reston, United States - Information Technology

    Information Technology
    Information Technology Reston, United States

    3 weeks ago

    Default job background
    Description

    Job Details

    Description

    Responsibilities and Duties:

  • The Data Engineer supports the maintenance, synchronization, wrangling, and migration of data across a hybrid environment.
  • Collaborates to understand, analyze, document, and efficiently implement data pipelines for event-based data replication, including micro-batch and batch-based data pipelines.
  • Establishes methods to enrich the platform with new sources of information that support analytics and web application data delivery needs.
  • Actively participates with broader team to enhance the capabilities of the data platform, including strengthening analytical workflows executing on our Business Intelligence tool (Looker).

    • Position Requirements:

  • 3+ years building scripts using SQL and at least one of the following: R, Python, Scala or Java.
  • 2+ years developing data flow and data orchestration workflows (e.g. Azure Data Factory).
  • 2+ years of experience using Azure Databricks to build scalable ETL pipelines.
  • Comfortable building data lake storage and data processing strategies for analytic workloads.
  • Experienced with developing automation workflows using RESTful APIs.
  • Experienced with change management workflows (Azure DevOps, Visual Studio, GitHub, TFS).
  • Familiarity with scripting and command line operations including execution using MS PowerShell.
  • Knowledgeable of techniques to automate business-oriented data capture (e.g. Power Platform).
  • Interest in developing streaming data pipelines using resources such as Azure Event Hubs, Spark-Streaming, Kafka, or Flink Streams.
  • Familiarity with Microsoft Dynamics Finance and Operations 365 highly desired.
  • Comfortable working in an Agile Kanban environment and using tools such as Jira or Trello.
  • Detailed oriented/self-motivated with the ability to learn and deploy new technology quickly.
  • Good communication skills. Able to present the technical solutions and understand business requirements.
  • Someone comfortable wearing multiple data hats on a small team, and continuously seeking new techniques to improve data processing and analytics.
  • Adhere to company safety standards and help promote a safe working environment.
  • Adhere to and promote the Mission, Vision, and Values of Stanley Martin Homes.