Google Cloud Data Engineer - Windsor Locks, United States - Diverse Lynx

    Diverse Lynx
    Diverse Lynx Windsor Locks, United States

    1 week ago

    Default job background
    Description
    Google Cloud Data Engineer
    Hartford - CT

    Job summary

    Should have strong knowledge of large scale search applications and building high volume data pipelines.
    Should have experience in building data transformation and processing solutions. Should have knowledge in Hadoop architecture, HDFS commands and experience designing & optimizing queries against data in the HDFS environment.
    Should have the ability to leverage multiple tools and programming languages to analyze and manipulate data sets from disparate data sources. Should have experience with 3.Experience 10to14Yrs 4.

    Required Skills ,Hive, Apache Spark, BigQuery 5.Nice to have skills ,Unix Shell Scripting, Python, PySpark, SQL, Cloud Composer, Cloud Dataflow, Cloud Dataproc, Google BigQuery, GCP CloudSQL

    Roles & Responsibilities
    • Develops large scale data structures, pipelines and efficient ETL (extract/load/transform) workflows to organize, collect and standardize data that helps generate insights and addresses reporting needs.
    • Collaborates with other data teams to transform data and integrate algorithms and models into automated processes.
    • Uses knowledge in Hadoop architecture, HDFS commands and experience designing & optimizing queries to build data pipelines.
    • Builds data marts and data models to support Data Science and other internal customers.
    • Analyzes current information technology environments to identify and assess critical capabilities and recommend solutions.
    • Experiments with available tools and advises on new tools in order to determine optimal solution given the requirements dictated by the model/use cases
    Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.