- Work as part of an implementation team from concept to operations, providing deep technical subject matter expertise for successfully deployment of Data Platform Implement methods for automation of all parts of the pipeline to minimize labor in development and production.
- Identify, develop, evaluate, and summarize Proof of Concepts to prove out solutions
- Test and compare competing solutions and report out a point of view on the best solution
- Experience with large scale solutioning and operationalization of data warehouses, data lakes and analytics platforms on GCP
- Design and build production data engineering solutions to deliver our pipeline patterns using Google Cloud Platform (GCP) services: o BigQuery, DataFlow (Apache Beam), Pub/Sub, BigTable, Data Fusion, DataProc, Cloud Composer (Apache Airflow), Cloud SQL, Compute Engine, Cloud Functions, and App Engine
- Migrate existing Big Data pipelines into Google Cloud Platform . Build new data products in GCP.
GCP Data Engineer - Dearborn, United States - Insight Global
Description
Job Description
GDIA Data Engineer Job Description A position is open for a data engineer in the GDI&A Customer 360.The successful candidate will be responsible for designing, developing the transformation and modernization of big data solutions on GCP cloud integrating native GCP services and 3rd party data technologies also build new data products in GCP.
We are looking for candidates who have a broad set of technology skills across these areas and who can demonstrate an ability to design and develop right solutions with appropriate combination of GCP and 3rd party technologies for deploying on GCP cloud.
Responsibilities Key Responsibilities:
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day.
Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances.
If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:.
Skills and Requirements
Minimum 3 Years of Experience in Java/python in-depth Minimum 2 Years of Experience in data engineering pipelines/ building data warehouse systems with ability to understand ETL principles and write complex sql queries.
Minimum 5 Years of GCP experience working in GCP based Big Data deployments (Batch/Real-Time) leveraging Big Query, Big Table, Google Cloud Storage, PubSub, Data Fusion, Dataflow, Dataproc Minimum 2 years of experience in development using Data warehousing, Big Data Eco System Hive (Hql) & Oozie Scheduler, ETL IBM Data Stage, Informatica IICS with Teradata 1 Year experience of deploying google cloud services using Terraform Minimum 3 Years of Experience in Java/python in-depth Minimum 2 Years of Experience in data engineering pipelines/ building data warehouse systems with ability to understand ETL principles and write complex sql queries.
Minimum 5 Years of GCP experience working in GCP based Big Data deployments (Batch/Real-Time) leveraging Big Query, Big Table, Google Cloud Storage, PubSub, Data Fusion, Dataflow, Dataproc Minimum 2 years of experience in development using Data warehousing, Big Data Eco System Hive (Hql) & Oozie Scheduler, ETL IBM Data Stage, Informatica IICS with Teradata 1 Year experience of deploying google cloud services using Terraform null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day.
Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances.
If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to