No more applications are being accepted for this job
- An associate degree, a bachelor's degree in computer science or equivalent courses
- At least 4 years of experience in Data Engineering with SQL, Python
- Experience with relational SQL and NoSQL databases
- Experience in the following Big Data frameworks: File Format (Parquet, AVRO, ORC etc.)
- At least 3 years of experience working with large data sets, streaming, experience working with distributed computing (Map Reduce, Hadoop, Hive, Apache Spark, Apache Kafka etc.)
- At least 2 years of experience with AWS cloud (with focus on Data services)
- Experience with Databricks a plus
- Experience with RESTful API development, Familiarity with HTTP and invoking web-APIs
- Working knowledge of data structures, SQL, XML, JSON, Data visualization tools, Version Control Systems, Programming, and Unix/Linux shell scripting
- Able to interpret user requirements and identify additional information needed in user requirements
- Able to see effects of current design with future requirements and see possible coding solutions to meet the requirements
- Detailed understanding of logical and physical data structures
- Highly skilled in tools, evaluates the need for various tools for continuous integration, testing, automation, deployment etc. and discuss with the team
- Highly skilled at designing tests for unfamiliar designs; Evaluates tests for weaknesses and continuously improves them
- Detailed understanding of how to effectively test against multiple tools/software
- Equivalent education and/or experience may be substituted for any of the above requirements
- Design, develop and implement data mining tools and analyses to sift through large amounts of data stored in a data warehouse or data mart to find relationships and patterns
- Be responsible for implementing the systems, processes and logic required to extract, transform, clean, and distribute data across one or more data stores from a wide variety of sources for systems with moderate complexity
- Work under general guidance and clear framework of accountability with substantial autonomy
- Use best practices and knowledge of internal or external business issues to improve products or services
- Solve complex problems; takes a new perspective using existing solutions #DICE
Data Engineer - Torrance, United States - C4 Technical Services
Description
Dallas, TX (hybrid onsite) Data Engineer
The team is is looking for a versatile Data Engineer who will provide data and report development services or technical support.
You will also develop data set processes and assist with design and identify ways to improve data reliability, efficiency, and quality.
As the Data Engineer you will work independently, receive minimal guidance, and have accountability for their work and work of junior members.
Responsibilities: