Jobs
>
New York City

    Data Engineer - New York, United States - Blackbird

    Blackbird
    Blackbird New York, United States

    6 days ago

    Default job background
    Description

    Blackbird.AI helps organizations discover emergent threats and stay one step ahead of real-world harm through our AI-powered Narrative and Risk Intelligence Platform. Our commitment is to prioritize safety and security, providing the tools to identify potential risks and ensure a safer environment proactively. No matter the job or where it's located, we're all connected by a shared vision: To lead and enhance the landscape of risk intelligence.

    Reporting to the Head of Data Engineering, the Data Engineer will optimize data pipelines for our cutting-edge real-time streaming cloud-hosted analytics platform. You will be the key driver of our collection, analysis, and visualization processes and will play a pivotal role in shaping the future of our platform.

    The Data Engineer will encompass writing ETL processes, designing database systems, and developing tools for query and analytic processing, all focusing on real-time streaming applications. Additionally, you will elevate performance tradeoffs, own the database architecture, and lead build automation, continuous integration, and deployment efforts while ensuring adherence to security requirements.

    As the Data Engineer, you'll have the chance to:

    • Write ETL processes to support the ingestion and normalization of various data formats from social media, news, and web sources.
    • Design robust database systems and developing tools for query and analytic processing, especially for real-time streaming applications.
    • Conduct performance analysis and empirical studies to evaluate tradeoffs (e.g., cost vs. throughput/latency).
    • Develop, manage, and own the database architecture for our real-time streaming cloud-hosted analytics platform.
    • Lead build automation, continuous integration, deployment, and performance optimization efforts while ensuring compliance with security requirements.
    • Lead design of test suites and inline instrumentation as needed to ensure data correctness

    What you'll bring:

    • Bachelor's degree in Computer Science or related field.
    • Proven success in deploying products in the cloud and SaaS model, with expertise in building optimized processing pipelines for streaming analytics applications and cloud-agnostic solutions (e.g., Kubernetes, Docker).
    • Expertise in databases and query optimization, including PostgresSQL, ElasticSearch, MongoDB, Redis, and Druid, with additional experience in NoSQL and graph databases being advantageous.
    • Experience in horizontally scaling databases.
    • Proficiency in Kafka and Airflow, with a deep understanding of runtime profiling tools to optimize throughput and latency, and able to establish comparative performance benchmarks.
    • Strong skills in build automation, continuous integration, and deployment (CI/CD) tools like Webpack, Buddy, Jenkins, and Docker.
    • Expert-level Python coding skills.
    • Experience collaborating with distributed teams.

    Helpful to have:

    • Technical background in Artificial Intelligence (AI) and Machine Learning (ML).
    • Experience designing and implementing interactive query-driven main-machine intelligence systems.
    • Solid skills in Java.

    We've outlined specific skills, experience, and requirements for this position, but don't stress if you don't meet every single one. Our Talent Team is dedicated to discovering exceptional individuals, and they might identify a relevant aspect of your background that suits this role or another opportunity within Blackbird.AI.

    If you have passion for the role, please still apply.

    What's in it for you:

    Blackbird.AI is embarking on an exciting growth journey with numerous opportunities for career development within the company. You will join a nurturing, inclusive, and experienced team.

    Join us as we soar to new heights

    Values:

    At Blackbird.AI, our core values shape how we work and make decisions. Our values inspire us to be authentic and continue improving.

    We embrace a strong sense of responsibility to society, recognizing the vital role our services play in empowering governments, communities, and individuals to foster critical thinking and empowerment. We believe in integrating personal and professional lives with societal needs, emphasizing the importance of creating an environment that attracts top talent and provides substantial growth opportunities. We are motivated by the potential of science and technology to impact humanity positively.

  • Understood

    Data Engineering

    6 days ago

    Understood New York, United States

    **About Understood**: · Understood is a nonprofit focused on shaping the world for difference. We raise awareness of the challenges and strengths of people who learn and think differently. Our resources help people navigate challenges, gain confidence, and find support and commun ...

  • LTIMindtree

    Specialist - Data Engineering

    4 days ago

    LTIMindtree New York, United States

    **Job Req Id**:1287937 · - **About Us**: · - **Job Title**:Specialist - Data Engineering (Geneva Developer) · - **Work Location**: New York, NY days onsite/week) · experience as developer. · - Strong functional experience in financial industry and Portfolio Accounting System Gene ...

  • EQUINOX

    Engineering Contractor, Data

    15 hours ago

    EQUINOX New York, United States Freelance

    **Engineering Contractor, Data**: · Overview · **OUR STORY** · Equinox Group is a high growth collective of the world's most influential, experiential, and differentiated lifestyle brands. We restlessly seek what is next for maximizing life - and boldly grow the lifestyle brands ...

  • FanDuel

    Data Engineering Director

    3 days ago

    FanDuel New York, United States

    **ABOUT FANDUEL** · FanDuel Group ("FanDuel") is an innovative sports-tech entertainment company that is changing the way consumers engage with their favorite sports, teams, and leagues. The premier gaming destination in the United States, FanDuel consists of a portfolio of leadi ...

  • Bayer Industry Services (Currenta GmbH & Co. OHG)

    Senior Data Engineer

    2 weeks ago

    Bayer Industry Services (Currenta GmbH & Co. OHG) New York, United States

    Das kannst Du bei uns bewegen · Große Datenmengen · sind Dein Ding – Du unterstützt den Data Steward bei der Anbindung immer neuer Datenquellen an unseren Data Lake, sorgst für die · Transformation der Rohdaten zu "Trusted Data" · und unterstützt die Data Scientists bei der D ...

  • CBRE

    IT Data Center Engineer

    1 week ago

    CBRE New York, United States

    Teilzeit (20 Wochenstunden) | unbefristet · Die Technische Universität Wien ist Österreichs größte Forschungs- und Bildungseinrichtung im technisch- naturwissenschaftlichen Bereich und leistet seit mehr als 200 Jahren einen unverzichtbaren Beitrag zur Sicherung der international ...

  • ESB Technologies

    Data Engineer

    6 days ago

    ESB Technologies New Jersey, United States

    Only apply if you are willing to W2 · Job Title: Senior Azure Data Engineer · Location: AZ, TX, NJ, NY, FL, (Any of these mentioned locations) · Job Description: · Position Overview: We are seeking a highly skilled and experienced Azure Data Engineer to join our team as a senior ...

  • Beauty by Imagination (BBI)

    Data Engineer

    1 week ago

    Beauty by Imagination (BBI) New York, United States

    Beauty by Imagination · Data Engineer · Hybrid · Position Overview: · We are seeking a highly skilled and motivated Data Engineer to create and manage a data warehouse, integrating data from various sources including NetSuite, IRI, and other sales data. This role is pivotal in ow ...

  • Oakridge Staffing

    Data Engineer

    1 week ago

    Oakridge Staffing New York, United States

    Global hospitality company is looking for an experienced Data Engineer for their Midtown NYC headquarters. · Responsibilities: · Lead the in-house development of ETL/ELT data pipelines and data science projects. · Create and own data products, such as a recommendation engine or ...

  • iO Associates - US

    Data Engineer

    1 week ago

    iO Associates - US New York, United States

    An exciting Cloud Cybersecruity start up is looking for a talented Data Engineer to join their team based in New York City. The firm has recently secured new seed funding and are working on not just detecting cloud cybersecurity problems but solving them as well. · The ideal cand ...

  • NR Consulting

    Data Engineer

    1 week ago

    NR Consulting New York, United States

    Data Engineer SME · Location: NYC / Charlotte (Candidate should be open for hybrid work model) · Duration: 12 months · Responsibilities: · A background in financial services, investment banking, regulated sectors and large enterprise · DevOps and Agile engineering practitioner w ...

  • Codex Tech-it

    Data Engineer

    1 week ago

    Codex Tech-it New York, United States

    Job Description · Job DescriptionCodex Tech-IT is a leading technology solutions provider specializing in data management and analytics. We are seeking a skilled Data Engineer to join our team and contribute to the development and maintenance of our data infrastructure. With a st ...

  • Stealth AI Startup

    Data Engineer

    2 weeks ago

    Stealth AI Startup New York, United States

    Are you fascinated by the potential of the intersection between artificial intelligence and blockchain technology? Do you thrive in fast-paced environments where innovation is the norm? Interested in being a founding member of an early-stage start-up? · If so, we want you to be ...

  • Net2Source Inc.

    Data Engineer

    1 week ago

    Net2Source Inc. New York, United States

    W2 Candidates Only · Position: Data Engineer – III · Locations: New York, NY Hybrid) · Duration: 12+ months Contract · Qualifications: · Expert-level understanding of Agile, SDLC, CI/CD, DevOps, GitOps, and ProdOps is required. · Expert-level expertise in SQL, Python, Scala or si ...

  • Benchmark IT - Technology Talent

    Data Engineer

    1 week ago

    Benchmark IT - Technology Talent New York, United States

    Our direct client, a fast-growing FinTech firm in New York City, is looking for a Data Engineer to join the Data and Analytics team. This role will help build the infrastructure to make data a central part of the firm. Data driven decisions are very critical for the business, as ...

  • OTG

    Data Engineer

    1 week ago

    OTG New York, United States

    YOUR NEXT OPPORTUNITY IS NOW BOARDING: · Join OTG as the Data Engineer – FLO Solutions now at PAS( Home Office) and drive a new type of hospitality. Explore career opportunities in a unique hospitality environment with some of the industry's best compensation and benefits, includ ...

  • ExodusPoint

    Data Engineer

    6 days ago

    ExodusPoint New York, United States

    ABOUT EXODUSPOINT · ExodusPoint Capital, founded in 2017 by Michael Gelband and Hyung Lee, began managing investor capital in 2018. The firm employs a global multi-strategy investment approach, seeking to deliver compelling asymmetric returns by combining complementary liquid str ...

  • New York Blood Center

    Data Engineer

    6 days ago

    New York Blood Center New York, United States

    · Data Engineer · Job Locations · US-NY-New York · Job ID · Category · Information Technology · Minimum Rate · USD $100,000.00/Yr. · Maximum Rate · USD $115,000.00/Yr. · Work Location Type · Hybrid · Overview · At New York Blood Center Enterprises (NYBCe), one of the most ...

  • Oxford Knight

    Data Engineer

    1 week ago

    Oxford Knight New York, NY, United States Full time

    Data Engineer wanted for systematic arm of globally recognised hedge fund to join the growing Data Engineering team where you will support the Commodities business. This opportunity will give you the chance to design, implement and evolve the data platform, including pipeline, re ...

  • Open Systems Technologies

    Data Engineer

    1 week ago

    Open Systems Technologies New York, United States

    A financial firm is looking for a Data Engineer to join their team in New York, NY. · Compensation: $150-200k · Responsibilities: · Build data pipelines · Work with data vendors such as Bloomberg, Refinitiv, etc. · Take vendor data and normalize it for firm consumption · Qualifi ...