AI/ML Engineering Director - Marietta - Boston Consulting Group

Description

Who We Are

Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we help clients with total transformation-inspiring complex change, enabling organizations to grow, building competitive advantage, and driving bottom-line impact.

Maximise your chances of a successful application to this job by ensuring your CV and skills are a good match.

To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge management consulting along with technology and design, corporate and digital ventures—and business purpose. We work in a uniquely collaborative model across the firm and throughout all levels of the client organization, generating results that allow our clients to thrive.

What You'll Do

Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we help clients with total transformation-inspiring complex change, enabling organizations to grow, building competitive advantage, and driving bottom-line impact.

To succeed, organizations must blend digital and human capabilities. Our diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark change. BCG delivers solutions through leading-edge management consulting along with technology and design, corporate and digital ventures—and business purpose. We work in a uniquely collaborative model across the firm and throughout all levels of the client organization, generating results that allow our clients to thrive.

What You'll Do

As the Director of AI/ML Engineering within BCG's Data Layer Team, you will lead, and operationalize the intelligence layer that transforms BCG's vast data landscape into AI-ready, high-value capabilities. You will direct the design, development, and optimization of AI/ML systems - particularly LLM-powered reasoning, retrieval, enrichment, and knowledge synthesis - that enable BCG's GenAI applications (e.g., PA GenAI tools, AI copilots) to deliver accurate, safe, and context-aware insights at scale.

You will own the technical strategy for how data becomes usable in advanced AI workflows: from ingestion, semantic modeling, embeddings, retrieval architectures, and model pipelines to the construction of reusable AI services and APIs/MCPs that support multiple use cases across the firm.

You will lead an engineering team and collaborate closely with Product Owners, Data Engineers, Solution Architects and domain experts to ensure our AI/ML systems are robust, scalable, secure, and aligned with enterprise architecture standards. This is a highly visible strategic role shaping BCG's long-term AI enablement agenda.

Key responsibilities

AI/ML Engineering Leadership & Strategy

• Define and own the AI/ML engineering strategy for the Intelligence Layer, ensuring alignment with Data Layer vision and enterprise architecture principles
• Architect end-to-end AI and LLM systems (RAG pipelines, embeddings, retrieval frameworks, model orchestration) to make BCG's data actionable and consumable by GenAI applications.

Technical Design & Delivery of AI Systems

• Lead the design and implementation of scalable pipelines for data preprocessing, feature/embedding generation, knowledge graph construction, semantic indexing, and retrieval optimization.
• Oversee integration of AI services into enterprise systems, ensuring reliability, security, observability, and compliance with internal standards.

Building the Intelligence Layer

• Partner with Product Owners, Data Engineers and Architects to ensure pipelines reflect highest standards of data quality, provenance, governance, and ML-readiness.
• Drive implementation of advanced retrieval, reasoning, and grounding systems to support BCG's internal GenAI products.

Cross-Functional Collaboration & Influence

• Collaborate with Product Owners and GenAI teams to translate user needs into technical capabilities, ensuring measurable, high-value outcomes for consulting teams and practice areas.
• Engage with TAL, PPL, and other engineering chapters to align resource strategy, scale best practices, and evolve AI/ML engineering capabilities.

Quality, Performance, and Innovation

• Ensure systems meet stringent performance, accuracy, stability, and resilience requirements -leveraging expertise such as prompt engineering, LLM optimization, evaluation frameworks, and multi-agent pipelines.
• Introduce emerging techniques (fine-tuning, model compression, retrieval optimization, evaluators, guardrails) to continuously improve system effectiveness.

Talent Leadership & Chapter Development

• Lead, mentor, and develop Data engineers driving excellence in modern engineering, responsible AI, and advanced ML craftsmanship.
• Contribute to community building, knowledge sharing, and uplift across the AI/ML and Data Layer engineering chapters.

What You'll Bring

• 12+ years of experience in AI/ML engineering, including 3–5 years leading teams building production-grade ML/LLM systems.
• Demonstrated expertise in LLM-based system design, including embeddings, retrieval architectures (VectorDBs, OpenSearch, semantic search), RAG patterns, and multi-agent systems.
• Strong software engineering background across Python, modern ML libraries, APIs/microservices, and cloud-native infrastructure (Containers, Kubernetes).
• Deep understanding of data engineering, modelling, and pipeline design to support ML workloads (aligns with AI/ML MSD3 requirement to design infrastructure for ML/AI systems).
• Proven ability to define architecture strategy, technical roadmaps, and solution blueprints for complex AI/ML systems at enterprise scale.
• Exceptional cross-functional leadership: able to influence senior stakeholders, resolve ambiguity, and set direction across distributed teams.
• Strong grounding in data governance, model evaluation, responsible AI, and security considerations for enterprise deployments.

Who You'll Work With

• GenAI Product Portfolio Lead (PPL), Technical Area Lead (TAL) and Data Layer Product Owner (PO) to shape AI/ML Data strategy and align on resourcing and architecture.
• Data Engineers, Solution Architects and QA engineers to build high-quality, ML-ready pipelines.
• GenAI product teams (CT/PA AI applications) to integrate intelligence-layer services into end-user experiences.
• AI Engineering & LLM Engineering chapters to share best practices, contribute to chapter growth, and align on standards.
• Agile Coaches and Product Owners to drive iterative delivery and ensure alignment with user needs. xhmxlyz

Additional info

This is a hybrid role, typically 2–3 days/week in office. Travel requirements are expected to be