Job Description

We are seeking a Senior Data Engineer (Databricks) to design, operate, and continuously improve our SAP data replication platform. The platform streams change data from SAP S/4HANA / ECC sources into downstream like Databricks. This is used across analytics, finance, supply chain, and operations. You will own end-to-end pipeline engineering roadmap.

Key Responsibilities

Pipeline Engineering

• Design, build, and continuously improve pipelines from SAP to downstream
• Develop and continuously improve on Kafka / Confluent topics, connectors, schemas (Avro / Schema Registry), and stream-processing jobs.
• Build automation for deployment, validation, refresh, and reconciliation.
• Tune capture and integrate jobs for throughput, lag, and resource efficiency.

Platform & DevOps

• Deploy and manage Kafka and HVR components on AKS / Kubernetes and Azure infrastructure.
• Maintain CI/CD pipelines (GitHub Actions / Azure DevOps) and IaC (Terraform / Bicep).
• Enforce security, RBAC, secrets management (Key Vault), and network segmentation.

Collaboration & Governance

• Partner with SAP Basis, Network, Security
• Author runbooks, architecture diagrams, and knowledge-base articles.
• Contribute to data-quality and contract-testing frameworks. Experience
• 5+ years in data engineering or platform engineering, with 3+ years in streaming / CDC pipelines preferred
• Hands-on production experience with CDC tool preferred (ex. HVR, Debezium)
• Hands-on production experience with Apache Kafka or Confluent Platform (topics, partitions, consumer groups, Schema Registry).

Job Qualifications

Experience

• 5+ years in data engineering or platform engineering, with 3+ years in streaming / CDC pipelines preferred
• Hands-on production experience with CDC tool preferred (ex. HVR, Debezium)
• Hands-on production experience with Apache Kafka or Confluent Platform (topics, partitions, consumer groups, Schema Registry).

Technical Skills

• Strong SQL (HANA, Spark SQL, or T-SQL) and understanding of SAP table structures (ACDOCA, BSEG, MARA, etc.).
• Proficiency in Python for automation, REST clients, and data tooling.
• Linux administration and shell scripting (Bash / PowerShell).
• Kubernetes / AKS workload deployment and troubleshooting.
• Cloud experience: Azure preferred (Storage, Key Vault, Monitor, Event Hubs); AWS/GCP equivalent acceptable.
• Observability stack: Grafana, Prometheus, log aggregation (ELK / Splunk / Log Analytics).
• Git and CI/CD pipelines (GitHub Actions, Azure DevOps, or Jenkins).
• Concepts
• Log-based vs trigger-based CDC; exactly-once vs at-least-once semantics.
• Schema evolution, backward/forward compatibility, contract testing.
• Event-driven and lakehouse (medallion) architecture patterns.

Soft Skills

• Strong written communication and documentation discipline.
• Comfort with on-call and production-incident ownership.
• Cross-functional collaboration in a regulated enterprise environment.

Preferred Qualifications (Nice-to-Have)

• Direct experience with Fivetran HVR6 (REST API, Hub/Agent, channel design).
• SAP S/4HANA / ECC functional knowledge (O2C, P2P, R2R) or basic ABAP / CDS view literacy.
• Databricks / Delta Lake / Unity Catalog development experience.
• Kafka Streams, ksqlDB, or Apache Flink.
• Terraform / Bicep infrastructure-as-code.
• Snowflake or other cloud data warehouse experience.
• Data quality frameworks (Great Expectations or custom DQ engines).
• Scala / Java for stream-processing internals.
• Prior experience operating 24/7 mission-critical replication for finance or supply-chain systems.

We offer

• P&G-sized projects and access to world leading IT partners and technologies from Day 1.
• Wide range of self-development possibilities (training and certifications paths).
• Competitive starting salary and benefits program (private health care, P&G stock, saving plans, sport cards).
• Regular salary increases and possible promotions - in line with your results and performance.
• Opportunity to change role every few years to be in the best place for you and best for P&G.