Senior Data Engineer London, United Kingdom (Hybrid)

Super Payments is seeking a Senior Data Engineer to join our global fintech team. We run an event-driven data platform on AWS and Snowflake, with data flowing from product services through EventBridge, Kinesis Firehose and S3 and ingested into Snowflake via Snowpipe. Batch and third-party data lands via Snowpark-based Python procedures. Our infrastructure is defined with Terraform and Terragrunt. Python (Poetry, Pydantic, pytest) is our pipeline language, and GitHub Actions powers CI/CD. Datadog provides end-to-end pipeline observability.

Key responsibilities: - Ingestion and Pipeline Engineering: own and evolve the EventBridge → Firehose → S3 → Snowpipe ingestion path; tune partitioning and throughput as event volumes grow across streams like fraud, payments and merchant. - Batch and Third-Party Integrations: design and deploy Snowpark-based Python procedures for file-based and third-party data loads, scheduled and monitored as Snowflake Tasks. - Schema Evolution: work with product engineers to manage upstream schema changes and late-arriving data without breaking downstream consumers. - Infrastructure and Reliability: own and extend Terraform/Terragrunt modules for ingestion infrastructure across multiple environments; design Datadog monitors and alerting for Firehose, EventBridge, DLQs and Snowpipe; triage and resolve ingestion failures. - Security and Compliance: manage PII handling (hashing, secure deletion), masking, data governance and access in Snowflake; support audit-ready, reconciled data for Finance and Comp

Senior Data Engineer London, United Kingdom (Hybrid)

Super Payments is seeking a Senior Data Engineer to join our global fintech team. We run an event-driven data platform on AWS and Snowflake, with data flowing from product services through EventBridge, Kinesis Firehose and S3 and ingested into Snowflake via Snowpipe. Batch and third-party data lands via Snowpark-based Python procedures. Our infrastructure is defined with Terraform and Terragrunt. Python (Poetry, Pydantic, pytest) is our pipeline language, and GitHub Actions powers CI/CD. Datadog provides end-to-end pipeline observability.

Key responsibilities: - Ingestion and Pipeline Engineering: own and evolve the EventBridge → Firehose → S3 → Snowpipe ingestion path; tune partitioning and throughput as event volumes grow across streams like fraud, payments and merchant. - Batch and Third-Party Integrations: design and deploy Snowpark-based Python procedures for file-based and third-party data loads, scheduled and monitored as Snowflake Tasks. - Schema Evolution: work with product engineers to manage upstream schema changes and late-arriving data without breaking downstream consumers. - Infrastructure and Reliability: own and extend Terraform/Terragrunt modules for ingestion infrastructure across multiple environments; design Datadog monitors and alerting for Firehose, EventBridge, DLQs and Snowpipe; triage and resolve ingestion failures. - Security and Compliance: manage PII handling (hashing, secure deletion), masking, data governance and access in Snowflake; support audit-ready, reconciled data for Finance and Comp