Overview
Data Engineering Lead Jobs in Johannesburg, South Africa at Blue Pearl HQ
Overview
Our client is seeking an experienced Data Engineering Lead to design, build, and lead enterprise-scale data platforms that enable business intelligence, advanced analytics, reporting, and data-driven decision‑making.
The successful candidate will provide technical leadership across the full data engineering lifecycle, ensuring scalable, secure, reliable, and high-performing data solutions. This role requires close collaboration with business stakeholders, solution architects, BI teams, governance specialists, and software engineering teams to deliver trusted and analytics-ready data assets.
The ideal candidate is a hands-on technical leader with extensive experience in cloud data platforms, ETL/ELT development, data architecture, SQL optimization, and modern data engineering practices.
Key Responsibilities
- Data Engineering Leadership:
Lead the design, development, implementation, and continuous improvement of enterprise-grade data engineering solutions. Architect scalable, secure, reusable, and high-performance data pipelines supporting batch and near real-time processing. Provide technical leadership across multiple data engineering initiatives and mentor junior and intermediate engineers. Establish and enforce enterprise data engineering standards, frameworks, and best practices. Drive innovation and continuous improvement across the organisation’s data platform capabilities. - Data Platform Architecture:
Design robust ingestion frameworks for structured, semi-structured, and unstructured data. Develop scalable storage architectures supporting data warehousing and lake house environments. Implement efficient ETL and ELT processing frameworks. Optimize cloud-native data architectures for performance, scalability, reliability, and cost efficiency. Design resilient solutions supporting analytics, reporting, AI, and machine learning initiatives. - Data Engineering & Development:
Build, maintain, and optimize enterprise data pipelines. Develop complex SQL queries, stored procedures, and transformation logic. Design and maintain reusable data integration components. Perform data transformation, cleansing, enrichment, and validation processes. Ensure consistent data availability and integrity across enterprise platforms. - Data Governance & Quality:
Embed data quality controls throughout the engineering lifecycle. Implement automated validation, monitoring, lineage, metadata management, and observability. Ensure compliance with organizational governance frameworks and regulatory requirements. Secure sensitive and confidential information through appropriate engineering controls. Maintain technical documentation including: source-to-target mappings, data dictionaries, data lineage, technical specifications, data models, and pipeline documentation. - Stakeholder Management:
Collaborate closely with business stakeholders, BI teams, data analysts, solution architects, application development teams, governance teams, and infrastructure teams. Translate business requirements into scalable technical solutions. Communicate technical concepts clearly to both technical and non-technical audiences. Provide regular project updates, technical recommendations, and risk assessments. - Operational Excellence:
Monitor production data platforms. Troubleshoot pipeline failures and performance issues. Optimize processing times and infrastructure costs. Lead root cause analysis and incident resolution. Support production deployments and platform upgrades.
Requirements
Minimum Requirements
Education
- Bachelor’s Degree in Computer Science, Information Systems, Information Technology, Software Engineering, Data Science, Engineering, Mathematics, Statistics, or a related discipline. Relevant postgraduate qualifications will be advantageous.
Experience
- 8–10+ years’ experience in Data Engineering. Minimum 3 years leading technical data engineering teams. Experience designing enterprise-scale cloud data platforms. Proven experience building scalable ETL/ELT solutions. Experience working within enterprise environments. Strong experience delivering complex data integration solutions. Experience supporting reporting, analytics, and data…
Title: Data Engineering Lead
Company: Blue Pearl HQ
Location: Johannesburg, South Africa
Category: