Overview
Data Engineer Jobs in Johannesburg Metropolitan Area at Indsafri
Title: Data Engineer
Company: Indsafri
Location: Johannesburg Metropolitan Area
We are seeking the Data Engineer plays a pivotal role in advancing the organization’s technical capabilities by developing and maintaining robust data infrastructure. You will be responsible for building, monitoring, and optimizing data pipelines including ingestion, provisioning, streaming, and API solutions to ensure the availability of clean, high-quality data. By supporting advanced analytics, machine learning, and AI initiatives, you will contribute directly to our strategic goal of becoming a data-driven organization, ensuring all data products align with enterprise architecture standards.
Responsibilities:
- Manage day-to-day data pipeline operations, including data profiling, cleaning, configuration, and validation to support data squads and epics
- Ensure the security, availability, and reliability of the data infrastructure
- Maintain end-to-end data pipelines, spanning data virtualization, ingestion, and provisioning
- Ensure monitoring systems are active and pipelines run successfully, with the ability to implement minor configuration changes
- Develop virtual databases and generate precise data extracts to meet evolving business requirements
- Partner with Data Analysts to perform data profiling, validation, and comprehensive documentation to support project epics
- Provide first-line support for the Data Warehouse, ensuring strict adherence to Service Level Agreements (SLAs) regarding data availability and system reliability
- Monitor and manage cloud-based compute and storage processes to ensure the seamless execution of cloud pipelines
- Generate daily operational reports to track job success rates and ensure the data warehouse is maintained to enterprise standards
- Collaborate with business stakeholders to gain domain knowledge, translate business requirements into technical solutions, and optimize business queries
Required Skills and Qualifications:
- Proven experience in data engineering, including data ingestion, transformation, and provisioning (ETL/ELT)
- Strong experience in building and maintaining data pipelines and streaming technologies
- Solid understanding of data virtualization, SQL, and database management
- Proficiency in monitoring and managing cloud infrastructure (compute and storage)
- Strong data profiling, quality assurance, and validation skills
- Ability to provide first-line technical support and troubleshoot complex pipeline issues
- Excellent collaboration skills with the ability to translate business needs into technical data solutions
Preferred Qualifications:
- Relevant degree in Computer Science, Information Technology, or a related quantitative field
- Experience working within an Agile/Scrum environment (Data Squads/Epics)
- Exposure to Big Data ecosystems and enterprise data architecture roadmaps
- Certification in cloud platforms (e.g., AWS, Azure, or GCP)