Data Engineer

The Deputy Manager of Engineering (Data Engineers) is responsible for developing, testing, and maintaining ETL (Extract, Transform, Load) processes and designing data pipelines using modern tools such as Apache Airflow and Apache Kafka which currently CDX Data team use. The role ensures data quality and integrity across all data stores, collaborates with DevOps to deploy and monitor data infrastructure, and manages data storage solutions including Data Lakes, Data Warehouses, and Big Lakes. PIC also expected to perform data mining and advanced statistical analysis towards several CDX Product projects

Duties and Responsibilities
Developing, testing, and maintaining ETL (Extract, Transform, Load) processes.
Designing and implementing data pipelines using modern tools such as Apache Airflow and Apache Kafka.
Ensuring data quality and integrity across all data stores.
Collaborating with DevOps to deploy and monitor data infrastructure.
Managing and optimizing data storage solutions, including Data Lakes, Data Warehouses, and Big Lakes.
Ensuring high availability and disaster recovery of data infrastructure.
Implementing and managing scalable cloud-based data solutions.
Performing exploratory data analysis to identify trends and patterns.
Creating and maintaining dashboards and reports using BI tools like Tableau and Power BI.
Working with stakeholders to gather requirements and deliver analytical solutions.
Building and deploying machine learning models to solve business problems.
Performing data mining and advanced statistical analysis.
Collaborating with data engineers to ensure models are integrated into production.
Managing data stewardship programs and ensuring data lineage and metadata management.
Monitoring and improving data quality across the data ecosystem.
Implementing data quality checks and validation routines.
Working with other teams to resolve data quality issues and ensure consistent data standards.
Automating the deployment of data applications and infrastructure.

Monitoring the performance and availability of data systems.
Troubleshooting and resolving infrastructure and application issues.

Qualification
 Degree in Computer Science or in any related discipline or equivalent experience. With at least 2 year experience in application development and support areas plus 1 year experience in management.

Years of Experience
 More than 2 year experience in application development and support areas
 More than 2 year experience in banking and management.

Specific Skills/Knowledge and Certification Required
 Must have supervisory experience, ability to plan and control projects, ability to articulate applications strategy and approach, and ability to communicate.
 Has Make-it-happen-fast attitude, acknowledged expertise, hands-on orientation and must command respect of Business Department Heads.