Data Engineer (Banking)

Build and maintain robust data pipelines for complex data projects with / without AI component:
- programmatically ingesting data from different static and real-time sources (incl. web scraping)
- define optimized schemas and data models for scalable implementation of a solution
- log granular usage of web / mobile apps for comprehensive product analytics
Perform ETL on large and complex datasets for AI applications - work closely with data scientists on performance optimization of large-scale ML/DL model training
Build data tools to facilitate fast data cleaning and statistical analysis
Industrialize ML / DL solutions and deploy and manage production services; proactively handle data issues arising on live apps
Resolve issues escalated from Business and Functional areas on data quality, accuracy, and availability
Work closely with APAC CDO and coordinate with a fully decentralized team across different locations in APAC and global HQ.

REQUIREMENTS:

5+ years' experience in data engineering role
Expert in Python programming especially handling large volumes of data both in traditional and distributed environments - pandas, numpy, scikitlearn, pySpark
Well versed with structured (SQL) and unstructured (noSQL) database in traditional and Big data environments (Hive, Spark)
Well versed with concepts like like containerization (Dockers, Kubernetes, Openshift), workflow management (AirFlow), streaming data (Kafka or Spark Streaming), CICD pipeline (Jenkins)
Experienced in shell scripting and writing schedulers
Some experience with Cloud deployment (AWS / Azure / GCP) would be a big plus
Strong knowledge of data security best practices

SEARCH ALLY PTE LTD | [EA License Number] 18C9289 | [Registration Number] R155129

Send job

Sign in to create job alerts

Sign in or create an account to start creating job alerts and receive personalised job recommendations straight to your inbox.