NRMA - Data Engineer
Role Responsibilities
· Build and maintain scalable data extraction, transformation, and loading workflows using Informatica
· Manage tables, datasets, partitions, and clustering inside Google BigQuery for large-scale analytics.
· Write and tune advanced SQL scripts and queries to boost execution speed and lower compute costs.
· Use Unix shell scripting to automate file transfers, pre- and post-load routines, and batch job sequencing.
· Develop custom Python scripts to parse files, orchestrate data movement, and interact with cloud APIs.
· Schedule, monitor, and troubleshoot integrated workflows across Unix cron jobs and cloud schedulers.
· Move legacy on-premise data assets into cloud environments via hybrid ETL pipelines.
· Implement error-handling, logging, and data validation rules using SQL and Python checks.
· Diagnose bottlenecks in Informatica mappings, BigQuery queries, and shell scripts.
· Maintain technical mapping specs, system architecture diagrams, and partner with analysts or data
Key Skills & Experience
· 4–7 years of hands-on data engineering experience designing, developing, and supporting production data pipelines.
· Experience building ETL/ELT workflows using Informatica, including mapping development, workflow scheduling, monitoring, and troubleshooting.
· Proven capability working with Google BigQuery, including dataset management, table design, partitioning, clustering, and query performance optimisation.
· Advanced SQL skills, including complex joins, window functions, stored procedures, query tuning, and data validation routines.
· Practical experience with Unix shell scripting for file handling, batch orchestration, job sequencing, and automation of operational tasks. (desired)
· Proficiency in Python for data processing, file parsing, API integration, automation, and development of reusable data engineering utilities.
· Experience supporting cloud migration or hybrid data integration initiatives, including movement of legacy on-premise data assets to cloud platforms. (desired)
· Good understanding of data quality, reconciliation, error handling, logging, and operational monitoring practices across batch and scheduled workloads.
· Ability to diagnose and resolve technical issues across Informatica mappings, SQL queries, shell scripts, and cloud data pipelines.
· Experience maintaining technical documentation, mapping specifications, operational runbooks, and working closely with analysts, testers, and business stakeholders.
· Familiarity with Agile delivery practices, code version control, release processes, and collaborative delivery across onshore and offshore teams.
· Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related discipline; relevant cloud, data, or Informatica certifications are desirable.
