Data Engineer

The role

We’re looking for a curious, growth-oriented Data Engineer to help us build, operate and evolve APRA’s data platforms. You don’t need experience across everything — we value mindset, adaptability, and eagerness to learn above all.

You’ll pair with experienced engineers and work across Databricks, modern data warehousing practices, and cloud-native engineering, using AI tools to accelerate development, testing, documentation, and problem-solving. The role blends BAU support with project delivery, giving you exposure to real-world engineering challenges at scale.

The team

You’ll be part of Enterprise Data Solutions in APRA’s Data, Technology & Security (DTS) division, reporting to the Applications Manager, Data. We build and run APRA’s core data and integration platforms, partnering closely with business teams to support regulatory outcomes that matter.

Key responsibilities

• Deliver reliable data pipelines: design, build and maintain ELT/ETL on Databricks (PySpark/Spark SQL); ensure data quality, lineage and observability.
• Operate at scale: optimise large datasets (partitioning, file formats, caching, cluster configs) with cost-aware design.
• Implement unit tests ,integration tests (end-to-end pipeline runs), and contract tests for API/data interfaces.
• Define data-quality tests (nulls, ranges, schema drift, referential integrity) using frameworks such as Great Expectations, dbt tests, or Databricks expectations.
• Embed tests into CI/CD with quality gates, test coverage, and automated backfill/rollback playbooks.
• Support BAU: monitor and troubleshoot pipelines, fix defects, backfill data, manage releases, and continuously improve reliability (SLOs, runbooks, automation).
• Enable analytics & warehousing: implement data models (lakehouse/warehouse patterns), curate reusable data products, manage schema evolution and CDC.
• Integrate systems: collaborate with integration teams on MuleSoft APIs and event flows; use Java fundamentals to understand upstream services and improve data contracts.

• Engineer with DevOps discipline: apply CI/CD, IaC patterns where applicable, observability (logs/metrics/traces), and change/go-live practices.

  
  Prototype prompt-engineering use cases (code/test generation, doc copilots, anomaly detection).

• Apply AI coding assistants, prompt engineering, and automation tools for: Code and test generation Documentation Data quality rule creation Troubleshooting and optimisation
• Prototype new AI-assisted workflows and share learnings with the team.
• Security & compliance first: implement controls for privacy, security, access management, and audit.
• Document & share: produce high-quality documentation, patterns, and playbooks; participate in peer reviews and knowledge sharing.

To work with us, you must be an Australian citizen with eligibility to gain a defence clearance through the Australian Government Security Vetting Agency.

About you

• Practical engineering experience gained through employment, internships, coursework, open-source contributions or personal projects, paired with a strong willingness to learn.
• Proficient in SQL and exposure to Databricks, Spark or similar data platforms.
• Understanding of data modelling concepts (warehouse, lakehouse, batch/streaming).
• Willingness to use AI tools as part of daily development workflow.
• Basic test automation skills or eagerness to learn quickly.
• Good engineering fundamentals (Git, code review practices, CI/CD concepts).

• Collaborative, proactive communicator who enjoys solving problems and learning fast.

   

Desired experience

• Experience with prompt engineering for code, tests or documentation.
• Cloud (Azure preferred) and IaC familiarity.
• API/event-driven patterns, Kafka, and streaming analytics.
• Java-based data processing or performance tuning experience.
• Experience in regulated environments (financial services, government).

About APRA

Australian Prudential Regulation Authority (APRA) was established in 1998 as an independent statutory authority that supervises almost 1,200 financial institutions that manage $8.6 trillion in assets for Australians across the banking, insurance and superannuation sectors.

In overseeing the safety, competitiveness and stability of the financial system, we seek to recruit, develop and retain highly skilled professionals, who want to help shape financial services and protect the financial wellbeing of the Australian community. Our employee base of almost 900 come predominantly from the commercial financial services industry or other government agencies; as such, we have the feel of a small corporate organisation that can work flexibly and with agility.

Why Work for APRA

We recognise the skills, experience and commitment that our staff bring to their professional lives, and we seek to reward them accordingly. We also recognise that for our staff to be able to perform at their best, we need to ensure that they are able to bring their best selves to work. Our commitment to wellbeing is having engaged people supported by resilient leaders within a values-aligned culture.

At APRA, we’re committed to providing an inclusive workplace where everyone belongs, feels valued and respected. We aspire to attract and foster diversity of background, thought, and experience, recognising that a broad range of perspectives, approaches and ideas makes us stronger, and better enables us to meet our obligation to protect the financial wellbeing of the Australian community. If you need any adjustments during the recruitment process, please inform at application stage so we can do our best to accommodate your requirements.