home/data projects/labour market
Data Pipeline — 01

Four in five working men are full-time. For women, it's barely one in two.

An end-to-end analytics pipeline on live Australian Bureau of Statistics data — Python ingests it, Azure SQL models it, and a 4-page Power BI report visualises it. The whole report is generated as code, not clicked together by hand.

Role
Solo build
Stack
Python · Azure SQL · Power BI
Source
ABS Data API (live)
Window
1978 – 2026
01 Overview

Every politician has a line about jobs. I wanted the number behind the line — not the headline rate, but the shape of who is working, in what, full-time or part, and how that's shifted.

So I built the whole stack an analyst actually uses: ingest → transform → store → model → visualise. Python pulls the ABS Labour Force series straight off the API, pandas cleans and reshapes it, Azure SQL stores and models it through a layered view pattern, and Power BI sits on top as a 4-page report covering the national picture, the states, industries, and the full-time / part-time split.

This page is that report, told as a story — the strongest findings first, the engineering that makes them trustworthy underneath.

02 The data

Everything here comes from the Australian Bureau of Statistics — the same Labour Force survey the RBA and Treasury quote. I hit the ABS Data API directly (no key required): the monthly Labour Force series for the national, state and full-time/part-time numbers, and the annual Labour Account for employment by industry. The national series runs back to February 1978 and is current to early 2026.

0
National unemployment rate — near historic lows.
0
Employed persons — a record high (14,737k).
0
Participation rate — also near record territory.
03 The build

The pipeline is deliberately boring, which is the point. The raw ABS output is messy multi-dimension data with coded labels and a wide shape; I parse the time periods, map the dimension codes to readable names, and reshape it long. The cleaned data lands in Azure SQL staging tables, then a staging → mart view layer shapes the metrics. Power BI only ever reads the curated mart views — never the raw tables.

The Power BI report itself is generated by a Python script, not hand-built in Desktop. One build_report.py writes the data model, every visual, the 15 DAX measures and a custom colour theme as text files. So the entire dashboard is diff-able, reviewable and version-controlled like code — if it ever corrupts, I regenerate it instead of fighting a binary .pbix.

It runs on serverless Azure SQL, which auto-pauses when idle — the whole thing costs roughly $2–6 a month, almost all of it storage rather than compute.

Ingest
ABS Data API
Python · live pull, no key
Transform
pandas
clean · map codes · reshape
Store + model
Azure SQL
staging → mart views (T-SQL)
Visualise
Power BI
generated as code · DAX
aus_job_dashboard.pbip — Overview
Power BI Overview page: national unemployment rate trend back to 1978, employed persons trend, and KPI cards for unemployment rate, employment, participation and emp-to-pop ratio.
Page 1 — Overview. The national headline series, seasonally adjusted: unemployment at 4.49%, employment at a record 14.7M, +0.88% year-on-year.
04 Key insights

Insight one: full-time work isn't shared evenly. The part-time economy is overwhelmingly female.

Full-time share of employment, by sex
% of jobs that are full-time
Male79.98%
Persons68.95%
Female56.92%
Read: 80% of working men hold a full-time job; for women it's 57%. The gap has narrowed over decades, but it's still a 23-point spread.
aus_job_dashboard.pbip — Full-time vs Part-time
Power BI Full-time vs Part-time page: stacked area of full-time vs part-time employment, full-time share KPIs, full-time share by sex over time, and an employment-by-sex donut.
Page 4 — Full-time vs Part-time. Full-time is 68.95% of all jobs (+0.19 ppt YoY). The by-sex lines tell the real story: male and female full-time shares have been drifting apart and back for forty years.

“The headline rate hides the composition. Who works full-time, and where, is the part that actually moved.”

Insight two: the labour market is tight almost everywhere — the states barely differ.

I expected wide regional gaps. Instead the spread between the highest and lowest state unemployment rate is under 0.9 of a percentage point — Tasmania at the top, Western Australia at the bottom, everyone bunched in between.

Unemployment rate by state — latest month
% · seasonally adjusted
Tasmania4.99%
Victoria4.82%
New South Wales4.44%
Queensland4.27%
South Australia4.19%
Western Australia4.12%
Read: a 0.87 ppt spread across the states. Honest note: the ABS feed returned 6 of 8 jurisdictions — ACT and NT were missing at source, so I left them off rather than draw a map with holes in it.
aus_job_dashboard.pbip — State Breakdown
Power BI State Breakdown page: ranked bars of unemployment rate and employed persons by state, plus an unemployment-rate trend line per state.
Page 2 — State Breakdown. Ranked bars instead of a map: a built-in Bing map rendered as a zoomed-out world view and read as broken, so I dropped it for bars that never look broken.

Insight three: health care is the country's biggest employer — but here's where I have to be honest about the data.

By headcount, Health Care & Social Assistance and Retail Trade sit at the top of the industry table. But the industry breakdown comes from the annual ABS Labour Account, which lags badly — it ends at 2022, while everything else on the dashboard is current. I confirmed via the live API that it's stale at source, so rather than quietly mix a 2022 series with current numbers, I labelled every industry title with its vintage. Knowing a source's recency before you build on it matters more than any chart.

aus_job_dashboard.pbip — Industry View
Power BI Industry View page: horizontal bar of employed persons across 19 ANZSIC divisions for 2022, a focus-industries trend line, and an industry detail table with year-on-year growth.
Page 3 — Industry View. All 19 ANZSIC divisions ranked by employment, with a growing/shrinking detail table. Titles say “2022, annual” on purpose — the source ends there.
05 What's next
  1. 01Replace the stale annual industry source with a current quarterly Labour Force Detailed “employed by industry” series, so every page is current to the same month.
  2. 02Fix the missing ACT & NT in the state mart view — chase the gap back through staging → mart filtering.
  3. 03Harden the Azure side: a budget + cost alert, secrets in Key Vault, and Azure AD auth to the SQL DB instead of a SQL login.
06 Take a look