Inventory Overview
This page visualises metadata about the data files stored at https://data.wa.aemo.com.au/public/
Data is gathered every 6 hours, though no listed data files are downloaded in the process.
Number of files modified by day
The following gives an indication of last time of update for all files found; you can filter the starting year using the dropdown box below.
Most Common File Types
Looking at file extensions simply by type alone can be misleading. In the following bar chart, CSV file types appear dominant; in reality, a small number of services regularly update (and seldom archive) CSV files.
However, if we review the file sizes in aggregate, it becomes clear the majority
of file size is attributable to zip files.
Typically, AEMO archives 1 day of JSON files for selected data models, each day.
How many files and directories are there?
See how many directories and files are being updated in the pre-canned periods
Total Directories
Total Files
Total Size (Gb)
Directories Active in Period
Loading...Files Active in Period
Loading...Which files are updated repeatedly?
It is useful to know which files are updating most regularly. This is calculated by identifying files that:
- Are listed in every inventory run
- Have been updated within a threshold of the inventory run commencement time
This manages to avoid including files that are recently updated on an inventory run but are later archived into a daily zip file.
Click on any of the table entries above to download the listed file.
File Growth
The following Bubble Chart shows the periods of high file size drops.
The most recent time periods (to the right of the graph) will always show large bubbles because of the temporary presence (typically 24 hrs) of very large JSON files per dispatch interval.
File Size Distribution
To get an idea of where file volume is hierarchically, a Sankey Diagram is useful to descend between paths with a visual indicator as to their aggregate size volume.
Below you can see that the recent WEM Dispatch Engine covers a large proportion of the overall data.
File Count Distribution
Let's repeat the same process but for number of files, instead of storage volume
This space intentionally left blank