ABC
How the ABC Made 91 Years of Archives Searchable in Milliseconds
91 Years of History, Searchable in Milliseconds
Kablamo migrated 6 petabytes of media and 3 million records for the ABC, replacing a three-week manual search process with a platform that returns results in milliseconds. The archive spans 91 years of Australian broadcasting history.
“The process was slow, manual, inefficient and unacceptable, so we had to fix it.”
ABC, Content Management
The Challenge
The ABC's archive spanned 91 years of Australian broadcasting history, with over 11 million hours of video and audio files spread across multiple siloed systems. Finding specific content meant navigating five disconnected on-premise storage locations, physical warehouses, and manual processes that could take up to three weeks to complete a single search. Thousands of content creators, journalists, and internal users relied on the archive daily, but the process was, as the ABC's content management team described it, "slow, manual, inefficient and unacceptable."
The organisation needed to overhaul its content library, moving away from scattered warehouses, disconnected metadata systems, and storage locations that had never been fully digitised. The goal was to consolidate metadata from all five systems, migrate petabytes of media into a single cloud platform, and deliver a search experience that could locate archives in seconds instead of weeks. The platform needed to serve the entire breadth of the ABC's operations, from historical research to live news and radio production, supporting the thousands of users who depended on the archive every day.
.png)
The Approach
The migration came in two parts. First, multiple metadata sources were merged into a single golden record following a new format designed collaboratively with the ABC. This metadata consolidation involved reconciling records from five separate legacy systems into a unified schema. Over three months, the process iterated with constant updates and user feedback, moving several gigabytes of metadata into AWS S3. The collaborative design process ensured the new format addressed the needs of archivists, journalists, and content producers across the organisation, resolving inconsistencies between the legacy systems and establishing a single source of truth for every record.
Second, video, audio, and photo media was migrated and aligned with the newly standardised metadata. Media was organised in S3 buckets with unique prefixes and filenames matching IDs, creating a consistent and predictable storage structure that could be accessed programmatically. The migration handled content spanning the full history of Australian broadcasting, from early radio recordings through to modern high-definition video. Each media file was linked to its corresponding metadata record, ensuring that search results could immediately surface the associated audio, video, or image alongside its descriptive information.
.png)
Using AWS Machine Learning services, Kablamo implemented Amazon Transcribe to ingest and process archival content for transcription, reducing reliance on manual tagging. ABC audio and video archives achieved transcription accuracy exceeding 90%, making decades of previously unsearchable audio and video content discoverable through text queries for the first time. These synchronised processes moved 3 million records into the CoDA (Content Digital Archive) system, totalling 6 petabytes of audio, video and photo content. The strategy was extended to support ingestion and export for other systems, including live news video desk editing and live radio editing, making CoDA the central hub for content flowing across the organisation.
The Results
A successful cloud archive search prototype was ready within six weeks, demonstrating that the approach was viable and that the new search experience represented a step change over the existing process. CoDA hit production within three months. The five legacy storage systems became obsolete, with all content housed in the AWS cloud platform for scalable, remote access and ongoing cost savings. Storage costs decreased as the organisation moved from maintaining physical infrastructure and on-premise servers to a pay-as-you-go cloud model.
In CoDA's first six months, nearly two million archives had been uploaded while content had been processed or downloaded more than two billion times. The platform eliminated the need for physical retrieval from warehouses, and staff across the country could access the same archive simultaneously from any location. Journalists working on breaking news stories could locate and pull archival footage in seconds, a process that had previously required days of coordination with archive staff.
The machine learning transcription pipeline continued to improve the discoverability of the archive over time. As more content was processed, the searchable corpus grew, making the platform increasingly valuable to producers and researchers who needed to find specific moments in the ABC's 91-year history.
Looking Forward
Today, the ABC supports lightning-fast search, access and editing of content, with archive search time reduced from weeks to milliseconds. The CoDA platform continues to serve as the backbone of ABC's content management operations, handling the daily demands of news production, radio editing, and historical research. The platform's integration with live news and radio workflows means it is not just an archive but an active part of the content creation process, with new material flowing into CoDA as it is produced.
The CoDA transformation was presented at the NFSA Digital Directions conference in 2019, sharing how the project changed the way Australia's national broadcaster manages its content heritage. The serverless cloud architecture ensures the platform can scale with the ABC's growing archive without requiring additional infrastructure investment. This project is also published as a case study on aws.amazon.com.
.png)
