National Computational Infrastructure and the Data Cube

The research infrastructure services and expertise provided by the National Computational Infrastructure (NCI) is essential to the continual development of the Australian Geoscience Data Cube (Data Cube).

NCI innovation

NCI's work on the Data Cube is assisting to overcome the global issue of making more satellite data available in a form that is quick and easy to use.

Work undertaken by NCI has helped solve the problem of limited access to sufficient compute and data resources, which inhibited the capacity to process quantities of time-series observational data in a performant way and at all scales.

NCI has dedicated a number of resources to the Data Cube, including:

  • A technical environment that tightly integrates Petascale quantities of computational and data-intensive resources, with a well-managed storage environment designed for modern high performance data analysis.
  • Access to a wide range of expertise and management practises on the effective organisation of data, and the improvement of Data Cube technology for accessing Earth Observation and other data collections at NCI.
  • Expert advice in data and technology support areas that encompass data management, Quality Assurance/Quality Control processes, and use of data formats for interoperability with NCI data services as well as a broad range of application programming interfaces, tools and libraries for the various research, government and industry communities.

The work completed by the NCI means that Data Cube users can now perform continental scale multidisciplinary analyses in hours or even minutes, which previously would have been done manually and taken years.

NCI funding

Throughout its work on the Data Cube, the NCI has drawn on national research infrastructure and services funded by the Australian Government through:

  • The National Collaborative Research Infrastructure Strategy (NCRIS).
  • Major investments by federal government science organisations, including the Bureau of Meteorology, the Commonwealth Scientific and Industrial Research Organisation, and Geoscience Australia.
  • Research-intensive universities, led by the Australian National University.

Since 2011 $46 million worth of NCRIS-funded infrastructure has been used by the NCI. NCI hardware infrastructure funded as part of the NCRIS NCI project has played a key role in the development and deployment of the Data Cube. Additionally, further capability and infrastructure has been developed at NCI with funding and support from the following NCRIS projects:

  • Research Data Services (RDS) project,
  • Australian National Data Service (ANDS) program, and
  • National eResearch Collaboration Tools and Resources (NeCTAR) project.