IN015: Compressing Scientific Data: Reducing Storage While Preserving Information
Conveners: John Clyne, Dorit Hammerling, and Allison Baker – National Center for Atmospheric Research
Due to rapid technological progress our ability to generate increasingly larger data sets from high resolution numerical models is outpacing our ability to store, manage and effectively access these vast volumes of data. Similar statements can be made with regard to observational data captured by a variety of advanced instruments. One potential solution to this Big Data dilemma is the use of compression. Lossless compression offers perfect reconstruction, but provides only limited compaction when confronted with floating point data. Lossy compression, however, is able to achieve substantial reduction, but by its very definition is unable to exactly reproduce original values. The challenge then becomes addressing the question: how much information loss may be tolerated without affecting the interpretation of results, and how can this best be achieved? This session will address a variety of scientific data compression topics including: novel technologies, applications, and methods for evaluation.
Abstracts are due 2 August 2017 at 23:59 ET (11:59 P.M. EST), and may be submitted via the AGU abstract submission page: https://fallmeeting.agu.org/20