Discovering the unusual or unexpected in scientific data is often the path to the most fundamental breakthroughs. Yet as scientific data become more complex, our ability to find the unexpected is rapidly diminishing. New scientific instruments will regularly create data sets of 50-100 TB or more in a single observing project, data sets which are far too large to be examined using existing methods. Yet close scrutiny of the data is required to discover unexpected signals or unusual events. A paradigm shift must occur in the approach used to view and analyze these massive data sets. NRAO is teaming with university, industry and other national laboratories to research, design, and test the algorithms and tools needed for modern data analysis.
- Innovations in Data Intensive Astronomy workshop (Held May 3-5, 2011)