Analytics & Visualization
Overview and Strategies
Data analysis and data mining tools are used to compare datasets or find features within a dataset. Visualization of data is one of the primary tools for data exploration, and may precede or inspire more formal data analyses. The technologies described above may be used individually or together to explore data. Data exploration discusses how analytics technologies can be integrated to provide a framework for discovery. In general, scientific data management and workflow management are enabling technologies. Scientific data management provides tools for efficient access to large amounts of data, as well as supporting data organization and security. Workflow management describes a systematic approach to data processing pipelines or the pre-processing and post-processing steps involved in running simulations. Workflow management tools can be used to automate repetitive processing tasks and make processing pipelines more robust.
discusses how analytics technologies can be integrated to provide a framework for discovery.
How to choose the right system at NERSC:
- Small datasets
- Large datasets
How to choose the right tools and software to use
Analytics and Visualization Software at NERSC
Data analysis and visualization are two steps in data understanding, often interleaved and symbiotic, so many of the available tools characterized as one category, end-up having some functionalities of the other. Bellow find a list of tools organized under Analytics or Visualization, but have in mind that they may have a yet large intersection in terms of their functions.
Visualization
Analytics
Visit
Matlab
Paraview
Mathematica
Python
AVS
R
Python tools - Numpy, Scipy, iPython, matplotlib
TCL/TK
SQL
Link to Visit Software page
Link to Paraview Software page
R
SQL
Matlab
Mathematica
Python tools - Numpy, Scipy, iPython, matplotlib
(This may be a restful query, a dynamic list of Visit pages.)
Case Studies
(Dani to decide what needs to be brought over)
Visualization
Workflow Management?
Data Analysis and Mining
Data Exploration
-
Case Study: Astrophysics
Quick Links: NERSC Tools for Data Exploration
discusses how analytics technologies can be integrated to provide a framework for discovery.
How to choose the right system at NERSC:
- Small datasets
- Large datasets
How to choose the right tools and software to use
Analytics and Visualization Software at NERSC
Data analysis and visualization are two steps in data understanding, often interleaved and symbiotic, so many of the available tools characterized as one category, end-up having some functionalities of the other. Bellow find a list of tools organized under Analytics or Visualization, but have in mind that they may have a yet large intersection in terms of their functions.
| Visualization | Analytics |
| Visit | Matlab |
| Paraview | Mathematica |
| Python | |
| AVS | R |
| Python tools - Numpy, Scipy, iPython, matplotlib | |
| TCL/TK | |
| SQL |
Link to Visit Software page
Link to Paraview Software page
R
SQL
Matlab
Mathematica
Python tools - Numpy, Scipy, iPython, matplotlib
(This may be a restful query, a dynamic list of Visit pages.)
Case Studies
(Dani to decide what needs to be brought over)
Visualization
Workflow Management?
Data Analysis and Mining
Data Exploration
-
Case Study: Astrophysics
Quick Links: NERSC Tools for Data Exploration
Visualization
Visualization facilitates data exploration. It often supports simulation since it allows inspection of the output varying in time or with changes in parameter values, or for the locations of interesting regions in large data sets. Read More »
Data Exploration
Among the modalities of data exploration, visual exploration is often necessary to guarantee that the algorithms are performing what they are supposed using some dataset. This page illustrates some of the scientific problems explored in terms of visualization tools. Read More »
Data Analysis and Mining
Data analysis techniques include post-processing (e.g., data statistics) of experimental datasets and/or simulation output, as well as the use of mathematical methods (e.g., filtering data) and statistical tests. Data mining usually refers to the application of more advanced mathematical techniques such as classification, clustering, pattern recognition, etc. Read More »
Strategies for Choosing Analytics and Visualization Software and Hardware at NERSC
Data analysis (or analytics) and visualization are two steps in data understanding, often interleaved and symbiotic, so that many of the available tools characterized as one category, end-up having some functionalities of the other. Bellow find links to software tools grouped under Analytics or Visualization, but have in mind that their functions may be interchangeable. Visualization Analytics Visit Matlab Python tools: Numpy, Scipy, iPython, matplotlib Paraview Mathematica Perl IDL Python… Read More »
Workflow Management
Workflow management refers to the process of connecting various software tools based on specific input and output parameters. The goal of workflow management is to automate a specific sets of tasks. Read More »
