PKU Scientific Visualization System

Advection-Based Sparse Data Management for Visualizing Unsteady Flow

Abstract


When computing integral curves and integral surfaces for large-scale unsteady flow fields, a major bottleneck is the widening gap between data access demands and the available bandwidth (both I/O and in-memory). In this work, we explore a novel advection-based scheme to manage flow field data for both efficiency and scalability. The key is to first partition flow field into blocklets (e.g. cells or very fine-grained blocks of cells), and then (pre)fetch and manage blocklets on-demand using a parallel key-value store. The benefits are (1) greatly increasing the scale of local-range analysis (e.g. source-destination queries, streak surface generation) that can fit within any given limit of hardware resources; (2) improving memory and I/O bandwidth-efficiencies as well as the scalability of naive task-parallel particle advection. We demonstrate our method using a prototype system that works on workstation and also in supercomputing environments. Results show significantly reduced I/O overhead compared to accessing raw flow data, and also high scalability on a supercomputer for a variety of applications.
Keywords: Flow visualization, Data management, High performance visualization, Key-value store

Figures


Figure 1. The pipeline of the advection-based sparse data management.

Figure 2. The data structures in the sparse data management: (a) the prefetching hint graph of a small dataset; (b) key-value store with prefetching hints. The prefeching hints are issued by the tracers, and reused by the parallel key-value store.

Citation


Hanqi Guo, Jiang Zhang, Richen Liu, Lu Liu, Xiaoru Yuan, Jian Huang, Xiangfei Meng, and Jingshan Pan. Advection-Based Sparse Data Management for Visualizing Unsteady Flow. IEEE Transactions on Visualization and Computer Graphics (SciVis'14), 20(12):2555-2564, 2014.

BibTeX

@Article{GuoZLLYHMP14,
Title     = {Advection-Based Sparse Data Management for Visualizing Unsteady Flow},
Author    = {Hanqi Guo and Jiang Zhang and Richen Liu and Lu Liu and Xiaoru Yuan and Jian Huang and Xiangfei Meng and Jingshan Pan},
Journal   = {{IEEE} Transactions on Visualization and Computer Graphics},
Year      = {2014},
Number    = {12},
Pages     = {2555--2564},
Volume    = {20},
Bibsource = {dblp computer science bibliography, http://dblp.org}
}