Visualization of three-dimensional image stacks from serial sectioned histology or from other modalities poses a challenge, and solutions are often limited to projections or to rotated views of the 3D stacks. Here, we present SparStVR, a virtual reality application for interactive 3D image data visualization. A virtual environment with textured 3D models and, when desired, embedded measurements is generated from a stack of labeled masks and stacks of color images. In addition, modeling whole sample from sample masks and embedding multimodal measurements from heatmaps are supported. The possibility to interactively and intuitively study 3D structures and measurements in virtual reality is a ground-breaking feature compared to traditional tools. SparStVR was designed especially for visualization of sparse 3D histology data, yet it can be used for any 3D image data with objects of interest to model. SparStVR enables discoveries from complex data of tissue structures and pathology in an unprecedented manner.