Author Archives: mngu012

Recent Advances in Online Computational Stereo Vision

We present a fully featured, web-based, online autostereogram creation system [bibcite key=webIvcnz11,minhPhd] that allows a user to upload their own stereo images, generate depth data via computational stereo vision, and then turn this depth data into an autostereogram. The system can also perform the reverse process and extract depth data from a given autosteregram or generate anaglyphs from them. By leveraging the parallelism of modern graphics processors (GPUs), the system can process video streams, creating depthmaps and converting them into autostereogram videos at real-time frame-rates transmitted over the internet. For usability the system provides automatic image rectification for the user provided stereo image pairs. These novel features place the system ahead of current alternatives and allows a wide variety of users to experience stereo reconstruction and autostereogram generation in a quick and easy manner. Additionally, the system could serve as a platform for online based visual perception studies.
Click here to go to the Project page.

Online Autostereogram Creation using Stereo Vision Technique

We present a fully featured, web-based, online autostereogram creation system that allows a user to upload their own stereo images, generate depth data via computational stereo vision, and then turn this depth data into an autostereogram.

The system can also perform the reverse process and extract depth data from a given autosteregram or generate anaglyphs from them.

By leveraging the parallelism of modern graphics processors (GPUs), the system can process video streams, creating depthmaps and converting them into autostereogram videos at real-time frame-rates transmitted over the internet.

For usability the system provides automatic image rectification for the user provided stereo image pairs.

These novel features place the system ahead of current alternatives and allows a wide variety of users to experience stereo reconstruction and autostereogram generation in a quick and easy manner.

Additionally, the system could serve as a platform for online based visual perception studies.

A Fast and Simple Method to Simulate 3D Scene View Navigation, Generated from a Pair of Stereo Images

Our versatile web-based system allows users to dynamically generate visible surfaces of three-dimensional (3D) scenes from stereo pairs from monocular or stereo cameras including web-cams.

The current advanced version accepts static images or live video sequences from different imaging sources (via direct or indirect Internet uploads), rectifies these images automatically, and processes the rectified images to reconstruct the scene by one of the available stereo matching algorithms.

Results of processing are returned to the user in multiple formats, such as a disparity map, an anaglyphic image, an autostereogram, a virtual WebGL-HTML5 or Java3D scene, a 3D .OBJ file, or a live depth video. The present system is portable, simple to set up and operate, and currently available online at http://www.ivs.auckland.ac.nz/quick_stereo.

A variety of possible applications include remote camera control, on-line calibration and rectification, simple 3D object and avatar reconstruction, web-based real-time stereo matching, artistic creation of auto-stereograms, etc.

Figure 1. 2D Morton key used to find the location of a particle in a Tree. We travel down the tree structure using a pair of bits at the time, the first bit from the y coordinate the second from the x coordinate, the final key location for the particle is 1.10.01.10

We propose the use of the Lagrangian method Smoothed Particle Hydrodynamics (SPH) to model the behaviour of fluids through a pore structure of a volcanic soil core sample. Such studies are of importance to simulate preferential flows which are essential in the leeching of organic and chemical compounds in underground aquifers. Our approach combines X-ray CT imaging, and image processing techniques to extract the 3D porous structure. SPH modelling simulates water transport in the reconstructed 3D pore network using Navier Stokes governing laws. In this work an isolated pore is considered as a case-based study.

To obtain the computational solution of the set of differential equations over time the SPH approach is used. In SPH a set of particles (differential volumes) are introduces to approximate the continuum solution.

In Fig. 1 is shown the volumetric flow of the fluid at different depth levels. SPH provides with a point by point solution of the fluid behaviour.

Brain region interactions could reveal much about the structure of cognitive systems, and might be measurable using time-lagged correlation of BOLD data collected using fMRI. Previous researchers have described methods for detecting time-lagged correlation between region of interest (ROI) activation, primarily variants on Granger causality (GC). With appropriate caveats, GC can draw inferences from temporal precedence about effective connectivity between ROIs in a way methods like SEM and DCM do not. Some studies examined circumstances where time-lagged correlation between ROI activation can help draw causal inferences from BOLD data, and tested the limits of poor temporal resolution of fMRI on this method. The current project examines whether time-lag analysis can estimate the direction of causation in frontal and parietal areas known to act together in spatial working memory tasks. Independent Component Analysis is used to identify components whose interactions are then examined using the GC method. The method successfully identified causal relationships on replicated, artificially simulated data, but has not yet significantly detected GC relationships between ROIs in the spatial working memory task. Changes under way to better detect GC relationships include multivariate Granger analysis, a frequency-domain approach, and optimizing selection of components.

Simulating the collision between fluids as solids under gravity.

The use of Graphics Processing Units (GPUs)in computationally intensive applications is increasing due to recent frameworks that allow general purpose programming to be done on the specialized hardware of the GPU. This project summarizes the use of a GPU to improve the performance of a Smoothed Particle Hydrodynamics (SPH) program.

SPH is a computational method of simulating the behaviour of particles. Particles can be used to represent objects as a collection of points where physical properties are known. The interaction of particles in a system of objects can be simulated by interpolation using local particle neighbourhoods.

The Compute Unified Device Archtecture (CUDA) was used to develop a parallel implementation of the SPH algorithm developed by Alfonso Gastelum Strozzi. CUDA is the framework that allows general purpose code to be executed on NVIDIA GPUs.

So far, the physical calculations have been ported to the GPU and current work involves developing parallel octrees to efficiently search for particle neighbourhoods. Performance on physical calculations have been approximately 4x. Expected performance is to be at least 10x. This will be the goal of optimisation part of this research where the algorithms will be refined and hardware support maximised.

Tsukuba

Venus

The belief propagation based stereo approach approximates the minimum energy solution on graphical models such as Markov Chains, or Markov Random Field (MRF) of disparities. Our approach exploits a symmetric Cyclopean matching model, which accounts for visibility conditions, to construct epipolar profiles which are close to the human perception. Unlike traditional asymmetric matching models, this model can construct disparity maps with respect to the left, right or Cyclopean reference frame, as well as a Cyclopean image of a 3D scene depicted in a stereo pair, simultaneously.

We focused on both one-dimensional (1D), and two-dimensional (2D) belief propagation. 1D belief propagation has the advantage of fast computation, and low memory usage, but suffers matching errors due to the lack of vertical information. 2D belief propagation is more memory intensive, and has slower computation speed, but it can achieve high quality results using the powerful 2D message passing, where matching information is passed around the MRF, and decisions are made using all the image information.

The results of symmetric 2D belief propagation are shown.

Here is a pair of stereo frames taken by the stereo webcam. The green bounding boxes indicate our tracking subject – the yellow ball.

Although most of the existing 3D sculpting programs (e.g. Pixologic Z-brush, Autodesk Mudbox, etc.) can be controlled by special hand-controlled hardware such as a conventional or 3D mouse, or a tablet, it will be more convenient and natural to let users sculpt 3D models just by motions and poses of bare hands. Such a control interface allows the traditional clay sculptors or people with no sculpting knowledge to utilize the above digital programs readily without prior training. Therefore, an image-based hand tracking system can turn the device-free control interface into reality.

This Masters project aims to develop a real-time stereo system for hand tracking by means of a pair of synchronized video cameras in order to operate a 3D sculpting program. To reduce the project’s complexity, we aim to use marker-based tracking, which requires users to wear gloves with special colour patterns to be tracked.

The tracking unit produced 3D coordinates by triangulation and sent to Blender via TCP connection to move the yellow cube inside the 3D viewport of Blender.

At this stage, we have a simple working prototype of the system created. This prototype is connected to the Minoru stereo web cam for live-stream video input. It is capable of tracking more than one object at the same time via the Continuous Adaptive Mean Shift (CAMShift) algorithm. The current CAMShift algorithm adapted source code from OpenCV Library.

3D coordinates generated by the tracking unit are sent continuously via a TCP connection to Blender. Blender is selected as our choice for 3D sculpting interaction because it is the only open source software package coming with sculpting support and has free access to its source codes for further customization. By means of a customized Blender Python script, the tracked coordinates are then used to control the movements of some basic 3D cubes within Blender in real time. Different programming assets such as camera calibration, real-time video streaming from camera, etc. from the Intelligent Vision Systems (IVS) research group of the Department of Computer Science were used to complete the current prototype.