Gaussian Mixture Model

Gaussian Mixture Model Many computer related vision technology, it is critical to identify moving objects from a sequence of videos frames. In order to achieve this, background subtraction is applied which mainly identifies moving objects from each portion of video frames. Background subtraction or segmentation is a widely used technique in video surveillance, target recognitions and banks. By using the Gaussian Mixture Model background model, frame pixels are deleted from the required video to achieve the desired results. The application of background subtraction involves various factors which involve developing an algorithm which is able to detect the required object robustly, it should also be able to react to various changes like illumination, starting and stopping of moving objects. Surveillance is the monitoring of the behaviour, activities or other changing information usually of people and often ina surreptitiousmanner. Video surveillance is commonly used for event detection and human identification. But it is not easy as think to detect the event or tracking the object. There are many techniques and papers introduced by many scientists for the backend process in the video surveillance. Different automated softwares are used for the analysis of the video footage. It tracks large body movements and objects. In this report we discuss the application of mixture of Gaussian (M.O.G) which is used as a background subtraction model in order to achieve object/target recognition; this method involves calculation the mean and standard deviation of each pixel frame which will be discussed more in detailed as the report progresses. 2. Background Subtraction (Literature Review) Figure 1 Background subtraction Flow Diagram [1] According to [1], the above figure shows the basic flow diagram on how background subtraction algorithm is based. The four important steps described by [1] and which play an important role in background subtraction are described below and include: 2.1 Preprocessing Temporal or spatial smoothing is used in the early pre processing stage to eliminate device noise which can be a factor under different light intensity. Smoothing technique also includes removing various other elements like environment such as rain and snow.   In real-time systems, frame size and frame rate are commonly adopted to reduce the data processing rate. Another key factor in preprocessing technique is the data format used by the background subtraction model. Most algorithms can handle luminance intensity which is one scalar value per each pixel.   As shown in the examples below [2]: Figure 2 Image on the left shows snowing and image on the right is a resultant of smoothing effect In the figure 2, shown are two images which shows snow on the left and whereas with the application of spatial and temporal smoothing on right image results in the elimination of snow producing an more clear and effective image for background subtraction. 2.2 Background Modeling This step uses the new video frame in order to calculate and update the background model. The main aim of developing a background model is that it should be robust against environmental changes in the background, but sensitive enough to identify all moving objects of interest. 2.3 Foreground Detection In this step, it identifies the pixels in the frame. Foreground detection compares the video frame with the background model, and identify candidate foreground pixels from the frame. Commonly- used approach for foreground detection is to check whether the pixel is significantly different from the corresponding background estimate as show below: A different foreground detection scheme is normalised threshold based statistics as shown below: Where,  µd and are the mean and standard deviation of the for all spatial locations (x,y). 2.4 Data Validation Finally, this step eliminated any pixels which are not connected to the image. It involves the process of improving the foreground mask based on the information obtained from the outside background model. Most background models lack three main points: 1. Ignoring any correlation between neighbouring pixels 2. The rate of adaption may not match the moving speed of the foreground object. 3. Non-stationary pixels, from moving leavers or shadow cast by moving objects are at times mistaken for true foreground objects. 3. Background Reading There are different types of background modelling which are as follows: 3.1 Simple Frame Difference Frame difference is considered the simplest form of background subtraction. The method involves the subtraction of previous pixel frame with current pixel frame and if the value obtained from this greater than a set threshold then it is considered as the foreground. Advantages: Less complex The method can easily and quickly adapt to changes Able to distinguish the background from foreground much affectively. Disadvantages: Not Robust, object has to continuously move. Only applicable when with fixed camera. Fails to handle illumination changes. 3.2 Moving Average Filtering Referring from [3], Moving Average filtering is a method in which an estimate value of a particular background pixel value is weighted as average of the previous values. Pixels in the far part are weighted at Zero and the weights increase smoothly.  Ã‚   3.3 Online Gaussian Model In this method, we assume the background as a Gaussian distribution rather than a single value. To calculate Gaussian for 1-D is as follows[4]: Where, x is the current pixel value,  µ is the mean of each pixel and sigma is the standard deviation of the pixel. Finally, the following equations are used in online updated of Gaussian mean and variance: In the above equations[4]:  µ(t) is the mean for each pixel where ÃŽÂ ± is the learning rate, t is the time of the previous pixel and x is the current pixel value. à Ã†â€™2(t) is the standard deviation of the each pixels. The foreground decision rule applied here is when the distance between each pixel value and Gaussian model is larger than a certain threshold, it is considered as the Foreground. 3.3 GMM Gaussian Mixture Model In order to use GMM the following equations are considered: In the above equations [4]:  µ(t) is the mean for each pixel where ÃŽÂ ± is the learning rate, t is the time of the previous pixel and x is the current pixel value. à Ã†â€™2(t) is the standard deviation of the each pixels. The description and implementation of GMM model will be discussed more in detail in the later part of the report 3.4 Codebook based Technique It is a method where a quantized and clustering technique is motivated by Kohonen to construct the background model from long observation sequences, without making any parameter assumptions. For each pixels, a codebook contains one or more codewords and mixed backgrounds can be modelled by using multiple codewords. Samples at each pixel are clustered into the set of codewords based on a color distortion metric together with a brightness ratio. [4] 4.0 Gaussian Mixture Model In this report we implement the Gaussian Mixture Model for background subtraction. This background subtraction model is more robust than other models discussed in previous section. Most importantly it can handle multi-modal situations e.g. trees and sky which is more effectively filtered by the GMM model. Each pixel value is modelled by a mixture of Gaussian rather than a particular type of distribution. Based on the variance of each of the Gaussian of the mixture, we determine which Gaussians may correspond to background colors. Pixel values that do not fix the background distributions are considered foreground until there is a Gaussian that includes them with sufficient consistent evidence supporting it [5]. This method is very adaptable with lighting changes, repetitive motions and slow moving objects. This method contains 2 main significant parameters -Alpha, the learning constant and T, the proportion of the data that should be accounted for by the background The Gaussian mixture components for a pixel have normalized weights calculated from the past observations. The likelihood that a pixel has a value of Xt is assumed to be:    [5] The parameters of the mixture components are updates with the new frames. A retrieved pixel value is compared with all the components of the mixture assigned to that pixel to find out if there is a match. A match is said to happen when the retrieved pixel value is within 2.5 times standard deviation of a mixture component. The update procedure is different for the matching component and other components. The mean values and the covariance matrices are updated for only the matching component. The value formulas for the matching components are: And the weights are updated with given formula    [6] In case there is not a match between the current pixel value and the mixture components related to that pixel .the component having the smallest likelihood with respect to the current pixel value is discarded. A new Gaussian component is created in place of the discarded one, having a mean value equal to the current pixel value, and a variance equal to a predetermined constant. To retrieve the foreground the following equation is used. Where, T is the threshold value for the sum of the weights.[6] 5.0 Algorithm In order to give a better understanding of the algorithm used for background subtraction the following steps were adopted to achieve the desired results: Firstly, we compare each input pixels to the mean mu of the associated components. If the value of a pixel is close enough to a chosen components mean, then that component is considered as the matched component. In order to be a matched component, the difference between the pixel and mean must be less than compared to the components standard deviation scaled by factor D in the algorithm. Secondly, update the Gaussian weight, mean and standard deviation (variance) to reflect the new obtained pixel value. In relation to non-matched components the weights w decreases whereas the mean and standard deviation stay the same. It is dependent upon the learning component p in relation to how fast they change. Thirdly, here we identify which components are parts of the background model. To do this a threshold value is applied to the component weights w. Fourthly, in the final step we determine the foreground pixels. Here the pixels that are identified as foreground dont match with any components determined to be the background. 6.0 Experimental Results For better results, median filter was used where the main purpose was to filter out unconnected pixels from the large connected pixels (e.g. people, vehicles) so that it was easy to distinguish between the background and foreground. With median filter the value of the output pixel is determined by the median of the neighboring pixels instead of the mean. Median has a much smaller insensitivity compared with mean to extreme values. The function used was medfilt2(A,[m n]). Figure 3 GMM Background Subtraction with high light intensity In the above image, the picture on the right shows the output of background subtraction using the GMM model. There is still false negative foreground detection which produces the disturbances in the image. Although it still shows the objects moving hence, making it suitable for object tracking. The reason for these disturbances can be narrowed down to illumination changes. Figure 4 GMM Background Subtraction with Low light intensity For the above, a short 30 second video was recorded where light intensity was moderate and the camera was still. It can be observed that the hand is visible and false negative detection is low which suggests that the model works robustly in low light intensities. From the above results, we can say that there are still improvements to be made in the algorithm mostly to work with illumination which increases the false negative background subtraction. 7.0 Further Improvements The GMM model used in this paper could be further tweaked to provide better results in terms of zero disturbances/noises. The use of morphological filters could be implemented into the algorithm to reduce these unconnected pixels which were seen in the figures above and make it more robust in object tracking. The algorithm has the capabilities of further improvements to run large size videos and importantly the ability to process more frames per seconds using larger number of Gaussians in the mixture model. Apart from the above improvements, we further plan to research the model using PTZ (Pan Tilt Zoom) technique to study the processing rate of frame/second and observe the robustness of the algorithm in terms of disturbances/noise. There are possibilities of false positive results where background could be recognised as foreground. 8.0 Conclusions In this project we implemented a very strong and widely used background subtraction method according to the paper Adaptive background mixture models for real-time tracking. This method is very adaptable for lighting changes and shadow removals. Also it finds the repetitive actions as well with the use of mixture of Gaussians. Alpha and T are the key parameters in this paper. The values of these parameters changes with the different cameras and different environment. So it is very important to get the best values for them to work with for different videos. Also we tried to improve the output quality by using some filtration methods like median filter. The guassain model is not perfect with the result since there are some disturbances after the modelling also. But there are many good post processing techniques are introduced and we referenced two of them. A new region Gaussian background model for video surveillance by Xun Cai and Long Jiang, and Improved Post-Processing for GMM based Adaptive Background Modeling by Deniz Turdu,Hakan Erdogan. But the 1st paper doesnt give any clear information about the techniques failed to explain the equations as well. Second paper introduces a very good technique and we couldnt able to get the result properly. But this assignment was very challenging for us and helped us to get a very good knowledge about different background subtraction methods used in Video surveillance. 9.0 References [1] Robust techniques for background subtraction in urban traffic video Sen-Ching S. Cheung and Chandrika Kamath, 2006/07 [2] Background Subtractions of Moving Objects https://computation.llnl.gov/casc/sapphire/background/background.html [3] Page444, Computer vision A modern approach, David A. Forsyth [4] Dr J. Zhang, CSC7006 Lecture 2 Slides, Queens University Belfast. [5] Adaptive background mixture models for real-time tracking, Chris Stafer, W.E.L Grimson [6] Improved Post processing for GMM based adaptive background modelling by Deniz Turdu, Hakan Erdogan