TrafficGem

= = toc

The Problem:
Every road is designed to support a certain volume of traffic. As the number of vehicles increase, the road network must be upgraded to avoid congestion. However, in some instances, upgrading the road network is difficult, in-feasible or outright impossible. A good example is downtown Karachi where the road network design is almost a century old and cannot be upgraded due to the relics of construction on either side of the road. Some of these buildings are so old that they are rightful candidates for heritage sights! As a result, this happens:



Traffic congestion occurs when there is a sudden and often unpredictable surge in the volume of traffic that "saturates" the capacity of the road network. This saturation is often caused by a small accident and the [|butterfly effect] makes sure that queues not unlike the ones shown above are formed. The situation is more volatile during rush hours when the volume of the traffic is already stretching the road network capacity and even the slightest disturbance renders the system out of control. Following is the "Big Bad List" of traffic jams according to the excellent [|Wikipedia article]:
 * As a non-productive activity for most people, congestion reduces regional economic health.
 * Delays, which may result in late arrival for employment, meetings, and education, resulting in lost business, disciplinary action or other personal losses.
 * Inability to forecast travel time accurately, leading to drivers allocating more time to travel "just in case", and less time on productive activities.
 * Wasted fuel increasing [|air pollution] and [|carbon dioxide emissions]owing to increased idling, acceleration and braking.
 * Wear and tear on vehicles as a result of idling in traffic and frequent acceleration and braking, leading to more frequent repairs and replacements.
 * Stressed and frustrated motorists, encouraging [|road rage]and reduced health of motorists
 * Emergencies: blocked traffic may interfere with the passage of emergency vehicles traveling to their destinations where they are urgently needed.
 * [|Spillover effect] from congested main arteries to secondary roads and side streets as alternative routes are attempted ('[|rat running]'), which may affect neighborhood [|amenity] and [|real estate] prices.

**Motivation and Need:**
 * Rapid growth of traffic flow caused traffic monitoring job become harder.
 * Current available surveillance system which is based on electronic sensor and manual observation by human operator may lead to unreliable traffic analysis. It is impossible for small group of operator to give constant attention while monitoring large number of screen in long time period. Besides that, current traditional method of traffic data collection requires different electronic sensor for different task and they're limited in functionality. Using different components for different task is expensive and hard to maintain.
 * This project is prompted by the need of reducing the cost of collecting transport data using manual observation with computer vision technology. The use of this technology has the potential to greatly reduce the cost because computer vision can provide unlimited application feature towards the existing surveillance system.
 * With the increasing capacity of digital storage, improved processor power, better internet connection infrastructures and new video compression standard make computer vision number one choice for automatic monitoring solution.
 * There are many methods for extracting moving object. There are many methods which can be used to detect vehicle in video. A proper research should be done before choosing any method to implement a computer vision problem depending on several factors such as lightning condition, weather, camera position, and average traffic flow statistic.
 * The ultimate need of this project is to improve current system and reduce human error by implementing computer vision technology. This project proposes an automatic visual surveillance system which uses video sequence of traffic footage to detect and track multiple vehicles.

**Abstract:** == There has been a rising in demand for computer vision solution in the current surveillance application relating to traffic management and analysis. Traditional method of monitoring and analysis is by manual traffic observation with the help of data collected from electronic sensors located at certain locations. Current feature is == limited for route planning, congestion detection and accident alert. With computer vision technology, current surveillance system can be use for traffic enforcement system such as speed measurement and vehicle counting. This project consists of three main parts, which is video processing unit, data analysis and web application. Video processing unit focuses on multiple object detection, object tracking, speed measurement and object counting based on video sequences obtained from a video camera feed located on the road.

Modules:

 * Video processing
 * Data Analysis
 * Web Application
 * *Embedding GSM with the website(depending on time constraints).

Computer Vision:
**Computer Vision **  is the science and technology of machines that see, where //see //  in this case means that the machine is able to extract information from an image that is necessary to solve some task. As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras etc. There has been widespread attention in advancing surveillance systems using computer vision technology. Most of the research mainly related to detecting moving objects for pedestrians and traffic surveillance. This research projected methods for background modeling, object detection, object tracking and object classification.

Software Interfaces:
**Hardware & Communication Interfaces:**
 * SQL server 2005
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Visual studio 2010
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">ASP.NET
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">C#.NET


 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Webcam/CCTV to be used for live video feed and process it for deducing the traffic congestion on roads
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">The communication interface includes e-mail, web browser (preferably Google Chrome or Firefox)

What is OpenCV?

 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real time [|computer vision], developed by [|Intel] ..
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">It contains many algorithms and examples for image processing and for machine vision.
 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Opencv is now provided in the form of a dynamic link library (DLL), the presented wrapper enables you to call the library function from within the modern .NET language C#.

C# Bindings for OpenCV:
<span style="display: block; font-family: arial,helvetica,sans-serif; font-size: 12px; line-height: 18px; text-align: left;">There are many platforms available for binding of opencv with C#.net some of them are:
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Emgucv.
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">OpenCVSharp.
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">OpenCVDotNet.
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Opencv Code Project.

<span style="color: black; display: block; font-family: arial,helvetica,sans-serif; font-size: 9pt; text-align: left;">In Traffic Monitoring System we used EmguCv and Code Project library for C#.net binding..

OpenCV vs Matlab:

 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Matlab itself is built upon Java. And Java is built upon C. So whenever one can run a Matlab program, computer is busy trying to interpret all that Matlab code. Then it turns it into Java, and then finally executes the code, so in that case Matlab is slow.
 * OpenCV was made for image processing. Each function and data structure was designed with the Image Processing coder in mind. Matlab, on the other hand, is quite generic. You get almost anything in the world in the form of toolboxes. All the way from financial toolboxes to highly specialized DNA toolboxes.
 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">On the other hand, Opencv is developed specially for C/C++ environment, that’s why we preferred Opencv for our project.

Trials:
Computer vision was pretty new and interesting for us to work on. Having no background of it we initially had to start understanding the basic concepts behind it. We started off with few wiki articles on computer vision and our external’s lecture, which helped us to proceed with the project. __**For this purpose we first found Harris Corner Detection algorithm on aforge.net.**__ **Snap shot of the output**
 * Our first task was to start up with OPENCV (Computer Vision) library. We choose C# for the development of the video processing and data analysis window application using VISUAL STUDIO 2010 as development tool.
 * Since OpenCV’s supported mainly C and C++ language, we then looked for its wrapper for C#.
 * EMGUCV is one of the opencv wrappers that we are using in our project.
 * Get emgucv installed we started playing with samples provided by it which took a period of three weeks,
 * Examples were
 * 1) Hello world
 * 2) Face Detection
 * 3) Camera Capture
 * 4) Shape detection
 * 5) Traffic sign recognition
 * 6) License plate recognition
 * 7) Finally, Motion Detection
 * Running these examples successfully was a big accomplishment for us and this booted our confidence to proceed with video processing and we attempted number of codes which can be viewed in trials.pdf document.
 * After testing the samples of emgucv our next task was to start searching for algorithms related to corner detection.

This worked excellent.

__**LK DEMO initial:**__

As our external advisor guided, we then looked for LK(Lucas-Kanade optical flow algorithm)-demo implementation in C# and came across the following code and tested a couple of videos on it.

**Snap shot of the output**



**__Conclusion__:** This output was not up to the requirement not all cars’ corners were detected.

__**Motion Detection using AFORGE.NET**__

Since the output of the above code was not up to the requirement then we looked for motion detection algorithms and found a very good one implemented in aforge.net at codeproject.com

**__Description__:** For description and code visit the following link **__http://www.codeproject.com/KB/audio-video/Motion_Detection.aspx__**

Additional work done on this project we used emgucv to run .avi videos and used motion detection of aforge.net

**Snap Shot of the output:**

**__Conclusion:__**
 * Though we didn’t use this for the project but implementing and learning aforge.net was a good experience.
 * Then we switched to CVGoodFeaturesToTrack to proceed to module 2 of the project Data Analysis in which we required corners to form clusters.

Video Processing [ First Module]:

 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Window app is developed using C# language. Video capturing and background modeling is based on open source computer vision library, “OpenCV”.
 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Video processing is done with tha help of Capture class of “EmguCv”(Opencv).
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Capture Class can take three parameters as argument.
 * 1) <span style="font-family: Arial,Helvetica,sans-serif; font-size: 9pt; line-height: 18px;">File Name of the recorded video.
 * 2) <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Null in case of Webcam
 * 3) <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Index in “integer” for any other kind of camera for Live Streaming..
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;"><span style="font-family: Arial,Helvetica,sans-serif; font-size: 9pt;">In our Project we are using Recorded Videos fo r testing purpose.
 * The Frame Rate for capturing frames is 1 sec,i.e after every one second frame is captured and then that frame is processed for Motion detection..
 * In our project we set total 60 frames for Processing..

Tracking the motion of vehicle:

 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">In order to track the Vehicle we use the function “Good Features to track”
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;"> “ __ cvgoodfeaturestotrack __ ” is also called “Shi & Tomasi” Method.

<span style="color: #464646; font-family: Arial,Helvetica,sans-serif; font-size: 12pt;">Tracking of Vehicles:

Every Object has certain distinct features with which the object can be tracked and worked upon. Thus by observing the characteristic properties of vehicles we observed that every vehicle can be tracked by its corners while moving on the road.

**<span style="color: #464646; font-family: Arial,Helvetica,sans-serif; font-size: 12pt;">Good Features to track: **

 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">The function first calculates the minimal eigenvalue for every pixel of the source image.
 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">The function cvGoodFeaturesToTrack finds corners with big eigenvalues in the image.
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">A corner can be defined as the intersection of two edges.




 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">The function ensures that all the corners found are distanced enough from one another by getting two strongest features and checking that the distance between the points is satisfactory.
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">This function also implements “Harris corner detection”..

Eigen Values:
The eigenvectors of a [|square matrix] are the non-zero [|vectors] that, after being [|multiplied] by the matrix, remain proportional to the original vector (i.e., change only in magnitude, not in direction). For each eigenvector, the corresponding eigenvalue is the factor by which the eigenvector changes when multiplied by the matrix.

Results of motion tracking of a vehicle:


In the image above the motion of the car is tracked. The points in yellow show the detected corners of the car and the arrows in red tell the flow fields.

Data Analysis [Second Module]:
The points that were tracked using “Good Features to Track” on the vehicles on road are then used to find out their velocity that is how far a vehicle has moved from one captured frame of the video to the second consecutive captured frame, this will tell the speed of the vehicles o the road. The distance between the points tracked will tell the distance between the cars on the road that is if the cars are very close to each other compared to a threshold value then it will indicate high congestion.
 * Data Analysis** is the second main module of the project. In this module the data gathered is processed and relevant information extracted to deduce the traffic congestion on the roads.
 * Processing The Data Obtained From Traffic Video**

The graph below shows the tracked points plotted for only one moving object (car) on the road. Here we can easily figure out the points and see and identify where each point lies n the graph. <span style="color: black; direction: ltr; display: block; font-family: arial,helvetica,sans-serif; font-size: 9pt; margin-bottom: 0pt; margin-left: 0.4in; margin-top: 4pt; text-align: left; text-indent: -0.28in; unicode-bidi: embed;">We saved the points tracked by the function of “Good Features To Track” and plotted the graphs as shown below.
 * <span style="color: black; direction: ltr; font-family: Arial,Helvetica,sans-serif; font-size: 9pt; margin-bottom: 0pt; margin-left: 0.4in; margin-top: 4pt; text-indent: -0.28in; unicode-bidi: embed;">When only one moving object in video <span style="color: black; direction: ltr; font-family: Arial,Helvetica,sans-serif; font-size: 9pt; line-height: 0px; margin-bottom: 0pt; margin-left: 0.4in; margin-top: 4pt; overflow: hidden; text-indent: -0.28in; unicode-bidi: embed;">[[image:alixedi/g1.jpg width="669" height="320"]]

When we plot the graph for multiple moving objects (that is many cars on the road) the points are so intermixed with each other that it is very difficult and nearly impossible to figure out where each point is.


 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 9pt;">When multiple moving objects in video




 * Noise Filtration of the Tracked Points** **by cvGoodFeaturestoTrack:**

The points that were tracked using “Good Features to Track” had points that were not required that is Noise which were to be removed to get those filtered points only that will contribute in providing the essential information about the traffic congestion on the road. These points included
 * The static points on trees and other non moving objects.
 * The points on pedestrians crossing the road
 * The points indicating any kind of abnormality such as a car cannot turn over an angle of 90 degrees.
 * The points with a random and abnormal change in speed which is practically not possible

All such points add to noise and thus are to be removed by efficient filtering. For this purpose we added we modified our code for the tracked points and obtained the constructive points.

== ==

=Clustering:= = = =What is Clustering?=


 * Cluster analysis** or **clustering** is the assignment of a set of observations into subsets (called //clusters//) so that observations in the same cluster are similar in some sense. Clustering is a method of [|unsupervised learning], and a common technique for [|statistical] [|data analysis] used in many fields, including [|machine learning] , [|data mining] , [|pattern recognition] , [|image analysis] , [|information retrieval] , and [|bioinformatics]

=Need Of Clustering In The Project:=

By finding the number of clusters of the tracked points we are in effect finding out the number of vehicles on road. If the count is above a certain threshold value for the road it will indicate High traffiic congestion, below the threshold values will indicate low traffic congestion and if it falls in a certain range it will indicate medium traffic congestion.
 * //“All the points located near to each other belong to the same object”.//** That is the points belonging to one car will be positioned close to each other. All the nearby points will form one cluster that is the cluster is actually indicating the car or any vehicle.

We have used the Quality Threshold Algorithm in C#.NET to find out the no of clusters from the tracked points after filtration.
 * How have we Extracted The Number of Clusters**

=Clustering code in C#= Reading the excellent Wikipedia [|article] on cluster analysis, you will find that QT (Quality Threshold) clustering algorithm is what we need. A quick google search reveals the following links to C# code implementing QT clustering:


 * An excellent article with a good description of the algorithm by [|C# Corner]
 * Discussion on [|Debugging.com]with snippets of code
 * Article covering Hierarchical clustering and K-Means clustering by [|Extreme Optimization] (Not needed for the project, but still a good article)

**Database:**

<span style="color: black; display: block; font-family: arial,helvetica,sans-serif; font-size: 9pt; text-align: left;">Our database consist of two tables
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Location
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">UserLogin


 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">Design of Location Table: **



**TraffixGem (website) [Third Module]:**
 * Design Of User Login Table:**

> > ===***Home Page of Website***===
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 12px; line-height: 18px;">The basic objective of this website is to improve current traffic system and provide users current traffic updates on click.
 * <span style="font-family: Arial,Helvetica,sans-serif; font-size: 9pt;">We cover Karachi Traffic Situation 24/7 round the clock ..
 * <span style="font-family: 'Times New Roman',serif; font-size: 12pt;">Traffic Updates powered by Google Maps.
 * <span style="color: #000000; display: block; font-family: arial,helvetica,sans-serif; font-size: 9pt; text-align: left;">We are using Google Map “dll” in order to embed Google maps in our website..

The dll we have is from googlemaps.subgrim.net. The user is required to select the location on the website of which he needs to know the current traffic situation. The location is shown on the maps.

We have implemented map, hybrid and satellite views of Google maps in the project. Once a location is selected the traffic congestion data is extracted from the database which keeps a record of the traffic situation of all the entered locations, a pop up then appears telling whether the congestion is high, medium or low currently in the entered location.



*GSM [Bonus Module]:
==<span style="color: black; direction: ltr; font-family: Arial,Helvetica,sans-serif; font-size: 9pt; margin-bottom: 0pt; margin-left: 0.4in; margin-top: 4pt; text-indent: -0.28in; unicode-bidi: embed;">We are planning to embed GSM service in our website. ==




 * <span style="color: black; font-family: Arial,Helvetica,sans-serif; font-size: 9pt;">For that purpose we made a registration form so that a user will get Traffic Updates from our site to his/her mobile phone

> === Here is the link: [] ===
 * ===Free Online API(for C#.net) to embed GSM service..===

Project Poster


**Project Source code(website,windows app and database backup)**

The code below has been contributed by all the group members.(Arisha Yazdani, Anam Siddiqui,Hira Batool, Zaira Azmat)

=Project Report= = Project Presentation =

**Resources** > > > > > >
 * [|OpenCV]
 * D. Beymer et al, "A real-time computer vision system for measuring traffic parameters". In Proceedings of IEEE CVPR 97, pages 495–501, 1997 [|(pdf)]
 * Emgucv []
 * C# [|bindings]for OpenCV
 * <span style="color: #2a2a2a; font-family: Tahoma,Verdana,Arial,sans-serif; line-height: normal;">Tracking vehicles.doc using lkdemo and clustering __ link __
 * <span style="color: #2a2a2a; font-family: Tahoma,Verdana,Arial,sans-serif; line-height: normal;">Computing Optical Flow using lkdemo [] & []
 * Motion Analysis and Object Tracking[]
 * Kalman Filters Color Tracking: @http://www.youtube.com/watch?v=1iwvL83Cumo
 * OpenCV Motion Analysis:[]
 * Motion Analysis & object tracking Manual [|http://www710.univ-lyon1.fr/~bouakaz/OpenCV-0.9.5/docs/ref/OpenCVRef_Motion_Tracking.htm#decl_cvCalcOpticalFlowLK]
 * Condensation Algorithm []
 * Particle Filter pdf: []
 * Particle Filter opencv video: []
 * Simple Particle Filter video []
 * Particle filter for positioning,tracking and navigating :(pdf)
 * Theory & implementation of Particle Filters "[[file:Theory and Implementation of Particle Filters.ppt]]"
 * About condensation algorithm: http://pinporelmundo.blogspot.com/2009/10/tracking-using-condensation-algorithm.html
 * Particle filter tracking [|http://www.youtube.com/watch?v=tx9S4Ha1wgk&NR=]
 * Particle Filter tutorial :[]
 * [](integrating rapid miner with visual studio)
 * []