The Problem:



Every road is designed to support a certain volume of traffic. As the number of vehicles increase, the road network must be upgraded to avoid congestion. However, in some instances, upgrading the road network is difficult, in-feasible or outright impossible. A good example is downtown Karachi where the road network design is almost a century old and cannot be upgraded due to the relics of construction on either side of the road. Some of these buildings are so old that they are rightful candidates for heritage sights! As a result, this happens:

external image Karachi_Traffic_1_PNN.jpg external image 20060818_02.jpg external image 2057491078_c9b966cfa5.jpg

Traffic congestion occurs when there is a sudden and often unpredictable surge in the volume of traffic that "saturates" the capacity of the road network. This saturation is often caused by a small accident and the butterfly effect makes sure that queues not unlike the ones shown above are formed. The situation is more volatile during rush hours when the volume of the traffic is already stretching the road network capacity and even the slightest disturbance renders the system out of control. Following is the "Big Bad List" of traffic jams according to the excellent Wikipedia article:
  • As a non-productive activity for most people, congestion reduces regional economic health.
  • Delays, which may result in late arrival for employment, meetings, and education, resulting in lost business, disciplinary action or other personal losses.
  • Inability to forecast travel time accurately, leading to drivers allocating more time to travel "just in case", and less time on productive activities.
  • Wasted fuel increasing air pollution and carbon dioxide emissionsowing to increased idling, acceleration and braking.
  • Wear and tear on vehicles as a result of idling in traffic and frequent acceleration and braking, leading to more frequent repairs and replacements.
  • Stressed and frustrated motorists, encouraging road rageand reduced health of motorists
  • Emergencies: blocked traffic may interfere with the passage of emergency vehicles traveling to their destinations where they are urgently needed.
  • Spillover effect from congested main arteries to secondary roads and side streets as alternative routes are attempted ('rat running'), which may affect neighborhood amenity and real estate prices.

Motivation and Need:
  • Rapid growth of traffic flow caused traffic monitoring job become harder.
  • Current available surveillance system which is based on electronic sensor and manual observation by human operator may lead to unreliable traffic analysis. It is impossible for small group of operator to give constant attention while monitoring large number of screen in long time period. Besides that, current traditional method of traffic data collection requires different electronic sensor for different task and they're limited in functionality. Using different components for different task is expensive and hard to maintain.
  • This project is prompted by the need of reducing the cost of collecting transport data using manual observation with computer vision technology. The use of this technology has the potential to greatly reduce the cost because computer vision can provide unlimited application feature towards the existing surveillance system.
  • With the increasing capacity of digital storage, improved processor power, better internet connection infrastructures and new video compression standard make computer vision number one choice for automatic monitoring solution.
  • There are many methods for extracting moving object. There are many methods which can be used to detect vehicle in video. A proper research should be done before choosing any method to implement a computer vision problem depending on several factors such as lightning condition, weather, camera position, and average traffic flow statistic.
  • The ultimate need of this project is to improve current system and reduce human error by implementing computer vision technology. This project proposes an automatic visual surveillance system which uses video sequence of traffic footage to detect and track multiple vehicles.

Abstract:

There has been a rising in demand for computer vision solution in the current surveillance application relating to traffic management and analysis. Traditional method of monitoring and analysis is by manual traffic observation with the help of data collected from electronic sensors located at certain locations. Current feature is

limited for route planning, congestion detection and accident alert. With computer vision technology, current surveillance system can be use for traffic enforcement
system such as speed measurement and vehicle counting. This project consists of three main parts, which is video processing unit, data analysis and web application. Video processing unit focuses on multiple object detection, object tracking, speed measurement and object counting based on video sequences obtained from a video camera feed located on the road.



Modules:

  • Video processing
  • Data Analysis
  • Web Application
  • *Embedding GSM with the website(depending on time constraints).

Proposed System:

traffic.JPG


Computer Vision:




Computer Vision is the science and technology of machines that see, where see in this case means that the machine is able to extract information from an image that is necessary to solve some task. As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras etc.
There has been widespread attention in advancing surveillance systems using computer vision technology. Most of the research mainly related to detecting moving objects for pedestrians and traffic surveillance. This research projected methods for background modeling, object detection, object tracking and object classification.

Software Interfaces:


  • SQL server 2005
  • Visual studio 2010
  • ASP.NET
  • C#.NET
Hardware & Communication Interfaces:

  • Webcam/CCTV to be used for live video feed and process it for deducing the traffic congestion on roads
  • The communication interface includes e-mail, web browser (preferably Google Chrome or Firefox)

What is OpenCV?

  • OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real time computer vision, developed by Intel ..
  • It contains many algorithms and examples for image processing and for machine vision.
  • Opencv is now provided in the form of a dynamic link library (DLL), the presented wrapper enables you to call the library function from within the modern .NET language C#.

C# Bindings for OpenCV:

There are many platforms available for binding of opencv with C#.net some of them are:
  • Emgucv.
  • OpenCVSharp.
  • OpenCVDotNet.
  • Opencv Code Project.

In Traffic Monitoring System we used EmguCv and Code Project library for C#.net binding..

OpenCV vs Matlab:


  • Matlab itself is built upon Java. And Java is built upon C. So whenever one can run a Matlab program, computer is busy trying to interpret all that Matlab code. Then it turns it into Java, and then finally executes the code, so in that case Matlab is slow.
  • OpenCV was made for image processing. Each function and data structure was designed with the Image Processing coder in mind. Matlab, on the other hand, is quite generic. You get almost anything in the world in the form of toolboxes. All the way from financial toolboxes to highly specialized DNA toolboxes.
  • On the other hand, Opencv is developed specially for C/C++ environment, that’s why we preferred Opencv for our project.

Trials:

Computer vision was pretty new and interesting for us to work on. Having no background of it we initially had to start understanding the basic concepts behind it.
We started off with few wiki articles on computer vision and our external’s lecture, which helped us to proceed with the project.
  • Our first task was to start up with OPENCV (Computer Vision) library. We choose C# for the development of the video processing and data analysis window application using VISUAL STUDIO 2010 as development tool.
  • Since OpenCV’s supported mainly C and C++ language, we then looked for its wrapper for C#.
  • EMGUCV is one of the opencv wrappers that we are using in our project.
  • Get emgucv installed we started playing with samples provided by it which took a period of three weeks,
  • Examples were
    1. Hello world
    2. Face Detection
    3. Camera Capture
    4. Shape detection
    5. Traffic sign recognition
    6. License plate recognition
    7. Finally, Motion Detection
  • Running these examples successfully was a big accomplishment for us and this booted our confidence to proceed with video processing and we attempted number of codes which can be viewed in trials.pdf document.
  • After testing the samples of emgucv our next task was to start searching for algorithms related to corner detection.
For this purpose we first found Harris Corner Detection algorithm on aforge.net.
Snap shot of the output
untitled.JPG

This worked excellent.

LK DEMO initial:

As our external advisor guided , we then looked for LK(Lucas-Kanade optical flow algorithm)-demo implementation in C# and came across the following code and tested a couple of videos on it.



Snap shot of the output

11.JPG

Conclusion:
This output was not up to the requirement not all cars’ corners were detected.

Motion Detection using AFORGE.NET

Since the output of the above code was not up to the requirement then we looked for motion detection algorithms and found a very good one implemented in aforge.net at codeproject.com

Description:
For description and code visit the following link
http://www.codeproject.com/KB/audio-video/Motion_Detection.aspx

Additional work done on this project we used emgucv to run .avi videos and used motion detection of aforge.net

Snap Shot of the output:
12.JPG

Conclusion:
  • Though we didn’t use this for the project but implementing and learning aforge.net was a good experience.
  • Then we switched to CVGoodFeaturesToTrack to proceed to module 2 of the project Data Analysis in which we required corners to form clusters.

Video Processing [ First Module]:

  • Window app is developed using C# language. Video capturing and background modeling is based on open source computer vision library, “OpenCV”.
  • Video processing is done with tha help of Capture class of “EmguCv”(Opencv).
  • Capture Class can take three parameters as argument.
  1. File Name of the recorded video.
  2. Null in case of Webcam
  3. Index in “integer” for any other kind of camera for Live Streaming..
  • In our Project we are using Recorded Videos for testing purpose.
  • The Frame Rate for capturing frames is 1 sec,i.e after every one second frame is captured and then that frame is processed for Motion detection..
  • In our project we set total 60 frames for Processing..

Tracking the motion of vehicle:


  • In order to track the Vehicle we use the function “Good Features to track”
  • cvgoodfeaturestotrack” is also called “Shi & Tomasi” Method.

Tracking of Vehicles:

Every Object has certain distinct features with which the object can be tracked and worked upon. Thus by observing the characteristic properties of vehicles we observed that every vehicle can be tracked by its corners while moving on the road.

Good Features to track:

  • The function first calculates the minimal eigenvalue for every pixel of the source image.
  • The function cvGoodFeaturesToTrack finds corners with big eigenvalues in the image.
  • A corner can be defined as the intersection of two edges.

traffic5.JPG

  • The function ensures that all the corners found are distanced enough from one another by getting two strongest features and checking that the distance between the points is satisfactory.
  • This function also implements “Harris corner detection”..

Eigen Values:

The eigenvectors of a square matrix are the non-zero vectors that, after being multiplied by the matrix, remain proportional to the original vector (i.e., change only in magnitude, not in direction). For each eigenvector, the corresponding eigenvalue is the factor by which the eigenvector changes when multiplied by the matrix.


Results of motion tracking of a vehicle:

traffic6.JPG

In the image above the motion of the car is tracked. The points in yellow show the detected corners of the car and the arrows in red tell the flow fields.

Data Analysis [Second Module]:


Data Analysis is the second main module of the project. In this module the data gathered is processed and relevant information extracted to deduce the traffic congestion on the roads.
Processing The Data Obtained From Traffic Video
The points that were tracked using “Good Features to Track” on the vehicles on road are then used to find out their velocity that is how far a vehicle has moved from one captured frame of the video to the second consecutive captured frame, this will tell the speed of the vehicles o the road.
The distance between the points tracked will tell the distance between the cars on the road that is if the cars are very close to each other compared to a threshold value then it will indicate high congestion.

The graph below shows the tracked points plotted for only one moving object (car) on the road. Here we can easily figure out the points and see and identify where each point lies n the graph.
We saved the points tracked by the function of “Good Features To Track” and plotted the graphs as shown below.
  • When only one moving object in videog1.jpg


When we plot the graph for multiple moving objects (that is many cars on the road) the points are so intermixed with each other that it is very difficult and nearly impossible to figure out where each point is.

  • When multiple moving objects in video

g2.jpg

Noise Filtration of the Tracked Points by cvGoodFeaturestoTrack:

The points that were tracked using “Good Features to Track” had points that were not required that is Noise which were to be removed to get those filtered points only that will contribute in providing the essential information about the traffic congestion on the road.
These points included
  • The static points on trees and other non moving objects.
  • The points on pedestrians crossing the road
  • The points indicating any kind of abnormality such as a car cannot turn over an angle of 90 degrees.
  • The points with a random and abnormal change in speed which is practically not possible

All such points add to noise and thus are to be removed by efficient filtering. For this purpose we added we modified our code for the tracked points and obtained the constructive points.

==
filtered_image.jpg
Actual Frame and Tracked Frame
==

Clustering:

What is Clustering?


Cluster analysis or clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense. Clustering is a method of unsupervised learning, and a common technique for statistical data analysis used in many fields, including machine learning, data mining, pattern recognition, image analysis, information retrieval, and bioinformatics

Need Of Clustering In The Project:


“All the points located near to each other belong to the same object”. That is the points belonging to one car will be positioned close to each other. All the nearby points will form one cluster that is the cluster is actually indicating the car or any vehicle.
By finding the number of clusters of the tracked points we are in effect finding out the number of vehicles on road. If the count is above a certain threshold value for the road it will indicate High traffiic congestion, below the threshold values will indicate low traffic congestion and if it falls in a certain range it will indicate medium traffic congestion.

How have we Extracted The Number of Clusters
We have used the Quality Threshold Algorithm in C#.NET to find out the no of clusters from the tracked points after filtration.


Clustering code in C#

Reading the excellent Wikipedia article on cluster analysis, you will find that QT (Quality Threshold) clustering algorithm is what we need. A quick google search reveals the following links to C# code implementing QT clustering:

  • An excellent article with a good description of the algorithm by C# Corner
  • Discussion on Debugging.comwith snippets of code
  • Article covering Hierarchical clustering and K-Means clustering by Extreme Optimization (Not needed for the project, but still a good article)

Database:

Our database consist of two tables
  • Location
  • UserLogin

Design of Location Table:

traffic2.JPG


Design Of User Login Table:
traffic3.JPG
TraffixGem (website) [Third Module]:

  • The basic objective of this website is to improve current traffic system and provide users current traffic updates on click.
  • We cover Karachi Traffic Situation 24/7 round the clock..

    *Home Page of Website*

hp.JPG
  • Traffic Updates powered by Google Maps.
  • We are using Google Map “dll” in order to embed Google maps in our website..

The dll we have is from googlemaps.subgrim.net. The user is required to select the location on the website of which he needs to know the current traffic situation. The location is shown on the maps.

We have implemented map, hybrid and satellite views of Google maps in the project. Once a location is selected the traffic congestion data is extracted from the database which keeps a record of the traffic situation of all the entered locations, a pop up then appears telling whether the congestion is high, medium or low currently in the entered location.




gm.JPG


*GSM [Bonus Module]:

We are planning to embed GSM service in our website.


gsm.jpg

Registration Form for GSM:

  • For that purpose we made a registration form so that a user will get Traffic Updates from our site to his/her mobile phone
2.jpg





Project Poster



poster.JPG


Project Source code(website,windows app and database backup)

The code below has been contributed by all the group members.(Arisha Yazdani, Anam Siddiqui,Hira Batool, Zaira Azmat)






Project Report


Project Presentation



Resources