Obstruction detection and tracking using OpenCV-Python
This post shall discuss on a use case where we attempt to use object detection and tracking algorithm to check for obstruction within a specified zone in a video feed using OpenCV-Python. The concept described is similar to that of an abandoned-object detection where we detect object and track object to determine static object for a prolonged period of time.
Problem statement
For this use case, there is a requirement to use video analytics to monitor a controlled space and detect if there are obstructions to accessway such as corridors, lift lobbies, emergency exits etc. The definition of obstruction in this use case is rather generic and is defined as object(s) used to block access to pathways. Some of these obstructions may be transient in nature and as such there is a time limit which the obstructions have to be removed from the space.
The primary source of the video is defined as the premise CCTV.
Desired Output
The desired output from this solution is to generate an alert once a potential blockage is detected.
Object Tracking
In this project, we are applying the centroid tracking algorithm to our input video streams for tracking the detected objects. Essentially this method relies on the Euclidean distance between (1) existing object centroids (i.e., objects the centroid tracker has already seen before) and (2) new object centroids between subsequent frames in a video.
This is implemented using openCV library. To further understand this algorithm, you can refer to this site. In this article, we will use the script from this github.
Getting Started
Before we begin, the following libraries has to be installed. This can be done via pip or conda.
We will start by importing the necessary dependencies for this use case
The next few steps involves construction of the argument parser, parsing the arguments and initializing variables that are used in the algorithm.
Line 16 of obstruct_initialize_var.py defines the maximum allowable duration for the detected static obstruction before alert gets generated.
Notice in lines 26–43 of obstruct_initialize_var.py, there is a list of coordinates which forms a polygon or region of interest in the video created. This polygon or points can be easily obtained using the LabelMe annotation tool where the polygon is drawn on a screengrab and the points are then saved in json format.
In line 41, the points need to be rescale to a smaller resolution as the defined by the output video in line 59 and 60.
We also define a function called NMS.py. Essentially this non-maximum suppression function does is able to ignore the smaller overlapping bounding boxes and return only the larger ones for detected objects in images or videos. And even if they did overlap, do the overlap ratio does not exceed the supplied threshold of 0.3 which can be modified in line 72 of the main code below.
Now for the main code. As we grab the first frame of the video input as the base image and compare with the rest of the frames in the video, ideally the first frame should be free from obstructions. The region of interest (ROI) will also be drawn from the list of coordinates from the initialization stage. As we compare the frames, only bounding rectangles within the ROI will be shown and we will also define the minimum size of the object by parsing the minimum area argument. The default size is 500 pixels.
In Line 122 to 131 of main_code_on_detect.py, the detection of the obstruction will appear if it dwells more than the allowable time as set in the initialization stage. We will also save out the snapshot of the frame containing the detected object. The object tracking tracks the centroid of the object and as long as the centroid stays within the drawn ROI, it will be flagged out as an obstruction.
Below shows a snapshot of the output of the code. The thin blue line indicates the ROI and the obstruction detected is indicated by the red bounding box.
Limitations
We are using a heuristic approach in this use case as there is no clear indication of what are the objects that could potentially be the obstructions. Hence, we will only be able to set a rule such that once the object is of certain size, we will consider that as a sizeable obstruction.
The potential to use object recognition or identification after detection will be high if we are able to have data on the types of objects detected and using it to train the model.
The other limitation of this approach is the use of the first frame as its base image. In video surveillance, we may not be able to determine the first frame and hence the algorithm would need to be modified to capture the “empty” frame as the base image. This approach is also limited to consistent lighting as it compares the frames with the base image. Further improvements can be made to use average frames over a longer period of time or using more advanced frame differencing algorithms to solve this lighting constraint.
You can download my complete codes here
