It took 4 minutes for 911 to be called, let’s fix that with AI
In dedication to Al Rehmtulla
Motivation
Over the past year, the pandemic has had a massive toll on small businesses, my parents being affected by that. Running PurePages, a hotel-based Internet Service Provider (ISP), they were forced into laying off most of their staff. With that being said, my Dad stayed at work late at night wearing multiple hats from tech support to server maintenance. On the daily I would help my father out with the business and work on an IoT project with him (app-controlled vending machine)
On the night of May 15th, during a cigarette break outside, he was caught in the crossfire of a shooting between 5 others, with one other victim being fatal.
It took a total of 4 minutes for 911 to be called, with one of the survivors calling while hiding behind the office. My dad made it to the hospital, where every second counts, but despite first responders attempts, he passed away within hours.
With my custom dataset using artificial intelligence, the police would have been notified within milliseconds! Making it only a 2-minute buffer, allowing the offenders to be arrested and EMT more time to treat my dad.
The AI 🤖
The creation process
Being completely new to Artificial Intelligence (AI), I had no idea where to even start. After some research, I found that TensorFlow would be the best language to use, combined with an algorithm called You Only Look Once (YOLO). Getting this to work is surprisingly simple! After a couple of days, I had trained my custom dataset to detect guns. As of now, it isn’t 100% accurate due to a lack of resources, but this will be fixed as I upgrade my PC.
TensorFlow
I chose TensorFlow over other libraries like PyTorch for several reasons, with the main being compatibility with YOLO. Tensorflow is less object-oriented but has several more options, not to mention it’s fast training time. TensorFlow has also enabled me to future-proof my code, in the coming months, I aim to create a full computer vision software for hospitality, commercial, and restaurants with features including people count per store, mask detectors and anti-shoplifting measure. While TensorFlow is harder to learn, it is effortless to scale, which is necessary for my long-term success.
YOLO
We all know computers don't think the same way people do, we need to first enable our machine to parse the images used, this is also referred to as Computer Vision.
Yolo divides the full image into a square grid, it then applies a single CNN (convolutional neural network). Using this neural network as opposed to alternatives like an RNN allows YOLO to be blazing fast, reaching 120 frames per second on high-end machines! After it has divided the image into a grid, the algorithm applies bounding boxes and a confidence score of the predicted object.
The entire process of applying grids to bounding boxes:
The IoT 🌐
Starting the code
As mentioned earlier, down to the last day with my dad, we worked on a project to control a vintage vending machine from your phone. In light of my intentions for this project, I tried to use what we worked on together in this script. My last week with him was by far the most significant and memorable as I finally surpassed his knowledge in coding (for Arduino and APIs at least 😉), making myself genuinely useful in the process.
YOLO side API
This part is completely from scratch since there haven’t been any proper projects utilizing an API with YOLO. By creating a fairly straightforward python script running Flask, I sent an image from my laptop to my desktop and received back the confidence score. Then I booted up an IP camera and did a GET command to quickly get real-time images and feed them through my custom dataset.
Flask
Flask is the communicator of the project. Once the .jpg has been saved from a security cam, we instantly pass it over here. Using flask we run the image through the instance of yolo and return the output with a list of detections and the confidence score. If a gun is detected we send a notification over the LAN for the Arduino to receive.
The actual functionality of the Flask application is very straightforward. It saves the image into YOLOs path, runs the detection and sends a response!
Arduino
While adapting the code, I instantly felt the frustration of my dad’s old school PHP work. I ended up doing all my API calls locally on the same computer and feeding the response directly to an Arduino turning on and off an LED. This is easy to replace with a calling module to notify 911, but I'm sure they wouldn’t appreciate all my tests :)
How you can do the same 🤯
Using Google Colab, we can take advantage of cloud GPUs and a Linux based Virtual Machine (VM).
Using this link, I have curated a line by line notebook to get everything working within minutes! I have set it to the standard datasets, but as I rebuild my custom, it will be swapped :)
To get my custom dataset run the following, but as mentioned throughout the article, it needs to be retrained for stability
!wget http://aleem.rehmtulla.com/yolov3-custom-final.weights
My next steps ➡️
The very first thing I will do is retrain the model. The resources I need won’t come for several weeks, but upon arrival, I will retrain my custom dataset to pick up on a gun being pulled out much quicker.
Shortly after I get the memory needed, a Graphical User Interface (GUI) will also be made, so this can actually be used. Artificial Intelligence has the ability to save countless lives. Let's legitimately implement it!
Why isn’t it open source? This is the ultimate goal but right now, I’m sorting out some major kinks so stay tuned when I publish the final repo.
