Object Detection Google TensorFlow
We at Anonsoftware like a challenge, so we set out to make a python script that would give a visual to objects around during a GTA V game, using TensorFlow Object Detection
During the process of building the script we learnt many new things, including gpu usage with python.
Over the course of the project we tested many different models provided by google, and settled with a happy medium between performance and accuracy of detection
Initially we ran this without using any GPU’s and noticed a the performance was not very good, nor of a standard to allow it to be used in any form of self driving vehicles or self playing AI, this is when we started using GPU’s in our case we used a GTX 1060 6GB card
Once we used GPU’s for the Object Detection script, it allowed us to get down to a few milliseconds to process each frame, allowing us to more reliably and comfortably sit with the basis of the script to progress to player movement or vehicle driving.
The following ideas we had for this script are
Self Driving Vehicle With Object Avoidance to mimic a player
GTA 5 Race Player(Script applies the inputs in order to complete a GTA 5 Race with visual and external inputs for the game)
Outcome And Experience
Once we had finished the initial project we realised how this could be used in he other project ideas we had, one is todo with the self driving vehicles.
We realised that with the detection boxes and the labels supplied we can pass the boxes to a second network that is designed to determine, Red Light/Green Light, or GTA 5 Race Checkpoints.
The aboved combined with edge detection in order to allow it to stay on tracks or roads, would be a fairly decent experience, however with this comes drawbacks.
Some draw backs are primarily performance, the more objects or areas it has to compute the slower it becomes to handle a frame, apply extras on top and the script starts to slow down even more.
While object detection is great fun, the practical use with most consumer hardware is not great, however with the new Nvidia Graphics Cards such as the RTX 2080ti, which contain tensor corse designed for neural networks, could bring more ai related applications for games or general use to be more frequent.