This little bugger loves candy.. So he tries to run them over! The DetectBot is programmed using the pre-made Nvidia JetPack 4.3 from the Nvidia Jetson ''Hello world'' there are just a few more installations needed to be done to get it up and running.
Note: This is built is in Alpha state - everything works, however hardware and software needs some improvements before being called a Beta build and ready for serious testing.
Due to the lack of time the hardware setup and program code is as of now just made to work it will be updated in the future.
The CandyNet detect program is only trained on about 100 images and whit a black surface, I decided to use 50x50 to 400x400 bounding boxes so the bot is unable to detect candy more than about 30cm from the camera, this can be remedied/improved with settings when creating a new detection model.
Multi object dates from SSD-Mobilenet-v2 using 91-COCO classes or by making your own dataset, you can set manual parameters for follow and avoidance - remote control and video transfer. In the demo I've taken pictures of a candy and the different object i'd like the bot to avoid.
This robot will after being trained be able to go on autopilot/classify objects and when finding pre-set objects it can find and identefy/follow, if the trained network has some problems you can whit a remote PC control the Bot whit a keyboard.
DetectBot can be remotely controlled using any Linux/Windows/Mac PC using noMachine, however it needs an HDMI connected before NoMachine can render the correct size of the screen, so connect an HDMI screen.. start the DetectBot and wait 1 min after that it can be disconnected. for video/keyboard streaming download NoMachine to both PC and Jetson Nano - https://www.nomachine.com/download
I decided to make a high end autonomous navigation system as a UNI final exam project, but there is a catch I had no idea how object detection worked, further more I had never programmed in Python or handled board outputs, so with this in mind I realized I had to start somewhere.. I found the Jetson Nano and wanted to make a Jetbot without using their chassi. Trying to follow the Jebtbot instructions I soon realized that it expected a webcam to be used whit Jupierlab and I were using the MSI camera, furthermore I like to have more control than one gets in Jupiterlab so I decided to do my own project using the Jetson Nano image.
Install JetPack 4.3 and build the repo from Nvidia Jetson nano "Hello World" edit the DetectBot.py text file if you want to change parameters for example key input, objects to detect, turning speed and so on. You might need to install some packages used in the file, as of now I am unsure what are pre-installed on the Jetson image you need to do the Nvidia Jetson setup to get some librarys and the nessesery files for object detection and GPIO library, if any packages are missing google for the download and follow instructions to install. Put the DetectBot.py in your home directory and the run with sudo./DetectBot.py. More instructions in the DetectBot.py file
Make your own object detection data set with Nvidia DIGITS follow the instructions here: https://www.coria.com/insights/blog/computer-vision/preparing-data-for-custom-object-detection-using-nvidia-digits
and to load/use model
When starting the program you have to put the comand window on top, I recommend to keep it as small as possible so not to disturb, after a few Sec clicks the window to activate it and Puch any button a few Sec later repeat.Now the Camera could have one image to show, if so the program is ready to be used.. See the video for a demo.
To start SSD-Mobilenet-v2 91-COCO detection just run DetectBot.py file
If you want to run the CandyNet you need to download the CandyNet.gz and put it in the home derecory, then run command:
./DetectBot.py --width=1280 --height=720 --threshold=0.3 \
W drive forward
S drive backward
A drive left
D drive right
P stop and quit the program
Backspace hold down to drive the automatic search and find protocoll
Sadly enught I dont have any instructions for asembly, get som double sided adhesive and a screwdriver and you'll be set. Some pichures on the assembeld bot - Note this is not the final product so it is stillin alfa state and looks like this after trial and error.
Go to in to the DetectBot.py file and read from PIN configuration, you can use any GPIO pins for setup however you need to change the numbers accordingly in the DetectBot.py file, or see the electrical diagram for refrence GPIO info.