Introduction
This is an autonomous robotics project that uses a TurtleBot3 to navigate its environment by recognizing visual cues. A Raspberry Pi V2 camera is mounted on the robot to capture a continuous live video feed, which is processed by a machine learning model to identify specific directional labels.
The robot then interprets these labels and uses ROS2 to execute corresponding commands. For example, recognizing a forward arrow triggers a command to move 10 inches forward, while a right curved arrow instructs the robot to rotate 90 degrees to the right. This project demonstrates the integration of computer vision and robotics for intelligent, image-based navigation.
Key Components
- TurtleBot3: The mobile robot platform.
Raspberry Pi V2 Camera: Captures real-time images of the environment.
Machine Learning Model: Recognizes and classifies the visual cues (labels).
ROS2: The framework used for sending movement commands to the robot based on the recognized labels.
Visual Labels: Pre-defined images (e.g., arrows) that the model is trained to recognize.
How it Works
1. Image Capture
The Raspberry Pi V2 camera continuously streams live video to the processing unit.
2. Visual Recognition
The live feed is analyzed by the machine learning model, which identifies any of the pre-trained visual labels.
3. Command Generation
Based on the recognized label, the system generates a specific movement command (e.g., move forward, turn right).
4. Robot Movement
The command is sent to the TurtleBot3's control system via ROS2, causing the robot to perform the desired action. This entire process allows the robot to make intelligent navigation decisions in real-time.
Software Setup
Ensure your Raspberry Pi is running Raspberry Pi OS Bullseye (64-bit) with ROS2 Humble or Foxy installed. Standard TurtleBot3 packages must be configured.
Installing AI Dependencies
# Update system dependencies
sudo apt-get update
sudo apt-get install python3-pycoral
# Install CV and ML tools
pip3 install opencv-contrib-python==4.11.0.86
pip3 install numpy==1.19.5
Camera Setup and Calibration
The camera must be mounted at a height where it can clearly see the navigation labels on the floor or walls.
- Ensure the camera lens is clean and focused using the adjustment tool.
- Check for glare on the labels, which can interfere with model accuracy.
- Verify the camera ribbon cable is securely connected to the CSI port.
Creating a Model
Use Google Teachable Machine to train a model to recognize your specific arrows or labels.
- Capture 100+ images for each direction (Forward, Left, Right, Stop).
- Include a "Background" class for when no label is present.
- Export the model as TensorFlow Lite - EdgeTPU.
- Deploy the `.tflite` file to the Raspberry Pi.
Running Code
The main navigation script coordinates the camera feed with the ROS2 publisher to send Twist messages to the `/cmd_vel` topic.
Tip: Start with a slow linear velocity (0.05 - 0.1 m/s) until you are sure the vision model is triggering turns correctly.