Yellow Toy Car

This repository contains code, documentation and other stuff related to yellow toy car project I made.

I also made Flutter mobile app for controlling the toy car, see YellowToyCarApp repository.

Hardware

Hardware consist of:

Microcontroller: ESP32-Cam AI-Thinker development board
- ESP32S chip
  - 2x 32-bit LX6 CPU; up to 240 MHz; 520 KB SRAM.
  - 802.11 b/g/n Wi-Fi and Bluetooth 4.2 BR/EDR with BLE
- PSRAM on board, adding 4 MB.
- OV2640 camera.
- MicroSD card slot (unused, as GPIOs are used for motors and flash LED).
- 2 LEDs: red internal pulled high, and bright white external, acting for camera flash.
Motors driver: L298N-based module, able to drive 2 DC motors.
4 brushed motors, controlled in pairs, attached by gears to wheels.
External antenna for ESP32 Wi-Fi connectivity is used.
Battery (3 cells of 4 V, total 12 V for main board, 8 V for motors used).
Additional circuitry:
- Voltage converter (down to 5V, red LED)
- Voltage stabilizer (down to 3.3V required for ESP32, green LED).
- Battery, motor drivers and programmer connectors.
- Switch for programming mode (ON to program, OFF to execute).
Plastic grid and packaging.

Software

Software consist of:

Espressif IoT Development Framework (ESP-IDF) is used, which includes modified FreeRTOS.
Networking related code (AP or STA)
Camera related code
JSON configuration interface functions
Main HTTP web server (port 80)
- Status JSON
- Configuration endpoint
- Basic (slow) controls
- Car camera frame capture
Stream HTTP web server (port 81)
- Camera stream only, since it's blocking multipart data stream.
- Separate server to allow concurrent requests for main server.
Simple HAL for the motors and the lights
UDP socket server for fast controls inputs (port 83)
- Used by external scripts, allowing to control from the computer.
- Used by dedicated mobile app (related project)

Web API (HTTP)

/ or /index or /index.html → Website presented for user to control the car.

/status → Basic status, including time, lights & motors state and other diagnostic data.

 {
 	"uptime": 123456, // Microseconds passed from device boot.
 	"time": "2023-01-12T23:49:03.348+0100", // Device time, synced using SNTP.
 	"rssi": -67, // Signal strength of AP the device is connected to, or 0 if not connected.

 	/* With `?details=1` querystring parameter, extended response is provided. */
 	"stations": ["a1:b2:c3:d4:e5:f6"], // list of stations currently connected to our AP
 }

/config → Endpoint for requests to set configuration (JSON GET/POST API)

 {
 	/* Control & config for motors and lights */
 	"control": {
 		/* Other */
 		"timeout": 2000, // Time in milliseconds counted from last control request/packet, after which movement should stop for safety reason
 		/* Input values */
 		"mainLight": 1,
 		"otherLight": 1,
 		"left": 12.3,  // The motors duty cycle are floats as percents,
 		"right": 12.3, // i.e. 12.3 means 12.3% duty cycle.
 		/* Calibration */
 		"calibrate": {
 			"left": 0.95, // Inputs will be multiplied by calibration values before outputting PWM signal.
 			"right": 1.05,
 			"frequency": 100, // Frequency to be used by PWMs
 		}
 	},
 	/* Networking related. Some things are not implemented, including: DNS and DHCP leases */
 	"network": {
 		"mode": "ap", // for Access Point or "sta" for station mode, or "nat" (to make it work like router)
 		"fallback": 10000, // duration after should fallback to hosting AP if cannot connect as station
 		"dns1": "1.1.1.1",
 		"dns2": "1.0.0.1",
 		"sta": {
 			"ssid": "YellowToyCar",
 			"psk": "AAaa11!!",
 			"static": 0, // 1 if static IP is to be used in STA mode
 			"ip": "192.168.4.1",
 			"mask": 24, // as number or IP
 			"gateway": "192.168.4.1"
 		},
 		"ap": {
 			"ssid": "YellowToyCar",
 			"psk": "AAaa11!!",
 			"channel": 0, // channel to use for AP, 0 for automatic
 			"hidden": 0,
 			"ip": "192.168.4.1",
 			"mask": 24, // as number or IP
 			"gateway": "192.168.4.1",
 			"dhcp": {
 				"enabled": 1,
 				"lease": ["192.168.4.1", "192.168.4.20"],
 			}
 		},
 		"sntp": {
 			"pool": "pl.pool.ntp.org",
 			"tz": "CET-1CEST,M3.5.0,M10.5.0/3",
 			"interval": 3600000
 		}
 	},
 	/* Camera settings. See this project or `esp32_camera` library sources for details. */
 	"camera": {
 		"framesize": 13,
 		"pixformat": 4,
 		"quality": 12,
 		"bpc": 0,
 		"wpc": 1,
 		"hmirror": 0,
 		"vflip": 0,
 		"contrast": 0,
 		"brightness": 0,
 		"sharpness": 0,
 		"denoise": 0,
 		"gain_ceiling": 0,
 		"agc": 1,
 		"agc_gain": 0,
 		"aec": 1,
 		"aec2": 0,
 		"ae_level": 0,
 		"aec_value": 168,
 		"awb": 1,
 		"awb_gain": 1,
 		"wb_mode": 0,
 		"dcw": 1,
 		"raw_gma": 1,
 		"lenc": 1,
 		"special": 0
 	}
 }

Returns JSON of current configuration, if not changing anything.

For AP mode, default IP/gateway should stay 192.168.4.1 for now, as DHCP settings are hardcoded to some default values.
DNS, SNTP and NAT settings are also not implemented yet.
When changing network settings, device might get disconnected, so no response will be sent.

/drive → Basic controls endpoint, might be lagging as it's over HTTP, which uses TCP, which might retransmit old requests).

Querystring API:

 ?mainLight=1    // Main light (external bright white LED)
 &otherLight=1   // Other light (internal small red LED)
 &left=255       // Left motor duty and direction (negative values for backward)
 &right=255      // Right motor duty and direction (negative values for backward)

Returns nothing.

/capture → Frame capture from the car camera.
:81/stream → Continuous frames stream from the car camera using MJPEG that exploits special content type: multipart/x-mixed-replace that informs the client to replace the image if necessary. Separate HTTP server is used (hence the non-standard port 81), as it easiest way to continously send parts (next frames) in this single one endless request.

Fast controls API (UDP)

Application waits for UDP packets on port 83.

Short control packet

	_Octet	₀	₁	₂	₃
_Octet	_Bits	_{0 1 2 3 4 5 6 7}	_{8 9 10 11 12 13 14 15}	_{16 17 18 19 20 21 22 23}	_{24 25 26 27 28 29 30 31}
0	0	(UDP) Source port		(UDP) Destination port
4	32	(UDP) Length		(UDP) Checksum
8	64	Packet type ^{(always 1)}	Flags ^{(see table below)}	Left motor duty	Right motor duty

Flags

Bit	Mask	Description
0	`0b00000001`	Main light (external bright white LED)
1	`0b00000010`	Other light (internal small red LED)
2	`0b00000100`	Reserved
3	`0b00001000`	Reserved
4	`0b00010000`	Reserved
5	`0b00100000`	Reserved
6	`0b01000000`	Left motor direction
7	`0b10000000`	Right motor direction

For motor direction in the flags, cleared bit (0) means forward, set bit (1) means backward.

Long control packet

	_Octet	₀	₁	₂	₃
_Octet	_Bits	_{0 1 2 3 4 5 6 7}	_{8 9 10 11 12 13 14 15}	_{16 17 18 19 20 21 22 23}	_{24 25 26 27 28 29 30 31}
0	0	(UDP) Source port		(UDP) Destination port
4	32	(UDP) Length		(UDP) Checksum
8	64	Packet type: 2	Flags ^{(see below)}	Time (in milliseconds) to smooth blend towards target motor values
12	96	Left motor duty, percent as float (i.e. `63.8f` equals to 63.3% duty cycle)
16	128	Right motor duty, percent as float (i.e. `63.8f` equals to 63.3% duty cycle)

The flags in long control packet are the same as in the short, but without motor directions flags respected.
Use negative float numbers for moving backwards.

Scripts

Some scripts were developed to ease development and usage.

Config

$ python .\scripts\config.py --help
usage: config.py [-h] [--status] [--status-only] [--config-file PATH] [--wifi-mode {ap,sta,apsta,nat,null}] [--ip IP] [--read-only] [--restart [RESTART]]

This script allows to send & retrieve config from the car.

optional arguments:
  -h, --help            show this help message and exit
  --status              Request status before sending/requesting config.
  --status-only         Only request status.
  --config-file PATH    JSON file to be send as config.
  --wifi-mode {ap,sta,apsta,nat,null}
                        Overwrite WiFi mode from config.
  --ip IP, --address IP
                        IP of the device. Defaults to the one used for AP mode from new config or 192.168.4.1.
  --read-only           If set, only reads the request (GET request instead POST).
  --restart [TIMEOUT]   Requests for restart after updating config/retrieving the config.

Control

$ python .\scripts\control.py --help       
usage: control.py [-h] [--ip IP] [--port PORT] [--interval INTERVAL] [--dry-run] [--show-packets] [--short-packet-type] [--no-blink] [--max-speed VALUE] [--min-speed VALUE] [--acceleration VALUE]

This script allows to control the car by continuously reading keyboard inputs and sending packets.

optional arguments:
  -h, --help            show this help message and exit
  --ip IP, --address IP
                        IP of the device. Default: 192.168.4.1
  --port PORT           Port of UDP control server. Default: 83
  --interval INTERVAL   Interval between control packets in milliseconds. Default: 100
  --dry-run             Performs dry-run for testing.
  --show-packets        Show sent packets (like in dry run).
  --short-packet-type   Uses short packet type instead long.
  --no-blink            Prevents default behaviour of constant status led blinking.

Driving model:
  --max-speed VALUE     Initial maximal speed. From 0.0 for still to 1.0 for full.
  --min-speed VALUE     Minimal speed to drive motor. Used to avoid motor noises and damage.
  --acceleration VALUE  Initial acceleration per second.

Note: The 'keyboard' library were used (requires sudo under Linux), and it hooks work also out of focus, which is benefit and issue at the same time, so please care.

Controls for the control script

Controls:
	WASD (or arrows) keys to move; QE to rotate;
	F to toggle main light; R to toggle the other light;   
	Space to stop (immediately, uses both UDP and HTTP);   
	V to toggle between vectorized (smoothed) and raw mode;
	+/- to modify acceleration; [/] to modify max speed;
	Shift to temporary uncap speed; ESC to exit.

Tasks

Friendly name	Name	Affinity	Priority	Source file	Description
IPC tasks	`ipcx`*	All*	0	(internal)	IPC tasks are used to implement the Inter-Processor Call feature.
Main	`main`	CPU0	1	`main.cpp`	Initializes everything, starts other tasks, then carries background logic.
Camera stream	`httpd`	CPU0	5	`camera.cpp`
LwIP		?
WiFi		CPU0
Events		?
Idle tasks	`ipcx`*	All*	24	(internal)	Idle tasks created for (and pinned to) each CPU.

* - Some tasks work on multiple CPUs, as separate tasks.

Notes

Known issues

The communication (to ESP32) seems to work best in AP mode with UDP packets.
C/C++ compiler used is quite old and includes decade old known GCC bug related to structs aggregate initializers. See discussion here. As solution I found out its easiest to use strncpy which gets inlined/optimized away.
The PlatformIO docs about embedding files suggest to use prefix _binary_src_ while accessing the start/end labels of embedded data blocks (like in GENERATE_HTTPD_HANDLER_FOR_EMBEDDED_FILE macro), its not true. The docs seems outdated or invalid in some areas, at least for esp-idf. However I found solution: Use both board_build.embed_files in platformio.ini and also EMBED_FILES in CMakeLists.txt. In code, use _binary_, without src_ part.
Code style is a bit mess, snake_case mixed with camelCase because we use C libraries from ESP-IDF and some parts use them a lot. It's even uglier to ride a single camel in the middle of snakes.
There is an issue with easy enabling ESP_LOGV and ESP_LOGD for single file, so I redefine those macros to ESP_LOGI as a workaround.
...

Interesting materials

ESP-IDF 4.4.3 for ESP32 - Programming Guide - including API references, examples, guides and other resources.
YouTube series about RTOS by Digi-Key. Great introduction to ESP32-flavoured RTOS, with exercises for viewer.

To-do

Detailed status output, including debug stuff
- Process list and stats.
- Memory heap usage & fragmentation.
- Networking stats (packet counts?)
Min-max tasks:
- CPU pins:
  - One core for HTTP and trash tasks
  - Other core for networking & fast control (UDP)
- Use mutex to lock frame buffer between capture and streaming.
- Trace tasks? vTaskList/uxTaskGetSystemState
Website
- Camera
- Basic controls
- Network settings
- Camera settings
- Motors calibration
Networking
- When network config is changed, make sure to send some kind of response before disconnecting.
- Allow set IP and DHCP settings for AP mode.
- Allow change DNS settings.
- Captive portal when in AP mode.
- Password protection (especially useful when connecting to open networks).
- If password was to be implemented, don't forget to secure UDP server somehow.
SNTP time sync
- Make pool server and timezone configurable
Create our own Kconfig file to keep optional features there, including some debugging. Also see https://esp32tutorials.com/esp32-static-fixed-ip-address-esp-idf/
Allow some calibration for motors
Allow changing frequency for PWM signals for motors
Control LEDs with PWM?
Should there be status/echo packet types for UDP?
How does JSMN JSON handle escaping characters? Some strings like SSID/PSK might be invalid...
How do we nicely pass understandable error, i.e. from parsing config to response? https://github.com/TartanLlama/expected 👀
Does STA mode groups packets before delivering?
Explore hidden features of the camera, see espressif/esp32-camera#203
Isn't COM8_AGC_EN in the camera registers definitions off by 1?
Camera parameters are better described in CircuitPython bindings docs for the esp32_camera library.
Create fast and C++ constexpr string to IP 4 function
NVS dump. See https://github.com/AFontaine79/Espressif-NVS-Analyzer
Expose nice console over serial monitor
- Basic WiFi config
- Allow uploading JSON to change config?
You can use NAT?!
- https://github.com/jonask1337/esp-idf-nat-example/blob/master/main/main.c
- https://github.com/espressif/esp-lwip/blob/6132c9755a43d4e04de4457f1558ced415756e4d/src/core/ipv4/ip4_napt.c#L228

Name		Name	Last commit message	Last commit date
Latest commit History 21 Commits
.vscode		.vscode
include		include
scripts		scripts
src		src
.gitignore		.gitignore
CMakeLists.txt		CMakeLists.txt
README.md		README.md
platformio.ini		platformio.ini
sdkconfig.debug		sdkconfig.debug
sdkconfig.esp32cam		sdkconfig.esp32cam

AgainPsychoX/YellowToyCar

Folders and files

Latest commit

History

Repository files navigation