ToF distance sensing
Measures nearby obstacles in milliseconds and activates haptic warnings without Wi-Fi or cloud AI.
- Distance classification
- Configurable thresholds
- Always-on local feedback
Concept · MVP in development
A second layer
of vision.
VisionCane is a safety-first smart cane concept for blind and low-vision people. Local distance sensing warns about nearby obstacles in real time, while AI adds useful context about objects, pathways, signs, and surroundings.
Design principleLocal sensors provide safety.
AI provides perception and context.
AI provides perception and context.
The need
Context beyond contact.
A white cane is an essential mobility tool, but physical contact alone cannot name what is ahead, read a sign, or explain the wider scene. VisionCane is designed to add that missing layer without taking control away from the user.
01What is directly in front of me?
02Is that a person, vehicle, door, or staircase?
03What does the nearby sign say?
04Where is the clearest path through this scene?
How it works
Two perception paths. One clear response.
Immediate obstacle sensing stays on the device. Visual interpretation runs only when requested. The ESP32-S3 coordinates both paths and prioritizes local warnings.
ESP32-S3
Runs the state machine, coordinates camera and sensors, manages Wi-Fi, and gives safety alerts priority over narration.
- Sensor fusion
- Fault recovery
- Audio & haptic control
Camera + Gemini Vision
Captures a selected frame and returns concise scene context, object identity, OCR, or an answer to a question.
- Scene understanding
- Object recognition
- Reading & Q&A
ToF says: 62 cmA measured local distance
AI says: motorcycle aheadIdentity and context—never a replacement for ToF distance
Safety first
The cloud can fail. The safety path must not.
VisionCane is designed so that network loss, API timeouts, or uncertain AI responses never disable the local ToF-to-haptic warning loop.
AI failure≠Safety failure
Network failure≠Safety failure
AI uncertainty≠Invented certainty
Immediate warnings stay on-device
The ToF sensor and vibration motor do not wait for an internet round trip.
Critical haptics interrupt narration
A nearby obstacle takes precedence even while an AI response is being spoken.
Uncertainty is spoken clearly
The system is designed to say when it cannot identify an object confidently.
Core capabilities
Useful perception, delivered with restraint.
The experience is intentionally concise: one press, one useful answer, and no constant stream of narration.
Obstacle warnings
Configurable distance zones translate into distinct vibration patterns, giving immediate physical feedback without relying on the cloud.
Scene understanding
A short press captures a frame for AI analysis, prioritizing mobility-relevant objects such as people, vehicles, doors, stairs, curbs, and pathways.
03 / HUMAN-CENTERED OUTPUT
Concise open-ear audio
- Short messages such as “Door on the right.”
- Bone-conduction or open-ear output preserves ambient awareness.
- Critical local warnings remain higher priority than speech.
OCR on request
When asked to read, VisionCane captures a frame and returns visible text—or clearly says when the text is not legible.
Ask about the current view
A long press enters question mode so the user can ask what is ahead or request specific visual context.
06 / PRIVACY BY DEFAULT
Capture only when needed
- No continuous video recording.
- No facial recognition.
- Frames are sent only for a requested AI task.
- Temporary image buffers are designed to be discarded after processing.
Hardware system
Every component has one clear job.
The prototype architecture uses widely available, low-power parts. Together they create an independent safety path and an on-demand perception path.
01 · CONTROLLER
ESP32-S3 with PSRAM
The main brain: coordinates sensors, camera, Wi-Fi, buttons, audio, haptics, and the runtime state machine.
Why it fits: compact, low power, camera-capable, connected, and suitable for reliable embedded control.
02 · VISION
OV2640 camera
Captures a forward-facing JPEG frame when the user requests a scan, reading task, or scene question.
Why it fits: proven ESP32 support, manageable image size, and enough detail for cloud scene analysis.
03 · DISTANCE
VL53L1X ToF sensor
Measures the actual distance to nearby obstacles and triggers haptic warnings locally—even with no internet.
Why it fits: fast, compact range sensing that does not confuse AI-estimated distance with measured distance.
04 · HAPTIC
Vibration motor
Turns distance zones into tactile patterns: one pulse for nearby, two pulses for warning, and rapid vibration for critical proximity.
Why it fits: immediate, private, and usable when audio is busy or the environment is noisy.
05 · CONTROL
Physical tactile button
Short press: scan. Long press: question mode. Double press: repeat the last response when enabled.
Why it fits: direct, discoverable control that does not depend on a touchscreen or precise gestures.
06 · AUDIO
Bone-conduction / open-ear audio
Speaks short scene summaries while leaving the ears open to traffic, voices, bicycles, and other environmental cues.
Why it fits: sealed headphones could mask critical ambient sound; open-ear output preserves awareness.
07 · POWER
Protected Li-ion / LiPo battery + charging circuit
Supplies regulated portable power to the controller, camera, sensors, haptics, and audio. The prototype target is 4–8 hours, subject to physical testing.
Why it fits: rechargeable energy density for a compact handle, with protection and regulation instead of direct battery-to-GPIO wiring.
08 · RECOVERY
Hardware watchdog
Monitors the firmware and resets the controller if a task stalls, helping the device recover from camera, network, or software faults.
Why it fits: embedded safety functions must recover rather than remain frozen after a transient failure.
Sensor fusion
Measured distance meets visual meaning.
Fusion happens at the ESP32-S3. The ToF measurement always remains authoritative for distance.
Local sensor62 cm
VL53L1X provides measured proximity.
Visual contextMotorcycle ahead
Gemini identifies the object and direction.
Prioritized outputRapid haptic + short audio
“Motorcycle nearby.”
Roadmap
From a build-ready system to field validation.
The architecture is defined. The next work is disciplined prototyping, testing, and learning with users—not claiming a finished product before it is ready.
Architecture & specification
Safety invariants, state machine, hardware roles, failure behavior, and acceptance criteria.
DefinedIntegrated prototype
ESP32-S3 firmware, ToF-to-haptic loop, camera capture, AI response parsing, and open-ear audio.
NextBench & safety tests
Offline operation, timeout recovery, malformed AI output, warning priority, power, and physical durability.
PlannedUser-led pilot
Validate language, haptic patterns, comfort, usefulness, and failure handling with expert supervision.
PlannedEarly collaboration
Help shape a safer second layer of vision.
We are looking to connect with accessibility experts, embedded engineers, mobility specialists, and future pilot partners who want to help test the assumptions behind VisionCane.