echolocation is a hardware device designed to as assistive technology to help visual impaired users use a standard keyboard. It does this by connecting to a standard keyboard and reading aloud to every keypress. It then connects to a computer as a standard keyboard, and if the user holds down a key it sends that as a keypress to the computer.
This document is written by a human and is the source of truth for AI agents so they know what to build.
The hardware used is M5Stacks, this allows us people to create the device without any hardware knowledge.
Specifically we are using the following modules.
- M5Stack CoreS3 ESP32S3
- M5GO Battery Bottom3 (for CoreS3 only)
- USB Module with MAX3421E v1.2
- M5Stack Audio Module (STM32G030)
You can connect a keyboard to the device, via bluetooth or usb. As soon as the keyboard is connected to the device it should start reading aloud every key that is pressed.
You can also connect a computer to the device, via usb or bluetooth. As soon as the computer is connected to the device it should start sending keypresses to the device when the user holds down a key (for the specified duration). It should simulate a normal keyboard keypress.
For each time the user holds the key down it should only send a single keypress to the computer.
The behaviour should be the same for modifier keys.
You should be able to seamlessly switch between different keyboards and computers.
It should work with any keyboard layout.
If the audio is playing and you press a new key it should stop the current audio and start playing the new audio.
A keyboard and a computer can be connected at the same time. Every keypress is spoken; only keys held for the configured duration are sent to the computer.
Settings are saved and persist across reboot.
On the CoreS3 screen it should show the current key that is being pressed. It should also have a battery level indicator. There should also be a settings button that opens a menu with the following configurable options:
- Volume
- Bluetooth
- Configure Keyboard Connection
- Configure Computer Connection
- Hold duration
- Factory reset
The screen should also report any errors that occur in a clear and easy to understand way.
The text to speech should be pre-generated and stored on a microSD. There should be a script to generate all the audio files needed to be loaded onto the microSD card. The script should easily be able to configure the voice of the text.
By default the audio should come out of the built in speaker. If you plug in a speaker it should seamlessly switch to the speaker and when you unplug it should switch back
Code should be kept as simple as possible and variables and functions should have names that very clear explain what they are from.
The code should be easy to alter in the future.
Wherever possible write tests. All tests should pass.
For hardware use C++. For scripts use NodeJS.
Code should have a brief guide on how to execute or load it.