Fly by Sound
Fly an FPV drone by ear
A free way for blind and low-vision people to learn to fly a simulated FPV drone using spatial sound and spoken lessons. No sight is needed: it works with a keyboard, a gamepad or an RC radio, and the app speaks for itself.
This is the first stage of an experiment. The sounds and lessons are being tested with pilots, and how a sound feels is for your ears to judge. Please tell us what helped and what annoyed you.
Start
Put on headphones and press the button. The app speaks to you and begins with a short headphone check (left, right, behind), then the first lessons: hovering, walls, a corridor and rings.
This opens the simulator in sound mode. A computer with a recent Chrome or Edge works best.
What you need
- Headphones. Wired ones are best: Bluetooth adds 180 to 250 milliseconds of delay. Switch off Windows Sonic or Dolby Atmos.
- A keyboard, a gamepad or an RC radio. A sighted helper sets up a radio once.
- A comfortable volume. If a sound hurts, take off the headphones and turn the volume down.
- A helper for the first time, for the radio and, if you want to record, for the consent.
The main keys
The app also tells you these keys aloud.
- Tstart or stop a recording
- Usend the last recording to the author
- Ihear where you are
- Zhear the last spoken line again
- Gopen the sound glossary
- Rgo back to the last checkpoint; hold it to start the lesson again
- Ngo to the next step
- Lcopy a link that opens this lesson, with the sound settings you changed, to share it
- Khear these keys
If a key does nothing, your screen reader may be in browse mode: press NVDA+Space in NVDA, or Insert+Z in JAWS, so the keys reach the simulator.
Recording and sending
- Recording is off until you agree. It is stored on your device only.
- Sending is a separate step: the app asks, and you confirm with a second key press, with the Send button, or by voice if you turned voice commands on.
- Pilots under 18 need a parent's or guardian's signed consent, confirmed by the helper, before anything can be sent.
- Voice commands use your browser's speech recognition, which sends audio to the browser's maker. They are off until you tick the box; the keys always work.
For helpers, teachers and schools
- A consent form template (not legal advice) and a test protocol are in the open-source repository.
- For a pilot under 18, the parent or guardian signs the consent form and the helper confirms it in the recordings panel before recording.
- Keep the volume at a safe level. A page cannot measure loudness, so a school device is calibrated once: 75 dBA is the reference for children and 80 dBA for adults.
Open source
The sound engine, the lessons, the session format and the report tool are an open-source package, Shramko Fly by Sound (MIT licence). The simulator that carries it is ShramkoGSFPV.
Questions and feedback
Tell us what helped and what annoyed you: answer the short survey after a recording, press B in the simulator for Feedback, or write to the author.