Hello, friends! For quite some time now I have been hiding away while working on my dissertation in Human Factors Psychology. Inspired by wisdom from Universal Design principles and the social model of disability, what I’ve created is a wearable device that converts components of speech to vibrating patterns presented on the arm, a novel encoding scheme to do that conversion, training software in the form of mini-games played on a PC, and a set of experiments to test the whole system.
I’ve been eager to share it here, but I can’t share too many details yet, as the extra knowledge could influence the results if any of my participants happen to read into what I post. I can share videos of the training games once all of my participants have been briefed, and even more details once data collection is complete if anyone is interested. I’ll keep this thread updated with details as I can share them. For now, I can share this:
The device, which I’m calling the “Vibey Transcribey” is based on the ESP32 and uses 16 LRAs. Vowels and consonants are rendered to different locations on the forearm, pitch is rendered to the bicep, and loudness moderates the intensity of the vibrations. In its current form the device only works with pre-processed speech. The speech-processing software is mostly written in Python and the training games were created with Unreal Engine 4. The device interacts directly with the training software, so you could call it a gaming peripheral. The PCBs were designed with LibrePCB, and fabricated/populated by PCBWay. Enclosures, jigs, and other parts were designed FreeCAD and Blender and 3D printed on my Prusa i3 MK3S.
When the project is completed, everything will be shared on the project’s Open Science Framework page. This includes the speech processing software, the experiment’s evaluation software, CAD designs, electronics schematics, statistical analysis code, etc. I intend to submit for publication, although I haven’t settled on any journals yet.
Now that I have approval from the university and my dissertation committee, I can finish building devices and collecting data, which is why I’m posting today. I have already recruited a few participants, but need more volunteers - twelve total. All of the participants will do a pre-test and, six weeks later, a post-test. Half of the participants will take one of my devices home with them along with the training software, while the other participants will receive no training and serve as a control group. Participants who are assigned to the trained group will be expected to play the training games at least 2-3 times per week for the six week period between the pre- and post-tests, and each training session should last at least 30 minutes.
To participate, you must:
- Be at least 18 years old
- Be a native English speakers
- Not be fluent in any other spoken language
- Not have any diagnosed language disorders
- Be right-hand dominant
- Be at least a moderate video-game player
- Have access to a modern gaming PC with a controller
If anyone here is interested in being a participant, I have a brief screener survey that you will need to complete. I was fortunate to receive a small amount of funding, but not enough to cover all of the hardware costs, so I’m unable to offer any compensation for participation other than my eternal gratitude and the opportunity to be a part of (what I consider to be) pretty cool science. Fair warning though, the pre-test and post-test are tedious and each take 2 hours. If you are assigned to the control group, you’ll only experience the pre- and post-tests and none of the training games. Please do not apply if you are not willing and able to be in either group.
Seems really heckin’ awesome, though! You’re doing great!!!