Wearable haptics reacting to determine direction of sound?
  • Unlucky_StrikesUnlucky_Strikes September 2016
    New forum member here,
    Hello everyone! First some basic backstory on myself, A few months ago I suffered Sudden Sensoneural Hearing Loss. (SSHL) This has left me completely deaf in my left ear. This has removed my ability to determine the direction any given sound is coming. I am a biologist and have always been interested in transhumanism and body modification. This forum seemed like a great place to learn and experience new ideas. Now on to the meat of the question.

    Humans require two points of reference (ears) to sense the direction a sound is coming from. Sound coming from one side is detected by that sides ear and the other ear picks of the sound a fraction of a second later, the brain then interprets the time difference to determine direction and to a much lesser degree, distance. Would it be possible to design a pair of wearables that work in concert with hearing aids to determine whatever sound is loudest and vibrate to give a rough sense a sounds direction?

    My initial thought is that a good position for this would be a necklace that vibrates on either the left or right side of the neck, input being sent from a pair of modified hearing aids such as a CROS system. While you wouldn't be able to pinpoint sounds you would at least know what side they were coming from.


  • BirdhandzBirdhandz September 2016
    @Unlucky_Strikes,
    Sounds like the idea could work to me.  If all you want from this device is to tell you which direction the loudest sound is coming from, using an Arduino and a couple sound detectors might be able to detect which microphone (left or right) has the highest signal.  From there, it should be easy to turn on a vibrating motor on whichever side you want.

    Background noise would probably be a problem but maybe you could just filter the microphone signals to only use sounds above a certain level to determine direction. 

    Those sound detector boards have an audio output which might be able to be used as the inputs for the hearing aid combining the two back to mono to go into your good ear. 

    I wonder if a belt mounted device would work almost as good as one around your neck and be more hidden.  Less robot-like.


  • MTSMTS September 2016
    cool idea and im sorry to hear about your hearing loss! and i agree with birdhandz. you could also vary the vibration based on the frequency of each sound. the higher the frequency the faster it vibrates, the lower the freq the slower the vibration, for example. adding frequency information could make the experience better/more informative and make it less boring or static.
  • ThomasEgiThomasEgi September 2016
    In theory the idea is pretty easy. In practice, things look a bit more difficult. So here are a few things I want to bring to your attention to help you solve problems you may run into.

    1. It's not only time delay. While you do get a time delay, your head and ears also plays a big part in locating sound. It dampens sounds depending on the direction of sound, your head and hair and the frequency of the sound itself. Resulting in a rather complex transfer function which can help your brain to identify sound directions even if there is no time delay. For example you can tell the difference between a sound from above, below, behind or in front of you, despite the sound waves arriving in perfectly the same time.  Kudo's to everyones brains btw, cause that's an incredible job it does there.

    2. In lab conditions, working based on timing and a bit of correlating of signals work pretty well. That is, you can easily locate the direction of an audio source if it is isolated. Especially if it's something like a loud bang which is easily and uniquely. This works best if the sound source you want to locate is a lot louder than the rest. If you want to locate the source of a gunshot, may it be civil or military use. you can pinpoint a sniper's position after his first shot by doing the math on a small set of microphones. Directions and distance can be easily calculated. But unless you  put yourself into a battlefield you probably won't be able to make use of this. Which brings us to point 3.

    3. Soundfields. Leaving the quiet lab, you'll have sounds hitting your head/microphone/whatever from pretty much every direction with pretty much all frequencies and phase angles all over the placer. If you want to locate the direction of the loudest sound, chances are you want to calculate the soundfield. Or in other words, put up a whole microphone array so you can reconstruct the actual pressure waves traveling across your microphone array. Since you'r probably ok with a limited resolution you might be able to get away with manually time-shifting the microphone signals according to the microphones positions in space and summing their signals up. Followed by FFT-ing the signal, weighting the amplitudes depending on the frequencies. Summing that up and at the end of the day you can decide which of the directions you calculated this for, contain the most powerful audio source.

    So when you think about building or developing this you probably want to plan for more than just 2 microphones. The more, the better.

    Btw, what caused your hearing loss? Damage to the auditory nerve? Or something inside the cochlear? Hair nerve cells? If the nerve's still intact you have some great bundle of now unconnected data-input lines to your body... unfortunately embedded in one of the densest piece of bone in your body too.

    Nevermind, here's what i'd suggest. Get a microcontroller with a bunch of analog input pins. Maybe some bigger Arduino variant. Aim for as much RAM as possible and a good amount of ADC-inputs, it'll make processing easier without pulling fancy tricks on your hardware. You'll also want a  bunch of cheap electret microphones and some op-amps to build a small amplifier circuit for each microphone (TL072 or so will do). Get into basic signal processing. Reading up on what FFTs do and how convolution work may help you along your project.  It's pretty much like a directional microphone, except you can define the direction in software, based on recordings.

    If a left/right thing is all you want to do, there are microphones that pick up sound depending on the direction. Of course this is frequency dependent to etc blabla yada yada but you may be able to get a working proof of concept device by simply choosing the right pair of microphones. Look for cardioid types.

    Also, if you make this a real thing using only 2 mics, build it in the shape of a sonic screwdriver.
  • Unlucky_StrikesUnlucky_Strikes September 2016
    Thank you for the input everyone!

    @ThomasEgi Thanks for giving a little more depth on the functions of hearing! I felt a little awkward having a big wall of text on the first post. As to the reason for hearing loss, idiopathic. I simply awoke one morning with total deafness on one side except for roaring tinnitus. Most likely suspects are an infection or autoimmune response that damaged the cochlear nerve.

    I think it may be time to hit up one or two of my engineering buddies. While I can safely do solder repairs I have never entertained something this complex. Also great idea on the cardioid mics, I was concerned about interference from my own clothes as I moved around wherever this rig gets placed, need more coffee.
  • BirdhandzBirdhandz September 2016
    This whole topic makes me wonder if it might be possible to "feel" sounds and actually understand what is being felt.

    Say you converted the sound into vibrations or even electrical stimulation pulses, would you eventually learn to "hear" those signals and be able to interpret different sounds or maybe even words?

    Since, I assume, you can still hear out of the other ear, maybe that hearing combined with a vibrating signal could be used to learn to hear vibrations as sound.  If the microphone was built into a device that fit your bad ear, you might even get the sense of direction back and use the ear itself as the reflector to make the microphone more directional.

    I'm interested in what you come up with.
  • Unlucky_StrikesUnlucky_Strikes September 2016
    @birdhandz,
    That's a neat concept, I guess it wouldn't be to bad with at least noting a difference between vowels and consonants.I believe all vowels are lower frequency than consonants, might have some trouble given everyone's voice box is different.
  • McSTUFFMcSTUFF September 2016
    Feeling sound:
    https://www.ted.com/talks/david_eagleman_can_we_create_new_senses_for_humans?language=en
    If you haven't heard David Eagleman talk about expanding senses or restoring senses through augmentation I highly recommend it.