Echolocation implant


You all may have heard by now that all humans use echolocation to some degree, and some of them, out of need, have managed to master the technique to remarkable heights. Fact is, some boost to our body equipment is needed to reach the level of precision of dolphins, bats and other animals. Research by academics has consistently shown that training is quite easier than expected, with sighted subjects starting to notice obstacles within hours, and to differentiate objects in about a week.

The solution would probably be either a cochlear implant or one straight to the V1, plus a sonar conveniently standing on a headset for signaling. Daniel Kish and co. have been working on such a device for several years (no news made public on the results so far, though), Warwick did some incursions into it, and now Cannon and Saver are tinkering with their magnet implants to develop their own "bottlenose" haptic echolocation-like system. There are commercial sonars for blind in the market, many handheld, most also haptic, and all of them failing to deliver significant results.

We are simply talking about automatically emitting a signature sound in the right set of frequencies and intervals by a sonar, and getting feedback from the surrounding at a much, much higher resolution than natural echolocators can manage to do (see Underwood, Bushway, Kish and others), straight to the inner ear or ideally implanted in the "visual cortex". The brain would do the rest, specially in those than lack visual input and therefore can use the processing power left unused to navigate using the new data flow. If people manage to get results by clicking their tongues, imagine what could they do with proper gadgets.

This would potentially mean being able to see in the dark, "structures such as bones, and even certain objects inside a bag", and more, as Professor Juan Antonio Martinez of Madrid's Alcala de Henares University reported in '09 , "this would be a new way of perceiving the world". As far as I see this is an area with huge potential, both for the blind and sighted people. Anyone willing to work together to make it happen please get in touch.
Tagged:
«13

Comments

  • Grindhouse is doing a sonar module *externally*. Sonar modules aren't even that hard to wire up. The tricky part in your proposal is wiring it into the brain.

    I'm in, of course. I'm always in.

    But I am not willing to experiment on my own brain, and I don't want to get caught up in any scandal. How do you want to go about this?
  • tipp: hands off the brain.

    cochlear implant's are terrible enough the way they currently work. and even that can't be implanted unless you have a highly qualified team of surgeons.

    the sonar idea itself is pretty easy to pull off, altho it does eat a lot of energy and will hardly work fully subdermal. but if you rely on some regular electrodes implanted under your skin somewhere, it should be do-able. or at least promising enough to try it.
  • What I was thinking was that it would be easy to utilize the idea of neural plasticity in combination with a magnetic implant. They seem great for sensing magnetic fields and whatnot, but why not wear a device that can induce vibrations in the implant through electro-magnetism in the same way that headphones work. if you could just convert the signal from a sonar apparatus into an analogue signal, it can then be turned into vibrations in the magnet. With tons of hours of "training" the mind by using multiple sense feedback, I imagine that you would be able to improve your ability to decode the vibrations in your hand. I think that they are doing something similar at Grindhouse Wetware, at least that's what I picked up from watching some of their videos.
  • Cochlear implants are awesome, but getting them costs you your natural hearing permanently.

    Braingate used the tongue to receive spatial data. They said it worked well. I wonder if it would be better than the ear?

    @ThomasEgi: Is the power consumption primarily the fault of the pulse making device or the sensor? Mechanical means could be used to make the clicks if it saves energy.
  • the sensor pretty much is a microphone, more or less. so it is not that demanding, it does eat some power tho. the more demanding part is the device that makes the pulses. altho if you aim for higher precision you'd probably want to avoid pulses and generate a more complex signal that's better suited for numeric correlation. this way you can use digital signal processing and get a very detailed "map" of the individual echoes, delay time and intensity.

    btw. as far as i am informed. "sound" quality of a cochlear implant is pretty much horrible compared to natural hearing.
  • I thought cochlear implant's sound quality issues were mostly software?
  • ahm.. you have like.. several thousand nerves connecting your ear to your brain. and a cochlear implant has 16 electrodes. maybe more recent ones have a few more but it is still no match at all. aside from that accessing the necessary nerves and stuff is incredibly difficult as all is cramped into a tiny room with lots of very dense bone around it. and the nerves you want to get your electrodes on are layered and wound up in a coil so you can't really target them precisely.
    such an implant will allow you to notice a car crash next to you. but even understanding word's will take tons of training. at least. that's my current knowledge bout this subject. you may be a lot better of with an electrode array in your earlobes or so.
  • Has anyone got links to the research the academics did? Specifically, what methods they used to train the sighted subjects. I figure, why not learn this as a skill. Could come in handy, like lip reading.
  • ThomasEgi: " you may be a lot better of with an electrode array in your earlobes or so" , That's actually a really cool idea, stimulation to the ears from a bottlenose like implant. I better shut up, way out of my league here...
  • rdbrdb
    edited August 2012
    There appears to be a surprising amount of information on the subject of learning (unaided) echolocation:
  • This also has some basic info: http://science.slashdot.org/story/09/07/04/2246211/you-too-can-learn-echolocation

    And good call, I'm going to give it a shot too and see how possible it is. Will probably write an article documenting the experience afterwards :-)
  • The site at 
    http://learnecholocation.blogspot.nl/ appears to be one of those 'test for sufficient interest' sites, that wants to confirm that there is a market, before writing and publishing the material.

    I could be wrong about this one, but don't hold your breath, at least about this specific publication.

    It is an interesting idea, though.


  • There are a whole bunch of free training lessons on the site I linked.  I haven't read through them extensively, so I can't comment on the quality, but it seems to contain plenty of useful information.
  • As an alternative, if it turns out that my subdermal bonephones don't hurt like hell, you could use those.  Then you could mount the emitter and mic on the back of your head and 'see' behind you.
  • a skull wasn't designed to be a speaker. conducting audio is one thing. emitting it is another. i sorta doubt the skull makes a decent ultrasonic emitter.
  • Ultrasound is being used for various brain modulation and cortical scanning techniques, which would suggest that the bone is more permeable to ultrasound than lower bands to some degree.  I could be wrong, though, I don't have the actual data.
  • edited August 2012
    Thanks for the links everyone, the whole idea seems to be in its infancy. I guess not many people see a need. Still, I think i'll give it a bit of practice. Who knows, maybe it'll come in handy.
  • edited August 2012
    Another cool device with similar function. 
    I like the dual rangefinders and the light detecting component. Also, the head-mounted device seems like it would be easy to aim and have a natural feel to it.
  • edited September 2012
    This reminded my of an episode of Quirks and Quarks (Canadian science program) I listened to a few years ago. Here is a link from the radio show http://www.cbc.ca/quirks/media/2004-2005/mp3/qq-2005-04-02a.mp3 .

    Here is a little applet that simulates seeing with sound. http://www.seeingwithsound.com/javoice.htm .

    I came upon this when I was looking into using sound as an interface to bringing data into the brain faster than reading it. Totally external to the body.

    edited: I should learn what haptics means before I use the word the wrong way.
  • you can simply use sound to generate tones and chords. like using braille with audio. a lot simpler than current screen readers.
  • I'm not sure about echolocation as 'hearing' - unless someone already has a hearing implant... Current cochlear implants don't really have great sound definition as far as I know - it's better than being deaf, but not worth sacrificing normal hearing...
    Btw. the cochlear implants still have 16 leads and higher definition than 16 tones is achieved by using electronics to simulate higher definition - instead of giving current from one electrode the implant uses two electrodes with varying current, the closer the tone to tone of electrode 1 the higher current in electrode 1 and lower in 2, while when the sound closes to electrode 2 tone the current on electrode 1 is reduced and on 2 increased and so on... Thanks to this there are 'virtual' electrodes between physical ones defined by how well the electronics are able to simulate it by adjusting currents.

    I personally would imagine an echolocation implant as a device implanted on hand or forearm (not sure what size the device does have) and implanting electrodes (wires as thin as possible to reduce trauma to the hand, possibly injected, or inserted by other method not as traumatic as cutting and separating skin from connective tissue underneath) into palm of a hand - for example 16 electrodes forming a 4x4 grid or 36 for 6x6 grid (or any other number we could fit inside), and then using a microchip to translate input from echolocation device to signals to electrodes, possibly adding the 'simulated' electrodes in ways similar to those used in cochlear implants.
    This way using those electrodes to produce a tingling sensation (or touch, or other - possibly not pain ;P ) stronger in places where something hard - more reflective to ultrasound is detected, weaker with things 'gray' on ultrasound and with no electrode activation when something is translucent to ultrasound, it would create something similar to a sensation of touching the surface being scanned with ultrasound - a sort of tactile map...

    I was thinking of using ultrasounds as in a ultrasonograph, to scan patient's body (I'll be a doctor soon) while simply touching him with hand, adding USG depth to regular palpation, but I don't see why something like this couldn't be used for echolocation in a similar way, just different wave frequency...

    I guess a sort of glove device could be made as prototype proof of concept before actually trying to implement something like this as it's quite an extensive implantation...

    Alternatively, to not touch palm of hand (which is difficult to cut into without specialized equipment and experienced hand surgeon, without risking loss of feeling in hand and other problems) the electrodes could possibly be implanted on inside of wrist or forearm (skin there doesn't have such feeling definition as on palm of hand, but should be sensitive enough - ultrasound isn't something that produces HD results anyway, just rough images)

    I still guess something like this is quite a big surgery, but on forearm it could be doable without a specialized hand surgeon ;P
  • @rdb - Thanks for posting the link to my blog.  I've been trying to learn echolocation for some time now and it is actually easier than one might think.  I've had some really good success and it's proven to be quite an interesting process of learning and discovery.  I'm a product design engineer myself and have tried to think of devices that might aid in the echolocation capability, but since it is so easy and accessible with only a little bit of practice, I don't think it's even worth a device.  It would end up functioning as well as one of those apple corer/slicers, or a plastic egg poacher... something that no one really needs, but they will use it if they don't have any interest in thinking for themselves.  :)

    I'm new to biohack, but it seems like a cool website with undoubtedly some good ideas.

    Anyhow, if you guys have any questions about echolocation, check out the blog, I'm always adding stuff there (http://learnecholocation.blogspot.com) and you can contact me directly if it floats your boat.
  • So.....anyone familiar with this guy's research? He did exactly what you are talking about doing. He implanted the chip that communicates with the sonar device in his spinal column.
  • @anthonynlee
    We have some people working on reproducing Warwick's experiment. We talk a little bit about it here:

    Our main hurdle right now is getting affordable multielectrode arrays, although re-reading the article just gave me an idea on how to possibly bypass that. I'll post something a little later for a feasibility critique.

    Welcome to biohack.me btw!
  • @anthonynlee:  Welcome to biohack.me!

    Yes, I think most of us are familiar with Warwick's work.  In fact, one of Grindhouse's devices, the Bottlenose, is very similar to the work that Warwick's students are doing with interfaces based on the finger magnets.  We are also looking into the possibility of getting our hands on a multielectrode array and implanting it, but the biggest hurdle there is the cost.
  • I saw that about the bottlenose. In fact, he was the first thing I thought of when I watched the video demonstration. I kind of figured you would know his work....I just didn't see him mentioned when you were discussing the issue here. :)
  • Wow Warwick's experiment is damn amazing!  Same with the bottlenose.

    Why limit this idea to echolocating though?  I've been contemplating this for a couple of days, and it seems like you would need an external system of some sort to gather data right?  So you would need to implant something to talk to that external input or rely on some sort of haptic feedback from the external device itself...

    So if we've got a bulky external device, why not super power it?  The best, and most available, device for mapping a 3d space is the Kinect.  You could potentially shoulder mount/wrist mount one and utilize it to send a 3d map to your internal system. 
    http://www.geeky-gadgets.com/3gear-dual-kinect-motion-controller-system-offers-precise-gesture-control-video-04-10-2012/

    There's already proven methods to observe the surrounding data utilizing a kinect, so the next question is how to map this into something your body can understand...Is there any potential to install decently sized magnets into your chest area?  If so you could put maybe 4 magents on both your front and back (one near each shoulder, one in a higher location in the middle and another below in the middle), then utilize a vest with magnet coils to stimulate the implants.  Utilizing multiple magnets would not only allow you to determine multiple objects, but also allow you to determine specific sizes, for example a person or a door frame.

    It would also be interesting to know if you could do a push and a pull on the magnet and be able to tell the difference.  Specifically for the blind, having the ability to just map where something is and knowing how fast it's moving or if it is coming or going.

    ps.  Just ordered my magnets, have been wanting to install them for months now and will hopefully have them in the next month or so.
  • If you have to have a big-ass external setup, then there's really no point in implanting magnets at all.  If you're wearing a vest, then you can get the same effect by just placing vibrators on your skin, or perhaps even something better; you'll have greater control over the sensation and you can avoid the risky procedure.  That starts to fall out of the realm of biohacking, though.

    On top of that, putting magnets in most locations won't do a lot of good because you need an area with high nerve density in order to be able to get a significant sensation.

    Also, you won't be able to sense static or slowly-changing fields with magnetic implants.
  • @rdb is absolutely correct.  We at Grindhouse are looking at building our device into a glove, so avoiding (relatively bulky) haptic motors makes sense, but if you're going to be building it into a vest anyway, haptic motors are probably the better option.  Also, magnets have never really worked anywhere but the fingertips and the genitals (and possibly the lips).

    It's not true that you can't sense slowly-changing fields, though (since you can feel it pulling the magnet slightly), it's just a much weaker sensation.  Either way, that problem is pretty easy to overcome, by instead pulsing the magnet, and changing the frequency of the pulses depending on the information received from the sensor in question.\

    Now, how to map the kinect data onto the magnets?  I imagine you'd need multiple magnets for that, but if you know all the data that the kinect outputs, coming up with an algorithm for mapping it is comparatively simple.

    ~Ian
  • rdb I had actually written a sentence along the lines of "Not really sure if this would be better than any other haptic feedback system" at the end, then removed it...lol

    At any rate I would think it falls into the same category as the glove.  Some wire run along a vest (for electromagnets) would be much thinner than vibration/pressure systems.  This potentially allowing the wearer to wear normal clothes. 

    Even with big magnets you don't think you could feel them vibrating?  Not even enough to tell if they are pulling in a direction?  I would think in the chest you could mount some pretty big magnets.  Electrodes on the skin may work too, though that seems more like an on/off sensation instead of a 3d sensation.  Vibration would be good for just range finding, though you would need a lot of motors to get a 3d feel and you probably couldn't get a high fidelity because you're not to sensitive to the different levels and you couldn't put them too close together.  Pressure plates could work to give greater differentiation, as well as telling push/pull.  Maybe a combination of vibration and pressure to give speed and distance? 

    Ian, ignoring the problem of location and size: I think it would be easy enough if you were comfortable with something like a raspberry pi (though it seems like at this point that device doesn't have enough power to do it).  Getting the data out is the easy part. 

    I'm not a physicist, so I'm basing some of this off of 3 college physics courses and a crapton of math courses.. One thought is to utilize standing waves to create the shape of the surrounding environment as the magnet field. 
    imageimage

    Or maybe this demonstrates what I'm thinking of better (skip to 1:00)


    The youtube is ferro fluid.  If you have mutliple magnets you can make different forms.  Would there be a potential way to map a height map to a magnetic field and then project that?
Sign In or Register to comment.