A theoretical approach to a pseudo-exocortex

13»

Comments

  • edited September 2015
    In order for us to get useful EMG data, we need to establish a baseline for signal at whatever monitor points we select, without any attempts to subvocalize. After that, taking some measurements when someone is speaking, and then when someone is attempting to subvocalize will likely give us a better idea of what we're looking for.

    On the other hand, try reading a word written on a piece of paper, rather than just thinking it.
  • I don't know anything about any of this so forgive me if this is a stupid idea.

    bciuser says spoken and mouthed speech give a usable signal. What if you keep your mouth closed but move your tongue like you're speaking out loud?
  • @bciuser What are you using to interface with the EMG machine? USB connection? External microphone port? Bluetooth?

  • The biopac setup I'm using is connected via Ethernet to the computer used to store the data. I don't have access to any other type of electrode, but I can cut these to make them smaller and try out various positions on the neck.Qualitatively the baseline is identical to the imagined speech, but I will try again next week after Grindfest.
  • From what i've heard, picking it up requires slightly more effort than just imagined speech, you have to make a small effort to get the signals actually down to the vocal chords.

    Difficult to describe in words, but hopefully you get what i'm saying.
  • edited September 2015
    Hey all,

    Simply imagining speech is not how subvocal recognition works. Shine has the right idea: You need to activiate the actual muscles that produce speech. Basically, you keep your mouth closed, but talk through your nose (you can also more easily do it with mouth open, but that would be obvious in public). In case that doesn´t really make any sense, let me explain. You move your tongue, jaw muscles, etc. as if you were actually speaking, but push the air out of your nose instead of your mouth. This should give you the most detectable EMG signal, while producing minimal sound and no visual clue you're using subvocal communication.

    See here: this is an interview with a researcher from NASA who works on subvocal recognition, and he explicitly says you measure the actual muscle contraction. http://thefutureofthings.com/3021-speaking-without-saying-a-word/

    "Q: Is there a difference between “thinking in words” and
    subvocal speech?  In other words, would you describe your device as a
    mind reading machine (even if a crude one at that)?

    A:
    Yes, there is a difference. Subvocal speech requires some
    activation of the speech muscles. It is not in any way a mind reading
    machine. Subvocal speech requires active cooperation and intentional
    stimulation of the speech muscles
    . Hence it is voluntary and private."

    Also, the website of that commerical thing is very confusing and definitely  wants to give readers the idea that it works by measuring nerve impulses (i.e. imagined speech)... but it doesn't, as is stated in their FAQ. Also, notice how the first question I quote has incorrect information and the second uses ambiguous language:

    "• What signals does the Audeo pick up?

    The Audeo picks up neurological signals (This is not true. It measures muscle activity. See below.) that are sent to the
    muscles of the vocal cords. These are purely control signals, which mean
    that a person has to activate these signals physically for the system
    to actually pick it up.



    • What activity does one have to do to get the Audeo to work?

    A person has to produce control signals that activate the muscles,
    in order for The Audeo to work (Again, quite vague. Why not leave out part of this sentence and make it clear what this device actually measures: 'A person has to activate muscles'. That's all it does. I don't know why they keep mentioning words like 'neurological signals' and 'control signals'). For example, if you try to move your
    tongue up and down, or your throat forward, muscle contractions will
    occur in the laryngeal muscles, which in turn will produce the required
    signal. With some practice, a user can intentionally produce these same
    signals without any visual indication of activity.

    • Does it work by just “thinking about speaking”?

    It is not enough for a user to just ‘think about speaking’. The
    user has to produce some physical signal which is what we term as a
    control signal. There are different examples of producing this control
    signal, two of which are mentioned above. Another easy way to produce a
    signal is to swallow. This helps a user to initially learn how to
    produce a signal. To start with this might be an easy enough method, but
    going down the line this is not the best way for continued use."


    So @bciuser : First, see if you can see a difference between rest and actual speech. Then: Compare actual speech with the 'nose'-speech I described (this is what you'd use with a subvocal communication device). They should theoretically give you a similar signal. Of course, it might be difficult to get a good signal from the neck, since your neck muscles are constantly firing to balance your head. Maybe try to rest your head on something like they often do in EEG research and see if that improves your speech signal. Also, visual inspection might not make it clear whether two conditions give different or similar signals. You might need to average over many trials... And after doing that you may find that the signal wasn't good enough to distinguish between rest and speech conditions and you will have wasted your time.. Electrode quality might simply not be sufficient to measure the muscles dedicated to speech.

    I wouldn't expect any good results. The people at NASA probably have some of the best equipment available and it took them years to distinguish a handful of phonemes.

  • Hi there Slach, and welcome.

    It seems you know a lot about the subject of SVR, so do you think it's even possible to pull off in the way I described in my first post?

    Bearing in mind that the point is to eventually use subdermal electrodes so that they stay precisely in the same place every time - something that has been a problem in SVR research.

    I've also pondered using a stethoscopic or throat microphone as per this:

    http://www.researchgate.net/publication/221489429_Non-audible_murmur_(NAM)_speech_recognition_using_a_stethoscopic_NAM_microphone

    This should make it possible to get a larger range of phonemes while still being fairly private - thoughts?
  • edited September 2015
    Thanks Gareth and hello,

    Let me just start by saying I am not an expert on SVR, so take everything I say with a grain of salt. Stuff like this is just something I'm interested in.

    To answer your first question: Theoretically, yes! This is basically what the people at NASA are doing, correct? You want a device that can register what you´re 'saying' from the musculature in your neck/throat? They're using non-invasive methods right now (just like you are). In cases where SVR can be an important tool (firemen, military, etc) I think we can also expect to see invasive SVR in the future, as that offers a more reliable signal without the need for daily installing or readjustment of electrodes.

    But like I said: SVR unfortunately is harder than one would think. I guess getting the right signal isn't even the biggest issue. Once you get a good signal, the main problem becomes classifying different phonomes. This requires a lot of trials! Also, with non-invasive methods, you need the electrodes to be in the exact same location every time. So, during classification, even small spatial deviations of electrode position can totally mess up your data for that session. In that respect, invasive electrodes would make your life a lot easier. Of course... actually getting electrodes implanted - especially when it's pretty much just for fun - is not feasible today.

    These days (neuro)surgeons have become quite adept at fixing electrodes in one location, so that shouldn't be a problem in the future. I actually wonder whether for instance the military isn't already using more advanced SVR stuff in the field today. All the technology and know-how is there basically. There's no reason why they wouldn't have it if they have use for it (unless I'm missing something??). Invasive SVR might even be able to pick up imaginary speech if inserted at the right location near the nerves innervating speech producing musculature.

    That stethoscope might do the trick. I'm not sure what this thing actually measures. According to this link you need to articulate (just like in SVR) and 'breathe' the speech out of your mouth (again, I can't think of any other way of explaining it). The device then amplifies the vibrations that it picks up through the soft tissue of your head. http://search.ieice.org/bin/summary.php?id=e89-d_1_1

    And according to this vid it's actually closer to whispering

    I have no idea how much volume/vibrations you need to produce to make that thing work. If whispering is acceptable for you - it should work.

    Btw don't be discouraged. If you have the hardware, playing around with this can be a lot of fun and actually very informative, even if you can't get the actual SVR to work.
  • I have been eyeballing some tech which could be modified for use in a speechless telephony setup. It is similar in ways to the video above. You would need to hook it up to earbuds or your cyberaudio to control the sound.





    http://www.ultravoice.com/how.htm

    So basically this emits a tone and you shape it with your mouth, much the same way your vocal cords emit a tone. Instead of an audible tone, we can emit something in ultrasound and use the mouth to shape those words. If our mic and processor can heterodyne the frequency to an audible range we can hear in our earbuds then we should be able to shape the words pretty good. Good enough for a speech to text system anyway. The audio would need some good filtering too. I guess it doesn't even need to be a denture. What do you guys think?
  • @bciuser which biopac system were you using?
Sign In or Register to comment.