Could you train yourself to perceive 4 spatial dimensions?
  • InvertInvert August 2016
    EDIT - MORE ACCURATE TITLE: Could you develop a sense of 4D spatial awareness through tactile exploration of a 4D virtual world?

    I've just heard about the Bach-y-Rita experiment with the stimulators in the chair and about BrainPort. My understanding of how it works may be simplistic, please correct me if I'm wrong and tell me if anyone has thought or done anything like this.

    It seems like people using these devices can gain a real sense of 3d space with the extra information, so they actually experience the objects in front of them like sighted do with vision. If you created a 3d virtual world and sent the first person view through these devices, they wouldn't be able to tell the difference (except for the fact that they can't interact with it). They would build a mental model and develop a sense of space from the virtual reality.

    My idea is to create a basic maze game with movement in 4 dimensions and feed the proper sensory data to the brain. If the brain could create a 3d mental map from playing a 3d maze game, then maybe it could create a 4d mental map from a 4d experience. Or maybe 3d spatial processing is hardwired into the structure of the brain and processing 4d data is impossible.

    Some questions: 

    What type of device would be able to transmit the data, and how much data could it send? The BrainPort looks like it has a 20x20 grid, and the cells in the grid can send a variable strength signal. That's at least 400 bits of data. The most minimalist maze could be done I think with 32 bits of sensory data total.

    How could I get one of these devices? Could I build my own?

    Does the physical location of the electrodes relative to each other matter? For instance, if you jumbled all of the pixels on the BrainPort, would the brain learn which pixels were next to each other and form a coherent image? Because with 4 dimensional data, I'm not sure you could put all spatially related impulses next to each other on a 2d surface.

    Thanks for any info or any other input. I realize I may be totally wrong and this may all be pointless.
  • chironexchironex August 2016
  • InvertInvert August 2016
    Why not? I don't mean perceiving 4 dimensions in the real world if that's what you're answering. If the brains concept of 3d space is based on its experience of sensory input from 3d reality, then it might be able to build a concept of 4d space if given 4d sensory input.
  • glimsglims August 2016
    Have you read Flatland? You can conceptualize a fourth dimension, but you can't actually see one of those with your three dimensional eyeball. Even our representations of the fourth dimension in games are not actually four dimensional things. They are 3 dimensional representations of a fourth dimensional thing, so we don't really have any 4th dimensional inputs. This is one of those limited by physics things.
  • InvertInvert August 2016
    Haven't read it but am familiar. I'm not trying to see a 4th dimension with my 3d eyeball, I'm trying to see (or feel) a 4 dimensional virtual world through a virtual 4 dimensional body. If a flatlander got fed all of the sensory information from a human 3d eyeball and all of the limbs and vestibular system, I think his brain would be able to understand it and create a 3d model of our world in his mind.

    So I want to see if I can hook up all of the sensory data (inertia, collision, maybe depth sensing as sight) from a computer generated 4d maze and use a xbox controller to navigate through it. My theory is that as I move around and receive feedback I will build a mental model of the 4d space and actually be able to feel like I am moving around in it, like I could a 3d maze. The inputs would just be movement along the 4 axis, which could be done on the controller's 2 joysticks.

    Does this make sense? I know it may not work like that, but I want to try it anyway. I'm figuring out now the best/easiest way to get the information to the brain. I assumed I would need electrostimulators or vibrators, but now I'm thinking I might be able to just use a computer screen and the controllers vibrate function.
  • glimsglims August 2016
    Again, we don't have 4 dimensional data. We have 3d representations of 4d situations. And what is going to receive the feedback? And what direction to you push an analog joystick to move ana and kata?
  • glimsglims August 2016
    I'm not saying this doesn't sound like a fun project btw
  • InvertInvert August 2016
    If I have a computer program that simulates a 4d environment, then I do have 4 dimensional data. The data does not need to be represented in 3d, the 4d data can go directly to the brain through touch, vision, or sound. This idea is based on the concept of sensory substitution. I may have a overly optimistic view of how it works, but I see it as the brain being a general pattern recognition device. You give it data (like the senses from the 3d environment) and it detects patterns and creates some sort of model of the pattern.

    If I made a 2d maze and fed the sensory data through the skin, so you could feel when you bump into a wall and feel yourself moving in each direction, you would eventually develop a sense of being that 2d body in the 2d maze.

    With 2 2D joysticks, I have input for all 4 axis of movement. For receiving the feedback, it could be anything. The best I think would be stimulus on the skin so you actually feel yourself bumping into walls. There would need to be one for each direction you can move. From what I understand it doesn't matter how the information is received. For each of the 8 possible directions a wall could be in, I will have a separate stimulus. As I move randomly and bump into walls, the brain learns that these 8 stimuli represent my sense of "touch" on each "side" of my hypercube body. I don't really know how it will work, but as long as I can program a 3d maze, adding a dimension should be simple.
  • McSTUFFMcSTUFF August 2016
    @Invert, I see the potential but I'm having trouble imagining how it would be used.
    Can you walk me through, step by step, a run through the maze? Let me know every important detail of each room and each turn. Every bump in every dimension.
    You've glossed over a few things with the idea that you'd tackle them when you got there but they're things that might not have a solution with current technology and even said, "I don't really know how it will work, but as long as I can program a 3d maze, adding a dimension should be simple." I don't see that as simple so can you explain what is involved?
  • MeanderpaulMeanderpaul August 2016
    Please forgive me but I'm very interested in this threads direction for mostly curiosity.

    What exactly are you going to tell the 3D simulation to make it show a 4d world? I'm having a hard time visualizing how your going to make something turn from 3D to 4d and what it is even going to show you in order to be 4d.

    My understanding, being very small on this, was that the 4d would be time and space. If I'm wrong please by all means inform me and I understand that's probably a very basic description if I'm right.
  • InvertInvert August 2016
    I can't really walk you through the maze but I can describe the data I guess. A 3d maze is mathematically similar to a 2d maze, simply with an added dimension. I can describe a maze in 2d, and you would be able to imagine what the 3d version would look like, and get some grasp on what the 4d version would be. So I'll just describe the 2d version and the 4d version will be like that but with 2 extra dimensions of movement. 

    I would like to create a detailed 4d environment with complex shapes and winding tunnels, but that would require a large amount of sensory data to read,  a large amount of processing power to calculate, and very complex math that would take months for me to grasp. So I plan on making the most basic world possible.

    In 2d it would look like a grid of squares, with some squares being passable, and others being impassable. Like a single block moving through the white space of a crossword  puzzle. For each direction you can move in, I want the following sensory information:
    -Movement in that direction. This would be like a sense of inertia.
    -Touch in that direction. This would be triggered on each side that is adjacent to an impassible block.
    -Resistance to movement. If you are trying to move against an impassible block, you should feel a different force then when just touching it.
    -Distance to impassible block in that direction. This one is optional and not binary. The closer you get to a wall, the stronger you would feel it. It would be like a depth-only vision.
    -There's many things I could add to get better sense of the world, but these I think would be sufficient to develop a sense of presence.

    They layout of the maze could be generated randomly. I would like to have some sort of "home base" area though that I can always return to, it might give me a better sense of things instead of constantly wandering around new unknown environments. 

    In terms of the programming, there are tons of 4d games: 
    They all visualize it in their own way, but programmaticly and mathematically, a 4d world would be no different than a 3d world. I will define a 4d array of cells, and define the player's body as an object with 1 height, 1 width, 1 depth, and 1 "whatever". Its position will be defined by xyzw coordinates. The control input will move it along these coordinates, and if it encounters an impassible cell, it won't be able to move further, and the appropriate sensory response will output.

    The main unknown is how the brain will interpret the data. It is my understanding that if the data is simply layed out in an arbitrary way on the skin, the brain will learn what each stimulus represents, and build a mental model.

    There is no 3d simulation. It is a virtual environment of 4 spatial dimensions. In there, time would be the 5th dimension. Imagine a brain in a vat with no experience of our world. It has no concept of 3d space, or even of space at all. The only concepts it has are what it learns from experience. If you gave it the stimulus of a 2d world, it would begin to "think" in 2 dimensions. The same could happen for 4 dimensions.
  • JohnDoeJohnDoe August 2016
    So I got totally lost after reading this.... After a Google search I have a few questions. In practice would this be a ability to see around a corner, or assuming with the ability to see 4D come the ability to move in 4D. You could enter a room with no doors or windows though a alternate dimension. While those abilitys would be nice I don't under stand how they could be possible at all.... Also what would be any real world application for this be, aside from a screwy way to see the world?

    Okay can someone give me a better explanation, after playing 2048 4D, everything I thought I knew is wrong.... I think....
  • InvertInvert August 2016
    I think that my title is misleading a lot of people. This has nothing to do with any ability to see 4 real world dimensions or have super human abilities in the real world. I don't think that a 4th spatial dimensions even exists in reality.

    A common statement people make when talking about a 4th spatial dimensions is that it is impossible to really picture in your mind. Our mind's spatial reasoning is done in 3D, so comprehending 4D space is impossible. We can have 3D visualizations of 4D shadows, or look at 3D slices of a 4D object, but we can't picture the whole object as one.

    If this were a scientific study my hypothesis is that there is no inherent 3D processing function of the brain, that 3D spatial reasoning is totally learned from experience, and you could therefore learn 4D spatial reasoning by having a "hands-on" experience with it. If this is possible, then it could be that many other seemingly incomprehensible concepts (such as consciousness, infinity, or space-time) could be grasped in an intuitive way if experienced from the right perspective. It would also just be cool to be able to say you can think in 4 spatial dimensions, it would be a clear example of the expanding of consciousness to a reality separate of our own.
  • MeanderpaulMeanderpaul August 2016
    I'm so lost now after a quick Google search absolutely nothing was made clear...I can definitely agree about not being able to conceptually see this or even imagine it. I'm just gonna hover on this thread till I read something I understand.

    @johndoe I'm with you on this.
  • InvertInvert August 2016
    Ignoring all of the 4D talk, you should understand how sensory substitution works. Here is a good document. Here's some excerpts

    "The TVSS (« tactile vision substitution system») makes it possible to convert the image captured by a video camera into a "tactile image". In the standard version, the tactile image is produced by a matrix of 400 activators (20 rows and 20 columns of solenoids of one millimeter diameter). The matrix is placed either on the back (first version), or on the chest, or on the brow"

    "Equipped with the TVSS, blind (or blindfolded) subjects are almost immediately able to detect simple targets and to orient themselves."

    "An essential observation is that this capacity to recognize forms is accompanied by a "projection" of the objects which are perceived as existing in an external space. Initially, the subject only feels a successions of stimulations on the skin. But after the learning process described above, the subject ends up by neglecting these tactile sensations, and is aware only of stable objects at a distance, "out there" in front of him. A number of experimental observations confirm this externalisation. For example, if the zoom of the camera is manipulated unknown to the subject, causing a sudden expansion of the tactile image, the subject takes characteristic evasive action"

    "Blind persons discover perceptive concepts which are quite new for them, such as parallax, shadows, and the interposition of objects. Certain classical optical illusions are also reproduced"

    These blind people were able to essentially "see" things in front of them. The data was passed from a video camera, through the skin via tiny vibrations, and into the brain. They did already have a sense of 3d space though, through touch and sound.

    The first idea I got from this was that the reality that is passed through the skin doesn't have to be from the real world, it could be from a computer generated virtual reality. If this is possible than trying it with a 4 dimensional virtual reality might allow you to "see", in your mind, a 4D world around you as you move through it.
  • ProteusProteus August 2016
    I'm also trying to wrap my head around this, because it sounds fun.  For the purposes of the maze, is the 4th dimension a way of 'jumping' through 3d?  

    I'm picturing the maze's 4th dimension as being a kind of 'intensity' that affects how you maneuver the 3 regular spatial dimensions.   Am I way off?

  • TheGreyKnightTheGreyKnight August 2016
    Are you thinking a 4th spatial dimension, as opposed to assigning something like time to the fourth dimensional slot?

    It's not perfect, but one implementation could be having 2 Different mazes that can only be solved by jumping between them from different points in the other mazes, and then overlaying them. And have your fourth "dimension" be something like "color". Make a red maze and a blue maze. 
  • Nick1Nick1 August 2016
    Im very interested in this. I've always wanted some tactile feedback in games, but this sounds way cooler.
  • InvertInvert August 2016
    @Proteus, Yes you would be able to jump "around" 3d space with 4d movement, but I don't want to treat the relation ship between the 3rd and 4th dimensions any different than the relationship between the 2nd and 3rd dimensions. All 4 dimensions exist equally, and a 3D space could exist among any 3 of the 4.

    @TheGreyKnight, Yes, a 4th spatial dimension. There are tons of games similar to the one you described. This one is very similar. It is not a very good representation of 4d space though, it's just a few 3D mazes next to eachother, with a few 4D paths between them. It doesn't really give a good sense of 4D space, which is my goal.

    @Nick1, Thanks. I'm just going to start the programming part now, and figure out the sensory transmission later. I could possibly just use the rumble function of my xbox controller, but I don't know how much differentiation there could be between different signals.

    I need to figure out how I can obtain a stimulator array like the one on the BrainPort device. Either electro or vibro would be good.
  • JohnDoeJohnDoe August 2016
    Would it be like flatland?
  • DiethylDiethyl August 2016
    I feel like I understand the concept, but to execute this properly you would have to accurately model the dimensionality of your simulated world and how does one model a 4D object? I'm not aware of any 4D modeling applications that are available, so you would end up having to make your own. Once you create your modeling application how do you actually create models in it without being able to perceive 4 dimensions, would you just have to basically make gibberish 4D objects? Once you have your 4D models you would have to find a way to render them and I don't know if transformations from 4D to 3D are the same as 3D to 2D, for example how to do you rotate a 4D object in 3D space. I think maybe a lot of these "4D" games have taken artistic liberties with the mathematical definition of 4D. Don't get me wrong I love the idea and I'm sure we all want to see it work ,but we're just not sure how. 
  • ZerbulaZerbula August 2016
    Ultimately, What I perceive from this is the equivalent of describing Tetrachromat to a Trichromat human... Just my own perception. ^^'

  • eggiteggit August 2016
    I have the same questions as Diethyl, how do you crate a 4d world when you can't perceive 4d?

    I also wonder fi you could use tricks similar to Doom. Doom is a 2d game, that gives the illusion of 3d. Perhaps you could make a 3d game that gives the illusion of 4d.
  • InvertInvert September 2016
    Sorry I missed all of your posts, stopped sending me reply notifications for some reason.

    @Diethyl I have created a very simple 4d "world" in the Unity game engine. It is just a 4D grid of cells. The cells can either contain a block or be empty. You play as a block yourself and move through the grid, "feeling" when you bump into the impassible blocks.

    Right now with the maze, I am able to navigate around a 3d obstacle via the 4th dimension. This is possible in all other 4d games, but it is not intuitive, and the 4th dimension feels like some special rule added to the 3d world instead of just another direction to move in. "Success" of this project would mean that I will be able to intuitively move around in 4d space without thinking of the 4th dimension as special, like there are just 4 dimensions to move in. And I will be able to have a mental image of the 4d space I am moving around in.

    The rumble output of the xbox controller is too course to create distinguishable signals for the different stimuli. I've been searching around for a while trying to find an vibro-stimulator array to buy, but can't find any. It looks like I will have to make one myself, which is a much larger project, and I don't have any electrical engineering experience. So I probably won't be doing much work until I have more free time.

    If this first project is successful, then I will look into creating a more "realistic" and detailed 4d world that is more than just a  4d grid, with curved surfaces and winding tunnels. I wouldn't be able to model it directly. I would just procedurally generate it on the computer. Just like this 2D image can be generated randomly, a 3d or 4d version could be generated and used as the geometry of the world.

    If I wanted to simulate vision in this world, my virtual body would need to have eyes with a 3-dimensional surface. I assume 4d space can be projected to 3d space the same way 3d can be projected to 2d. I'm not sure if I would need more than 2 eyes for depth perception in 4d. I might be able to just have 1 eye and encode the depth information in the light rays directly. This type of simulation would be very complicated and computationally demanding, and I probably don't have the expertise to create it fully. But if the first project is successful, then I might invest some time in it.

    It would be like rewiring a 3D human's body to the brain of a flatlander, and the flatlander learning how to navigate through 3d space that way.

    I don't want to create an illusion of 4d, there are tons of games that do this already. I want to train the brain to create an actual mental model of a 4d space.
  • DiethylDiethyl October 2016
    Very cool it sounds like you have a good handle of what you are doing, keep at it I would love to see/perceive/play this game. Only question I have in regard to the procedurally generated levels is if would be possible to accidentally generate a level that could block you in, the 4D equivalent of being trapped in a cell? Otherwise this is an amazing idea, it seems like less of a game and more like evolutionary training. I once heard something of an ancient mystery school/cult/invisible college whose goal it was to properly perceive the 4th dimension by contemplation of the tesseract and once this was accomplished the initiate was said to be a "transcended master". Not sure who this applied to, but I have a feeling it was the school/cult of Pythagoras.
  • DiethylDiethyl October 2016
    Is there any chance of screen shots of your progress so far? Oh, also an interesting tid-bit. A substitute teacher once told me that the cube is the shadow that a tessaract(hypercube) would cast. I thought that was a pretty cool concept, but that's assuming that light interacts with the 4th dimension as it does in the 3rd.
  • hertafeldhertafeld February 27
    This looks neat. 

    It seems like there are actually two questions you're looking at:

    Can the human brain develop a spacial intuition in 4D?

    Can the human brain model a virtual space from arbitrary but consistent tactile input?

    Your method seems to be simply combining these questions into one. The Bach-y-Rita experiment seems to answer the first question: Yes, human neuroplaciticity lets us generalize our inputs and learn to interpret them instinctively as long as they're consistent.

    The second question I think you're on your way to answering. You listed many 4-D games and you're working on something similar yourself. My suggestion is to just play through them all, and see if you begin to intuit this fourth dimension rather than, as you mentioned, see it as a gimmick or movement between 3D worlds. If one game especially gives you a "feel" for 4D, try implementing something similar and more nuanced (using the much higher-bandwidth inputs of sight and sound) and see how far you can take it.

    My main point is that these two ideas seem easily separable. If you're really sold on tactile + 4D, I'd suggest really tackling each on their own before putting it together. Find a 4D exploration paradigm that's you've found to slide easily into intuition with existing senses, and then try to distill it to the lowest bandwidth/resolution you can while maintaining the intuition. In parallel, play with the tactile input in a familiar domain (2D/3D/words/sounds/etc), so that you can see how much information can reliably be pushed through this medium. How arbitrary can you make your input - eg. can you scramble spacial pixels and re-learn a space? 

    Just my own thoughts. Keep me posted and best of luck!

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Login with Facebook Sign In with Twitter Sign In with OpenID Sign In with Google

Log In Sign Up

In this Discussion