2 posts tagged “theoretical computer science”

Last week on the TWiT's Futures in Biotech episode 10 Marc Pelletier interviews Dr. Carla Shatz about the human brain. The most fascinating part of the entire interview was a brief part discussing how the brain manages to connect the retina to the visual cortex without ending up completely random. This wasn't the first time I had read about the mechanism – V. S. Ramachandran has at least mentioned it in one of his books – but I had pretty much forgotten.

What happens is that the retina projects nerves to the back of the brain to connect with the occipital lobes to connect into the primary visual cortex. The connects are for the most part random, which is of course bad  because
there's no sense to the signals. Waves of activation spread across the retina, neighboring neurons lighting up with electrical activity while more distant ones remain quiet, sending signals back to the visual cortex. The visual cortex then is able to coordinate the input, since even though the receiving neurons seem to randomly light up, they only light up because they neighbor one another in the retina. I suspect it uses some form of Hebbian learning, where neurons that fire simultaneously tend more often to be affected by one another's activation and those that don't are less affected.

To illustrate I offer a simple example using a string of characters. Suppose you have a string that's been scrambled, say "     bbeenoooorttt". With information about neighbor status (the first t is next to the start of the string and next to the first o, the r is next to the second o and the first space, the second space is next to second and third t's, etc.) you can reconstruct the original: "To be or not to be". By using the neighbor status of the letters you can very simply say what they are near and thus by knowing what everything is near, you know where everything is.

there are a number of interesting applications that this might have, such as perhaps self building microchips, but the one I'm particularly interested in is artificial neural networks. Self organizing properties would make it significantly easier to design neural networks because it would remove the necessity to design the thing down to the tiniest detail. Instead they could be designed in a broader sense, by function rather than specific wiring. All that's left to do is get cybernetic implants to work properly and we'll have working cyberbrains! But that's another project...

I know this blog has primarily been about my devving, and this post isn't going to be different, but so far I've only focused on Jade and the related projects. I'm going to diverge a bit, now and in my next post, so brace yourselves!

This post is about parallel computations, not in the traditional sense of breaking a problem into multiple independent subtasks and performing them on multiple processors simultaneously, but rather something very different. Instead I mean a system in which the result of computation is independent of the order in which the parts are computed (which does indeed mirror traditional parallel computing), but in which the answer does dependent on what inputs are present. So while it is, in an absolute sense, completely true that traditional parallel computing does result in this, it's only in a strict sense. Sure, if you made a list of all the results of computation, that list would be different depending on which computations were and weren't done, but it's only different as a single whole. No two items are dependent on one another.

I contrast this with what I'm talking about in that there is interdependence; the answer may indeed be a collection of data, but each piece of data is influenced by a number of different inputs, not just one, which is the case with traditional computation. It's more a difference between how input maps to output, I suppose you can say. To modify the parallel computing notation, traditional computing models in general are SISO, single-input/single-output. Whether it's serial, with one instruction being processed at a time, and producing one output, or in parallel, with multiple processors doing multiple things, but each providing a single output at any given time. What I refer to instead could be thought of as MISO or MIMO, multiple-input/single-output or multiple-input/multiple-output. (Interestingly, you could also have a SIMO, single-input/multiple-output model, which creates divergent results through from process, perhaps random mutation, and this would model one of the processes involved in evolution.)

The reason I bring this topic up is because of a particular problem. If you try to simulate a non-SISO computation on an SISO machine, it's relatively easy to do. Because the results are order independent, the calculations can be done granularly in any particular order. But trying to do the reverse, in particular, trying to simulate an SISO computation on an MIMO machine seems to be especially difficult, because an MIMO machine has no instruction cue like an SISO computation does. Serial, and traditionally parallel, computations have some predefined list of instructions, but non-SISO parallel computations have no such predefined list of instructions, so how can you simulate SISO computations on non-SISO machines? It's an interesting problem, especially because it's essentially the question of how the brain, which is massively parallel MIMO device, can create our conscious experience, which seems to have some serialized nature, especially with things like mathematics, which is inherently serial. This difficulty in simulating MIMO computations might even explain why math is such a hard thing to learn.

One possible method, the only one that I've been able to think of, is that you might have a processor that produces a particular output from a particular input, and you might have a second processor that produces the next input, thus creating a feedback loop. This might not necessarily require that the entire input-output flow be mapped before hand, where you already know what outputs lead to what inputs, and aren't really computing anything new. Instead you could set it up so that the simple presence of the output, along with the current state of the processor and the input generator together produce a new input. This way all you'd have to do is have an input generator that depends on the completion of one process, and on the information of which input it previously provided, to generate a new input in a sequential manner. Of course it would be difficult to truly get a process-based system working, perhaps even impossible, since at no does the process really stop performing the computation. Every input to the processor changes the output, and the same goes for the input-generator, so theres a constant feedback.

I think this is a beginning for how it might work, not necessarily the way it does in fact work. But it is a starting point, something to use in creating models of computation.