You wouldn't SIMULATE a fruit fly?

Replicating results without the original code, on a laptop, for fun and zero profit

It feels kinda funny, looking at this thing as she hops along in her virtual world.

As far as I can reasonably tell, she’s not aware that she isn’t biological. I didn’t want to say “not real”, because she’s probably about as real as she could be: it’s just that her body lives in a world unlike our own.

She lives inside my laptop, within its humming circuits, and she does not know it.

I’ve simulated the brain and body of a fruit fly on my laptop.

As many of you would probably know, some time ago I wrote about my adventures in downloading the brain of a fruit fly and rendering a visualisation of it on my little ol’ laptop.

For those that don’t remember:

That was fun and all, but I was hoping to take it further and simulate parts of the fruit fly brain itself, to see if I could mimic any known real-life firing patterns in the neural circuitry.

Turns out, that’s a real challenge, because no one has been able to obtain high-resolution recordings of the activity of all the individual neurons in a real living fruit-fly brain. The reason is actually quite simple: it’s really freakin hard to record the activity of just one neuron in a biological brain, let alone all the neurons individually across the whole thing at the same time. So, with nothing to compare it to, I’d have no idea if my simulations were a reasonable simulacrum, or total nonsense.

However, one alternative approach I was pursuing was using a kind of virtual body, perhaps with a rendered 3D “world”. The different virtual body parts might receive signals from a simulated central nervous system connected to motor neurons and allow the fly to move around, while sending data in to its sensory neurons where it can “see”, “smell”, “taste” and “touch” the rendered 3D world we have created for it.

I kinda got stuck at that point, because the idea of building such a virtual environment to provide my little virtual fly brain with all of its sensory inputs, and then figuring out the correct mappings for the correct neurons, felt kind of overwhelming; so I put it aside for a while.

I’m a little annoyed that I did, because just the other day, someone sort-of beat me to the punch, and got a whoooole lot of press about it. Oddly, though, they didn’t release their code, so no one can really verify what they did.

I got even more annoyed by that fact. Why do something like this without releasing the code?!

So, as one does, I decided to replicate it by mah damn self, or at least part of it, based on the descriptions given in their company blog post, which wasn’t all that helpful if I’m honest.

It took me about 3 days. I found a pre-released version of their connectome code - just the brain, no integration, no body - and hooked it up with the virtual body powered by NeuroMechFly. I was then able to render a 5-second (in-world time) clip of the fly sort of hop-shuffling forwards, and then turning slightly to the left.

I was quite proud of myself for this little feat. I’m not a neuroscientist, and I have very limited knowledge of the connectome itself. I also hate python, like, viscerally. The company also hasn’t shared much technical information on exactly how they did this, either, so I had to make use of what they did provide, combine that with various publications and annotated datasets on the neurons themselves to identify the correct inputs and outputs, and sort-of vibe-code the rest.

At first, I was under the impression that merely stimulating the sugar-sensing neurons - ones which tell the fly that there is a source of sweet, sweet sugary goodness very nearby - was the thing giving impetus to forward locomotion. That was an incredibly exciting idea: the fly was choosing to move toward a sugar source! “Wow”, I thought, “this is real, autonomous behaviour!”

Sadly, I was to be disappointed.

In the connectome used by the Eon team in their simulation, it seems they are force-stimulating a special neuron called P9. This neuron seems to force a signal out through the descending neuron structure that would normally connect to the Ventral Nerve Cord (like a spinal cord for flies) and tell it to walk in the forward direction. Although we don’t have a full VNC simulated here, the signal is used to instruct NeuroMechFly to proceed forwards.

In my quest to replicate their results, I of course included stimulation of P9. In their post, they did make it pretty obvious what they were doing, but I just wasn’t really paying attention.

Curiously, that slight left-turn the fly makes at the end of the clip above was unexpected. Is it possible the fly is adjusting its direction of motion by choice? I don’t honestly know.

So I had another go. This time, using some of the Jupyter Notebook experiments in the original codebase, I tried to connect all the relevant bits for those behaviours to the actuators of NMF so we could observe the expected physical behaviour.

First, I tried activating grooming, by stimulating the sensory nerves of its antennae.

We all know what fly grooming looks like, of course, and it does not resemble this:

Nor this:

One theory is the way I’m calculating the new front-leg positions, and overlaps/collisions, is causing all kinds of mayhem.

Tommy Blanchard said I was just activating the well-known Drosophila Break-dance Neuron.

We’re now 5 days in to this run of experimentation. I am pushing ahead, but it is ridiculously slow to run these on a laptop. When you’re having to make a lot of tweaks and adjustments and retrying over and over, the iteration loop is painfully slow with the hardware I’m using. It takes at least 10 minutes to render 5 seconds worth of simulation.

I will keep this going for a while and see how far I can get. Ideally, I’d like to be able to simulate the entire VNC as well - for which I already have some working code, but the hooking up of the two connectomes is going to be a trick, and might require much more powerful hardware than I have currently available to simulate.

If anyone wants to donate some compute resources to help me make this go faster, I’d be very appreciative. Leave me a comment, or reply to this email (if you received this in your inbox).

Meanwhile, you can observe and/or participate in my tomfoolery over on the forked codebase: https://github.com/MiracleBlue/fly-brain

Watch this space, more to come. In the near future, I will do a more detailed technical write-up of exactly how this works.