How to build a spiking neuron?

Question

Hello Kyma People,

I want to build a spiking neuron in Kyma. Please don't ask me why, I'll let you know once I've discovered if I can do anything cool with them :)

For anyone unfamiliar with neural network tech, this is a bit different to the activation-level kind of unit that all the (not-so) new-fangled deep-learning things use. Spiking neurons model something like membrane potential but also respond to and encode timing information as well as just average activation.

The simplest model used to be known as a "leaky integrate and fire" and this is what I'm trying to model right now because they're reasonably simple. I've got the leaky integrator working ok (with some help from Gustav Scholda, ta :). Now I want to add the "fire" stage. Can anyone suggest a way I can do this? I currently think I that I need to make this thing stateful - with states something like:

a) Resting - accumulating and leaking input (the bit I already have)
b) Firing - after exceeding threshold, spikes to full then dips to below zero
c) Refactory period - recovering until it's ready to fire again

These correspond to "resting state", "action potential" and "refactory period" on this diagram https://upload.wikimedia.org/wikipedia/commons/4/4a/Action_potential.svg

Can anyone give me any hints how to build something like this that's stateful but, ideally, runs at audio rate?

My efforts so far - simple leaky integrators - https://www.dropbox.com/s/2z6by7nn0ixslei/Spikes.kym?dl=0

Matt

Comments

not sure if it helps but the picture looks pretty much like a swarmSmoother on a gate. The swarmSmoother is basically a physical model of a spring btw

---

Excellent suggestion. That's a good way to think of it.. massive positive spike that will then overcompensate in the opposite direction. Putting it like that also makes me think of a low pass with a bit of negative feedback, for that kind of resonant "hunting" behaviour.

---

If you were to build your own SwarmSmoother you would discover that it is in fact very similar to a LowPass with feedback, you got it! btw once you won't to port this all to WF style, I've got you covered with swarmSmoothing etc. ;)

---

Brilliant, thanks :)

Answer 1

Hi Matt,

Have you looked at LossyIntegrator, Hysteresis and SoftThreshold in the Prototypes? The S curve of the soft threshold is modeled on biological thresholds and the LossyIntegrator is exactly what you are looking for — a leaky integrator modeling the buildup and dissipation of chemical levels. You should be able to model the behavior using LossyIntegrator-->Hysteresis-->Soft Threshold.

Comments

I'd seen the lossy integrator but not the hysteresis or soft threshold. They sound perfect for this, thanks.

---

I can't see the SoftThreshold prototype anywhere?

---

Ah.. Saturator as Smooth Threshold?