Activation Function in Neural Networks

We always use a simple Hard Limit / Binary Step function with an adaptive threshold as our Activation Function. Instead of the most commonly used Sigmoid Function.

The adaptive threshold models a behaviour over time, which is elementar for the function of a Spiking Neural Network, and which can’t be achieved with other methods.

(You still may want to use a Sigmoid Function to model the adaptive behaviour of the threshold. Or anything else, it really doesn’t matter that much.)

And here comes the Big Trick:

If you don’t know what it means, then you’ll find out later..