Apr 28, 2013

nnForge v1.0.2

I have finally published nnForge v1.0.2

This release contains the single major feature: Performance tuning for Kepler GK110 (GeForce Titan, Tesla K20). I have also improved the performance for Fermi cards.

What about Kepler GK104 (Tesla K10, Geforce 680, 670 e t.c.)? Almost all the optimizations I applied for GK110 are applicable to GK104, though I didn't test it. I don't have GK104 card so I didn't even run the code on it.

Initially I planned to add support for 1D convolutional layers, but ended up adding it for testers and hessian calculators only. The reason is simple: It is better to have an example on which I would be able to test new functionality. Otherwise I might just add a lot of code which doesn't work.

Apr 26, 2013

Grumbling a little

I constantly find out that it is rather easy to make a mistake when implementing forward/backward propagation and weights update for neural network layers. For example, mistakes with offsets or iteration count. In the best case I get cuda-memcheck error and thus I am able to identify the problem and fix it right now.

In other cases the network will work almost fine. For example, if I accidentally set the iteration count to a value lesser than required, then I might end up just not updating some weights. And network will accommodate to such accidental restrictions and will still learn pretty fast. Until I encounter network schema where this bug affects too much weights and network starts learning too slow. *sigh*

Apr 6, 2013

NVIDIA GeForce Titan

Just bought GeForce Titan. I will play Bioshock Infinite first then will proceed with optimizing nnForge for GK110 :)