nnForge v1.1.11

Hi, I am releasing nnForge v1.1.11 with a number of significant changes:

• Padding added to sparse convolutional layers
• Sparse convolutional layers implemented in GPU backend (Kepler+ only)
• Fixed bug with dropout when error function is fuzed with last activation function
• Array with random numbers extended to 256K elements (for dropout)

nnForge v1.1.10

Hi, here is nnForge v1.1.10. The main new feature is zero-padding for convolutional layers, I should have implemented it long before. The full list of changes:

• You can now specify zero-padding for input data for convolutional layers
• Memory usage calculations improved
• Learning rates is per part now (was per parameter) - training consumes less memory, bigger networks might be trained
• Dropout implementation is simplified
• Minor fixes

nnForge v1.1.9

I released nnForge v1.1.9:

• More sparse cases supported in GPU backend for convolutional layers, improved perf
• convert_data_type_transformer added
• Hessian based learning algo is removed
• Galaxy Zoo example removed. Use previous releases to get it
• Reporting average weights/updates after each batch
• Image classifier demo added, improved perf for running single entry through the tester

nnForge v1.1.8

Hi, nnForge v1.1.8 is released:

• Sparse (in feature map dimension) convolutional layer added, with full support in CPU backend and fully connected (spatial) 1x1 support in GPU backend
• You can use -std=c++11 now with CUDA 6.5 toolkit
• Gradient check added
• GTSRB switched to batch training
• Boost and OpenCV libs default paths are /usr now
• Improved performance for 1x1 convolutions in GPU backend
• Minor fixes

nnForge v1.1.7

It is a big release. I added a number of useful features you would expect a NN lib should have. Here is the full list:

• Mini-batches added
• Weight decay added
• Momentum added
• Cross Entropy error function is renamed to Negative Losss Likelihood, true Cross Entropy added
• Sigmoid layer added, with correct biases initialization for the classifier
• Splitting single epoch into multiple epochs through epoch_count_in_training_set parameter
• max_subsampling layer supports 1D and 4D in GPU backend (was 2D and 3D only)
• rotate_band_data_transformer is extended to all dimensions (was 2D only)
• extract_data_transformer extended to data of any dimension in case input and output windows match
• snapshot_data: added scaling and 3D (video)
• Sigmoid+Coss-entropy and Softmax+Negative-log-likelihood fusion implemented in CPU and GPU backends to increase accuracy
• Max L2 bound on incoming weights implementation is dropped (*)
• Conversion to bw image fixed in GTSRB example
• max subsampling updater and hessian - corner cases fixed in CPU backend

(*) I did that because L2 bound on incoming weights didn't improve quality in any problem I worked on. Supporting it is not free. So I decided to drop it.

Jetson TK1

I got my own Jetson TK1, with Ubuntu 14.04 LTS running on it!

nnForge v1.1.6

I implemented a number of quite useful features in nnForge recently:

• Stochastic Gradien Descent training method is added
• Resume training fuctionality added
• Duplicating output to log file
• Logging current settings at the toolset initialization
• rgb_to_yuv_convert_layer_tester added in CPU backend
• Readers are redesign to allow variable data readers
• classifier_result is extended to top-N
• Added possibility yo split single reader into multiple epochs
• Multiple fixes