Jul 12, 2014

nnForge v1.1.7

It is a big release. I added a number of useful features you would expect a NN lib should have. Here is the full list:
  • Mini-batches added
  • Weight decay added
  • Momentum added
  • Cross Entropy error function is renamed to Negative Losss Likelihood, true Cross Entropy added
  • Sigmoid layer added, with correct biases initialization for the classifier
  • Splitting single epoch into multiple epochs through epoch_count_in_training_set parameter
  • max_subsampling layer supports 1D and 4D in GPU backend (was 2D and 3D only)
  • rotate_band_data_transformer is extended to all dimensions (was 2D only)
  • extract_data_transformer extended to data of any dimension in case input and output windows match
  • snapshot_data: added scaling and 3D (video)
  • Sigmoid+Coss-entropy and Softmax+Negative-log-likelihood fusion implemented in CPU and GPU backends to increase accuracy
  • Max L2 bound on incoming weights implementation is dropped (*)
  • Conversion to bw image fixed in GTSRB example
  • max subsampling updater and hessian - corner cases fixed in CPU backend
(*) I did that because L2 bound on incoming weights didn't improve quality in any problem I worked on. Supporting it is not free. So I decided to drop it.