- Mini-batches added
- Weight decay added
- Momentum added
- Cross Entropy error function is renamed to Negative Losss Likelihood, true Cross Entropy added
- Sigmoid layer added, with correct biases initialization for the classifier
- Splitting single epoch into multiple epochs through epoch_count_in_training_set parameter
- max_subsampling layer supports 1D and 4D in GPU backend (was 2D and 3D only)
- rotate_band_data_transformer is extended to all dimensions (was 2D only)
- extract_data_transformer extended to data of any dimension in case input and output windows match
- snapshot_data: added scaling and 3D (video)
- Sigmoid+Coss-entropy and Softmax+Negative-log-likelihood fusion implemented in CPU and GPU backends to increase accuracy
- Max L2 bound on incoming weights implementation is dropped (*)
- Conversion to bw image fixed in GTSRB example
- max subsampling updater and hessian - corner cases fixed in CPU backend
(*) I did that because L2 bound on incoming weights didn't improve quality in any problem I worked on. Supporting it is not free. So I decided to drop it.
