Released Neural Network Libraries v1.35.0 and nnabla-rl v0.13.0!
We have released Neural Network Libraries v1.35.0!
Along with this update, nnabla-rl has also been updated to v0.13.0!
Please see “Spotlight” for important changes.
Spotlight
Lion optimizer
We’ve added the new optimizer Lion, published in February, as a Solver! This optimizer has excellent memory efficiency since it only holds momentum as a state, and it has been reported to achieve reduced training time and high performance in several experiments. Please give it a try. For detailed technical information, refer to the following paper.
Xiangning Chen, et al. Symbolic Discovery of Optimization Algorithms. arXiv 2023.
prune the constant branches
The nnabla-converter supports various model conversions, but the tflite converter has significant improvements in this release!
# How to install nnabla-converter
pip install nnabla nnabla-converter
# convert nnp to tflite
nnabla_cli convert -b 1 input.nnp output.tflite
# convert nnp to tensorflow, then remove redundant operators on tensorflow model
nnabla_cli convert input.nnp output.pb
nnabla_cli optimize output.pb improved.pb- Precompute the constant branches
Andoid GPU delegates for TFLite don’t accept the operators with 2 constant inputs. We improve the tflite converter to eliminate those branches by precomputation. - Remove Split and Slice operators if the input shape and output shape are the same
Until previous versions, the converted TFLite model contained a few redundant operators. By eliminating these operators, the model is more compact and has better performance. - Replace operators for quantized model conversion
The different backends (or versions) have different support statuses, and the quantization model operators are a subset of TFLite operators. The quantized TFLite supports the core set operators, but some operators are not supported. So, these unsupported operators are replaced with supported operators. - Support TRANSPOSE_CONV operator for quantized model
TRANSPOSE_CONV seems to be supported on the Android GPU delegate for TFLite, so we optionally support this operator in the tflite converter. - Improve TensorFlow to TensorFlow optimization to remove redundant Transpose
The functionality to remove redundant operators with the “optimize” subcommand has been supported, and now it also supports TensorFlow v2.x.