AI/ML

ADA-Tucker: Compressing deep neural networks via adaptive dimension adjustment tucker decomposition.




Icon for Elsevier Science Related Articles

ADA-Tucker: Compressing deep neural networks via adaptive dimension adjustment tucker decomposition.

Neural Netw. 2018 Nov 13;110:104-115

Authors: Zhong Z, Wei F, Lin Z, Zhang C

Abstract
Despite recent success of deep learning models in numerous applications, their widespread use on mobile devices is seriously impeded by storage and computational requirements. In this paper, we propose a novel network compression method called Adaptive Dimension Adjustment Tucker decomposition (ADA-Tucker). With learnable core tensors and transformation matrices, ADA-Tucker performs Tucker decomposition of arbitrary-order tensors. Furthermore, we propose that weight tensors in networks with proper order and balanced dimension are easier to be compressed. Therefore, the high flexibility in decomposition choice distinguishes ADA-Tucker from all previous low-rank models. To compress more, we further extend the model to Shared Core ADA-Tucker (SCADA-Tucker) by defining a shared core tensor for all layers. Our methods require no overhead of recording indices of non-zero elements. Without loss of accuracy, our methods reduce the storage of LeNet-5 and LeNet-300 by ratios of 691× and 233 ×, respectively, significantly outperforming state of the art. The effectiveness of our methods is also evaluated on other three benchmarks (CIFAR-10, SVHN, ILSVRC12) and modern newly deep networks (ResNet, Wide-ResNet).

PMID: 30508807 [PubMed - as supplied by publisher]

Source link







Related posts

Artificial Intelligence for Mental Health and Mental Illnesses: an Overview.

Newsemia

University of Iowa Healthcare rolls out first autonomous AI diagnostic …

Newsemia

Innovations in inclusive housing

Newsemia

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept Read More

Privacy & Cookies Policy