General Bitwidth Assignment for Efficient Deep Convolutional Neural Network Quantization

Quantized Convolutional Neural Networks Robustness under Perturbation

Contemporary machine learning models are increasingly becoming restricted by size and subsequent operations per forward pass, demanding increasing compute requirements. Quantization has emerged as a convenient approach to addressing this, in which weights and activations are mapped from their conventionally used floating-point 32-bit numeric representations to lower precision integers. This process introduces significant reductions in inference time and simplifies the hardware requirements. It is a well-studied result that the performance of such reduced precision models is congruent with their floating-point counterparts. However, there is a lack of literature that addresses the performance of quantized models in a perturbed input space, as is common when stress testing regular full-precision models, particularly for real-world deployments. We focus on addressing this gap in the context of 8-bit quantized convolutional neural networks (CNNs). We study three state-of-the-art CNNs: ResNet-18, VGG-16, and SqueezeNet1_1, and subject their floating point and fixed point forms to various noise regimes with varying intensities. We characterize performance in terms of traditional metrics, including top-1 and top-5 accuracy, as well as the F1 score. We also introduce a new metric, the Kullback-Liebler divergence of the two output distributions for a given floating-point/fixed-point model pair, as a means to examine how the model's output distribution has changed as a result of quantization, which, we contend, can be interpreted as a proxy for model similarity in decision making. We find that across all three models and under each perturbation scheme, the relative error between the quantized and full-precision model was consistently low. We also find that Kullback-Liebler divergence was on the same order of magnitude as the unperturbed tests across all perturbation regimes except Brownian noise, where significant divergences were observed for VGG-16 and SqueezeNet1_1.

Latent Weight Quantization for Integerized Training of Deep Neural Networks

Existing methods for integerized training speed up deep learning by using low-bitwidth integerized weights, activations, gradients, and optimizer buffers. However, they overlook the issue of full-precision latent weights, which consume excessive memory to accumulate gradient-based updates for optimizing the integerized weights. In this paper, we propose the first latent weight quantization schema for general integerized training, which minimizes quantization perturbation to training process via residual quantization with optimized dual quantizer. We leverage residual quantization to eliminate the correlation between latent weight and integerized weight for suppressing quantization noise. We further propose dual quantizer with optimal nonuniform codebook to avoid frozen weight and ensure statistically unbiased training trajectory as full-precision latent weight. The codebook is optimized to minimize the disturbance on weight update under importance guidance and achieved with a three-segment polyline approximation for hardware-friendly implementation. Extensive experiments show that the proposed schema allows integerized training with lowest 4-bit latent weight for various architectures including ResNets, MobileNetV2, and Transformers, and yields negligible performance loss in image classification and text generation. Furthermore, we successfully fine-tune Large Language Models with up to 13 billion parameters on one single GPU using the proposed schema.

Toward Efficient Convolutional Neural Networks With Structured Ternary Patterns

High-efficiency deep learning (DL) models are necessary not only to facilitate their use in devices with limited resources but also to improve resources required for training. Convolutional neural networks (ConvNets) typically exert severe demands on local device resources and this conventionally limits their adoption within mobile and embedded platforms. This brief presents work toward utilizing static convolutional filters generated from the space of local binary patterns (LBPs) and Haar features to design efficient ConvNet architectures. These are referred to as Structured Ternary Patterns (STePs) and can be generated during network initialization in a systematic way instead of having learnable weight parameters thus reducing the total weight updates. The ternary values require significantly less storage and with the appropriate low-level implementation, can also lead to inference improvements. The proposed approach is validated using four image classification datasets, demonstrating that common network backbones can be made more efficient and provide competitive results. It is also demonstrated that it is possible to generate completely custom STeP-based networks that provide good trade-offs for on-device applications such as unmanned aerial vehicle (UAV)-based aerial vehicle detection. The experimental results show that the proposed method maintains high detection accuracy while reducing the trainable parameters by 40%-80%. This work motivates further research toward good priors for nonlearnable weights that can make DL architectures more efficient without having to alter the network during or after training.

Quantization via Distillation and Contrastive Learning

Quantization is a critical technique employed across various research fields for compressing deep neural networks (DNNs) to facilitate deployment within resource-limited environments. This process necessitates a delicate balance between model size and performance. In this work, we explore knowledge distillation (KD) as a promising approach for improving quantization performance by transferring knowledge from high-precision networks to low-precision counterparts. We specifically investigate feature-level information loss during distillation and emphasize the importance of feature-level network quantization perception. We propose a novel quantization method that combines feature-level distillation and contrastive learning to extract and preserve more valuable information during the quantization process. Furthermore, we utilize the hyperbolic tangent function to estimate gradients with respect to the rounding function, which smoothens the training procedure. Our extensive experimental results demonstrate that the proposed approach achieves competitive model performance with the quantized network compared to its full-precision counterpart, thus validating its efficacy and potential for real-world applications.

PSAQ-ViT V2: Toward Accurate and General Data-Free Quantization for Vision Transformers

Data-free quantization can potentially address data privacy and security concerns in model compression and thus has been widely investigated. Recently, patch similarity aware data-free quantization for vision transformers (PSAQ-ViT) designs a relative value metric, patch similarity, to generate data from pretrained vision transformers (ViTs), achieving the first attempt at data-free quantization for ViTs. In this article, we propose PSAQ-ViT V2, a more accurate and general data-free quantization framework for ViTs, built on top of PSAQ-ViT. More specifically, following the patch similarity metric in PSAQ-ViT, we introduce an adaptive teacher-student strategy, which facilitates the constant cyclic evolution of the generated samples and the quantized model (student) in a competitive and interactive fashion under the supervision of the full-precision (FP) model (teacher), thus significantly improving the accuracy of the quantized model. Moreover, without the auxiliary category guidance, we employ the task- and model-independent prior information, making the general-purpose scheme compatible with a broad range of vision tasks and models. Extensive experiments are conducted on various models on image classification, object detection, and semantic segmentation tasks, and PSAQ-ViT V2, with the naive quantization strategy and without access to real-world data, consistently achieves competitive results, showing potential as a powerful baseline on data-free quantization for ViTs. For instance, with Swin-S as the (backbone) model, 8-bit quantization reaches 82.13 top-1 accuracy on ImageNet, 50.9 box AP and 44.1 mask AP on COCO, and 47.2 mean Intersection over Union (mIoU) on ADE20K. We hope that accurate and general PSAQ-ViT V2 can serve as a potential and practice solution in real-world applications involving sensitive data. Code is released and merged at: https://github.com/zkkli/PSAQ-ViT.

Robust noise-aware algorithm for randomized neural network and its convergence properties

The concept of randomized neural networks (RNNs), such as the random vector functional link network (RVFL) and extreme learning machine (ELM), is a widely accepted and efficient network method for constructing single-hidden layer feedforward networks (SLFNs). Due to its exceptional approximation capabilities, RNN is being extensively used in various fields. While the RNN concept has shown great promise, its performance can be unpredictable in imperfect conditions, such as weight noises and outliers. Thus, there is a need to develop more reliable and robust RNN algorithms. To address this issue, this paper proposes a new objective function that addresses the combined effect of weight noise and training data outliers for RVFL networks. Based on the half-quadratic optimization method, we then propose a novel algorithm, named noise-aware RNN (NARNN), to optimize the proposed objective function. The convergence of the NARNN is also theoretically validated. We also discuss the way to use the NARNN for ensemble deep RVFL (edRVFL) networks. Finally, we present an extension of the NARNN to concurrently address weight noise, stuck-at-fault, and outliers. The experimental results demonstrate that the proposed algorithm outperforms a number of state-of-the-art robust RNN algorithms.

FAT: Frequency-Aware Transformation for Bridging Full-Precision and Low-Precision Deep Representations

Learning low-bitwidth convolutional neural networks (CNNs) is challenging because performance may drop significantly after quantization. Prior arts often quantize the network weights by carefully tuning hyperparameters such as nonuniform stepsize and layerwise bitwidths, which are complicated since the full- and low-precision representations have large discrepancies. This work presents a novel quantization pipeline, named frequency-aware transformation (FAT), that features important benefits: 1) instead of designing complicated quantizers, FAT learns to transform network weights in the frequency domain to remove redundant information before quantization, making them amenable to training in low bitwidth with simple quantizers; 2) FAT readily embeds CNNs in low bitwidths using standard quantizers without tedious hyperparameter tuning and theoretical analyses show that FAT minimizes the quantization errors in both uniform and nonuniform quantizations; and 3) FAT can be easily plugged into various CNN architectures. Using FAT with a simple uniform/logarithmic quantizer can achieve the state-of-the-art performance in different bitwidths on various model architectures. Consequently, FAT serves to provide a novel frequency-based perspective for model quantization.

Visual Dissemination of Intangible Cultural Heritage Information Based on 3D Scanning and Virtual Reality Technology

In order to meet the needs of modern people for the acquisition of intangible cultural heritage information, the authors propose a research method that combines 3D scanning and virtual reality technology. Taking the production process of Xiuyu as an example, using Unity3D virtual reality technology combined with a digital platform, 3D modeling of Xiuyu is carried out, so that people can view the intangible cultural heritage information intuitively. The experimental results show that after using this method, more than 60% of more than 1000 people surveyed in the questionnaire want to experience intangible cultural heritage. In a survey of visualization platforms conducted at the same time, 90% of users are willing to combine jade carving technology with 3D scanning virtual reality technology. Conclusion. 3D scanning and virtual reality technology can further promote the process of inheritance and dissemination of intangible cultural heritage, accelerate the cultivation of intangible cultural heritage talents through the visualization platform, and promote the sustainable development of intangible cultural heritage, in order to better pass down the life memory and cultural genes of our ancient nation.