Libraries for applying sparsification recipes to neural networks with a few lines of code, enabling faster and smaller models

A curated list for Efficient Large Language Models

[NeurIPS 2023] LLM-Pruner: On the Structural Pruning of Large Language Models. Support Llama-3/3.1, Llama-2, LLaMA, BLOOM, Vicuna, Baichuan, TinyLlama, etc.

Automated Identification of Redundant Layer Blocks for Pruning in Large Language Models

A research library for pytorch-based neural network pruning, compression, and more.

[AAAI 2024] Fluctuation-based Adaptive Structured Pruning for Large Language Models

Model optimizer used in Adlik.

Caffe/Neon prototxt training file for our Neurocomputing2017 work: Fuzzy Quantitative Deep Compression Network

This project provides tools to load and prune large language models using a structured pruning method.

KEN: Unleash the power of large language models with the easiest and universal non-parametric pruning algorithm

Official code for "EC-SNN: Splitting Deep Spiking Neural Networks on Edge Devices" (IJCAI2024)

Hierarchical Ensemble Pruning

[EMNLP 2024] Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning

This repository has the porpouse of give a solution to the travelling sales man problem

[JCST 2023] "Inductive Lottery Ticket Learning for Graph Neural Networks" by Yongduo Sui, Xiang Wang, Tianlong Chen, Meng Wang, Xiangnan He, Tat-Seng Chua.

This repository contains scripts to prune Wav2vec2 using a neuroevolution-based method. More details about this method can be found in the paper Compressing Wav2vec2 for Embedded Applications.

[PRL 2024] This is the code repo for our label-free pruning and retraining technique for autoregressive Text-VQA Transformers (TAP, TAP†).

This project implements Genetic Programming (GP) to evolve and optimize mathematical expressions that best fit given data. It leverages tree-based evolutionary algorithms to generate, evaluate, and refine expressions using selection, crossover, and mutation.

Play the Othello board game against the algorithm and try to win!

attention pruning