Pinned Loading

1. LanguageModelFromScratch LanguageModelFromScratch Public

   An autoregressive transformer language model from first principles, built without relying on ML libraries. Inspired by Vaswani et al.'s seminal 2017 work, "Attention Is All You Need".

   Python

2. SoftMoE SoftMoE Public

   Soft Mixture of Experts Vision Transformer, addressing MoE limitations as highlighted by Puigcerver et al., 2023.

   Python 15 1

3. Monolith Monolith Public template

   A starting point for rapid web development - a full-stack monolith using Next.js, Prisma, and SQLite.

   TypeScript 5

4. VisionTransformer VisionTransformer Public

   Implementation of the Vision Transformer (Dosovitskiy et al., 2021)

   Python

5. NeuronWithBackprop NeuronWithBackprop Public

   A simple implementation of an artificial neuron with backpropagation in Python

   Jupyter Notebook

6. GPTReplica GPTReplica Public

   GPT (Generative Pretrained Transformer) model with a decoder-only architecture using PyTorch

   Jupyter Notebook