今天在无意间找一个pytorch代码和注释的Github项目。
先上项目：
https://github.com/labmlai/annotated_deep_learning_paper_implementations
这个项目还有个网站，地址：https://nn.labml.ai/
这个项目将论文和pytorch代码结合起来，大大方便了大家的学习。
论文实现
Transformers
Multi-headed attention
Transformer building blocks
Transformer XL
Relative multi-headed attention
Rotary Positional Embeddings (RoPE)
Attention with Linear Biases (ALiBi)
RETRO
Compressive Transformer
GPT Architecture
GLU Variants
kNN-LM: Generalization through Memorization
Feedback Transformer
Switch Transformer
Fast Weights Transformer
FNet
Attention Free Transformer
Masked Language Model
MLP-Mixer: An all-MLP Architecture for Vision
Pay Attention to MLPs (gMLP)
Vision Transformer (ViT)
Primer EZ
Hourglass
Eleuther GPT-NeoX
Generate on a 48GB GPU
Finetune on two 48GB GPUs
LLM.int8()
Diffusion models
Denoising Diffusion Probabilistic Models (DDPM)
Denoising Diffusion Implicit Models (DDIM)
Latent Diffusion Models
Stable Diffusion
Generative Adversarial Networks
Original GAN
GAN with deep convolutional network
Cycle GAN
Wasserstein GAN
Wasserstein GAN with Gradient Penalty
StyleGAN 2
Recurrent Highway Networks
LSTM
HyperNetworks - HyperLSTM
ResNet
ConvMixer
Capsule Networks
U-Net
Sketch RNN
Graph Neural Networks
Graph Attention Networks (GAT)
Graph Attention Networks v2 (GATv2)
Reinforcement Learning
Proximal Policy Optimization with Generalized Advantage Estimation
Deep Q Networks with with Dueling Network, Prioritized Replay and Double Q Network.
Counterfactual Regret Minimization (CFR)
Solving games with incomplete information such as poker with CFR.
Kuhn Poker
Optimizers
Adam
AMSGrad
Adam Optimizer with warmup
Noam Optimizer
Rectified Adam Optimizer
AdaBelief Optimizer
Normalization Layers
Batch Normalization
Layer Normalization
Instance Normalization
Group Normalization
Weight Standardization
Batch-Channel Normalization
DeepNorm
Distillation
Adaptive Computation
PonderNet
Uncertainty
Evidential Deep Learning to Quantify Classification Uncertainty
Activations
Fuzzy Tiling Activations
Language Model Sampling Techniques
Greedy Sampling
Temperature Sampling
Top-k Sampling
Nucleus Sampling
Scalable Training/Inference
Zero3 memory optimizations
查看实例
我们一起看一下ResNet的例子，地址：https://nn.labml.ai/resnet/index.html
这是Block。
这是Block里面的内容。
这样的方式理解pytorch代码是不是简单了许多。
