torch.nn.ELU

CLASS torch.nn.ELU(alpha=1.0, inplace=False)

paper: Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs).

参数

• alpha ([float]) – $\alpha$ 默认为1.0
• inplace ([bool] ) – 内部做, 默认为 False

ELU 定义

$$\text{ELU}(x)=\{\begin{array}{ll}x,& \text{ if }x>0\\ \alpha \ast (\exp (x)-1),& \text{ if }x\le 0\end{array}$$

图

代码

import torch
import torch.nn as nn
m = nn.ELU()
input = torch.randn(2)
output = m(input)
print("input: ", input)  # input:  tensor([-2.2888,  1.2803])
print("output: ", output) # output:  tensor([-0.8986,  1.2803])

【参考】

ELU — PyTorch 1.13 documentation