5. LSTM的C++实现

news2025/1/18 4:36:16

[C++ 基于Eigen库实现CRN前向推理]

第三部分：TransposedConv2d实现 （含dilation）

  • 前言：(Eigen库使用记录)
  • 第一部分：WavFile.class (实现读取wav/pcm,实现STFT)
  • 第二部分：Conv2d实现
  • 第三部分：TransposedConv2d实现 (mimo,padding,stride,dilation,kernel,outpadding)
  • 第四部分：NonLinearity （Sigmoid，Tanh，ReLU，ELU，Softplus）
  • 第五部分：LSTM
  • GITHUB仓库

1. LSTM介绍

1.1 pytorch LSTM介绍

Lstm是RNN网络中最有趣的结构之一，不仅仅使得模型可以从长序列中学习，还创建了长短期记忆模块，模块中所记忆的数值在需要时可以得到更改。

  • 遗忘门
    遗忘单元可以将输入信息和隐藏信息进行信息整合，并进行信息更替，更替步骤如右图公式，其中与乘上权重矩阵后，加上偏置项后，经过激活函数，此时输出值为位于[0,1]之间,并将上一个时间步的与激活函数输出值相乘，更新为
    在这里插入图片描述

  • 输入门
    当有输入进入时，输入门会结合输入信息与隐藏信息进行整合，并对信息进行更替
    过程与 过程类似，中间公式使用了tanh函数，可以将输出缩放到[-1,1]之间，再更新
    在这里插入图片描述

  • 输出门
    输出门也会对输出过程进行控制，与输入门不同的是，输出门使用tannh激活函数
    在这里插入图片描述

1.2 LSTM递推公式

pytorch的lstm递推公式如下图所示。
在pytorch中，4个权重矩阵Wii,Wif,Wig,Wio被合并为一个权重矩阵Wih,Whh也类似，方便一步计算。
在这里插入图片描述

1.3 python实现手动lstm

可以根据公式简单的写出手动实现的版本
这是一个两层的lstm，w和b都写死了，就是固定两层的参数。hidden为1024.

def test_lstm(input, wih0, bih0, whh0, bhh0, wih1, bih1, whh1, bhh1):
    # 手动模拟
    B, T, F = input.shape
    hidden_size = 1024
    inp_pointer = input
    for layer in range(2):
        h_t, c_t = (torch.zeros(B, hidden_size).cuda(), torch.zeros(B, hidden_size).cuda())
        output = torch.zeros(B, T, hidden_size).cuda()
        batch, time, freq = output.shape
        if layer == 0:
            cur_w_ih = wih0
            cur_w_hh = whh0
            cur_b_ih = bih0
            cur_b_hh = bhh0
        else:
            cur_w_ih = wih1
            cur_w_hh = whh1
            cur_b_ih = bih1
            cur_b_hh = bhh1
        for t in range(time):
            x_t = inp_pointer[:, t, :]
            gates = x_t @ cur_w_ih.T + cur_b_ih + h_t @ cur_w_hh.T + cur_b_hh
            i_t, f_t, g_t, o_t = (
                torch.sigmoid(gates[:, :hidden_size]),  # input
                torch.sigmoid(gates[:, hidden_size:hidden_size * 2]),  # forget
                torch.tanh(gates[:, hidden_size * 2:hidden_size * 3]),
                torch.sigmoid(gates[:, hidden_size * 3:]),  # output
            )

            c_t = f_t * c_t + i_t * g_t
            h_t = o_t * torch.tanh(c_t)
            output[:, t, :] = h_t
        inp_pointer = output
    return inp_pointer

另外，还实现了一个双向LSTM的版本，用了一个小样本进行测试，同样参数都是写死了。

def test_lstm():
    input_size = 4
    hidden_size = 6
    num_layer = 2
    bidirectional = True
    direction = 2 if bidirectional else 1
    input = torch.Tensor([[[[0.896227, 0.713551],
                            [0.605188, 0.0700275],
                            [0.827175, 0.186436]],
                           [[0.872269, 0.032015],
                            [0.259925, 0.517878],
                            [0.224867, 0.943635]]],
                          [[[0.290171, 0.0767354],
                            [0.251816, 0.31538],
                            [0.828251, 0.730255]],
                           [[0.24641, 0.757985],
                            [0.354927, 0.694123],
                            [0.990138, 0.946459]]]]).float().transpose(1, 2).reshape(2, 3, 4)
    B, T, F = input.shape

    lstm = nn.LSTM(input_size=input_size, hidden_size=hidden_size, num_layers=num_layer, batch_first=True,
                   bidirectional=bidirectional)

    state = OrderedDict()
    state['weight_ih_l0'] = torch.ones([4 * hidden_size, input_size])
    state['weight_hh_l0'] = torch.ones([4 * hidden_size, hidden_size]) * 2
    state['bias_ih_l0'] = torch.zeros(4 * hidden_size) + 0.5
    state['bias_hh_l0'] = torch.zeros(4 * hidden_size) + 1.0
    state['weight_ih_l1'] = torch.ones([4 * hidden_size, hidden_size * direction]) * 2
    state['weight_hh_l1'] = torch.ones([4 * hidden_size, hidden_size]) * 3
    state['bias_ih_l1'] = torch.zeros(4 * hidden_size) + 0.5
    state['bias_hh_l1'] = torch.zeros(4 * hidden_size) + 1.0
    state['weight_ih_l0_reverse'] = torch.ones([4 * hidden_size, input_size])
    state['weight_hh_l0_reverse'] = torch.ones([4 * hidden_size, hidden_size]) * 2
    state['bias_ih_l0_reverse'] = torch.zeros(4 * hidden_size) + 0.5
    state['bias_hh_l0_reverse'] = torch.zeros(4 * hidden_size) + 1.0
    state['weight_ih_l1_reverse'] = torch.ones([4 * hidden_size, hidden_size * direction]) * 2
    state['weight_hh_l1_reverse'] = torch.ones([4 * hidden_size, hidden_size]) * 3
    state['bias_ih_l1_reverse'] = torch.zeros(4 * hidden_size) + 0.5
    state['bias_hh_l1_reverse'] = torch.zeros(4 * hidden_size) + 1.0
    lstm.load_state_dict(state, strict=False)

    # 手动模拟
    inp_pointer = input
    for layer in range(num_layer):
        h_t, c_t = (torch.zeros(B, hidden_size), torch.zeros(B, hidden_size))
        h_t_reverse, c_t_reverse = (torch.zeros(B, hidden_size), torch.zeros(B, hidden_size))
        output = torch.zeros(B, T, hidden_size)
        output_reverse = torch.zeros(B, T, hidden_size)
        batch, time, freq = output.shape
        cur_w_ih = state['weight_ih_l{}'.format(layer)]
        cur_w_ih_reverse = state['weight_ih_l{}_reverse'.format(layer)]
        cur_w_hh = state['weight_hh_l{}'.format(layer)]
        cur_w_hh_reverse = state['weight_hh_l{}_reverse'.format(layer)]
        cur_b_ih = state['bias_ih_l{}'.format(layer)]
        cur_b_ih_reverse = state['bias_ih_l{}_reverse'.format(layer)]
        cur_b_hh = state['bias_hh_l{}'.format(layer)]
        cur_b_hh_reverse = state['bias_hh_l{}_reverse'.format(layer)]
        for t in range(time):
            x_t = inp_pointer[:, t, :]
            r_t = inp_pointer[:, time - t - 1, :]
            gates = x_t @ cur_w_ih.T + cur_b_ih + h_t @ cur_w_hh.T + cur_b_hh
            gates_r = r_t @ cur_w_ih_reverse.T + cur_b_ih_reverse + h_t_reverse @ cur_w_hh_reverse.T + cur_b_hh_reverse
            i_t, f_t, g_t, o_t = (
                torch.sigmoid(gates[:, :hidden_size]),  # input
                torch.sigmoid(gates[:, hidden_size:hidden_size * 2]),  # forget
                torch.tanh(gates[:, hidden_size * 2:hidden_size * 3]),
                torch.sigmoid(gates[:, hidden_size * 3:]),  # output
            )
            i_r, f_r, g_r, o_r = (
                torch.sigmoid(gates_r[:, :hidden_size]),  # input
                torch.sigmoid(gates_r[:, hidden_size:hidden_size * 2]),  # forget
                torch.tanh(gates_r[:, hidden_size * 2:hidden_size * 3]),
                torch.sigmoid(gates_r[:, hidden_size * 3:]),  # output
            )
            c_t = f_t * c_t + i_t * g_t
            c_t_reverse = f_r * c_t_reverse + i_r * g_r
            h_t = o_t * torch.tanh(c_t)
            h_t_reverse = o_r * torch.tanh(c_t_reverse)
            output[:, t, :] = h_t
            output_reverse[:, time - t - 1, :] = h_t_reverse

        inp_pointer = torch.cat([output, output_reverse], dim=2)
    print(inp_pointer.view(2, 3, 2, -1).transpose(1, 2))
    th_out, (h, c) = lstm(input)
    print(th_out)

2. 基于Eigen实现C++ 的LSTM推理

2.1 Layer_LSTM.h
//
// Created by 65181 on 2022/10/31.
//

#ifndef CRN_LAYER_LSTM_H
#define CRN_LAYER_LSTM_H

#include "Eigen"
#include "mat.h"
#include "Eigen/CXX11/Tensor"

class Layer_LSTM {
public:
    Layer_LSTM();

    Layer_LSTM(int64_t inp_size, int64_t hid_size, int64_t num_layer = 2, bool bidirectional = false);

    void LoadState(MATFile *pmFile, const std::string &state_preffix);

    void LoadTestState();

    Eigen::Tensor<float_t, 3> forward(Eigen::Tensor<float_t, 3> &input, std::vector<Eigen::Tensor<float_t, 2>> &h_t,
                                      std::vector<Eigen::Tensor<float_t, 2>> &c_t);

private:
    int64_t input_size;
    int64_t hidden_size;
    int64_t num_layers;
    int64_t direction;
    bool bidirectional;
    std::vector<Eigen::Tensor<float_t, 2>> weight_ih, weight_hh;
    std::vector<Eigen::Tensor<float_t, 2>> weight_ih_reverse, weight_hh_reverse;
    std::vector<Eigen::Tensor<float_t, 2>> bias_ih, bias_hh;
    std::vector<Eigen::Tensor<float_t, 2>> bias_ih_reverse, bias_hh_reverse;

    Eigen::Tensor<float_t, 2> _load_mat(MATFile *pmFile, const std::string &state_name);

    Eigen::Tensor<float_t, 3> _uni_lstm(Eigen::Tensor<float_t, 3> &input, std::vector<Eigen::Tensor<float_t, 2>> &h_t,
                                        std::vector<Eigen::Tensor<float_t, 2>> &c_t);

    Eigen::Tensor<float_t, 3> _bi_lstm(Eigen::Tensor<float_t, 3> &input, std::vector<Eigen::Tensor<float_t, 2>> &h_t,
                                       std::vector<Eigen::Tensor<float_t, 2>> &c_t);


    void print2(Eigen::Tensor<float_t, 2> input);

    void print3(Eigen::Tensor<float_t, 3> input);
};


#endif //CRN_LAYER_LSTM_H
2.2 Layer_LSTM.cpp
//
// Created by 65181 on 2022/10/31.
//

#include "iostream"
#include "../include/Layer_LSTM.h"

using namespace std;

Layer_LSTM::Layer_LSTM() {
    this->input_size = 64;
    this->hidden_size = 64;
    this->num_layers = 2;
    this->direction = 1;
}

Layer_LSTM::Layer_LSTM(int64_t inp_size, int64_t hid_size, int64_t num_layer, bool bidirectional) {
    this->input_size = inp_size;
    this->hidden_size = hid_size;
    this->num_layers = num_layer;
    this->bidirectional = bidirectional;
    this->direction = bidirectional ? 2 : 1;
}

void Layer_LSTM::LoadState(MATFile *pmFile, const std::string &state_preffix) {
    for (int layer_idx = 0; layer_idx < this->num_layers; layer_idx++) {
        std::string weight_ih_name = state_preffix + "_weight_ih_l" + std::to_string(layer_idx);
        std::string bias_ih_name = state_preffix + "_bias_ih_l" + std::to_string(layer_idx);
        std::string weight_hh_name = state_preffix + "_weight_hh_l" + std::to_string(layer_idx);
        std::string bias_hh_name = state_preffix + "_bias_hh_l" + std::to_string(layer_idx);

        this->weight_ih.push_back(_load_mat(pmFile, weight_ih_name));
        this->bias_ih.push_back(_load_mat(pmFile, bias_ih_name));
        this->weight_hh.push_back(_load_mat(pmFile, weight_hh_name));
        this->bias_hh.push_back(_load_mat(pmFile, bias_hh_name));

        if (this->bidirectional) {
            std::string w_ih_reverse = state_preffix + "_weight_ih_l" + std::to_string(layer_idx) + "_reverse";
            std::string b_ih_reverse = state_preffix + "_bias_ih_l" + std::to_string(layer_idx) + "_reverse";
            std::string w_hh_reverse = state_preffix + "_weight_hh_l" + std::to_string(layer_idx) + "_reverse";
            std::string b_hh_reverse = state_preffix + "_bias_hh_l" + std::to_string(layer_idx) + "_reverse";

            this->weight_ih_reverse.push_back(_load_mat(pmFile, w_ih_reverse));
            this->bias_ih_reverse.push_back(_load_mat(pmFile, b_ih_reverse));
            this->weight_hh_reverse.push_back(_load_mat(pmFile, w_hh_reverse));
            this->bias_hh_reverse.push_back(_load_mat(pmFile, b_hh_reverse));
        }
    }
}

Eigen::Tensor<float_t, 2> Layer_LSTM::_load_mat(MATFile *pmFile, const std::string &state_name) {
    mxArray *pa = matGetVariable(pmFile, state_name.c_str());
    auto *values = (float_t *) mxGetData(pa);
    long long dim1 = mxGetM(pa);
    long long dim2 = mxGetN(pa);
    Eigen::Tensor<float_t, 2> matrix(dim1, dim2);
    int idx = 0;
    for (int i = 0; i < dim2; i++) {
        for (int j = 0; j < dim1; j++) {
            matrix(j, i) = values[idx++];
        }
    }
    return matrix;
}


void Layer_LSTM::LoadTestState() {
    for (int layer = 0; layer < this->num_layers; layer++) {
        int64_t _ih_DIM = layer == 0 ? this->input_size : this->hidden_size * this->direction;
        Eigen::Tensor<float_t, 2> state_w_ih(this->hidden_size * 4, _ih_DIM);
        Eigen::Tensor<float_t, 2> state_w_hh(this->hidden_size * 4, this->hidden_size);
        Eigen::Tensor<float_t, 2> state_b_ih(1, this->hidden_size * 4);
        Eigen::Tensor<float_t, 2> state_b_hh(1, this->hidden_size * 4);
        state_w_ih.setConstant(2);
        state_w_hh.setConstant(2);
        state_b_ih.setConstant(1.0);
        state_b_hh.setConstant(1.0);
        this->weight_ih.push_back(state_w_ih);
        this->weight_hh.push_back(state_w_hh);
        this->bias_ih.push_back(state_b_ih);
        this->bias_hh.push_back(state_b_hh);

//        Eigen::Tensor<float_t, 2> state_w_ih_reverse(this->hidden_size * 4, _ih_DIM);
//        Eigen::Tensor<float_t, 2> state_w_hh_reverse(this->hidden_size * 4, this->hidden_size);
//        Eigen::Tensor<float_t, 2> state_b_ih_reverse(1, this->hidden_size * 4);
//        Eigen::Tensor<float_t, 2> state_b_hh_reverse(1, this->hidden_size * 4);
//        state_w_ih_reverse.setConstant(layer + 1);
//        state_w_hh_reverse.setConstant(layer + 2);
//        state_b_ih_reverse.setConstant(0.5);
//        state_b_hh_reverse.setConstant(1.0);
//        this->weight_ih_reverse.push_back(state_w_ih_reverse);
//        this->weight_hh_reverse.push_back(state_w_hh_reverse);
//        this->bias_ih_reverse.push_back(state_b_ih_reverse);
//        this->bias_hh_reverse.push_back(state_b_hh_reverse);
    }
}


Eigen::Tensor<float_t, 3> Layer_LSTM::forward(Eigen::Tensor<float_t, 3> &input,
                                              std::vector<Eigen::Tensor<float_t, 2>> &h_t,
                                              std::vector<Eigen::Tensor<float_t, 2>> &c_t) {
    Eigen::Tensor<float_t, 3> output;
    if (this->bidirectional) {
        output = this->_bi_lstm(input, h_t, c_t);
    } else {
        output = this->_uni_lstm(input, h_t, c_t);
    }
    return output;
}

Eigen::Tensor<float_t, 3> Layer_LSTM::_uni_lstm(Eigen::Tensor<float_t, 3> &input,
                                                vector<Eigen::Tensor<float_t, 2>> &h_t,
                                                vector<Eigen::Tensor<float_t, 2>> &c_t) {
    Eigen::Tensor<size_t, 3>::Dimensions dim_inp = input.dimensions();
    Eigen::Tensor<float_t, 3> out_pointer = input;
    if (h_t.empty() || c_t.empty()) {
        for (int idx_layer = 0; idx_layer < this->num_layers; idx_layer++) {
            Eigen::Tensor<float, 2> ht_zeros(dim_inp[0], this->hidden_size);
            Eigen::Tensor<float, 2> ct_zeros(dim_inp[0], this->hidden_size);
            ht_zeros.setZero();
            ct_zeros.setZero();
            h_t.push_back(ht_zeros);
            c_t.push_back(ct_zeros);
        }
    }
    for (int idx_layer = 0; idx_layer < this->num_layers; idx_layer++) {
        Eigen::Tensor<size_t, 3>::Dimensions dim_cur = out_pointer.dimensions();
        int64_t N_BATCH = dim_cur[0], N_TIME = dim_cur[1], N_FREQ = dim_cur[2], N_HIDDEN = this->hidden_size;
        Eigen::Tensor<float_t, 2> cur_w_ih = this->weight_ih[idx_layer];
        Eigen::Tensor<float_t, 2> cur_w_hh = this->weight_hh[idx_layer];
        Eigen::Tensor<float_t, 2> cur_b_ih = this->bias_ih[idx_layer].broadcast(Eigen::array<int64_t, 2>{N_BATCH, 1});
        Eigen::Tensor<float_t, 2> cur_b_hh = this->bias_hh[idx_layer].broadcast(Eigen::array<int64_t, 2>{N_BATCH, 1});
        Eigen::Tensor<float, 2> &cur_ht = h_t[idx_layer];
        Eigen::Tensor<float, 2> &cur_ct = c_t[idx_layer];
//        print2(cur_w_ih);
//        print2(cur_w_hh);
//        print2(cur_b_ih);
//        print2(cur_b_hh);
//        print2(cur_ht);
//        print2(cur_ct);

        Eigen::Tensor<float_t, 3> output(N_BATCH, N_TIME, N_HIDDEN);
        Eigen::Tensor<float_t, 2> X_t;
        Eigen::Tensor<float_t, 2> gates;
        Eigen::Tensor<float_t, 2> i_t, f_t, g_t, o_t;
        Eigen::array<Eigen::IndexPair<int>, 1> product_dims = {Eigen::IndexPair<int>(1, 1)};
        Eigen::array<int64_t, 2> gate_patch = Eigen::array<int64_t, 2>{N_BATCH, N_HIDDEN};
        for (int t = 0; t < N_TIME; t++) {
            X_t = out_pointer.chip(t, 1);
//            print2(X_t);
//            print2(X_t.contract(cur_w_ih, product_dims) + cur_b_ih);
//            print2(cur_ht.contract(cur_w_hh, product_dims) + cur_b_hh);
            gates = X_t.contract(cur_w_ih, product_dims) + cur_b_ih + cur_ht.contract(cur_w_hh, product_dims) +
                    cur_b_hh;
//            print2(gates);
            i_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 0}, gate_patch).sigmoid();
            f_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 1}, gate_patch).sigmoid();
            g_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 2}, gate_patch).tanh();
            o_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 3}, gate_patch).sigmoid();
//            print2(i_t);
//            print2(f_t);
//            print2(g_t);
//            print2(o_t);
            cur_ct = f_t * cur_ct + i_t * g_t;
            cur_ht = o_t * cur_ct.tanh();
//            print2(cur_ct);
//            print2(cur_ht);
            output.chip(t, 1) = cur_ht;
        }
        out_pointer = output;
    }
    return out_pointer;
}

Eigen::Tensor<float_t, 3> Layer_LSTM::_bi_lstm(Eigen::Tensor<float_t, 3> &input,
                                               vector<Eigen::Tensor<float_t, 2>> &h_t,
                                               vector<Eigen::Tensor<float_t, 2>> &c_t) {
    Eigen::Tensor<size_t, 3>::Dimensions dim_inp = input.dimensions();
    Eigen::Tensor<float_t, 3> out_pointer = input;
    if (h_t.empty() || c_t.empty()) {
        for (int idx_layer = 0; idx_layer < this->num_layers * this->direction; idx_layer++) {
            Eigen::Tensor<float, 2> ht_zeros(dim_inp[0], this->hidden_size);
            Eigen::Tensor<float, 2> ct_zeros(dim_inp[0], this->hidden_size);
            ht_zeros.setZero();
            ct_zeros.setZero();
            h_t.push_back(ht_zeros);
            c_t.push_back(ct_zeros);
        }
    }
    for (int idx_layer = 0; idx_layer < this->num_layers; idx_layer++) {
        Eigen::Tensor<size_t, 3>::Dimensions dim_cur = out_pointer.dimensions();
        int64_t N_BATCH = dim_cur[0], N_TIME = dim_cur[1], N_FREQ = dim_cur[2], N_HIDDEN = this->hidden_size;
        Eigen::Tensor<float_t, 2> cur_w_ih = this->weight_ih[idx_layer];
        Eigen::Tensor<float_t, 2> cur_w_ih_reverse = this->weight_ih_reverse[idx_layer];
        Eigen::Tensor<float_t, 2> cur_w_hh = this->weight_hh[idx_layer];
        Eigen::Tensor<float_t, 2> cur_w_hh_reverse = this->weight_hh_reverse[idx_layer];
        Eigen::Tensor<float_t, 2> cur_b_ih = this->bias_ih[idx_layer].broadcast(Eigen::array<int64_t, 2>{N_BATCH, 1});
        Eigen::Tensor<float_t, 2> cur_b_ih_reverse = this->bias_ih_reverse[idx_layer].broadcast(
                Eigen::array<int64_t, 2>{N_BATCH, 1});
        Eigen::Tensor<float_t, 2> cur_b_hh = this->bias_hh[idx_layer].broadcast(Eigen::array<int64_t, 2>{N_BATCH, 1});
        Eigen::Tensor<float_t, 2> cur_b_hh_reverse = this->bias_hh_reverse[idx_layer].broadcast(
                Eigen::array<int64_t, 2>{N_BATCH, 1});

        Eigen::Tensor<float, 2> &cur_ht = h_t[idx_layer * 2];
        Eigen::Tensor<float, 2> &cur_ht_reverse = h_t[idx_layer * 2 + 1];
        Eigen::Tensor<float, 2> &cur_ct = c_t[idx_layer * 2];
        Eigen::Tensor<float, 2> &cur_ct_reverse = c_t[idx_layer * 2 + 1];
        // cout << "cur_w_ih" << endl << cur_w_ih << endl;
        // cout << "cur_w_hh" << endl << cur_w_hh << endl;
        // cout << "cur_b_ih" << endl << cur_b_ih << endl;
        // cout << "cur_b_hh" << endl << cur_b_hh << endl;
        // cout << "cur_ht" << endl << cur_ht << endl;
        // cout << "cur_ct" << endl << cur_ct << endl;

        Eigen::Tensor<float_t, 3> output(N_BATCH, N_TIME, N_HIDDEN);
        Eigen::Tensor<float_t, 3> output_reverse(N_BATCH, N_TIME, N_HIDDEN);
        Eigen::Tensor<float_t, 2> X_t;
        Eigen::Tensor<float_t, 2> X_t_reverse;
        Eigen::Tensor<float_t, 2> gates;
        Eigen::Tensor<float_t, 2> gates_reverse;
        Eigen::Tensor<float_t, 2> i_t, f_t, g_t, o_t;
        Eigen::Tensor<float_t, 2> i_t_reverse, f_t_reverse, g_t_reverse, o_t_reverse;
        Eigen::Tensor<float_t, 2> cur_cat;
        Eigen::array<Eigen::IndexPair<int>, 1> product_dims = {Eigen::IndexPair<int>(1, 1)};
        Eigen::array<int64_t, 2> gate_patch = Eigen::array<int64_t, 2>{N_BATCH, N_HIDDEN};
        for (int t = 0; t < N_TIME; t++) {
            X_t = out_pointer.chip(t, 1);
            X_t_reverse = out_pointer.chip(N_TIME - t - 1, 1);
            // cout << "X_t" << endl << X_t << endl;
            // cout << "X_t_reverse" << endl << X_t_reverse << endl;
            gates = X_t.contract(cur_w_ih, product_dims) + cur_b_ih + cur_ht.contract(cur_w_hh, product_dims) +
                    cur_b_hh;
            gates_reverse = X_t_reverse.contract(cur_w_ih_reverse, product_dims) + cur_b_ih_reverse +
                            cur_ht_reverse.contract(cur_w_hh_reverse, product_dims) +
                            cur_b_hh_reverse;
            i_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 0}, gate_patch).sigmoid();
            f_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 1}, gate_patch).sigmoid();
            g_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 2}, gate_patch).tanh();
            o_t = gates.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 3}, gate_patch).sigmoid();
            i_t_reverse = gates_reverse.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 0}, gate_patch).sigmoid();
            f_t_reverse = gates_reverse.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 1}, gate_patch).sigmoid();
            g_t_reverse = gates_reverse.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 2}, gate_patch).tanh();
            o_t_reverse = gates_reverse.slice(Eigen::array<int64_t, 2>{0, N_HIDDEN * 3}, gate_patch).sigmoid();


            cur_ct = f_t * cur_ct + i_t * g_t;
            cur_ht = o_t * cur_ct.tanh();
            cur_ct_reverse = f_t_reverse * cur_ct_reverse + i_t_reverse * g_t_reverse;
            cur_ht_reverse = o_t_reverse * cur_ct_reverse.tanh();
            output.chip(t, 1) = cur_ht;
            output_reverse.chip(N_TIME - t - 1, 1) = cur_ht_reverse;
        }

        out_pointer = output.concatenate(output_reverse, 2);
    }
    return out_pointer;
}

void Layer_LSTM::print2(Eigen::Tensor<float_t, 2> input) {
    const Eigen::Tensor<size_t, 2>::Dimensions &dim_inp = input.dimensions();
    std::cout << "Variable:" << std::endl;
    // 0 0
    std::cout << input(0, 0) << " " << input(0, 1) << " " << input(0, 2) << " ";
    std::cout << input(0, dim_inp[1] - 3) << " " << input(0, dim_inp[1] - 2) << " "
              << input(0, dim_inp[1] - 1);
    std::cout << std::endl;


    // 0 -1
    std::cout << input(dim_inp[0] - 1, 0) << " " << input(dim_inp[0] - 1, 1) << " "
              << input(dim_inp[0] - 1, 2) << " ";
    std::cout << input(dim_inp[0] - 1, dim_inp[1] - 3) << " " << input(dim_inp[0] - 1, dim_inp[1] - 2)
              << " "
              << input(dim_inp[0] - 1, dim_inp[1] - 1);
    std::cout << std::endl;
}

void Layer_LSTM::print3(Eigen::Tensor<float_t, 3> input) {
    const Eigen::Tensor<size_t, 3>::Dimensions &dim_inp = input.dimensions();
    std::cout << "Variable:" << std::endl;
    // 0 0
    std::cout << input(0, 0, 0) << " " << input(0, 0, 1) << " " << input(0, 0, 2) << " ";
    std::cout << input(0, 0, dim_inp[2] - 3) << " " << input(0, 0, dim_inp[2] - 2) << " "
              << input(0, 0, dim_inp[2] - 1);
    std::cout << std::endl;


    // 0 -1
    std::cout << input(0, dim_inp[1] - 1, 0) << " " << input(0, dim_inp[1] - 1, 1) << " "
              << input(0, dim_inp[1] - 1, 2) << " ";
    std::cout << input(0, dim_inp[1] - 1, dim_inp[2] - 3) << " " << input(0, dim_inp[1] - 1, dim_inp[2] - 2)
              << " "
              << input(0, dim_inp[1] - 1, dim_inp[2] - 1);
    std::cout << std::endl;
}

3.参考链接

[1] 实现LSTM-pytorch版

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/48247.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

你知道不同U盘在ARM+Linux下的读写速率吗？

你知道不同U盘在ARM+Linux下的读写速率吗？

优秀的产品离不开完善的测试&#xff0c;即使一个简单的USB接口也要确保稳定性及兼容性。不同的U盘在ARMLinux板卡下的兼容性、速率怎么样呢&#xff1f;本文将为大家提供测试参考数据及详细测试步骤&#xff01; 1. 测试准备 主控选用最近发布的64位Cortex-A55核心板&#xff…
阅读更多...
设计模式-day01

设计模式-day01

1&#xff0c;设计模式概述 1.1 软件设计模式的产生背景 "设计模式"最初并不是出现在软件设计中&#xff0c;而是被用于建筑领域的设计中。 1977年美国著名建筑大师、加利福尼亚大学伯克利分校环境结构中心主任克里斯托夫亚历山大&#xff08;Christopher Alexand…
阅读更多...
深入理解SR-IOV和IO虚拟化

深入理解SR-IOV和IO虚拟化

一、背景 SR-IOV&#xff08;Single Root I/O Virtualization&#xff09;是由PCI-SIG组织定义的PCIe规范的扩展规范《Single Root I/O Virtualization and Sharing Specification》&#xff0c;目的是通过提供一种标准规范&#xff0c;为VM&#xff08;虚拟机&#xff09;提供…
阅读更多...
springboot+vue职称评审管理系统

springboot+vue职称评审管理系统

开发语言&#xff1a;Java 框架&#xff1a;Springbootssm(SpringSpringMVCMyBatis) JDK版本&#xff1a;JDK1.8 前端框架&#xff1a;vue.js 服务器&#xff1a;tomcat 数据库&#xff1a;mysql 数据库工具&#xff1a;Navicat11 开发软件&#xff1a;eclipse/idea都支持 Mave…
阅读更多...
前端页面的性能测试

前端页面的性能测试

介绍 随着 Web 应用的空前发展&#xff0c;前端业务逐渐复杂&#xff0c;为了处理这些复杂业务&#xff0c;前后端分离&#xff0c;出现了专门应对这种分离架构的应用开发框架&#xff0c;比如 Angular&#xff0c;React&#xff0c;Vue 等&#xff0c;从而也导致 Web 应用的复…
阅读更多...
高效率开发Web安全扫描器之路（一）

高效率开发Web安全扫描器之路（一）

一、背景 经常看到一些SRC和CNVD上厉害的大佬提交了很多的漏洞&#xff0c;一直好奇它们怎么能挖到这么多漏洞&#xff0c;开始还以为它们不上班除了睡觉就挖漏洞&#xff0c;后来有机会认识了一些大佬&#xff0c;发现它们大部分漏洞其实是通过工具挖掘的&#xff0c;比如说下…
阅读更多...
linux服务器监控性能测试

linux服务器监控性能测试

一、服务器实时内存监控 1、top&#xff1a;能够实时监控系统的运行状态&#xff0c;并且可以按照cpu以及内存等进行排序。 输入&#xff1a;top -help就可以出现下面的例子&#xff1a; 命令例子&#xff1a;top -hv | -bcHiOSs -d secs -n max -u|U user -p pid(s) -o fiel…
阅读更多...
计算机网络——分层结构，协议接口，服务

计算机网络——分层结构，协议接口，服务

分层结构 主机进行资源共享时需满足以下条件&#xff1a; &#xff08;1&#xff09;发起通信的计算机要将数据通路进行激活 &#xff08;2&#xff09;告诉网络如何识别主机 &#xff08;3&#xff09;发起通信的主机要查明目的主机是否开机等 &#xff08;4&#xff09;发起…
阅读更多...
Java 中你绝对没用过的一个关键字？

Java 中你绝对没用过的一个关键字？

这节课给大家介绍一个 Java 中的一个关键字 Record&#xff0c;那 Record 关键字跟不可变类有什么关系呢&#xff1f;看完今天的文章你就知道了。友情提示 Record 关键字在 Java14 过后才支持的&#xff0c;所以是不是被我说中了&#xff0c;还在使用 Java 8 的你一定没用过&am…
阅读更多...
vpp hqos分析

vpp hqos分析

vpp支持两套qos实现&#xff0c;一套是基于policer实现的qos&#xff0c;另外一套是基于dpdk的qos套件实现的hqos。 &#xff08;免费订阅,永久学习&#xff09;学习地址: Dpdk/网络协议栈/vpp/OvS/DDos/NFV/虚拟化/高性能专家-学习视频教程-腾讯课堂 更多DPDK相关学习资料有…
阅读更多...
DPU网络开发SDK——DPDK(一)

DPU网络开发SDK——DPDK(一)

随着软件定义网络SDN的不断发展&#xff0c;网络数据转发面的需求越来越多样化&#xff0c;这体现在更快的数据包处理速率&#xff0c;更高的网络吞吐带宽&#xff0c;更灵活的自定义网络协议。传统的硬件设备无法满足网络协议的自定义&#xff0c;而基于Linux内核网络协议栈的…
阅读更多...
故障电弧探测器的必要性及组网方案 安科瑞 时丽花

故障电弧探测器的必要性及组网方案 安科瑞 时丽花

摘要】&#xff1a;电气设备是建筑中不可缺少的一部分&#xff0c;具有较为重要的作用和意义&#xff0c;在应用过程中不仅能够提升建筑本身实用性能&#xff0c;而且可为消费者提供更加优良的生活环境。但设备一旦在运行过程中出现故障&#xff0c;不仅会影响居民正常生活&…
阅读更多...
艾美捷细胞失巢凋亡检测试剂盒测定原理化验方案

艾美捷细胞失巢凋亡检测试剂盒测定原理化验方案

对细胞外基质&#xff08;ECM&#xff09;的粘附对于许多粘附细胞的生存和繁殖至关重要细胞。细胞与ECM的粘附丧失或不当粘附导致的细胞凋亡定义为“anoikis”。Anoikis&#xff0c;来自希腊语无家可归的意思&#xff0c;与生理学有关组织更新和细胞稳态的过程。 癌症发展和生长…
阅读更多...
做测试8年，33岁前只想追求大厂高薪，今年只求稳定收入

做测试8年，33岁前只想追求大厂高薪，今年只求稳定收入

疫情3年&#xff0c;每一个行业的危机&#xff0c;每一个企业的倒下&#xff0c;背后都是无数人的降薪、降职和失业。这也暴露了人生的残酷真相&#xff1a;人活一辈子&#xff0c;总有“丰年”和“荒年” 优秀的测试既过得了丰年&#xff0c;也受得住荒年 一个测试宝妈&…
阅读更多...
冒烟测试的7个好处，你是否经常用到它？

冒烟测试的7个好处，你是否经常用到它？

以下为作者观点&#xff1a; 冒烟测试(smoke testing)是在开发的早期阶段评估基本的软件组件&#xff0c;以检查它们是否 “着火”&#xff08;有问题&#xff09;&#xff0c;本文旨在介绍冒烟测试及其在程序开发过程中的作用。 什么是冒烟测试&#xff1f; 冒烟测试是在开…
阅读更多...
Linux文件权限

Linux文件权限

Linux文件权限 文件权限介绍 Linux文件权限有三种&#xff1a; 权限对应字符可读(read)r可写(write)w可执行(execute)x 文件权限身份也有三种&#xff1a; 权限身份对应字符文件所有者(user)u文件所有者所在组(group)g其他(other)&#xff0c;即除了文件所有者和其所在组的…
阅读更多...
简明误差卡尔曼滤波器（ESKF）及其推导过程

简明误差卡尔曼滤波器（ESKF）及其推导过程

文章目录1. 简明ESKF简介ESKF基本过程及优点ESKF参数含义连续时间上的 ESKF状态方程误差状态方程推导误差状态的旋转项误差状态的速度项完整误差变量的运动学方程离散时间上的ESKF运动学方程ESKF的运动过程ESKF的更新过程ESKF的误差状态后续处理小结1. 简明ESKF 简介 本文主要…
阅读更多...
电商新趋势：Starday拿下黑色星期五的制胜法宝是物流速度

电商新趋势：Starday拿下黑色星期五的制胜法宝是物流速度

国内电商“双十一”购物狂欢季活动已经闭幕&#xff0c;“双十二”又将袭来&#xff0c;但更多人却将眼光放在蓬勃发展的跨境电商行业中。当下跨境电商卖家们正在各大跨境电商服务平台的带领下全力备战&#xff0c;在“黑色星期五”期间推出各类大促活动&#xff0c;奋力冲刺20…
阅读更多...
灌区量测水监测系统解决方案 灌区量测水系统解决方案 农业水价综合改革解决方案

灌区量测水监测系统解决方案 灌区量测水系统解决方案 农业水价综合改革解决方案

平升电子灌区量测水监测系统解决方案/灌区量测水系统解决方案/农业水价综合改革解决方案&#xff0c;对灌区的渠道水位、流量、水雨情、土壤墒情、气象等信息进行监测&#xff0c;同时对泵站、闸门进行远程控制&#xff0c;对重点区域进行视频监控&#xff0c;实现了信息的采集…
阅读更多...
网络面试知识

网络面试知识

客户端与服务端的网络通信 bind函数 把一个本地协议地址和套接口绑定&#xff0c;比如把本机的2222端口绑定到套接口。注意&#xff1a;为什么在上图中客户端不需要调用bind函数&#xff1f;这是因为如果没有调用bind函数绑定一个端口的话&#xff0c;当调用connect函数时&…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023