腾讯云SDK并发调用优化方案

news2024/11/24 14:31:39

目录

一、概述

二、 网关的使用

2.1 核心代码

三、腾讯云SDK依赖包的改造


一、概述

     此网关主要用于协调腾讯云SDK调用的QPS消耗,使得多个腾讯云用户资源能得到最大限度的利用。避免直接使用腾讯云SDK 时,在较大并发情况下导致接口调用异常。网关的工作流程如下图所示:

     如上图所示,各个客户端在发起腾讯云SDK调用时,请求统一先发到网关,网关会根据现有的腾讯云账户资源使用情况,通过负载均衡算法,选择一个合适的腾讯云账户来执行请求,将请求转发到腾讯云服务,从而保证了腾讯云用户资源的最大利用。在这个过程中,如果暂时未找到可用的腾讯云用户,则会阻塞线程,直到有可用的账户时再将线程唤醒放行,避免了在较大并发量时直接调用SDK,而导致接口报错的情况发生。

二、 网关的使用

2.1 核心代码

   RequestLimitFilter.java

package com.tencentcloudapi.gateway.filter;


import com.tencentcloudapi.common.Sign;
import com.tencentcloudapi.common.exception.TencentCloudSDKException;
import com.tencentcloudapi.gateway.api.dto.UserInfo;
import com.tencentcloudapi.gateway.api.service.UserManageService;
import io.micrometer.common.util.StringUtils;
import jakarta.annotation.Resource;
import lombok.extern.slf4j.Slf4j;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.core.Ordered;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.stereotype.Component;
import org.springframework.util.CollectionUtils;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;

import javax.xml.bind.DatatypeConverter;
import java.nio.charset.StandardCharsets;
import java.sql.Date;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.Semaphore;

/**
 * @Description 限流过滤器
 * @Author miller.Lai
 * @Date 2023-11-06 10:23
 */
@Component
@Slf4j
public class RequestLimitFilter implements GlobalFilter, Ordered {


    @Resource
    private UserManageService userManageService;


    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        ServerHttpRequest request = exchange.getRequest();
        // 打印请求路径
        String requestUri = request.getPath().pathWithinApplication().value();
        log.info("接受到请求,请求路径:{}", requestUri);

        List<String> authorizationList = request.getHeaders().get("authorization");
        List<String> actionList = request.getHeaders().get("X-TC-Action");
        // 如果请求头中存在 authorization 信息,则要进行替换,以适应现有的接口TPS限制策略
        if (!CollectionUtils.isEmpty(authorizationList) && StringUtils.isNotEmpty(authorizationList.get(0))&&
                !CollectionUtils.isEmpty(actionList) && StringUtils.isNotEmpty(actionList.get(0))) {
            UserInfo userInfo = new UserInfo();
            // 接口名称
            String action = null;
            try {
                action = actionList.get(0);
                log.info("当前调用API名称:{}", action);
                // 获取可用的腾讯秘钥,这是一个阻塞方法
                userInfo = userManageService.getAvailableUserInfo(action);
                String secretId = userInfo.getSecretInfo().getSecretId();
                String secretKey = userInfo.getSecretInfo().getSecretKey();
                // 根据可用的秘钥对请求头中的认证信息做重新生成
                String signedHeaders = "content-type;host";
                SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
                sdf.setTimeZone(TimeZone.getTimeZone("UTC"));

                // 接口请求所在秒钟
                String timestamp = String.valueOf(System.currentTimeMillis() / 1000L);
                String date = sdf.format(new Date(Long.valueOf(timestamp + "000")));
                String service = request.getHeaders().get("service") != null ?  request.getHeaders().get("service").get(0) : "";
                String credentialScope = date + "/" + service + "/tc3_request";
                String stringToSign = request.getHeaders().get("stringToSign") != null ? new String(Base64.getDecoder().decode(Objects.requireNonNull(request.getHeaders().get("stringToSign")).get(0).getBytes())) : "";
                try {
                    byte[] secretDate = Sign.hmac256(("TC3" + secretKey).getBytes(StandardCharsets.UTF_8), date);
                    byte[] secretService = Sign.hmac256(secretDate, service);
                    byte[] secretSigning = Sign.hmac256(secretService, "tc3_request");
                    String signature = DatatypeConverter.printHexBinary(Sign.hmac256(secretSigning, stringToSign)).toLowerCase();
                    String authorization = "TC3-HMAC-SHA256 Credential=" + secretId + "/" + credentialScope + ", SignedHeaders=" + signedHeaders + ", Signature=" + signature;
                    exchange.getRequest().mutate().headers(httpHeaders -> {
                        // 去除自定义的请求头
                        httpHeaders.remove("service");
                        httpHeaders.remove("stringToSign");
                        // 去除不合法的认证信息
                        httpHeaders.remove("authorization");
                        // 塞入有效的认证信息
                        httpHeaders.add("authorization", authorization);

                    });
                } catch (TencentCloudSDKException e) {
                    throw new RuntimeException(e);
                }
                log.info("线程 {} 的请求时间:{} 毫秒,secretId:{}",Thread.currentThread().getName(),System.currentTimeMillis(),secretId);
                UserInfo finalUserInfo = userInfo;
                String finalAction = action;


                // 如果上述过着正常获取信号量的许可
                String hasAcquired = userManageService.getHasAcquiredThreadLocal().get();
                userManageService.getHasAcquiredThreadLocal().remove();
                return chain.filter(exchange).then( Mono.fromRunnable(() -> {
                    // 接口逻辑执行完毕后释放信号量锁
                    if("1".equals(hasAcquired)){
                        finalUserInfo.getInterfaceInfo(finalAction).getSemaphore().release();
                        log.info("线程 {} 已释放线程锁",Thread.currentThread().getName());
                    }
                }));

            } catch (Exception e) {
                // 在发生错误时释放信号量锁
                // 将当前线程标记为已获取许可
                String hasAcquired = userManageService.getHasAcquiredThreadLocal().get();
                if("1".equals(hasAcquired)){
                    userInfo.getInterfaceInfo(action).getSemaphore().release();
                    userManageService.getHasAcquiredThreadLocal().remove();
                    log.info("线程 {} 已释放线程锁",Thread.currentThread().getName());
                }

                throw new RuntimeException(e);
            }
        }else{
            return chain.filter(exchange);
        }
    }

    @Override
    public int getOrder() {
        return 1;
    }
}
UserManageServiceImpl.java
package com.tencentcloudapi.gateway.api.service;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.tencentcloudapi.gateway.api.dto.UserInfo;
import com.tencentcloudapi.gateway.api.dto.InterfaceInfo;
import io.micrometer.common.util.StringUtils;
import jakarta.annotation.PostConstruct;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.TimeUnit;

/**
 * @author Miller.Lai
 * @description:  腾讯云用户管理实现类
 * @date 2024-01-25 17:13:10
 */
@Service
public class UserManageServiceImpl implements UserManageService {

    private  List<UserInfo>  userInfos;


    @Value("${tencentcloud.api.authorization.file:}")
    private String authorizationJsonUrl;


    private ThreadLocal<String> hasAcquiredThreadLocal ;


    public ThreadLocal<String> getHasAcquiredThreadLocal() {
        return hasAcquiredThreadLocal;
    }


    @PostConstruct
    public void init(){
        try {
            InputStream inputStreams = null;
            // 如果有指定authorization文件路径,则按指定的路径找配置文件
            if (StringUtils.isNotBlank(authorizationJsonUrl)) {
                inputStreams = new FileInputStream(authorizationJsonUrl);
            } else {
                // 从resource目录下加载JSON文件
                Resource resource = new ClassPathResource("authorization.json");
                inputStreams = resource.getInputStream();
            }
            // 使用Jackson的ObjectMapper将JSON数组内容映射为List对象
            ObjectMapper objectMapper = new ObjectMapper();
            userInfos = objectMapper.readValue(inputStreams, new TypeReference<>() {});
            hasAcquiredThreadLocal =new ThreadLocal<>();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    /**
     * 随机获取一个可用的用户
     * @param action
     * @return
     */
    @Override
    public UserInfo getAvailableUserInfo(String action) {

        // 可用的用户列表
        List<UserInfo> availableUserInfos = new ArrayList<>();

        // 计算总权重,即总并发量
        int totalWeight = 0;

        // 过滤可用的腾讯用户,过滤条件: action匹配
        for (int i = 0; i < userInfos.size(); i++) {
            UserInfo userInfo =userInfos.get(i);
            // 当前用户是否可用
            boolean available = false;
            for (int j = 0; j < userInfo.getInterfaces().size(); j++) {
                InterfaceInfo interfaceInfo  = userInfo.getInterfaces().get(j);
                if(interfaceInfo.getAction().equals(action)){
                    available = true;
                    totalWeight += interfaceInfo.getMaxTPS();
                    break;
                }
            }
            // 如果当前用户可用,则加入列表
            if(available){
                availableUserInfos.add(userInfo);
            }
        }

        // 如果没找到可用用户,说明用户接口配置文件存在问题
        if(availableUserInfos.size() == 0 ){
           throw new RuntimeException("未找到可用的腾讯用户,请检查接口配置文件");
        }

        // 根据总权重生成权重随机数,0 ~ totalWeight-1
        int randomWeight = new Random().nextInt(totalWeight);

        // 根据权重值选择对应的用户
        int currentWeight = 0;
        for (int i = 0; i < availableUserInfos.size(); i++) {
            UserInfo userInfo = availableUserInfos.get(i);
            for (int j = 0; j < userInfo.getInterfaces().size(); j++) {
                InterfaceInfo interfaceInfo = userInfo.getInterfaces().get(j);
                // 找到对应的接口
                if (interfaceInfo.getAction().equals(action)) {
                    currentWeight += interfaceInfo.getMaxTPS();
                    // 如果当前接口的当前权重 > 随机权重, 则使用当前用户
                    if (currentWeight > randomWeight) {
                        // 给线程加锁,这是一个阻塞方法,如果当前用户的并发数达到上限,则当前线程会被阻塞
                        try {
                            interfaceInfo.getSemaphore().tryAcquire(30, TimeUnit.SECONDS);
                            // 将当前线程标记为已获取许可
                            hasAcquiredThreadLocal.set("1");
                        } catch (InterruptedException e) {
                            throw new RuntimeException(e);
                        } finally {
                            return userInfo;
                        }

                    }
                    break;
                }
            }
        }

        return null;

    }
}

authorization.json

[
  {
    "secretInfo": {
      "secretId": "AKIDpiYK2xxxxxxxxxxxxxxxxxxbUyOk2W",
      "secretKey": "dFlSKDBDXXXXXXXXXXXXXXXXXXYdyBE5"
    },
    "interfaces": [
      {
        "action": "RecognizeTableAccurateOCR",
        "maxTPS": 2
      },
      {
        "action": "VehicleLicenseOCR",
        "maxTPS": 10
      },
      {
        "action": "DriverLicenseOCR",
        "maxTPS": 10
      },
      {
        "action": "MLIDPassportOCR",
        "maxTPS": 5
      },
      {
        "action": "HmtResidentPermitOCR",
        "maxTPS": 20
      },
      {
        "action": "MainlandPermitOCR",
        "maxTPS": 20
      }
    ]
  },
  {
    "secretInfo": {
      "secretId": "AKIDJh9fxxxxxxxxxxxxxxxxxxxxxv8IOR",
      "secretKey": "00HaowzxxxxxxxxxxxxxxxxxxxxxjMp8b"
    },
    "interfaces": [
      {
        "action": "RecognizeTableAccurateOCR",
        "maxTPS": 2
      },
      {
        "action": "VehicleLicenseOCR",
        "maxTPS": 10
      },
      {
        "action": "DriverLicenseOCR",
        "maxTPS": 10
      },
      {
        "action": "MLIDPassportOCR",
        "maxTPS": 5
      },
      {
        "action": "HmtResidentPermitOCR",
        "maxTPS": 20
      },
      {
        "action": "MainlandPermitOCR",
        "maxTPS": 20
      }
    ]
  }
]

application-dev.yml

spring:
  application:
    name: uap-gateway
  cloud:
    gateway:
      routes:
        - id: tencentcloud-route
          uri: https://ocr.tencentcloudapi.com
          predicates:
            - Path=/tencentcloudapi/**
          filters:
            - StripPrefix=1
  main:
    web-application-type: reactive
server:
  port: 9000

# 腾讯云用户接口权限配置文件
#tencentcloud:
#  api:
#    authorization:
#      file: d://authorization.json

三、腾讯云SDK依赖包的改造

修改 com.tencentcloudapi.common.AbstractClient 类中 REMOTE_SERVER_ADDRESS 的值,指向实际的网关地址,如下所示:

/*
 * Copyright (c) 2018 THL A29 Limited, a Tencent company. All Rights Reserved.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing,
 * software distributed under the License is distributed on an
 * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
 * KIND, either express or implied.  See the License for the
 * specific language governing permissions and limitations
 * under the License.
 */

package com.tencentcloudapi.common;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import com.google.gson.JsonSyntaxException;
import com.google.gson.reflect.TypeToken;
import com.tencentcloudapi.common.exception.TencentCloudSDKException;
import com.tencentcloudapi.common.http.HttpConnection;
import com.tencentcloudapi.common.profile.ClientProfile;
import com.tencentcloudapi.common.profile.HttpProfile;
import okhttp3.*;
import okhttp3.Headers.Builder;

import javax.crypto.Mac;
import javax.net.ssl.SSLContext;
import javax.xml.bind.DatatypeConverter;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.lang.reflect.Type;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.URLEncoder;
import java.nio.charset.StandardCharsets;
import java.security.SecureRandom;
import java.sql.Date;
import java.text.SimpleDateFormat;
import java.util.*;

public abstract class AbstractClient {
  /**
   * 远程服务地址,根据实际情况修改
   * 格式为:http://ip:port/tencentcloudapi
   */
  public static String REMOTE_SERVER_ADDRESS = "http://localhost:9000/tencentcloudapi";
  public static final int HTTP_RSP_OK = 200;
  public static final String SDK_VERSION = "SDK_JAVA_3.1.699";

  private Credential credential;
  private ClientProfile profile;
  private String endpoint;
  private String service;
  private String region;
  private String path;
  private String sdkVersion;
  private String apiVersion;
  public Gson gson;
  private TCLog log;
  private HttpConnection httpConnection;
  public AbstractClient(String endpoint, String version, Credential credential, String region) {
    this(endpoint, version, credential, region, new ClientProfile());
  }

  static {
    String remoteServerAddress = System.getenv("REMOTE_SERVER_ADDRESS");
    if(remoteServerAddress != null){
      AbstractClient.REMOTE_SERVER_ADDRESS = remoteServerAddress;
    }
  }


  public AbstractClient(
      String endpoint,
      String version,
      Credential credential,
      String region,
      ClientProfile profile) {
    this.credential = credential;
    this.profile = profile;
    this.endpoint = endpoint;
    this.service = endpoint.split("\\.")[0];
    this.region = region;
    this.path = "/";
    this.sdkVersion = AbstractClient.SDK_VERSION;
    this.apiVersion = version;
    this.gson = new GsonBuilder().excludeFieldsWithoutExposeAnnotation().create();
    this.log = new TCLog(getClass().getName(), profile.isDebug());
    this.httpConnection = new HttpConnection(
            this.profile.getHttpProfile().getConnTimeout(),
            this.profile.getHttpProfile().getReadTimeout(),
            this.profile.getHttpProfile().getWriteTimeout()
    );
    this.httpConnection.addInterceptors(this.log);
    this.trySetProxy(this.httpConnection);
    warmup();
  }

  public void setRegion(String region) {
    this.region = region;
  }

  public String getRegion() {
    return this.region;
  }

  public void setClientProfile(ClientProfile profile) {
    this.profile = profile;
  }

  public ClientProfile getClientProfile() {
    return this.profile;
  }

  public void setCredential(Credential credential) {
    this.credential = credential;
  }

  public Credential getCredential() {
    return this.credential;
  }

  /**
   * Use post/json with tc3-hmac-sha256 signature to call any action. Ignore request method and
   * signature method defined in profile.
   *
   * @param action Name of action to be called.
   * @param jsonPayload Parameters of action serialized in json string format.
   * @return Raw response from API if request succeeded, otherwise an exception will be raised
   *     instead of raw response
   * @throws TencentCloudSDKException
   */
  public String call(String action, String jsonPayload) throws TencentCloudSDKException {
    HashMap<String, String> headers = this.getHeaders();
    headers.put("X-TC-Action", action);
    headers.put("Content-Type", "application/json; charset=utf-8");
    byte[] requestPayload = jsonPayload.getBytes(StandardCharsets.UTF_8);
    String authorization = this.getAuthorization(headers, requestPayload);
    headers.put("Authorization", authorization);
    String url = REMOTE_SERVER_ADDRESS + this.path;
    return this.getResponseBody(url, headers, requestPayload);
  }

  /**
   * Use post application/octet-stream with tc3-hmac-sha256 signature to call specific action.
   * Ignore request method and signature method defined in profile.
   *
   * @param action Name of action to be called.
   * @param headers Parameters of the action, will be put in http header.
   * @param body octet-stream binary body.
   * @return Raw response from API if request succeeded, otherwise an exception will be raised
   *     instead of raw response
   * @throws TencentCloudSDKException
   */
  public String callOctetStream(String action, HashMap<String, String> headers, byte[] body)
      throws TencentCloudSDKException {
    headers.putAll(this.getHeaders());
    headers.put("X-TC-Action", action);
    headers.put("Content-Type", "application/octet-stream; charset=utf-8");
    String authorization = this.getAuthorization(headers, body);
    headers.put("Authorization", authorization);
    String url = REMOTE_SERVER_ADDRESS + this.path;
    return this.getResponseBody(url, headers, body);
  }

  private HashMap<String, String> getHeaders() {
    HashMap<String, String> headers = new HashMap<String, String>();
    String timestamp = String.valueOf(System.currentTimeMillis() / 1000);
    headers.put("X-TC-Timestamp", timestamp);
    headers.put("X-TC-Version", this.apiVersion);
    headers.put("X-TC-Region", this.getRegion());
    headers.put("X-TC-RequestClient", SDK_VERSION);
    headers.put("Host", this.getEndpoint());
    String token = this.credential.getToken();
    if (token != null && !token.isEmpty()) {
      headers.put("X-TC-Token", token);
    }
    if (this.profile.isUnsignedPayload()) {
      headers.put("X-TC-Content-SHA256", "UNSIGNED-PAYLOAD");
    }
    if (null != this.profile.getLanguage()) {
      headers.put("X-TC-Language", this.profile.getLanguage().getValue());
    }
    return headers;
  }

  private String getAuthorization(HashMap<String, String> headers, byte[] body)
      throws TencentCloudSDKException {
    String endpoint = this.getEndpoint();
    // always use post tc3-hmac-sha256 signature process
    // okhttp always set charset even we don't specify it,
    // to ensure signature be correct, we have to set it here as well.
    String contentType = headers.get("Content-Type");
    byte[] requestPayload = body;
    String canonicalUri = "/";
    String canonicalQueryString = "";
    String canonicalHeaders = "content-type:" + contentType + "\nhost:" + endpoint + "\n";
    String signedHeaders = "content-type;host";

    String hashedRequestPayload = "";
    if (this.profile.isUnsignedPayload()) {
      hashedRequestPayload = Sign.sha256Hex("UNSIGNED-PAYLOAD".getBytes(StandardCharsets.UTF_8));
    } else {
      hashedRequestPayload = Sign.sha256Hex(requestPayload);
    }
    String canonicalRequest =
        HttpProfile.REQ_POST
            + "\n"
            + canonicalUri
            + "\n"
            + canonicalQueryString
            + "\n"
            + canonicalHeaders
            + "\n"
            + signedHeaders
            + "\n"
            + hashedRequestPayload;

    String timestamp = headers.get("X-TC-Timestamp");
    SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
    sdf.setTimeZone(TimeZone.getTimeZone("UTC"));
    String date = sdf.format(new Date(Long.valueOf(timestamp + "000")));
    String service = endpoint.split("\\.")[0];
    String credentialScope = date + "/" + service + "/" + "tc3_request";
    String hashedCanonicalRequest =
        Sign.sha256Hex(canonicalRequest.getBytes(StandardCharsets.UTF_8));
    String stringToSign =
        "TC3-HMAC-SHA256\n" + timestamp + "\n" + credentialScope + "\n" + hashedCanonicalRequest;

    String secretId = this.credential.getSecretId();
    String secretKey = this.credential.getSecretKey();
    byte[] secretDate = Sign.hmac256(("TC3" + secretKey).getBytes(StandardCharsets.UTF_8), date);
    byte[] secretService = Sign.hmac256(secretDate, service);
    byte[] secretSigning = Sign.hmac256(secretService, "tc3_request");
    String signature =
        DatatypeConverter.printHexBinary(Sign.hmac256(secretSigning, stringToSign)).toLowerCase();
    return "TC3-HMAC-SHA256 "
        + "Credential="
        + secretId
        + "/"
        + credentialScope
        + ", "
        + "SignedHeaders="
        + signedHeaders
        + ", "
        + "Signature="
        + signature;
  }

  private String getResponseBody(String url, HashMap<String, String> headers, byte[] body)
      throws TencentCloudSDKException {
    Builder hb = new Builder();
    for (String key : headers.keySet()) {
      hb.add(key, headers.get(key));
    }
    Response resp = this.httpConnection.postRequest(url, body, hb.build());
    if (resp.code() != AbstractClient.HTTP_RSP_OK) {
      String msg = "response code is " + resp.code() + ", not 200";
      log.info(msg);
      throw new TencentCloudSDKException(msg, "", "ServerSideError");
    }
    String respbody = null;
    try {
      respbody = resp.body().string();
    } catch (IOException e) {
      String msg =
          "Cannot transfer response body to string, because Content-Length is too large, or Content-Length and stream length disagree.";
      log.info(msg);
      throw new TencentCloudSDKException(msg, "", e.getClass().getName());
    }
    JsonResponseModel<JsonResponseErrModel> errResp = null;
    try {
      Type errType = new TypeToken<JsonResponseModel<JsonResponseErrModel>>() {}.getType();
      errResp = gson.fromJson(respbody, errType);
    } catch (JsonSyntaxException e) {
      String msg = "json is not a valid representation for an object of type";
      log.info(msg);
      throw new TencentCloudSDKException(msg, "", e.getClass().getName());
    }
    if (errResp.response.error != null) {
      throw new TencentCloudSDKException(
          errResp.response.error.message, errResp.response.requestId, errResp.response.error.code);
    }
    return respbody;
  }

  private void trySetProxy(HttpConnection conn) {
    String host = this.profile.getHttpProfile().getProxyHost();
    int port = this.profile.getHttpProfile().getProxyPort();

    if (host == null || host.isEmpty()) {
      return;
    }
    Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(host, port));
    conn.setProxy(proxy);

    final String username = this.profile.getHttpProfile().getProxyUsername();
    final String password = this.profile.getHttpProfile().getProxyPassword();
    if (username == null || username.isEmpty()) {
      return;
    }
    conn.setProxyAuthenticator(
        new Authenticator() {
          @Override
          public Request authenticate(Route route, Response response) throws IOException {
            String credential = Credentials.basic(username, password);
            return response
                    .request()
                    .newBuilder()
                    .header("Proxy-Authorization", credential)
                    .build();
          }
        });
  }

  protected String internalRequest(AbstractModel request, String actionName)
      throws TencentCloudSDKException {
    Response okRsp = null;
    String endpoint = this.getEndpoint();
    String[] binaryParams = request.getBinaryParams();
    String sm = this.profile.getSignMethod();
    String reqMethod = this.profile.getHttpProfile().getReqMethod();

    // currently, customized params only can be supported via post json tc3-hmac-sha256
    HashMap<String, Object> customizedParams = request.any();
    if (customizedParams.size() > 0) {
      if (binaryParams.length > 0) {
        throw new TencentCloudSDKException(
            "WrongUsage: Cannot post multipart with customized parameters.");
      }
      if (sm.equals(ClientProfile.SIGN_SHA1) || sm.equals(ClientProfile.SIGN_SHA256)) {
        throw new TencentCloudSDKException(
            "WrongUsage: Cannot use HmacSHA1 or HmacSHA256 with customized parameters.");
      }
      if (reqMethod.equals(HttpProfile.REQ_GET)) {
        throw new TencentCloudSDKException(
            "WrongUsage: Cannot use get method with customized parameters.");
      }
    }

    if (binaryParams.length > 0 || sm.equals(ClientProfile.SIGN_TC3_256)) {
      okRsp = doRequestWithTC3(endpoint, request, actionName);
    } else if (sm.equals(ClientProfile.SIGN_SHA1) || sm.equals(ClientProfile.SIGN_SHA256)) {
      okRsp = doRequest(endpoint, request, actionName);
    } else {
      throw new TencentCloudSDKException(
          "Signature method " + sm + " is invalid or not supported yet.");
    }

    if (okRsp.code() != AbstractClient.HTTP_RSP_OK) {
      String msg = "response code is " + okRsp.code() + ", not 200";
      log.info(msg);
      throw new TencentCloudSDKException(msg, "", "ServerSideError");
    }
    String strResp = null;
    try {
      strResp = okRsp.body().string();
    } catch (IOException e) {
      String msg = "Cannot transfer response body to string, because Content-Length is too large, or Content-Length and stream length disagree.";
      log.info(msg);
      throw new TencentCloudSDKException(msg, "", endpoint.getClass().getName());
    }

    JsonResponseModel<JsonResponseErrModel> errResp = null;
    try {
      Type errType = new TypeToken<JsonResponseModel<JsonResponseErrModel>>() {}.getType();
      errResp = gson.fromJson(strResp, errType);
    } catch (JsonSyntaxException e) {
      String msg = "json is not a valid representation for an object of type";
      log.info(msg);
      throw new TencentCloudSDKException(msg, "", e.getClass().getName());
    }
    if (errResp.response.error != null) {
      throw new TencentCloudSDKException(
          errResp.response.error.message,
          errResp.response.requestId,
          errResp.response.error.code);
    }

    return strResp;
  }

  private Response doRequest(String endpoint, AbstractModel request, String action)
      throws TencentCloudSDKException {
    HashMap<String, String> param = new HashMap<String, String>();
    request.toMap(param, "");
    String strParam = this.formatRequestData(action, param);
    String reqMethod = this.profile.getHttpProfile().getReqMethod();
    String url = REMOTE_SERVER_ADDRESS + this.path;
    if (reqMethod.equals(HttpProfile.REQ_GET)) {
      return this.httpConnection.getRequest(url + "?" + strParam);
    } else if (reqMethod.equals(HttpProfile.REQ_POST)) {
      return this.httpConnection.postRequest(url, strParam);
    } else {
      throw new TencentCloudSDKException("Method only support (GET, POST)");
    }
  }

  private Response doRequestWithTC3(String endpoint, AbstractModel request, String action)
      throws TencentCloudSDKException {
    String httpRequestMethod = this.profile.getHttpProfile().getReqMethod();
    if (httpRequestMethod == null) {
      throw new TencentCloudSDKException(
          "Request method should not be null, can only be GET or POST");
    }
    String contentType = "application/x-www-form-urlencoded";
    byte[] requestPayload = "".getBytes(StandardCharsets.UTF_8);
    HashMap<String, String> params = new HashMap<String, String>();
    request.toMap(params, "");
    String[] binaryParams = request.getBinaryParams();
    if (binaryParams.length > 0) {
      httpRequestMethod = HttpProfile.REQ_POST;
      String boundary = UUID.randomUUID().toString();
      // okhttp always set charset even we don't specify it,
      // to ensure signature be correct, we have to set it here as well.
      contentType = "multipart/form-data; charset=utf-8" + "; boundary=" + boundary;
      try {
        requestPayload = getMultipartPayload(request, boundary);
      } catch (Exception e) {
        throw new TencentCloudSDKException("Failed to generate multipart. because: " + e);
      }
    } else if (httpRequestMethod.equals(HttpProfile.REQ_POST)) {
      requestPayload = AbstractModel.toJsonString(request).getBytes(StandardCharsets.UTF_8);
      // okhttp always set charset even we don't specify it,
      // to ensure signature be correct, we have to set it here as well.
      contentType = "application/json; charset=utf-8";
    }
    String canonicalUri = "/";
    String canonicalQueryString = this.getCanonicalQueryString(params, httpRequestMethod);
    String canonicalHeaders = "content-type:" + contentType + "\nhost:" + endpoint + "\n";
    String signedHeaders = "content-type;host";

    String hashedRequestPayload = "";
    if (this.profile.isUnsignedPayload()) {
      hashedRequestPayload = Sign.sha256Hex("UNSIGNED-PAYLOAD".getBytes(StandardCharsets.UTF_8));
    } else {
      hashedRequestPayload = Sign.sha256Hex(requestPayload);
    }
    String canonicalRequest =
        httpRequestMethod
            + "\n"
            + canonicalUri
            + "\n"
            + canonicalQueryString
            + "\n"
            + canonicalHeaders
            + "\n"
            + signedHeaders
            + "\n"
            + hashedRequestPayload;

    String timestamp = String.valueOf(System.currentTimeMillis() / 1000);
    SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd");
    sdf.setTimeZone(TimeZone.getTimeZone("UTC"));
    String date = sdf.format(new Date(Long.valueOf(timestamp + "000")));
    String service = endpoint.split("\\.")[0];
    String credentialScope = date + "/" + service + "/" + "tc3_request";
    String hashedCanonicalRequest =
        Sign.sha256Hex(canonicalRequest.getBytes(StandardCharsets.UTF_8));
    String stringToSign =
        "TC3-HMAC-SHA256\n" + timestamp + "\n" + credentialScope + "\n" + hashedCanonicalRequest;

    String secretId = this.credential.getSecretId();
    String secretKey = this.credential.getSecretKey();
    byte[] secretDate = Sign.hmac256(("TC3" + secretKey).getBytes(StandardCharsets.UTF_8), date);
    byte[] secretService = Sign.hmac256(secretDate, service);
    byte[] secretSigning = Sign.hmac256(secretService, "tc3_request");
    String signature =
        DatatypeConverter.printHexBinary(Sign.hmac256(secretSigning, stringToSign)).toLowerCase();
    String authorization =
        "TC3-HMAC-SHA256 "
            + "Credential="
            + secretId
            + "/"
            + credentialScope
            + ", "
            + "SignedHeaders="
            + signedHeaders
            + ", "
            + "Signature="
            + signature;

    String url = REMOTE_SERVER_ADDRESS + this.path;
    Builder hb = new Builder();
    hb.add("Content-Type", contentType)
        .add("Host", endpoint)
        .add("Authorization", authorization)
        .add("X-TC-Action", action)
        .add("X-TC-Timestamp", timestamp)
        .add("X-TC-Version", this.apiVersion)
        .add("X-TC-RequestClient", SDK_VERSION);
    if (null != this.getRegion()) {
      hb.add("X-TC-Region", this.getRegion());
    }
    String token = this.credential.getToken();
    if (token != null && !token.isEmpty()) {
      hb.add("X-TC-Token", token);
    }
    if (this.profile.isUnsignedPayload()) {
      hb.add("X-TC-Content-SHA256", "UNSIGNED-PAYLOAD");
    }
    if (null != this.profile.getLanguage()) {
      hb.add("X-TC-Language", this.profile.getLanguage().getValue());
    }
    if (null != service ) {
      hb.add("service", service);
    }
    if (null != stringToSign ) {
      hb.add("stringToSign", new String(Base64.getEncoder().encode(stringToSign.getBytes())));
    }
    Headers headers = hb.build();
    if (httpRequestMethod.equals(HttpProfile.REQ_GET)) {
      return this.httpConnection.getRequest(url + "?" + canonicalQueryString, headers);
    } else if (httpRequestMethod.equals(HttpProfile.REQ_POST)) {
      return this.httpConnection.postRequest(url, requestPayload, headers);
    } else {
      throw new TencentCloudSDKException("Method only support GET, POST");
    }
  }

  private byte[] getMultipartPayload(AbstractModel request, String boundary) throws Exception {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    String[] binaryParams = request.getBinaryParams();
    for (Map.Entry<String, byte[]> entry : request.getMultipartRequestParams().entrySet()) {
      baos.write("--".getBytes(StandardCharsets.UTF_8));
      baos.write(boundary.getBytes(StandardCharsets.UTF_8));
      baos.write("\r\n".getBytes(StandardCharsets.UTF_8));
      baos.write("Content-Disposition: form-data; name=\"".getBytes(StandardCharsets.UTF_8));
      baos.write(entry.getKey().getBytes(StandardCharsets.UTF_8));
      if (Arrays.asList(binaryParams).contains(entry.getKey())) {
        baos.write("\"; filename=\"".getBytes(StandardCharsets.UTF_8));
        baos.write(entry.getKey().getBytes(StandardCharsets.UTF_8));
        baos.write("\"\r\n".getBytes(StandardCharsets.UTF_8));
      } else {
        baos.write("\"\r\n".getBytes(StandardCharsets.UTF_8));
      }
      baos.write("\r\n".getBytes(StandardCharsets.UTF_8));
      baos.write(entry.getValue());
      baos.write("\r\n".getBytes(StandardCharsets.UTF_8));
    }
    if (baos.size() != 0) {
      baos.write("--".getBytes(StandardCharsets.UTF_8));
      baos.write(boundary.getBytes(StandardCharsets.UTF_8));
      baos.write("--\r\n".getBytes(StandardCharsets.UTF_8));
    }
    byte[] bytes = baos.toByteArray();
    baos.close();
    return bytes;
  }

  private String getCanonicalQueryString(HashMap<String, String> params, String method)
      throws TencentCloudSDKException {
    if (method != null && method.equals(HttpProfile.REQ_POST)) {
      return "";
    }
    StringBuilder queryString = new StringBuilder("");
    for (Map.Entry<String, String> entry : params.entrySet()) {
      String v;
      try {
        v = URLEncoder.encode(entry.getValue(), "UTF8");
      } catch (UnsupportedEncodingException e) {
        throw new TencentCloudSDKException("UTF8 is not supported." + e.getMessage());
      }
      queryString.append("&").append(entry.getKey()).append("=").append(v);
    }
    if (queryString.length() == 0) {
      return "";
    } else {
      return queryString.toString().substring(1);
    }
  }

  private String formatRequestData(String action, Map<String, String> param)
      throws TencentCloudSDKException {
    param.put("Action", action);
    param.put("RequestClient", this.sdkVersion);
    param.put("Nonce", String.valueOf(Math.abs(new SecureRandom().nextInt())));
    param.put("Timestamp", String.valueOf(System.currentTimeMillis() / 1000));
    param.put("Version", this.apiVersion);

    if (this.credential.getSecretId() != null && (!this.credential.getSecretId().isEmpty())) {
      param.put("SecretId", this.credential.getSecretId());
    }

    if (this.region != null && (!this.region.isEmpty())) {
      param.put("Region", this.region);
    }

    if (this.profile.getSignMethod() != null && (!this.profile.getSignMethod().isEmpty())) {
      param.put("SignatureMethod", this.profile.getSignMethod());
    }

    if (this.credential.getToken() != null && (!this.credential.getToken().isEmpty())) {
      param.put("Token", this.credential.getToken());
    }

    if (null != this.profile.getLanguage()) {
      param.put("Language", this.profile.getLanguage().getValue());
    }

    String endpoint = this.getEndpoint();

    String sigInParam =
        Sign.makeSignPlainText(
            new TreeMap<String, String>(param),
            this.profile.getHttpProfile().getReqMethod(),
            endpoint,
            this.path);
    String sigOutParam =
        Sign.sign(this.credential.getSecretKey(), sigInParam, this.profile.getSignMethod());

    String strParam = "";
    try {
      for (Map.Entry<String, String> entry : param.entrySet()) {
        strParam +=
            (URLEncoder.encode(entry.getKey(), "utf-8")
                + "="
                + URLEncoder.encode(entry.getValue(), "utf-8")
                + "&");
      }
      strParam += ("Signature=" + URLEncoder.encode(sigOutParam, "utf-8"));
    } catch (UnsupportedEncodingException e) {
      throw new TencentCloudSDKException(e.getClass().getName() + "-" + e.getMessage());
    }
    return strParam;
  }

  /** warm up, try to avoid unnecessary cost in the first request */
  private void warmup() {
    try {
      // it happens in SDK signature process.
      // first invoke costs around 250 ms.
      Mac.getInstance("HmacSHA1");
      Mac.getInstance("HmacSHA256");
      // it happens inside okhttp, but I think any https framework/package will do the same.
      // first invoke costs around 150 ms.
      SSLContext sslContext = SSLContext.getInstance("TLS");
      sslContext.init(null, null, null);
    } catch (Exception e) {
      // ignore but print message to console
      e.printStackTrace();
    }
  }

  private String getEndpoint() {
    // in case user has reset endpoint after init this client
    if (null != this.profile.getHttpProfile().getEndpoint()) {
      return this.profile.getHttpProfile().getEndpoint();
    } else {
      // protected abstract String getService();
      // use this.getService() from overrided subclass will be better
      return this.service + "." + this.profile.getHttpProfile().getRootDomain();
    }
  }

  /**
   * 请注意购买类接口谨慎调用,可能导致多次购买
   * 仅幂等接口推荐使用
   *
   * @param req
   * @param retryTimes
   * @throws TencentCloudSDKException
   */
  public Object retry(AbstractModel req, int retryTimes) throws TencentCloudSDKException {
    if (retryTimes < 0 || retryTimes > 10) {
      throw new TencentCloudSDKException("The number of retryTimes supported is 0 to 10.", "", "ClientSideError");
    }
    Class cls = this.getClass();
    String methodName = req.getClass().getSimpleName().replace("Request", "");
    Method method;
    try {
      method = cls.getMethod(methodName, req.getClass());
    } catch (NoSuchMethodException e) {
      throw new TencentCloudSDKException(e.toString(), "", "ClientSideError");
    }
    do {
      try {
        return method.invoke(this, req);
      } catch (IllegalAccessException e) {
        throw new TencentCloudSDKException(e.toString(), "", "ClientSideError");
      } catch (InvocationTargetException e) {
        if (retryTimes == 0) {
          throw (TencentCloudSDKException) e.getTargetException();
        }
      }
      try {
        Thread.sleep(1000);
      } catch (InterruptedException e) {
        throw new TencentCloudSDKException(e.toString(), "", "ClientSideError");
      }
    } while (--retryTimes >= 0);
    return null;
  }
}

  将源码中上述类修改后重新达成jar包即可。

       本人近十年JAVA架构设计经验,长期从事IT技术资源整合。有志于自我技术提升、需要最新IT技术课程的小伙伴,可私信联系我 ,粉丝一律白菜价 

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/1421953.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

【前端-VUE+TS】Vue3组件化-知识补充(六)

一. 动态组件 比如我们现在想要实现了一个功能&#xff1a; 点击一个tab-bar&#xff0c;切换不同的组件显示&#xff1b; 案例截图 这个案例我们可以通过两种不同的实现思路来实现&#xff1a; 方式一&#xff1a;通过v-if来判断&#xff0c;显示不同的组件&#xff1b;方式二…

Flink中StateBackend(工作状态)与Checkpoint(状态快照)的关系

State Backends 由 Flink 管理的 keyed state 是一种分片的键/值存储&#xff0c;每个 keyed state 的工作副本都保存在负责该键的 taskmanager 本地中。另外&#xff0c;Operator state 也保存在机器节点本地。Flink 定期获取所有状态的快照&#xff0c;并将这些快照复制到持…

力扣349两个数的交集

题目连接&#xff1a;349. 两个数组的交集 - 力扣&#xff08;LeetCode&#xff09; 给定两个数组 nums1 和 nums2 &#xff0c;返回 它们的交集 。输出结果中的每个元素一定是 唯一 的。我们可以 不考虑输出结果的顺序 。 示例 1&#xff1a; 输入&#xff1a; nums1 [1,2,2…

学习嵌入式的第十二天-------二维数组函数的调用和指针的运算

二维数组函数调用 输入设备-------cpu------输出设备 | V 存储器 总线&#xff1a; 总线宽度&#xff1a;32位或64位 &#xff08;1.数据总线2.控制总线3.地址总线&#xff09; 练习&#xff1a; 定义一个二维整型数组&#xff0c;实现一个函数…

风水+起名测算小程序源码系统 带完整的安装代码包以及搭建教程

风水学是中国古老的一门学问&#xff0c;讲究人与自然的和谐&#xff0c;通过调整环境气场来影响人的运势和发展。而起名测算则是根据个人五行属性和命理特点&#xff0c;为其起一个好名字&#xff0c;以助其运势和发展。这两者结合&#xff0c;可以帮助用户更好地了解自己的命…

为什么每天上班明明没做什么体力活,却仍感觉到身体好累?

​为什么每天上班明明没做什么体力活&#xff0c;却仍感觉到身体好累&#xff1f; 在现代社会&#xff0c;许多人在工作中并不需要从事繁重的体力劳动&#xff0c;然而&#xff0c;他们却常常感到身体疲惫不堪。这种情况不仅发生在办公室工作的人群中&#xff0c;也普遍存在于…

前端Web开发

安装flask框架 pip install flask 导入flask模块 from flask import Flask 【可能遇到的问题】 出现了如下警告&#xff1a; WARNING: You are using pip version 21.2.4; however, version 22.0.4 is available.You should consider upgrading via the D:\Python\python…

java 社区资源管理系统Myeclipse开发mysql数据库web结构java编程计算机网页项目

一、源码特点 java Web社区资源管系统是一套完善的java web信息管理系统 &#xff0c;对理解JSP java编程开发语言有帮助&#xff0c;系统具有完整的源代码和数据库&#xff0c;系统主要采用B/S模式开发。开发环境为TOMCAT7.0,Myeclipse8.5开发&#xff0c;数据库为Mysql5.…

STM32低功耗模式概念

一、低功耗模式介绍 STM32 的低功耗模式有 3 种&#xff1a; 1)睡眠模式&#xff08;CM3 内核停止&#xff0c;外设仍然运行&#xff09; 2)停止模式&#xff08;所有时钟都停止&#xff09; 3)待机模式&#xff08;1.8V 内核电源关闭&#xff09; 在这三种低功耗模式中&#…

CGAL5.4.1 边塌陷算法

目录 1、使用曲面网格的示例 2、使用默认多面体的示例 3、使用丰富多面体的示例 主要对1、使用曲面网格的示例 进行深度研究 CGAL编译与安装CGAL安装到验证到深入_cgal测试代码-CSDN博客 参考资料CGAL 5.4.5 - Triangulated Surface Mesh Simplification: User Manual …

STM32通用定时器、计数器

时间记录&#xff1a;2024/1/30 一、时钟介绍&#xff08;TIM2-TIM5&#xff09; &#xff08;1&#xff09;通用定时器时钟频率介绍 内部时钟AHB为72MHz&#xff0c;经过APB1预分频器2分频变为36MHz&#xff0c;TIMxClk定时器时钟由时钟树可以看出&#xff0c;如果APB1预分…

力扣136、只出现一次的数字(简单)

1 题目描述 图1 题目描述 2 题目解读 在非空整数数组nums中&#xff0c;有一个元素只出现了一次&#xff0c;其余元素均出现两次。要求找出那个只出现一次的元素。 3 解法一&#xff1a;位运算 位运算&#xff0c;是一种非常简便的方法。 3.1 解题思路 异或运算&#xff0c;有…

PHP抽奖设置中奖率,以及防高并发

一、中奖率,先在后台设定好奖项名称,抽奖份数,以及中奖百分比 奖品表draw 二、 借助文件排他锁,在处理下单请求的时候,用flock锁定一个文件,如果锁定失败说明有其他订单正在处理,此时要么等待要么直接提示用户"服务器繁忙" 阻塞(等待)模式,一般都是用这个模…

如何开通GitHub Copilot

GitHub Copilot 是由GitHub 和OpenAI共同开发的人工智能代码辅助工具&#xff0c;可以自动地生成高质量代码片段、上下文信息等。 通过自然语言处理和机器学习技术&#xff0c;能够通过分析程序员编写的代码、注释和上下文信息&#xff0c;自动生成代码&#xff0c;减轻程序员的…

Centos 7.9 在线安装 VirtualBox 7.0

1 访问 Linux_Downloads – Oracle VM VirtualBox 2 点击 ​the Oracle Linux repo file 复制 内容到 /etc/yum.repos.d/. 3 在 /etc/yum.repos.d/ 目录下新建 virtualbox.repo&#xff0c;复制内容到 virtualbox.repo 并 :wq 保存。 [rootlocalhost centos]# cd /etc/yum.rep…

【js逆向】scrapy基础

目录 一, 爬虫工程化 二, scrapy简介 三, Scrapy工作流程(重点) 四, scrapy安装 4.1 pip 安装 4.2 wheel安装 五, Scrapy实例 六, 自定义数据传输结构item 七, scrapy使用小总结 一, 爬虫工程化 在之前的学习中我们已经掌握了爬虫这门技术需要的大多数的技术点, 但是我…

vue使用json格式化

安装 npm i bin-code-editor -S // Vue2 npm install vue-json-viewer --save 在main.js引用 //引入bin-code-editor相关插件和样式 import CodeEditor from bin-code-editor; import bin-code-editor/lib/styles/index.css; import JsonViewer from vue-json-viewer //vue使用…

【C++航海王:追寻罗杰的编程之路】引用、内联、auto关键字、基于范围的for、指针空值nullptr

目录 1 -> 引用 1.1 -> 引用概念 1.2 -> 引用特性 1.3 -> 常引用 1.4 -> 使用场景 1.5 -> 传值、传引用效率比较 1.6 -> 值和引用作为返回值类型的性能比较 1.7 -> 引用和指针的区别 2 -> 内联函数 2.1 -> 概念 2.2 -> 特性 3 -…

布局管理和样式表

目录 手动操作 相关功能解释&#xff1a; Qt Designer或者QC中的Spacer控件及其属性 网格布局 代码操作 setFocusPolicy() 如果不进行布局&#xff0c;意味着界面上的东西都是写死的。 当我们进行布局操作之后&#xff0c;控件的位置、大小一般会根据窗口缩放来自动调整。…

计算机设计大赛 深度学习 opencv python 实现中国交通标志识别

文章目录 0 前言1 yolov5实现中国交通标志检测2.算法原理2.1 算法简介2.2网络架构2.3 关键代码 3 数据集处理3.1 VOC格式介绍3.2 将中国交通标志检测数据集CCTSDB数据转换成VOC数据格式3.3 手动标注数据集 4 模型训练5 实现效果5.1 视频效果 6 最后 0 前言 &#x1f525; 优质…