【Python模块】——request.request的重试问题

news2024/9/29 13:18:10

前言

在工作中，经常遇到调用request.request去返回某个网站报超时，并且报超时的内容不一样。一种是HTTPSConnectionPool(host='www.xxx.com', port=443): Read timed out. (read timeout=10)；另一种是HTTPSConnectionPool(host='www.xxx.com', port=443): Max retries exceeded with url: /api/1/test (Caused by ConnectTimeoutError)。很显然，这是两种不同的超时报错。

超时详解

超时分类

超时可分为连接超时和读取超时两种。从requests.request源码可知，timeout支持浮点类型也支持元组。timeout=float()，表示在等待服务端返回数据前的等待时间；timeout=tuple(connect timeout, read timeout)，前者指的是连接超时，后者是读取超时。连接超时是指客户端连接服务端等待时间超过设定值的超时；读取超时是指客户端连接到服务端，等待服务端请求返回的等待时间。

def request(method, url, **kwargs):
    """Constructs and sends a :class:`Request <Request>`.
	...
    :param timeout: (optional) How many seconds to wait for the server to send data
        before giving up, as a float, or a :ref:`(connect timeout, read
        timeout) <timeouts>` tuple.
    :type timeout: float or tuple
    """

本地flask服务

from flask import Flask
import time


app = Flask(__name__)

@app.route("/")
def index():
	print("congratulation coming this website")
    time.sleep(3)
    return "hello world"

if __name__ == '__main__':
    app.run("127.0.0.1", port=4444)

设置timeout=float

连接超时

import requests

host = "http://192.168.0.1:4444"

try:
    response = requests.get(host, timeout=3)
    print(response.status_code)

except ConnectionError as e:
    raise Exception(e)
except Exception as e:
    raise Exception(e)

本地连接192.168.0.1:4444，返回结果为HTTPConnectionPool(host='192.168.0.1', port=4444): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x106e47910>, 'Connection to 192.168.0.1 timed out. (connect timeout=3)'))。本地ping不通192.168.0.1这个ip，即无法连接。所以超过3s后，返回无法连接到这个ip

读取超时

import requests

host = "http://localhost:4444"

try:
    response = requests.get(host, timeout=3)
    print(response.status_code)

except ConnectionError as e:
    raise Exception(e)
except Exception as e:
    raise Exception(e)

返回结果为HTTPConnectionPool(host='localhost', port=4444): Read timed out. (read timeout=3)。这是因为本地flask服务设置time.sleep(3)，服务端的日志已经打出congratulation coming this website，等待3s后再打印出127.0.0.1 - - [29/Sep/2024 10:32:47] "GET / HTTP/1.1" 200 -。这说明已经请求到服务端了，只是因为客户端从服务端读取相应数据超时了。

设置timeout=tuple

连接超时

import requests

host = "http://192.168.0.1:4444"

try:
    response = requests.get(host, timeout=(2.0,  3.0))
    print(response.status_code)

except ConnectionError as e:
    raise Exception(e)
except Exception as e:
    raise Exception(e)

返回结果为Exception: HTTPConnectionPool(host='192.168.0.1', port=4444): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x106233910>, 'Connection to 192.168.0.1 timed out. (connect timeout=2.0)'))。

读取超时

import requests

host = "http://localhost:4444"

try:
    response = requests.get(host, timeout=(2.0, 3.0))
    print(response.status_code)

except ConnectionError as e:
    raise Exception(e)
except Exception as e:
    raise Exception(e)

返回结果为Exception: HTTPConnectionPool(host='localhost', port=4444): Read timed out. (read timeout=3.0)。

如果将timeout改为(2.0, 4.0)，响应结果为200，说明读取超时=4.0已生效

requests超时重试

import requests
import logging 

if "__name__" == "__main__":
	s = requests.Session()
    s.mount("http://", HTTPAdapter(max_retries=3))
    s.mount("https://", HTTPAdapter(max_retries=3))
    try:
        r = requests.get(url, timeout=(2.0, 3.0))
        if r.status_code == 200:
            return r.json().get('configurations')
        time.sleep(3)
    except Exception as e:
        logging.error(e)