在学习python爬虫,在公司运行代码没有问题,但是下班回来把代码拉下来运行,却出现问题。
问题:
requests.exceptions.SSLError: HTTPSConnectionPool(host=‘campusgateway.51job.com’, port=443): Max retries exceeded with url: /jobone-api/school/getSchoolByCity?pageIndex=1&pageSize=21&city=%E5%8C%97%E4%BA%AC (Caused by SSLError(SSLError(1, ‘[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1000)’)))
上网查了各种方法,
- 添加verify=False -> 报错依旧
- 安装pip install requests == 2.19.1和pip install certifi==2018.8.13,变成另一个报错,ModuleNotFoundError: No module named ‘urllib3.packages.six.moves’。
百思不得其解,突然想到一句话,“代码一模一样但是运行不成功的话,要看环境有没配好”,然后我对比了公司和我电脑的Python解释器,我个人电脑的是Python 3.12版本,公司电脑我下载的是Python 3.6。果然不一样,然后我换成了3.6的,就可以成功运行了!!!
附上代码:
import requests
# 地址
url = "https://campusgateway.51job.com/jobone-api/school/getSchoolByCity"
# 查询参数
params = {
"pageIndex": "1",
"pageSize": "21",
"city": "北京"
}
# 请求头
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36"
}
# 请求发送,响应接收
response = requests.get(url, params=params, headers=headers)
# 提取响应
# print(response.text)
# json格式数据,可以转字典,进行key value提取
items = response.json()
for data in items['data']['records']:
# print(data)
name = data.get('schname')
print(name)