专栏导读
🌸 欢迎来到Python办公自动化专栏—Python处理办公问题,解放您的双手 🏳️🌈 博客主页:请点击——> 一晌小贪欢的博客主页求关注 👍 该系列文章专栏:请点击——>Python办公自动化专栏求订阅 🕷 此外还有爬虫专栏:请点击——>Python爬虫基础专栏求订阅 📕 此外还有python基础专栏:请点击——>Python基础学习专栏求订阅 文章作者技术和水平有限,如果文中出现错误,希望大家能指正🙏 ❤️ 欢迎各位佬关注! ❤️
库的介绍
库的安装
库 用途 安装 requests 爬虫库 pip install requests -i https://pypi.tuna.tsinghua.edu.cn/simple/
os 获取绝对路径 内置库无需安装
代码
"""
@Project :测试代码
@File :爬虫.py
@IDE :PyCharm
@Author :一晌小贪欢
@Date :2024/9/29 10:18
"""
import json
import os
import time
import requests
url = 'https://www.xxxxxx.com/'
headers = {
"Cookie" : "" ,
'Accept' : 'application/json, text/plain, */*' ,
'Accept-Encoding' : 'gzip, deflate' ,
'Accept-Language' : 'zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6' ,
'User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36 Edg/128.0.0.0'
}
params = {
"key1" : "value" ,
"key2" : "value" ,
"key2" : "value" ,
"key2" : "value" ,
"key2" : "value"
}
max_retries = 5
retry_delay = 5
def fetch_data ( url, params, headers) :
attempt = 0
while attempt < max_retries:
try :
response = requests. get( url, params= params, headers= headers, timeout= 10 )
response. raise_for_status( )
return response. json( )
except requests. exceptions. RequestException as e:
print ( f"请求失败: { e} ,尝试第 { attempt + 1 } 次重试..." )
attempt += 1
time. sleep( retry_delay)
print ( "已达到最大重试次数,请求失败。" )
return None
def save_data ( data, filename) :
"""将数据保存到JSON文件中"""
with open ( filename, 'w' , encoding= 'utf-8' ) as f:
json. dump( data, f, ensure_ascii= False , indent= 4 )
print ( f"数据已保存到 { filename} " )
def main ( ) :
data = fetch_data( url, params, headers)
if data:
save_data( data, 'result.json' )
if __name__ == "__main__" :
main( )
总结
希望对初学者有帮助 致力于办公自动化的小小程序员一枚 希望能得到大家的【一个免费关注】!感谢 求个 🤞 关注 🤞 此外还有办公自动化专栏,欢迎大家订阅:Python办公自动化专栏 求个 ❤️ 喜欢 ❤️ 此外还有爬虫专栏,欢迎大家订阅:Python爬虫基础专栏 求个 👍 收藏 👍 此外还有Python基础专栏,欢迎大家订阅:Python基础学习专栏