爬取斗鱼4300W流量的小姐姐并为颜值打分
↑ 关注 + 星标 ，每天学Python新技能
后台回复【大礼包】送你Python自学大礼包
1.爬取小姐姐照片
2.调用api为小姐姐诶你颜值打分
3.颜值排行
这是我们要爬取的目标：
这是我们爬取的过程：
https://www.douyu.com/gapi/rknc/directory/yzRec/1
https://www.douyu.com/gapi/rknc/directory/yzRec/2
https://www.douyu.com/gapi/rknc/directory/yzRec/3
https://www.douyu.com/gapi/rknc/directory/yzRec/4
url = f'https://www.douyu.com/gapi/rknc/directory/yzRec/1'
headers = {
'cookie':'xxxxx',
'referer':'xxxxx',
'user-agent':'xxxxx'
}
#获取浏览器返回的JSON结果
resp = requests.get(url,headers = headers).json()
#获取每一页的120个颜值小姐姐信息
girls = resp['data']['rl']
'''
[{'rid': 3477773, 'rn': '唱歌主播 无人听懂曲中意', 'uid': 160088008, 'nn': '鹿鹿酱in', 'cid1': 2, 'cid2': 311, 'cid3': 1749, 'iv': 0, 'av': 'avatar_v3/202104/31e0b77f8fa84f779f5e965c9ae3acf2', 'ol': 178516, 'url': '/3477773', 'c2url': '/directory/game/XX', 'c2name': '颜值', 'icdata': {'548': {'url': '', 'w': 0, 'h': 0}}, 'dot': 2103, 'subrt': 0, 'topid': 0, 'oaid': 0, 'bid': 0, 'gldid': 0, 'rs1': 'https://rpic.douyucdn.cn/live-cover/roomCover/2020/12/22/fbc91f6c4baf895bc7c91e6fa299803d_small.png/dy2', 'rs16': 'https://rpic.douyucdn.cn/live-cover/roomCover/2020/12/22/fbc91f6c4baf895bc7c91e6fa299803d_big.png/dy1', 'utag': [], 'rpos': 0, 'rgrpt': 1, 'rkic': '', 'rt': 2103, 'ot': 0, 'clis': 2, 'chanid': 0, 'icv1': [[{'id': 548, 'url': 'https://sta-op.douyucdn.cn/dy-listicon/4d1d7ef013162713c5f1f601697bcb2f.png', 'score': 1001, 'w': 0, 'h': 0}], [], [], []], 'ioa': 0, 'od': '', 'isShowUp': 0, 'authInfo': None}]
for girl in girls:
pic_links = girl['rs1'] # 所有照片链接
room_title = girl['rn'] # 房间标题
room_name = girl['nn'] # 房间名称
category = girl['c2name'] # 主播分类
print(pic_links,room_title,room_name,category)
'''
https://rpic.douyucdn.cn/live-cover/appCovers/2021/03/07/7750351_20210307090509_big.jpg/dy2 这是个靓女的直播间~ Thea柒清 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/10/11/638575_20201011212022_big.jpg/dy2 小姨子： 幸运之神 丽亚celiahunne 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/03/24/7909303_20210324172547_big.jpg/dy2 有颜有才，尽在斗歌台 斗歌台官方直播间
https://rpic.douyucdn.cn/live-cover/appCovers/2020/07/27/8922441_20200727190717_big.jpg/dy2 5月15日月庆聚会 睡懵的渣皇 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/03/04/9384662_20210304004743_big.jpg/dy2 想要成为那个被你保护的人。 新来的恩宠 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/21/7fb646bba059b6a9f78b094d13fcde68_small.png/dy2 身怀绝技 百里挑一 星势力官方直播间
https://rpic.douyucdn.cn/live-cover/appCovers/2021/04/15/9737095_20210415195744_big.jpg/dy2 08年妹妹第一天直播求关注9737095 一只酸奶牛new 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2020/07/22/f865fc38018b758f92ca65413d01f611_small.png/dy2 我学会孤独 娇儿的36D吖 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/07/23/4125556_20200723164324_big.jpg/dy2 我回来啦！！681888 Skr丶大伟伟 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/02/22/1010383_20210222125342_big.jpg/dy2 双倍亲密，三倍上限，来了不想走呦~ 王羽杉abc 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/13/42e209e1cb9f15b27b3f40e5b3224fb4_small.png/dy2 我和我的猫都很想你 小初困了吗 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/05/03/8086045_20210503111144_big.jpg/dy2 时间不语，但会回答一切 豆乐乐咩 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/05/07/5b840cb58099af5f0912c716d30ded13_small.png/dy2 一条小虞er~想被哥哥带上岸 虞Oxygen 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/05/07/946df9dfa50623ac9fcb10d943d5678b_small.png/dy2 520周年庆欢迎大家来玩喔❤️Day8 林芊又 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/05/11/9720412_20210511195844_big.jpg/dy2 新人主播请多关照！！9720412 开心市民小邵 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/16/4c932e400f6e74c4fe62219013f43e7a_small.png/dy2 医生讲，你不要熬夜了 奉奉儿 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2019/04/08/5271808_20190408082749_big.jpg/dy2 消费查询，恭喜空空荣升超帝 酥小A 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/05/12/2243218_20200512082834_big.jpg/dy2 哦 那我走呗 八分饱ww 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/29/cfd9f29d923b5e4159ef3e77402fde76_small.png/dy2 每一天都期待能和你见面 9775190 尤田T 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/02/06/9064900_20210206191857_big.jpg/dy2 今天是你喜欢的美女子吗 玉玉子sway 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/09/c90f603c9305f85dc5efbd3d24134ea6_small.png/dy2 四川新主播略略略略略略 翼遥遥丶 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/11/26/4566947_20201126195330_big.jpg/dy2 可可爱爱 没有脑袋 4566947 关晓羽 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2020/06/20/4652236_20200620175813_big.jpg/dy2 身高170长发及腰的武汉小姐姐~~ 小火郭丶 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2019/04/01/6df644b55178d672257996028418bf3b_small.jpg/dy2 2021 一切顺利~ 林cute 颜值
https://rpic.douyucdn.cn/live-cover/roomCover/2021/04/06/2796d0dfbbaff1b4a7f537829658b6a4_small.png/dy2 小猪开播了 金善智 颜值
https://rpic.douyucdn.cn/live-cover/appCovers/2021/04/27/6532591_20210427105108_big.jpg/dy2 随意来去 开心就好 林以安阿 颜值
.........
'''
ws = op.Workbook()
wb = ws.create_sheet(index=0)
wb.cell(row=1, column=1, value='照片链接')
wb.cell(row=1, column=2, value='房间标题')
wb.cell(row=1, column=3, value='房间名称')
wb.cell(row=1, column=4, value='主播分类')
ws.save('斗鱼颜值.xlsx')
def down_dy(save_pic):
count = 0
for num,(pic_img,pic_name) in enumerate(save_pic):
r = requests.get(pic_img)
pic = r.content
try:
with open('./pictures/{}.jpg'.format(pic_name), 'wb') as fin:
print(f'正在爬取第{count}张图片')
fin.write(pic)
print('{}.jpg----下载成功'.format(pic_name))
except:
print('下载失败！')
for page in range(1,6+1):
print(f'--------正在爬取第{page}页--------')
url = f'https://www.douyu.com/gapi/rknc/directory/yzRec/{page}'
def get_beauty(img_base64):
# client_id 为官网获取的AK， client_secret 为官网获取的SK
host = 'https://aip.baidubce.com/oauth/2.0/token'
data = {
'grant_type': 'client_credentials',
'client_id': 'GD9bcCHcQETq37OdxwImqH6X',
'client_secret': 'tbtX8c5GG7BGkSXe874LLVtaOi3exAQN'
}
response = requests.get(host, params=data)
token = response.json()['access_token']
'''
人脸检测与属性分析
'''
request_url = f"https://aip.baidubce.com/rest/2.0/face/v3/detect?access_token={token}"
params = {
'image': img_base64, # 读取图片
'image_type': 'BASE64',
'face_field': 'beauty',
}
headers = {'content-type': 'application/json'}
response = requests.post(request_url, data=params, headers=headers)
try:
beauty= response.json()['result']['face_list'][0]['beauty']
return beauty
except:
return '识别失败！'
# 读取单张图片
# f = open(r'pictures\7z流川枫.jpg', mode='rb')
# 识别成功的照片存入list使用sort后续排名
lis = []
# 读取所有图片
files = os.listdir('pictures1\\')
print('正在识别人脸，颜值检测中......')
for file in tqdm(files[:100]):
img_file = 'pictures1\\' + file
img_name = file.split('.')[0]
f = open(img_file, mode='rb')
# 转base64
img_base64 = base64.b64encode(f.read())
beauty = get_beauty(img_base64)
# 颜值排名
if beauty != '识别失败！':
dit = {
'主播': img_name,
'颜值': beauty
}
# 字典添加到空列表
lis.append(dit)
#print(f'{img_name}的颜值评分是{beauty}')
lis.sort(key=lambda x:x['颜值'], reverse=True)
num = 1
for index in lis:
print(f'颜值排名第{num}的主播{index["主播"]},颜值评分是{index["颜值"]}')
'''
0%| | 0/100 [00:00100%|██████████| 100/100 [00:56<00:00, 1.76it/s]
颜值排名第1的主播Bear幼宥,颜值评分是91.63
颜值排名第2的主播club17796794,颜值评分是88.84
颜值排名第3的主播K鬼鬼K,颜值评分是88.54
颜值排名第4的主播wuli萄萄,颜值评分是88.45
颜值排名第5的主播Lulu没烦恼,颜值评分是86.48
颜值排名第6的主播伊伊iiiiii,颜值评分是86.46
颜值排名第7的主播一颗小络秋a,颜值评分是86.39
颜值排名第8的主播不思悔改橘某人,颜值评分是85.39
颜值排名第9的主播Liii晴沁,颜值评分是84.79
颜值排名第10的主播一条肥肥啾,颜值评分是84.46
颜值排名第11的主播亦柔Lris,颜值评分是84.39
。。。。。。。。。。。
'''
推荐阅读
推荐阅读
您看此文用 
分 
秒，转发只需1秒哦
您看此文用
评论