视频1 视频21 视频41 视频61 视频文章1 视频文章21 视频文章41 视频文章61 推荐1 推荐3 推荐5 推荐7 推荐9 推荐11 推荐13 推荐15 推荐17 推荐19 推荐21 推荐23 推荐25 推荐27 推荐29 推荐31 推荐33 推荐35 推荐37 推荐39 推荐41 推荐43 推荐45 推荐47 推荐49 关键词1 关键词101 关键词201 关键词301 关键词401 关键词501 关键词601 关键词701 关键词801 关键词901 关键词1001 关键词1101 关键词1201 关键词1301 关键词1401 关键词1501 关键词1601 关键词1701 关键词1801 关键词1901 视频扩展1 视频扩展6 视频扩展11 视频扩展16 文章1 文章201 文章401 文章601 文章801 文章1001 资讯1 资讯501 资讯1001 资讯1501 标签1 标签501 标签1001 关键词1 关键词501 关键词1001 关键词1501 专题2001
requests库的基本使用
2020-11-27 15:03:40 责编:小采
文档


1. response.content和response.text的区别

response.content是编码后的byte类型(“str”数据类型),response.text是unicode类型。这两种方法的使用要视情况而定。注意:unicode -> str 是编码过程(encode()); str -> unicode 是解码过程(decode())。示例如下:

# --coding:utf-8-- #
import requests
response = requests.get("https://baidu.com/")
print response.url
print type(response.content)
with open("C:\Users\Administrator\Desktop\content.html", "w") as f:
 f.write(response.content)
 print "content保存成功"
print type(response.text)
with open("C:\Users\Administrator\Desktop\text.html", "w") as f:
 # 返回url的编码方式
 print response.encoding
 f.write(response.text.encode("ISO-8859-1"))
 print "text保存成功"

2. 发送get请求,直接调用“resquests.get" 就可以了。response的一些属性:response.text; response.content; response.url; response.encoding; response.status_code

# --coding:utf-8-- #
import requests
params = {
 "wd": "中国"
}
headers = {
 "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win; x) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36"
}
response = requests.get("https://baidu.com/s", params=params, headers=headers)
print response.url
with open("C:\Users\Administrator\Desktop\get.html", "w") as f:
 f.write(response.content)
 print "保存成功"

3. 发送post请求:传入data信息。注意get请求传入的是params信息。示例如下:

# --coding:utf-8-- #
import requests
data = {
 "first": "true",
 "pn": "1",
 "wd": "python"
}
headers = {
 "Referer": "https://www.lagou.com/jobs/list_python?labelWords=&fromSearch=true&suginput=",
 "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win; x) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36"
}
response = requests.post("https://www.lagou.com/jobs/positionAjax.json?needAddtionalResult=false", data=data, headers=headers)
print response.encoding
print type(response.content)
with open("C:\Users\Administrator\Desktop\post.html", "w") as f:
 f.write(response.content)
 print "保存成功"

4. 使用代理。在get方法中增加proxy参数即可。示例代码如下:

# --coding:utf-8-- #
import requests
proxy = {
 "http": "124.42.7.103"
}
response = requests.get("http://httpbin.org/ip", proxies=proxy)
print response.content

5. requests处理cookies信息。使用requests.Session()方法即可。示例代码如下:

# --coding:utf-8-- #
import requests
url = "http://www.renren.com/PLogin.do"
# url = "http://www.renren.com/SysHome.do"
data = {"email": "账号", "password": "密码"}
headers = {
 "User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win; x) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.62 Safari/537.36"
}
session = requests.Session()
session.post(url, data=data, headers=headers)
response = session.get("http://www.renren.com/543484094/profile")
with open("C:\Users\Administrator\Desktop\Liwei.html", "w") as fp:
 fp.write(response.content)
 print "保存成功"

6. 处理不信任的SSL证书。与上面的代码相比,多了一个verify=False参数,为了处理SSL证书不受信用的问题。

示例代码如下:

response = session.get("http://www.renren.com/543484094/profile", verify=False)

下载本文
显示全文
专题