我一直在玩漂亮的汤和解析网页几天。在编写的所有脚本中,我一直使用一行代码作为救星。代码行是:
r = requests.get('some_url', auth=('my_username', 'my_password')).
但是…
我想对(OPEN A URL WITH AUTHENTICATION)做同样的事情:
(1) sauce = urllib.request.urlopen(url).read() (1) (2) soup = bs.BeautifulSoup(sauce,"html.parser") (2)
我无法打开网址并阅读需要身份验证的网页。我如何实现这样的目标:
(3) sauce = urllib.request.urlopen(url, auth=(username, password)).read() (3) instead of (1)
看看官方文档中的如何使用urllib软件包获取Internet资源:
# create a password manager password_mgr = urllib.request.HTTPPasswordMgrWithDefaultRealm() # Add the username and password. # If we knew the realm, we could use it instead of None. top_level_url = "http://example.com/foo/" password_mgr.add_password(None, top_level_url, username, password) handler = urllib.request.HTTPBasicAuthHandler(password_mgr) # create "opener" (OpenerDirector instance) opener = urllib.request.build_opener(handler) # use the opener to fetch a URL opener.open(a_url) # Install the opener. # Now all calls to urllib.request.urlopen use our opener. urllib.request.install_opener(opener)