urllib2 for making HTTP requests. Don’t use
httplib is no fun
There are some dumb limitations:
HTTPConnection can only be used once
- You can only call
getresponse() on an
- There’s no
HTTPRequest object as such; you throw all your args into a single
There are one or two little gotchas:
### NO NO NO
conn = httplib.HTTPConnection("www.example.com/foo")
conn = httplib.HTTPConnection("www.example.com")
resp = conn.request("GET", "/foo")
All in all
httplib was cumbersome for interactive development.
A friend speculated that this was a thin wrapper around the C APIs. I don’t have anything against C, but this is Python.
urllib and urllib2 instead
urllib2 have what you need.
quote(), which you want if you want to URL encode a string. More convenient is
urlencode(), which takes a map or sequence, and outputs the usual
urllib2 supplies a
Request object. The constructor is a bit of a bear, but most of it is optional; you can call
add_header() for to add stuff later.
After that, call
urllib2.urlopen() with your
Request object. The
Response object is a file-like. The body is the file, and methods like
info() give you headers, et al.
It seems like on such as a 403 you’ll get an exception (
HTTPError) and your response will be empty. IMHO this is dumb. An exception should be in the case of a malformed, incomplete, or otherwise bogus response. A 403 is a legit response. It may have headers, etc. I should be able to look at that through the response object, not forced to paw through exception info trying to find the right fields.