我尝试使用asyncore做简单的异步http客户端:
这段代码工作正常,输出是(快速):
www.gmail.com : recv http code: 301
www.yandex.ru : recv http code: 200
www.python.org : recv http code: 200
www.google.ru : recv http code: 200
www.gravatar.com : recv http code: 302
www.com.com : recv http code: 302
www.yahoo.com : recv http code: 302
www.bom.com : recv http code: 301
但是,与我不存在主机的取消注释:
#c = AsyncHTTP('http://www.no-such-host.ru') #!this line breaks execution!
执行中断,代码挂起一段时间,输出部分数据并挂起,没有最后的数据输出:
connection error: [Errno -5] No address associated with hostname
www.gmail.com : recv http code: 301
www.yandex.ru : recv http code: 200
www.yahoo.com : recv http code: 302
www.com.com : recv http code: 302
www.bom.com : recv http code: 301
www.gravatar.com : recv http code: 302
…
一些主机在这里丢失,并在开始时长时间延迟.
为什么会这样,以及如何解决这个问题?
# coding=utf-8
import asyncore
import string,socket
import StringIO
import mimetools,urlparse
class AsyncHTTP(asyncore.dispatcher):
# HTTP requestor
def __init__(self,uri):
asyncore.dispatcher.__init__(self)
self.uri = uri
# turn the uri into a valid request
scheme,host,path,params,query,fragment = urlparse.urlparse(uri)
assert scheme == "http","only supports HTTP requests"
try:
host,port = string.split(host,":",1)
port = int(port)
except (TypeError,ValueError):
port = 80 # default port
if not path:
path = "/"
if params:
path = path + ";" + params
if query:
path = path + "?" + query
self.request = "GET %s HTTP/1.0\r\nHost: %s\r\n\r\n" % (path,host)
self.host = host
self.port = port
self.status = None
self.header = None
self.http_code = None
self.data = ""
# get things going!
self.create_socket(socket.AF_INET,socket.SOCK_STREAM)
#self.connect((host,port))
#return
try:
self.connect((host,port))
except Exception,e:
self.close()
self.handle_connect_expt(e)
def handle_connect(self):
self.send(self.request)
def handle_expt(self):
print "handle_expt error!"
self.close()
def handle_error(self):
print "handle_error error!"
self.close()
def handle_connect_expt(self,expt):
print "connection error:",expt
def handle_code(self):
print self.host," : ","recv http code: ",self.http_code
def handle_read(self):
data = self.recv(2048)
#print data
if not self.header:
self.data = self.data + data
try:
i = string.index(self.data,"\r\n\r\n")
except ValueError:
return # continue
else:
# parse header
fp = StringIO.StringIO(self.data[:i+4])
# status line is "HTTP/version status message"
status = fp.readline()
self.status = string.split(status," ",2)
self.http_code = self.status[1]
self.handle_code()
# followed by a rfc822-style message header
self.header = mimetools.Message(fp)
# followed by a newline,and the payload (if any)
data = self.data[i+4:]
self.data = ""
#header recived
#self.close()
def handle_close(self):
self.close()
c = AsyncHTTP('http://www.python.org')
c = AsyncHTTP('http://www.yandex.ru')
c = AsyncHTTP('http://www.google.ru')
c = AsyncHTTP('http://www.gmail.com')
c = AsyncHTTP('http://www.gravatar.com')
c = AsyncHTTP('http://www.yahoo.com')
c = AsyncHTTP('http://www.com.com')
c = AsyncHTTP('http://www.bom.com')
#c = AsyncHTTP('http://www.no-such-host.ru') #!this line breaks execution!
asyncore.loop()
ps:我的系统是ubuntu 11.10 python 2.7.2
最佳答案