我制作了一个解析文本文件和并行下载数据的程序.当在9个或更少的线程中运行下载方法时,程序没有错误.但是当在10个或更多线程中运行该方法时,程序抛出“`initialize”:getaddrinfo:名称或服务未知(SocketError)“错误.我尝试了一些并行运行的算法,但同样的问题也出现了.
当将“名称或服务未知”错误发生时,我将传递给’open’方法(open-uri)的url放入浏览器并确认此url有效并收到正确的数据.这是部分代码.
当将“名称或服务未知”错误发生时,我将传递给’open’方法(open-uri)的url放入浏览器并确认此url有效并收到正确的数据.这是部分代码.
jobs = [] aps = [] .... #jobs are pushed into jobs[] .... max_thread = 15 loop do ary_threads = [] max_thread.times do |i| break if jobs.size == 0 job = jobs.pop ary_threads << Thread.start { begin request(job[0],job[1]).each do |ap| #in "request" method,open(url)are called aps.push(ap) end end } end ary_threads.each { |th| th.join } break if jobs.size == 0 end
而错误是
/usr/lib/ruby/1.9.1/net/http.rb:762:in `initialize': getaddrinfo: Name or service not known (SocketError) from /usr/lib/ruby/1.9.1/net/http.rb:762:in `open' from /usr/lib/ruby/1.9.1/net/http.rb:762:in `block in connect' from /usr/lib/ruby/1.9.1/timeout.rb:54:in `timeout' from /usr/lib/ruby/1.9.1/timeout.rb:99:in `timeout' from /usr/lib/ruby/1.9.1/net/http.rb:762:in `connect' from /usr/lib/ruby/1.9.1/net/http.rb:755:in `do_start' from /usr/lib/ruby/1.9.1/net/http.rb:744:in `start' from /usr/lib/ruby/1.9.1/open-uri.rb:306:in `open_http' from /usr/lib/ruby/1.9.1/open-uri.rb:775:in `buffer_open' from /usr/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop' from /usr/lib/ruby/1.9.1/open-uri.rb:201:in `catch' from /usr/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop' from /usr/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri' from /var/lib/gems/1.9.1/gems/open-uri-cached-0.0.5/lib/open-uri/cached.rb:10:in `open_uri' from /usr/lib/ruby/1.9.1/open-uri.rb:677:in `open' from /usr/lib/ruby/1.9.1/open-uri.rb:33:in `open' from Test1.rb:42:in `request' from Test1.rb:77:in `block (3 levels) in <main>'
为什么会这样?有谁遇到过类似的问题?
请帮我!
第一个问题后3小时,我找到了临时解决方案.
如果我在’begin’方法中将’open’方法与’begin~scue-retry~end’夹在一起,那么第二次’open’调用时就不会发生错误.这就是代码.
begin response = open(url) rescue Exception puts url puts "retrying" retry end
解决方法
我想这可能是因为线程之间的竞争条件.尝试原子地进行操作.把互斥锁.
@mutex = Mutex.new @mutex.syncronize do ... ary_threads << Thread.start { begin request(job[0],open(url)are called aps.push(ap) end end } ... end