[#1711] Re: open-uri patch, added progress_proc hook — "T. Onoma" <transami@...>

Tanaka Akira:

22 messages 2003/11/19
[#1737] Re: open-uri patch, added progress_proc hook — Mathieu Bouchard <matju@...> 2003/11/23

[#1739] Re: open-uri patch, added progress_proc hook — Mathieu Bouchard <matju@...> 2003/11/23

[#1740] Re: open-uri patch, added progress_proc hook — "T. Onoma" <transami@...> 2003/11/23

On Sunday 23 November 2003 08:26 pm, Mathieu Bouchard wrote:

[#1741] Re: open-uri patch, added progress_proc hook — Mathieu Bouchard <matju@...> 2003/11/23

[#1718] Re: open-uri patch, added progress_proc hook — Elliott Hughes <ehughes@...>

22 messages 2003/11/21
[#1722] Re: open-uri patch, added progress_proc hook — Tanaka Akira <akr@...17n.org> 2003/11/22

In article <AD4480A509455343AEFACCC231BA850F17C434@ukexchange>,

[#1724] Re: open-uri patch, added progress_proc hook — "T. Onoma" <transami@...> 2003/11/22

On Saturday 22 November 2003 04:34 pm, Tanaka Akira wrote:

[#1726] Re: open-uri patch, added progress_proc hook — Tanaka Akira <akr@...17n.org> 2003/11/23

In article <200311221024.05642.transami@runbox.com>,

[#1731] Re: open-uri patch, added progress_proc hook — "T. Onoma" <transami@...> 2003/11/23

On Sunday 23 November 2003 02:24 am, Tanaka Akira wrote:

[#1732] Re: open-uri patch, added progress_proc hook — Tanaka Akira <akr@...17n.org> 2003/11/23

In article <200311230325.21687.transami@runbox.com>,

[#1733] Re: open-uri patch, added progress_proc hook — "T. Onoma" <transami@...> 2003/11/23

On Sunday 23 November 2003 03:10 pm, Tanaka Akira wrote:

[#1750] Re: open-uri patch, added progress_proc hook — Tanaka Akira <akr@...17n.org> 2003/11/24

In article <200311230648.41003.transami@runbox.com>,

[#1759] Re: open-uri patch, added progress_proc hook — Sean E Russell <ser@...> 2003/11/24

On Monday 24 November 2003 03:19, Tanaka Akira wrote:

[#1762] Re: open-uri patch, added progress_proc hook — "Nathaniel Talbott" <nathaniel@...> 2003/11/24

Sean E Russell [mailto:ser@germane-software.com] wrote:

[#1753] gc_sweep under 1.8 ... not syck.so — Richard Kilmer <rich@...>

We still encountered a gc_sweep in our use of Ruby 1.8 on Linux (v8).

16 messages 2003/11/24
[#1754] Re: gc_sweep under 1.8 ... not syck.so — ts <decoux@...> 2003/11/24

>>>>> "R" == Richard Kilmer <rich@infoether.com> writes:

[#1757] Re: gc_sweep under 1.8 ... not syck.so — Richard Kilmer <rich@...> 2003/11/24

Yes, there are several (Ruby) threads working during this gc_sweep.

[#1758] Re: gc_sweep under 1.8 ... not syck.so — ts <decoux@...> 2003/11/24

>>>>> "R" == Richard Kilmer <rich@infoether.com> writes:

[#1763] Re: gc_sweep under 1.8 ... not syck.so — Richard Kilmer <rich@...> 2003/11/24

of course this effects 300 machines ;-)

[#1755] Re: Controlled block variables — Jamis Buck <jgb3@...>

On Mon, 2003-11-24 at 02:04, T. Onoma wrote:

26 messages 2003/11/24
[#1756] Re: Controlled block variables — "T. Onoma" <transami@...> 2003/11/24

On Monday 24 November 2003 05:22 pm, Jamis Buck wrote:

[#1760] Re: Controlled block variables — Sean E Russell <ser@...> 2003/11/24

On Monday 24 November 2003 11:51, T. Onoma wrote:

[#1761] Re: Controlled block variables — "T. Onoma" <transami@...> 2003/11/24

On Monday 24 November 2003 06:40 pm, Sean E Russell wrote:

open-uri patch, added progress_proc hook

From: "T. Onoma" <transami@...>
Date: 2003-11-14 23:09:51 UTC
List: ruby-core #1674
hi, i am not familiear with any procedures for submitting changes to a ruby lib, so fogive me if I have not done this correctly. please email me if i need to do something differnetly. thanks.

i have made a small modification to open-uri to allow it to execute a proc in the data fetching loops. this has proved very useful to me in providing a progress reported download. (i used Ruby/ProgressBar). Note that in the future it might be nice to get open-uri to work more like a real IO.

- T. Onoma

here is the diff -u (let me know if i need to prove in different way):

--- open-uri.rb	2003-05-15 20:19:09.000000000 +0200
+++ /usr/lib/ruby/1.8/open-uri.rb	2003-11-14 09:18:36.000000000 +0100
@@ -54,6 +54,9 @@
 #
 #Author:: Tanaka Akira <akr@m17n.org>

+#ChangeLog
+#  2003-11-14 Added pogress_proc hook.  (T. Onoma <transami@runbox.com>)
+
 require 'uri'
 require 'stringio'
 require 'time'
@@ -63,7 +66,7 @@
   alias open_uri_original_open open # :nodoc:

   # makes possible to open URIs.
-  # If the first argument is URI::HTTP, URI::FTP or
+  # If the first argument is URI::HTTP, URI::FTP or
   # String beginning with http:// or ftp://,
   # the URI is opened.
   # The opened file object is extended by OpenURI::Meta.
@@ -93,6 +96,7 @@
   def OpenURI.open_uri(name, *rest) # :nodoc:
     uri = URI::Generic === name ? name : URI.parse(name)
     mode, perm, rest = OpenURI.scan_open_optional_arguments(*rest)
+    progress_proc = rest.shift if !rest.empty? && Proc === rest.first
     options = rest.shift if !rest.empty? && Hash === rest.first
     raise ArgumentError.new("extra arguments") if !rest.empty?

@@ -102,7 +106,7 @@
       raise ArgumentError.new("invalid access mode #{mode} (#{uri.class} resource is read only.)")
     end

-    io = open_loop(uri, options || {})
+    io = open_loop(uri, options || {}, progress_proc)
     if block_given?
       begin
         yield io
@@ -114,7 +118,7 @@
     end
   end

-  def OpenURI.open_loop(uri, options) # :nodoc:
+  def OpenURI.open_loop(uri, options, progress_proc=nil) # :nodoc:
     header = {}
     options.each {|k, v|
       if String === k
@@ -140,9 +144,9 @@
     begin
       buf = Buffer.new
       if proxy_uri = find_proxy.call(uri)
-        proxy_uri.proxy_open(buf, uri, header)
+        proxy_uri.proxy_open(buf, uri, header, progress_proc)
       else
-        uri.direct_open(buf, header)
+        uri.direct_open(buf, header, progress_proc)
       end
     rescue Redirect
       loc = $!.uri
@@ -353,15 +357,18 @@
   end

   class HTTP
-    def direct_open(buf, header) # :nodoc:
-      proxy_open(buf, request_uri, header)
+    def direct_open(buf, header, progress_proc) # :nodoc:
+      proxy_open(buf, request_uri, header, progress_proc)
     end

-    def proxy_open(buf, uri, header) # :nodoc:
+    def proxy_open(buf, uri, header, progress_proc=nil) # :nodoc:
       require 'net/http'
-      resp = Net::HTTP.start(self.host, self.port) {|http|
-               http.get(uri.to_s, header) {|str| buf << str}
-             }
+      resp = Net::HTTP.start(self.host, self.port) { |http|
+        http.get(uri.to_s, header) { |str|
+          buf << str
+          progress_proc.call(buf.io.pos) if progress_proc
+        }
+      }
       io = buf.io
       io.rewind
       io.status = [resp.code, resp.message]
@@ -382,9 +389,9 @@
   end

   class FTP
-    def direct_open(buf, header) # :nodoc:
+    def direct_open(buf, header, progress_proc=nil) # :nodoc:
       require 'net/ftp'
-      # xxx: header is discarded.
+      # xxx: header is discarded.
       # todo: extract user/passwd from .netrc.
       user = 'anonymous'
       passwd = nil
@@ -392,7 +399,10 @@

       ftp = Net::FTP.open(self.host)
       ftp.login(user, passwd)
-      ftp.getbinaryfile(self.path, '/dev/null', Net::FTP::DEFAULT_BLOCKSIZE) {|str| buf << str}
+      ftp.getbinaryfile(self.path, '/dev/null', Net::FTP::DEFAULT_BLOCKSIZE) { |str|
+         buf << str
+         progress_proc.call(buf.io.pos) if progress_proc
+      }
       ftp.close
       buf.io.rewind
     end



In This Thread

Prev Next