ruby-list

ちょっと前に話題になったものです．parser generator はおいといて，lex
もどきをつくってみました．すんごい手抜きだけど，そう悪くないかも．

# 行頭に `#' がある行だけはコメントになるはずです．
# 最初のモード宣言までは好きな ruby の式が書けます．まるのまま出力にコ
# ピーされる．

@var = nil                              # Scanner というクラスの
                                        # initialize になるのでそのつ
                                        # もりで．

# <name> というのが行頭にあるとモード宣言とみなします．`name' はそのま
# まメソッド名になるのでちゃんとするように．必ずただ一つの <default>
# が必要です．無かったり，複数あったりするとどうなるかわかりません．

<default>

# 行頭が `#' と空白文字以外の場合はその行全てがパターン．

pattern

# 空白文字で始まる行は pattern に対する action．

    expression
    RETURN value, :new_mode

# RETURN は value を返し新たなモード（デフォルトは default）に遷移しま
# す．

<new_mode>
pattern
    expression
    GOTO :default

# GOTO は状態の遷移のみを行います．

    * マッチした文字列は matched という変数でアクセスできます．
    * みての通りばかばかしい程簡単な構造です．
    * そのため入力が期待している形に外れると激しく破綻します．

シンプルというか，冗談みたい (^^;

# ところで最後につけたサンプルから作った scanner は何故か余分に true
# を返してくるようなのですが，何か変でしょうか?

===========================================================================
  柳川和久 @ 東大阪市 . 大阪府
  kjana@os.xaxon.ne.jp                                  December 25, 1998
「くだらないダジャレで気が抜けて成仏するのではなく?」
「ではなく!!」


#!/usr/local/bin/ruby

if ARGV[0]
  inf = File.open(ARGV[0])
  ARGV.shift
else
  inf = $stdin
end

if ARGV[0]
  out = File.open(ARGV[0], "w")
  ARGV.shift
else
  out = $stdout
end

out.puts <<SCANNER_HEAD
# Scanner generated by #{$0}

class ScanGoto < Exception
end

class ScanReturn < Exception
  def initialize(val)
    @val = val
  end

  attr_reader :val
end

class Scanner
  def initialize(__in__ = $stdin)
    case __in__
    when IO
      @__line__ = ""
      @__inf__ = __in__
    when String
      @__line__ = __in__
      @__inf__ = nil
    else
      raise TypeError, "IO or String is expected."
    end

    @__mode__ = :default

SCANNER_HEAD

while inf.gets
  break if /^<([_a-zA-Z]\w*)>/          # mode declarations
  next if /^\s*$/

  # until first mode declaration, any string is copied literally
  # as initialize sequence.
  out.puts "    #{$_.chomp!}"
end

if $1
  out.puts "  end"
  out.puts
  out.puts "  def #{$1}"
  out.puts "    case @__line__"
else
  out.puts "  end"
end

mode = []

while inf.gets
  case $_.chomp!
  when /(?:^\s*$)|(?:^#.*)/
    # ignore
  when /^<([_a-zA-Z]\w*)>/              # mode declarations
    mode << $1
    out.puts "    else"
    out.puts "      c = @__line__[0]"
    out.puts "      @__line__ = @__line__[1..-1]"
    out.puts "      return_with_val c"
    out.puts "    end"
    out.puts "  end"
    out.puts
    out.puts "  def #{$1}"
    out.puts "    case @__line__"
  when /^\s+.*/                         # actions
    # action strings are copied literally.
    out.puts "      #{$_}"
  else                                  # patterns
    out.puts "    when /\\A(#{$_})/"
    out.puts "      @__line__ = $'"
    out.puts "      matched = $1"
  end
end

out.puts "    else"
out.puts "      c = @__line__[0]"
out.puts "      @__line__ = @__line__[1..-1]"
out.puts "      return_with_val c"
out.puts "    end"
out.puts "  end"

out.puts <<SCANNER_TAIL

  def scan
    begin
      @__line__ = @__inf__.gets if @__line__.empty? and not @__inf__.nil?
      return nil if @__line__.nil?
      while not @__line__.empty?
        send @__mode__
      end
      true
    rescue ScanGoto
      retry
    rescue ScanReturn
      return $!.val
    end
  end

  def goto(__mode__)
    raise TypeError, "no such mode." if not self.respond_to? __mode__
    @__mode__ = __mode__
    raise ScanGoto.new
  end

  def return_with_val(__val__, __mode__ = :default)
    raise TypeError, "no such mode." if not self.respond_to? __mode__
    @__mode__ = __mode__
    raise ScanReturn.new(__val__)
  end

  alias GOTO goto
  alias RETURN return_with_val
end

if $0 == __FILE__
  lex = Scanner.new

  while tok = lex.scan
    p tok
  end
end
SCANNER_TAIL

########################################################################

# sample input

@part = ""

<default>
[a-zA-Z_]\w*
	RETURN [:id, matched]
\d*\.\d+(?:[eE][+-]?\d+)?
	RETURN [:float, matched]
\d+
	RETURN [:int, matched]
"
	@part << "\""
	GOTO :string
\s+
	# silently ignored
.
	RETURN matched

<string>
\\"
	@part << "\""
"
	@part << "\""
	part = @part
	@part = ""
	RETURN [:string, part], :default
[^"\\]+
	@part << matched

Thread

Prev Next

In This Thread

Prev Next

[#11156] How to delete methods from superclass? — Clemens Hintze <c.hintze@...>

[#11164] ruby-pcap — Masaki Fukushima <fukusima@...>

[#11165] usage of Time class — Noritsugu Nakamura <nnakamur@...>

[#11168] find.rb replacement(?) — sekita-n@... (Sekitani Nobutaka)

[#11170] 安直な perl 拡張 — Yoshinori Toki <toki@...>

[#11172] sample/cal.rb — Tadayoshi Funaba <tadf@...>

[#11199] about ext/{tk,tcltklib}/Makefile — Noritsugu Nakamura <nnakamur@...>

[#11203] announcement of ruby list? — ozawa@...

[#11206] variable $0 — ISII takesi <isii@...>

[#11207] $LOAD_PATH — ISII takesi <isii@...>

[#11208] ruby の本の出版はまだでしょうか？ — "Hirotaka Ishida" <h-ishida@...>

[#11216] manual bug for option -l — Inaba Hiroto <inaba@...>

[#11219] クラス定数の継承 — Yoshiki WADA <wada@...>

[#11225] ruby and ISP — Noritsugu Nakamura <nnakamur@...>

[#11245] undefined local variable or method `__END__' (Cygwin 版 1.1c9) — たむら けんいち <t9655832@...>

[#11250] Ruby 用語集 — matz@... (Yukihiro Matsumoto)

[#11252] Re: Ruby 用語集 — greentea@...2.so-net.ne.jp (Tomoyuki Kosimizu)

[#11253] shttpsrv-1.0.4 + servlet — Masatoshi SEKI <m_seki@...>

[#11257] comp.lang.* — Noritsugu Nakamura <nnakamur@...>

[#11258] require error? — OHARA Shigeki <os@...>

[#11267] CGI.rb, httplib.rb, ... — Wakou Aoyama <wakou@...>

[#11269] 京都 (Re: [ruby-dev:3789] Re: List()) — matz@... (Yukihiro Matsumoto)

[#11272] Re: 京都 (Re: [ruby-dev:3789] Re: List()) — tomokuni@... (Tetsuo TOMOKUNI)

[#11276] pstore.rb — keiju@... (Keiju ISHITSUKA)

[#11292] [CFP] UML'99 — Jean-Michel.Bruel@... (Jean-Michel BRUEL)

[#11293] ruby-talk — Sinichiro Dezawa <dezawa@...>

[#11309] ext/ICQ 0.1 released — jkuro@...

[#11316] find.rb bug(?) fix — sekita-n@... (Sekitani Nobutaka)

[#11323] How to use GTK::timeout? — Takao KAWAMURA <kawamura@...>

[#11326] Forward: ANNOUNCEMENT: filelock.rb (long) — matz@... (Yukihiro Matsumoto)

[#11337] Re: ruby on Win32 (Re: Ruby 用語集 ) — 助田 雅紀 <masaki.suketa@...>

[#11348] sub or gsub の第２引数について — 石田 宏香 <h-ishida@...>

[#11350] Re: ruby on Win32 (Re: Ruby 用語集 ) — 助田 雅紀 <masaki.suketa@...>

[#11371] irb 0.2.1.24 — keiju@... (Keiju ISHITSUKA)

[#11373] ruby 1.1d0 released — matz@... (Yukihiro Matsumoto)

[#11382] Hi. — kuma@... (Koji Kumagai)

[#11387] exceptions — gotoken@... (GOTO Kentaro)

[#11390] fep.rb — ttate@...

[#11391] gtk — Takehama Hirohisa <thiro@...>

[#11393] mod_ruby — shugo@... (Shugo Maeda)

[#11397] fold.rb — kjana@... (YANAGAWA Kazuhisa)

[#11404] djgpp 版と cygnusWin32 版の違いについて — 石田 宏香 <h-ishida@...>

[#11438] "Perl Conference" article on UNIX USER #1 — Ryo HAYASAKA <hayasaka@...>

[#11444] ruby 1.2 released — matz@... (Yukihiro Matsumoto)

[#11447] "\\1".upcase — gotoken@... (GOTO Kentaro)

[#11454] Dir クラスの delete メソッド — URA Takefumi <tura@...>

[#11456] qsort() for SunOS 4.1.x — gotoken@... (GOTO Kentaro)

[#11458] Scanner generator — kjana@... (YANAGAWA Kazuhisa)

[#11464] ruby and IDE — Noritsugu Nakamura <nnakamur@...>

[#11467] Ruby/IRC Library — Hiroshi Igarashi <igarashi@...>

[#11468] Ruby 1.2 FreeBSD port — Yasuhiro Fukuma <yasuf@...>

[#11475] subscribe — Kozaki Tsuneaki <zr2t-kzk@...>

[#11481] UNIX MAGAZINE — ttate@...

[#11483] ftpmirror-1.1 — Masatoshi SEKI <m_seki@...>

[#11491] Ruby.texinfo — Koji Arai <JCA02266@...>

[ruby-list:11458] Scanner generator

Thread

In This Thread

[#11245] undefined local variable or method `END' (Cygwin 版 1.1c9) — たむら　けんいち <t9655832@...>

[#11337] Re: ruby on Win32 (Re: Ruby 用語集 ) — 助田　雅紀 <masaki.suketa@...>

[#11348] sub or gsub の第２引数について — 石田宏香 <h-ishida@...>

[#11350] Re: ruby on Win32 (Re: Ruby 用語集 ) — 助田　雅紀 <masaki.suketa@...>

[#11404] djgpp 版と cygnusWin32 版の違いについて — 石田宏香 <h-ishida@...>