From 73da367263ac9b59ebdb531d7d4cc9302932099f Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sun, 1 Sep 2013 00:28:21 +0200 Subject: cleanup changelog for 1.1 --- Changes.textile | 69 +++++++++++++++++++++++++++++++++++---------------------- 1 file changed, 43 insertions(+), 26 deletions(-) diff --git a/Changes.textile b/Changes.textile index cd4443a..f8081da 100644 --- a/Changes.textile +++ b/Changes.textile @@ -4,40 +4,57 @@ p=. _This files lists all changes in the CodeRay library since the 0.9.8 release h2. Changes in 1.1 -* New scanner: Lua [#21, #22, thanks to Quintus] -* New scanner: Sass [#93] -* New scanner: Go [#28, thanks to Eric Guo and Nathan Youngman] -* New scanner: Taskpaper [#39, thanks to shimomura] +New scanners: + +* Go [#28, thanks to Eric Guo and Nathan Youngman] +* Lua [#21, #22, thanks to Quintus] +* Sass [#93] +* Taskpaper [#39, thanks to shimomura] + +More new stuff: + +* Add .xaml file type [#121, thanks to Kozman Bálint] +* New token type @:id@ for CSS/Sass [#27] +* New token type @:done@ for Taskpaper [#39] +* New token type @:map@ for Lua, introducing a nice nested-shades trick [#22, thanks to Quintus and Nathan Youngman] +* New token type @:unknown@ for Debug scanner +* New DebugLint encoder that checks for empty tokens and correct nesting + +Improvements: + +* CSS scanner uses @:id@ and @:tag@ now [#27] * Diff scanner: Highlight inline changes in multi-line changes [#99] * JavaScript scanner: Highlight multi-line comments in diff correctly +* HTML scanner displays style tags and attributes now [#145] * Ruby scanner: Accept %i and %I symbol lists (Ruby 2.0) [thanks to Nathan Youngman] * Ruby scanner: Accept keywords as Ruby 1.9 hash keys [#126] -* HTML scanner displays style tags and attributes now [#145] -* Remove double-click toggle handler from HTML table output + +Removed: + +* @Tokens#dump@, @Tokens.load@, @Tokens::Undumping@, and @zlib@ dependency +* Double-click toggle handler from HTML table output + +Fixes: + * Fixes to CSS scanner (floats, pseudoclasses, nth-child) [#143] * Fixed empty tokens and unclosed token groups in HTML, CSS, Diff, Goovy, PHP, Raydebug, Ruby, SQL, and YAML scanners [#144] -* Added @:keep_state@ functionality to more scanners [#116] -* CSS scanner uses @:id@ and @:tag@ now [#27] -* Removed @Tokens#dump@, @Tokens.load@, @Tokens::Undumping@, and @zlib@ dependency. Nobody was using this, right? -* Add .xaml file type [#121, thanks to Kozman Bálint] -* @CodeRay::TokenKinds@ should not be frozen [#130, thanks to Gavin Kistner] -* New token type @:id@ for CSS/Sass [#27] -* New token type @:done@ for Taskpaper [#39] -* New token type @:map@ for Lua, introducing a nice nested-shades trick [#22, thanks to Quintus and Nathan Youngman] -* New token type @:unknown@ for Debug scanner -* Display line numbers in HTML @:table@ mode even for single-line code (remove special case) [#41, thanks to Ariejan de Vroom] -* Override Bootstrap's @pre { word-break: break-all }@ styling for line numbers [#102, thanks to lightswitch05] * Fixed @:docstring@ token type style * Fixed several problems related to Hash caches and dynamic Symbol creation that might have been exploited by an attacker [#148] -* @PluginHost@ now works with Strings instead of Symbols internally (to avoid using @#to_sym@) -* The @Debug@ scanner maps unknown token kinds to @:unknown@ (to avoid creating Symbols based on possibly unsafe input) -* The @Raydebug@ scanner highlights unknown token kinds as @:plain@ -* @Plugin@ does not warn about fallback when default is defined -* Fixed @HTML@ encoder when output is a StringIO (eg. when using @-HTML@) -* @HTML@ encoder will not warn about unclosed token groups at the end of the stream -* @Debug@ encoder refactored; use @DebugLint@ if you want strict checking now -* @Debug@ encoder will not warn about errors in the token stream -* New @DebugLint@ encoder that checks for empty tokens and correct nesting +* Fixed HTML encoder when output is a StringIO (eg. when using @-HTML@ as a command line parameter) +* TokenKinds should not be frozen [#130, thanks to Gavin Kistner] +* Display line numbers in HTML @:table@ mode even for single-line code (remove special case) [#41, thanks to Ariejan de Vroom] +* Override Bootstrap's @pre { word-break: break-all }@ styling for line numbers [#102, thanks to lightswitch05] +* HTML encoder will not warn about unclosed token groups at the end of the stream + +Internals: + +* The Debug scanner maps unknown token kinds to @:unknown@ (to avoid creating Symbols based on possibly unsafe input) +* The Raydebug scanner highlights unknown token kinds as @:plain@ +* Debug encoder refactored; use DebugLint if you want strict checking now +* Debug encoder will not warn about errors in the token stream +* Plugin does not warn about fallback when default is defined +* PluginHost now works with Strings instead of Symbols internally (to avoid using @#to_sym@) +* Added @:keep_state@ functionality to more scanners [#116] h2. Changes in 1.0.9 -- cgit v1.2.1 From a48037b85a12228431b32103786456f36beb355f Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sun, 1 Sep 2013 01:01:35 +0200 Subject: final cleanup --- Changes.textile | 59 ++++++++++++++++++++++++++-------------------- README.markdown | 2 +- lib/coderay/scanners/go.rb | 1 - 3 files changed, 34 insertions(+), 28 deletions(-) diff --git a/Changes.textile b/Changes.textile index f8081da..8e388e0 100644 --- a/Changes.textile +++ b/Changes.textile @@ -13,48 +13,58 @@ New scanners: More new stuff: -* Add .xaml file type [#121, thanks to Kozman Bálint] -* New token type @:id@ for CSS/Sass [#27] -* New token type @:done@ for Taskpaper [#39] -* New token type @:map@ for Lua, introducing a nice nested-shades trick [#22, thanks to Quintus and Nathan Youngman] -* New token type @:unknown@ for Debug scanner -* New DebugLint encoder that checks for empty tokens and correct nesting +* @.xaml@ file type [#121, thanks to Kozman Bálint] +* recognize @Guardfile@, @Vagrantfile@, and @Appraisals@ as Ruby files [#121, thanks to Kozman Bálint] +* new token kind @:id@ for CSS/Sass [#27] +* new token kind @:done@ for Taskpaper [#39] +* new token kind @:map@ for Lua, introducing a nice nested-shades trick [#22, thanks to Quintus and Nathan Youngman] +* new token kind @:unknown@ for Debug scanner +* new DebugLint encoder that checks for empty tokens and correct nesting Improvements: * CSS scanner uses @:id@ and @:tag@ now [#27] * Diff scanner: Highlight inline changes in multi-line changes [#99] * JavaScript scanner: Highlight multi-line comments in diff correctly +* JSON scanner: simplify key/value heuristic, using look-ahead instead of a stack * HTML scanner displays style tags and attributes now [#145] -* Ruby scanner: Accept %i and %I symbol lists (Ruby 2.0) [thanks to Nathan Youngman] -* Ruby scanner: Accept keywords as Ruby 1.9 hash keys [#126] +* Ruby scanner: Accept @%i(…)@ and @%I(…)@ symbol lists (Ruby 2.0) [thanks to Nathan Youngman] +* Ruby scanner: Accept keywords as Ruby hash keys [#126] +* performance improvements to several scanners and encoders, especially Terminal and HTML +* added @:keep_state@ functionality to more scanners so they work nicely with diff now [#116] +* refactoring and cleanup to achieve better "Code Climate" ratings (but I don't really care) +* updated and cleaned up the documentation, +* documented list of TokenKinds +* Alpha style: tweaked colors for @.binary@, @.local-variable@, and @.predefined-type@ +* @rake generate@ supports Git now instead of Subversion Removed: * @Tokens#dump@, @Tokens.load@, @Tokens::Undumping@, and @zlib@ dependency -* Double-click toggle handler from HTML table output +* double-click toggle handler from HTML table output +* @rake_helpers@, @sample@ directories and several other ancient garbage Fixes: -* Fixes to CSS scanner (floats, pseudoclasses, nth-child) [#143] -* Fixed empty tokens and unclosed token groups in HTML, CSS, Diff, Goovy, PHP, Raydebug, Ruby, SQL, and YAML scanners [#144] -* Fixed @:docstring@ token type style -* Fixed several problems related to Hash caches and dynamic Symbol creation that might have been exploited by an attacker [#148] -* Fixed HTML encoder when output is a StringIO (eg. when using @-HTML@ as a command line parameter) +* fixes to CSS scanner (floats, pseudoclasses, nth-child) [#143] +* fixed empty tokens and unclosed token groups in HTML, CSS, Diff, Goovy, PHP, Raydebug, Ruby, SQL, and YAML scanners [#144] +* fixed @:docstring@ token type style +* fixed several infinite Hash caches and dynamic Symbol creation that might have been exploited by an attacker [#148] +* fixed HTML encoder when output is a StringIO (eg. when using @-HTML@ as a command line parameter) * TokenKinds should not be frozen [#130, thanks to Gavin Kistner] -* Display line numbers in HTML @:table@ mode even for single-line code (remove special case) [#41, thanks to Ariejan de Vroom] -* Override Bootstrap's @pre { word-break: break-all }@ styling for line numbers [#102, thanks to lightswitch05] +* display line numbers in HTML @:table@ mode even for single-line code (remove special case) [#41, thanks to Ariejan de Vroom] +* override Bootstrap's @pre { word-break: break-all }@ styling for line numbers [#102, thanks to lightswitch05] * HTML encoder will not warn about unclosed token groups at the end of the stream +* fixed problem with coderay/version.rb being loaded twice Internals: -* The Debug scanner maps unknown token kinds to @:unknown@ (to avoid creating Symbols based on possibly unsafe input) -* The Raydebug scanner highlights unknown token kinds as @:plain@ -* Debug encoder refactored; use DebugLint if you want strict checking now -* Debug encoder will not warn about errors in the token stream -* Plugin does not warn about fallback when default is defined -* PluginHost now works with Strings instead of Symbols internally (to avoid using @#to_sym@) -* Added @:keep_state@ functionality to more scanners [#116] +* The Debug scanner maps unknown token kinds to @:unknown@ (to avoid creating Symbols based on possibly unsafe input). +* The Raydebug scanner highlights unknown token kinds as @:plain@. +* The Debug encoder refactored; use DebugLint if you want strict checking now.. +* The Debug encoder will not warn about errors in the token stream. +* Plugin does not warn about fallback when default is defined. +* PluginHost now works with Strings instead of Symbols internally (to avoid using @#to_sym@). h2. Changes in 1.0.9 @@ -464,6 +474,3 @@ The helper classes were cleaned up; see above for details. * *CHANGED* @Plugin@ API was simplified and stripped of all unnecessary features. * *CHANGED* Moved @GZip@ and @FileType@ libraries into @CodeRay@; cleaned them up. - - - diff --git a/README.markdown b/README.markdown index f333655..e23f603 100644 --- a/README.markdown +++ b/README.markdown @@ -28,4 +28,4 @@ html = CodeRay.scan("puts 'Hello, world!'", :ruby).div(:line_numbers => :table) ## Documentation -See [http://coderay.rubychan.de/doc/](http://coderay.rubychan.de/doc/). +See [rubydoc](http://rubydoc.info/gems/coderay). diff --git a/lib/coderay/scanners/go.rb b/lib/coderay/scanners/go.rb index 59473f6..99fdd63 100644 --- a/lib/coderay/scanners/go.rb +++ b/lib/coderay/scanners/go.rb @@ -1,7 +1,6 @@ module CodeRay module Scanners - # Scanner for Go, copy from c class Go < Scanner register_for :go -- cgit v1.2.1 From e93aae88985667189bb5b24ad0d5f54cb5fdba70 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Fri, 20 Sep 2013 17:46:39 +0200 Subject: remove outdated, misleading and buggy "samples" --- rake_tasks/test.rake | 6 - test/samples/README | 1 - test/samples/cache.actual | 2 - test/samples/cache.expected | 2 - test/samples/cache.rb | 12 - test/samples/count.expected | 1 - test/samples/count.rb | 10 - test/samples/css.actual | 127 ----- test/samples/css.expected | 127 ----- test/samples/css.rb | 4 - test/samples/div.actual | 17 - test/samples/div.expected | 17 - test/samples/div.rb | 19 - test/samples/encoder.actual | 65 --- test/samples/encoder.expected | 65 --- test/samples/encoder.rb | 40 -- test/samples/global_vars.actual | 0 test/samples/global_vars.diff | 5 - test/samples/global_vars.expected | 3 - test/samples/global_vars.rb | 13 - test/samples/global_vars2.expected | 10 - test/samples/global_vars2.rb | 28 -- test/samples/highlight.expected | 175 ------- test/samples/highlight.rb | 14 - test/samples/html.expected | 919 ------------------------------------- test/samples/html.rb | 394 ---------------- test/samples/html2.expected | 185 -------- test/samples/html2.rb | 11 - test/samples/html_list.expected | 160 ------- test/samples/html_list.rb | 12 - test/samples/load_encoder.expected | 8 - test/samples/load_encoder.rb | 25 - test/samples/load_scanner.expected | 8 - test/samples/load_scanner.rb | 25 - test/samples/more.expected | 2 - test/samples/more.rb | 205 --------- test/samples/scanner.expected | 16 - test/samples/scanner.rb | 36 -- test/samples/server.rb | 110 ----- test/samples/simple.expected | 1 - test/samples/simple.rb | 10 - test/samples/stream.rb | 25 - test/samples/stream2.expected | 2 - test/samples/stream2.rb | 8 - test/samples/suite.rb | 86 ---- test/samples/tokens.expected | 1 - test/samples/tokens.rb | 3 - 47 files changed, 3015 deletions(-) delete mode 100644 test/samples/README delete mode 100644 test/samples/cache.actual delete mode 100644 test/samples/cache.expected delete mode 100644 test/samples/cache.rb delete mode 100644 test/samples/count.expected delete mode 100644 test/samples/count.rb delete mode 100644 test/samples/css.actual delete mode 100644 test/samples/css.expected delete mode 100644 test/samples/css.rb delete mode 100644 test/samples/div.actual delete mode 100644 test/samples/div.expected delete mode 100644 test/samples/div.rb delete mode 100644 test/samples/encoder.actual delete mode 100644 test/samples/encoder.expected delete mode 100644 test/samples/encoder.rb delete mode 100644 test/samples/global_vars.actual delete mode 100644 test/samples/global_vars.diff delete mode 100644 test/samples/global_vars.expected delete mode 100644 test/samples/global_vars.rb delete mode 100644 test/samples/global_vars2.expected delete mode 100644 test/samples/global_vars2.rb delete mode 100644 test/samples/highlight.expected delete mode 100644 test/samples/highlight.rb delete mode 100644 test/samples/html.expected delete mode 100644 test/samples/html.rb delete mode 100644 test/samples/html2.expected delete mode 100644 test/samples/html2.rb delete mode 100644 test/samples/html_list.expected delete mode 100644 test/samples/html_list.rb delete mode 100644 test/samples/load_encoder.expected delete mode 100644 test/samples/load_encoder.rb delete mode 100644 test/samples/load_scanner.expected delete mode 100644 test/samples/load_scanner.rb delete mode 100644 test/samples/more.expected delete mode 100644 test/samples/more.rb delete mode 100644 test/samples/scanner.expected delete mode 100644 test/samples/scanner.rb delete mode 100644 test/samples/server.rb delete mode 100644 test/samples/simple.expected delete mode 100644 test/samples/simple.rb delete mode 100644 test/samples/stream.rb delete mode 100644 test/samples/stream2.expected delete mode 100644 test/samples/stream2.rb delete mode 100644 test/samples/suite.rb delete mode 100644 test/samples/tokens.expected delete mode 100644 test/samples/tokens.rb diff --git a/rake_tasks/test.rake b/rake_tasks/test.rake index 371214a..b15b999 100644 --- a/rake_tasks/test.rake +++ b/rake_tasks/test.rake @@ -1,9 +1,4 @@ namespace :test do - desc 'run all sample tests' - task :samples do - ruby './test/samples/suite.rb' - end - desc 'run functional tests' task :functional do ruby './test/functional/suite.rb' @@ -85,4 +80,3 @@ Please rename or remove it and run again to use the GitHub repository: end task :test => %w(test:functional test:units test:exe) -task :samples => 'test:samples' \ No newline at end of file diff --git a/test/samples/README b/test/samples/README deleted file mode 100644 index b0ab6e4..0000000 --- a/test/samples/README +++ /dev/null @@ -1 +0,0 @@ -These demos rely on Ruby 1.8.5, so the tests might fail on other versions. \ No newline at end of file diff --git a/test/samples/cache.actual b/test/samples/cache.actual deleted file mode 100644 index c131857..0000000 --- a/test/samples/cache.actual +++ /dev/null @@ -1,2 +0,0 @@ -test <test> -test <test> diff --git a/test/samples/cache.expected b/test/samples/cache.expected deleted file mode 100644 index c131857..0000000 --- a/test/samples/cache.expected +++ /dev/null @@ -1,2 +0,0 @@ -test <test> -test <test> diff --git a/test/samples/cache.rb b/test/samples/cache.rb deleted file mode 100644 index 0c0b847..0000000 --- a/test/samples/cache.rb +++ /dev/null @@ -1,12 +0,0 @@ -require 'coderay' - -html_encoder = CodeRay.encoder :html - -scanner = Hash.new do |h, lang| - h[lang] = CodeRay.scanner lang -end - -for lang in [:ruby, :html] - tokens = scanner[lang].tokenize 'test ' - puts html_encoder.encode_tokens(tokens) -end diff --git a/test/samples/count.expected b/test/samples/count.expected deleted file mode 100644 index 7f493b6..0000000 --- a/test/samples/count.expected +++ /dev/null @@ -1 +0,0 @@ -2 out of 4 tokens have the kind :integer. diff --git a/test/samples/count.rb b/test/samples/count.rb deleted file mode 100644 index bcb7c2d..0000000 --- a/test/samples/count.rb +++ /dev/null @@ -1,10 +0,0 @@ -require 'coderay' - -stats = CodeRay.encoder(:statistic) -stats.encode("puts 17 + 4\n", :ruby) - -puts '%d out of %d tokens have the kind :integer.' % [ - stats.type_stats[:integer].count, - stats.real_token_count -] -#-> 2 out of 4 tokens have the kind :integer. diff --git a/test/samples/css.actual b/test/samples/css.actual deleted file mode 100644 index be73a7f..0000000 --- a/test/samples/css.actual +++ /dev/null @@ -1,127 +0,0 @@ -.CodeRay { - background-color: hsl(0,0%,95%); - border: 1px solid silver; - color: black; -} -.CodeRay pre { - margin: 0px; -} - -span.CodeRay { white-space: pre; border: 0px; padding: 2px; } - -table.CodeRay { border-collapse: collapse; width: 100%; padding: 2px; } -table.CodeRay td { padding: 2px 4px; vertical-align: top; } - -.CodeRay .line-numbers { - background-color: hsl(180,65%,90%); - color: gray; - text-align: right; - -webkit-user-select: none; - -moz-user-select: none; - user-select: none; -} -.CodeRay .line-numbers a { - background-color: hsl(180,65%,90%) !important; - color: gray !important; - text-decoration: none !important; -} -.CodeRay .line-numbers pre { - word-break: normal; -} -.CodeRay .line-numbers a:target { color: blue !important; } -.CodeRay .line-numbers .highlighted { color: red !important; } -.CodeRay .line-numbers .highlighted a { color: red !important; } -.CodeRay span.line-numbers { padding: 0px 4px; } -.CodeRay .line { display: block; float: left; width: 100%; } -.CodeRay .code { width: 100%; } - -.CodeRay .debug { color: white !important; background: blue !important; } - -.CodeRay .annotation { color:#007 } -.CodeRay .attribute-name { color:#b48 } -.CodeRay .attribute-value { color:#700 } -.CodeRay .binary { color:#549 } -.CodeRay .binary .char { color:#325 } -.CodeRay .binary .delimiter { color:#325 } -.CodeRay .char { color:#D20 } -.CodeRay .char .content { color:#D20 } -.CodeRay .char .delimiter { color:#710 } -.CodeRay .class { color:#B06; font-weight:bold } -.CodeRay .class-variable { color:#369 } -.CodeRay .color { color:#0A0 } -.CodeRay .comment { color:#777 } -.CodeRay .comment .char { color:#444 } -.CodeRay .comment .delimiter { color:#444 } -.CodeRay .constant { color:#036; font-weight:bold } -.CodeRay .decorator { color:#B0B } -.CodeRay .definition { color:#099; font-weight:bold } -.CodeRay .delimiter { color:black } -.CodeRay .directive { color:#088; font-weight:bold } -.CodeRay .docstring { color:#D42; } -.CodeRay .doctype { color:#34b } -.CodeRay .done { text-decoration: line-through; color: gray } -.CodeRay .entity { color:#800; font-weight:bold } -.CodeRay .error { color:#F00; background-color:#FAA } -.CodeRay .escape { color:#666 } -.CodeRay .exception { color:#C00; font-weight:bold } -.CodeRay .float { color:#60E } -.CodeRay .function { color:#06B; font-weight:bold } -.CodeRay .function .delimiter { color:#024; font-weight:bold } -.CodeRay .global-variable { color:#d70 } -.CodeRay .hex { color:#02b } -.CodeRay .id { color:#33D; font-weight:bold } -.CodeRay .include { color:#B44; font-weight:bold } -.CodeRay .inline { background-color: hsla(0,0%,0%,0.07); color: black } -.CodeRay .inline-delimiter { font-weight: bold; color: #666 } -.CodeRay .instance-variable { color:#33B } -.CodeRay .integer { color:#00D } -.CodeRay .imaginary { color:#f00 } -.CodeRay .important { color:#D00 } -.CodeRay .key { color: #606 } -.CodeRay .key .char { color: #60f } -.CodeRay .key .delimiter { color: #404 } -.CodeRay .keyword { color:#080; font-weight:bold } -.CodeRay .label { color:#970; font-weight:bold } -.CodeRay .local-variable { color:#963 } -.CodeRay .namespace { color:#707; font-weight:bold } -.CodeRay .octal { color:#40E } -.CodeRay .operator { } -.CodeRay .predefined { color:#369; font-weight:bold } -.CodeRay .predefined-constant { color:#069 } -.CodeRay .predefined-type { color:#0a5; font-weight:bold } -.CodeRay .preprocessor { color:#579 } -.CodeRay .pseudo-class { color:#00C; font-weight:bold } -.CodeRay .regexp { background-color:hsla(300,100%,50%,0.06); } -.CodeRay .regexp .content { color:#808 } -.CodeRay .regexp .delimiter { color:#404 } -.CodeRay .regexp .modifier { color:#C2C } -.CodeRay .reserved { color:#080; font-weight:bold } -.CodeRay .shell { background-color:hsla(120,100%,50%,0.06); } -.CodeRay .shell .content { color:#2B2 } -.CodeRay .shell .delimiter { color:#161 } -.CodeRay .string { background-color:hsla(0,100%,50%,0.05); } -.CodeRay .string .char { color: #b0b } -.CodeRay .string .content { color: #D20 } -.CodeRay .string .delimiter { color: #710 } -.CodeRay .string .modifier { color: #E40 } -.CodeRay .symbol { color:#A60 } -.CodeRay .symbol .content { color:#A60 } -.CodeRay .symbol .delimiter { color:#630 } -.CodeRay .tag { color:#070 } -.CodeRay .type { color:#339; font-weight:bold } -.CodeRay .value { color: #088 } -.CodeRay .variable { color:#037 } - -.CodeRay .insert { background: hsla(120,100%,50%,0.12) } -.CodeRay .delete { background: hsla(0,100%,50%,0.12) } -.CodeRay .change { color: #bbf; background: #007 } -.CodeRay .head { color: #f8f; background: #505 } -.CodeRay .head .filename { color: white; } - -.CodeRay .delete .eyecatcher { background-color: hsla(0,100%,50%,0.2); border: 1px solid hsla(0,100%,45%,0.5); margin: -1px; border-bottom: none; border-top-left-radius: 5px; border-top-right-radius: 5px; } -.CodeRay .insert .eyecatcher { background-color: hsla(120,100%,50%,0.2); border: 1px solid hsla(120,100%,25%,0.5); margin: -1px; border-top: none; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px; } - -.CodeRay .insert .insert { color: #0c0; background:transparent; font-weight:bold } -.CodeRay .delete .delete { color: #c00; background:transparent; font-weight:bold } -.CodeRay .change .change { color: #88f } -.CodeRay .head .head { color: #f4f } diff --git a/test/samples/css.expected b/test/samples/css.expected deleted file mode 100644 index be73a7f..0000000 --- a/test/samples/css.expected +++ /dev/null @@ -1,127 +0,0 @@ -.CodeRay { - background-color: hsl(0,0%,95%); - border: 1px solid silver; - color: black; -} -.CodeRay pre { - margin: 0px; -} - -span.CodeRay { white-space: pre; border: 0px; padding: 2px; } - -table.CodeRay { border-collapse: collapse; width: 100%; padding: 2px; } -table.CodeRay td { padding: 2px 4px; vertical-align: top; } - -.CodeRay .line-numbers { - background-color: hsl(180,65%,90%); - color: gray; - text-align: right; - -webkit-user-select: none; - -moz-user-select: none; - user-select: none; -} -.CodeRay .line-numbers a { - background-color: hsl(180,65%,90%) !important; - color: gray !important; - text-decoration: none !important; -} -.CodeRay .line-numbers pre { - word-break: normal; -} -.CodeRay .line-numbers a:target { color: blue !important; } -.CodeRay .line-numbers .highlighted { color: red !important; } -.CodeRay .line-numbers .highlighted a { color: red !important; } -.CodeRay span.line-numbers { padding: 0px 4px; } -.CodeRay .line { display: block; float: left; width: 100%; } -.CodeRay .code { width: 100%; } - -.CodeRay .debug { color: white !important; background: blue !important; } - -.CodeRay .annotation { color:#007 } -.CodeRay .attribute-name { color:#b48 } -.CodeRay .attribute-value { color:#700 } -.CodeRay .binary { color:#549 } -.CodeRay .binary .char { color:#325 } -.CodeRay .binary .delimiter { color:#325 } -.CodeRay .char { color:#D20 } -.CodeRay .char .content { color:#D20 } -.CodeRay .char .delimiter { color:#710 } -.CodeRay .class { color:#B06; font-weight:bold } -.CodeRay .class-variable { color:#369 } -.CodeRay .color { color:#0A0 } -.CodeRay .comment { color:#777 } -.CodeRay .comment .char { color:#444 } -.CodeRay .comment .delimiter { color:#444 } -.CodeRay .constant { color:#036; font-weight:bold } -.CodeRay .decorator { color:#B0B } -.CodeRay .definition { color:#099; font-weight:bold } -.CodeRay .delimiter { color:black } -.CodeRay .directive { color:#088; font-weight:bold } -.CodeRay .docstring { color:#D42; } -.CodeRay .doctype { color:#34b } -.CodeRay .done { text-decoration: line-through; color: gray } -.CodeRay .entity { color:#800; font-weight:bold } -.CodeRay .error { color:#F00; background-color:#FAA } -.CodeRay .escape { color:#666 } -.CodeRay .exception { color:#C00; font-weight:bold } -.CodeRay .float { color:#60E } -.CodeRay .function { color:#06B; font-weight:bold } -.CodeRay .function .delimiter { color:#024; font-weight:bold } -.CodeRay .global-variable { color:#d70 } -.CodeRay .hex { color:#02b } -.CodeRay .id { color:#33D; font-weight:bold } -.CodeRay .include { color:#B44; font-weight:bold } -.CodeRay .inline { background-color: hsla(0,0%,0%,0.07); color: black } -.CodeRay .inline-delimiter { font-weight: bold; color: #666 } -.CodeRay .instance-variable { color:#33B } -.CodeRay .integer { color:#00D } -.CodeRay .imaginary { color:#f00 } -.CodeRay .important { color:#D00 } -.CodeRay .key { color: #606 } -.CodeRay .key .char { color: #60f } -.CodeRay .key .delimiter { color: #404 } -.CodeRay .keyword { color:#080; font-weight:bold } -.CodeRay .label { color:#970; font-weight:bold } -.CodeRay .local-variable { color:#963 } -.CodeRay .namespace { color:#707; font-weight:bold } -.CodeRay .octal { color:#40E } -.CodeRay .operator { } -.CodeRay .predefined { color:#369; font-weight:bold } -.CodeRay .predefined-constant { color:#069 } -.CodeRay .predefined-type { color:#0a5; font-weight:bold } -.CodeRay .preprocessor { color:#579 } -.CodeRay .pseudo-class { color:#00C; font-weight:bold } -.CodeRay .regexp { background-color:hsla(300,100%,50%,0.06); } -.CodeRay .regexp .content { color:#808 } -.CodeRay .regexp .delimiter { color:#404 } -.CodeRay .regexp .modifier { color:#C2C } -.CodeRay .reserved { color:#080; font-weight:bold } -.CodeRay .shell { background-color:hsla(120,100%,50%,0.06); } -.CodeRay .shell .content { color:#2B2 } -.CodeRay .shell .delimiter { color:#161 } -.CodeRay .string { background-color:hsla(0,100%,50%,0.05); } -.CodeRay .string .char { color: #b0b } -.CodeRay .string .content { color: #D20 } -.CodeRay .string .delimiter { color: #710 } -.CodeRay .string .modifier { color: #E40 } -.CodeRay .symbol { color:#A60 } -.CodeRay .symbol .content { color:#A60 } -.CodeRay .symbol .delimiter { color:#630 } -.CodeRay .tag { color:#070 } -.CodeRay .type { color:#339; font-weight:bold } -.CodeRay .value { color: #088 } -.CodeRay .variable { color:#037 } - -.CodeRay .insert { background: hsla(120,100%,50%,0.12) } -.CodeRay .delete { background: hsla(0,100%,50%,0.12) } -.CodeRay .change { color: #bbf; background: #007 } -.CodeRay .head { color: #f8f; background: #505 } -.CodeRay .head .filename { color: white; } - -.CodeRay .delete .eyecatcher { background-color: hsla(0,100%,50%,0.2); border: 1px solid hsla(0,100%,45%,0.5); margin: -1px; border-bottom: none; border-top-left-radius: 5px; border-top-right-radius: 5px; } -.CodeRay .insert .eyecatcher { background-color: hsla(120,100%,50%,0.2); border: 1px solid hsla(120,100%,25%,0.5); margin: -1px; border-top: none; border-bottom-left-radius: 5px; border-bottom-right-radius: 5px; } - -.CodeRay .insert .insert { color: #0c0; background:transparent; font-weight:bold } -.CodeRay .delete .delete { color: #c00; background:transparent; font-weight:bold } -.CodeRay .change .change { color: #88f } -.CodeRay .head .head { color: #f4f } diff --git a/test/samples/css.rb b/test/samples/css.rb deleted file mode 100644 index 52e4bcc..0000000 --- a/test/samples/css.rb +++ /dev/null @@ -1,4 +0,0 @@ -require 'coderay' - -# print the default stylesheet for HTML codes -puts CodeRay::Encoders[:html]::CSS.new.stylesheet diff --git a/test/samples/div.actual b/test/samples/div.actual deleted file mode 100644 index d1e692a..0000000 --- a/test/samples/div.actual +++ /dev/null @@ -1,17 +0,0 @@ -
- 
for a in 0..255
-        a = a.chr
-        begin
-                x = eval("?\\#{a}")
-                if x == a[0]
-                        next
-                else
-                        print "#{a}: #{x}"
-                end
-        rescue SyntaxError => boom
-                print "#{a}: error"
-        end
-        puts
-end
-
-
diff --git a/test/samples/div.expected b/test/samples/div.expected deleted file mode 100644 index d1e692a..0000000 --- a/test/samples/div.expected +++ /dev/null @@ -1,17 +0,0 @@ -
- 
for a in 0..255
-        a = a.chr
-        begin
-                x = eval("?\\#{a}")
-                if x == a[0]
-                        next
-                else
-                        print "#{a}: #{x}"
-                end
-        rescue SyntaxError => boom
-                print "#{a}: error"
-        end
-        puts
-end
-
-
diff --git a/test/samples/div.rb b/test/samples/div.rb deleted file mode 100644 index 27b6f32..0000000 --- a/test/samples/div.rb +++ /dev/null @@ -1,19 +0,0 @@ -require 'coderay' - -puts CodeRay.scan(DATA.read, :ruby).div - -__END__ -for a in 0..255 - a = a.chr - begin - x = eval("?\\#{a}") - if x == a[0] - next - else - print "#{a}: #{x}" - end - rescue SyntaxError => boom - print "#{a}: error" - end - puts -end diff --git a/test/samples/encoder.actual b/test/samples/encoder.actual deleted file mode 100644 index 8bd83a9..0000000 --- a/test/samples/encoder.actual +++ /dev/null @@ -1,65 +0,0 @@ -Encoders Demo: puts 17 + 4 - -Statistic: - -Code Statistics - -Tokens 8 - Non-Whitespace 4 -Bytes Total 12 - -Token Types (4): - type count ratio size (average) -------------------------------------------------------------- - TOTAL 8 100.00 % 1.5 - space 4 50.00 % 1.0 - integer 2 25.00 % 1.5 - ident 1 12.50 % 4.0 - operator 1 12.50 % 1.0 - - -Original text: -[{"type":"text","text":"puts","kind":"ident"},{"type":"text","text":" ","kind":"space"},{"type":"text","text":"17","kind":"integer"},{"type":"text","text":" ","kind":"space"},{"type":"text","text":"+","kind":"operator"},{"type":"text","text":" ","kind":"space"},{"type":"text","text":"4","kind":"integer"},{"type":"text","text":"\n","kind":"space"}] - -YAML: ---- -- - puts - - :ident -- - " " - - :space -- - "17" - - :integer -- - " " - - :space -- - + - - :operator -- - " " - - :space -- - "4" - - :integer -- - | - - - - :space - -Dump: -"x\234\355\3121\n\2000\f@\321\335StLp\022\204\236G0H\226\266\304\364\376\235\304K\374\365\361\374\266\2262f\276Z\274\245=\026rT-}X\\\331C\366\337O\335\234N\247\323\351t:\235N\247\323\351t:\235N\377\372\002\2613\031\257" -compressed: 79 byte < 1200 byte - -Undump: - -Code Statistics - -Tokens 800 - Non-Whitespace 400 -Bytes Total 1200 - -Token Types (4): - type count ratio size (average) -------------------------------------------------------------- - TOTAL 800 100.00 % 1.5 - space 400 50.00 % 1.0 - integer 200 25.00 % 1.5 - ident 100 12.50 % 4.0 - operator 100 12.50 % 1.0 - diff --git a/test/samples/encoder.expected b/test/samples/encoder.expected deleted file mode 100644 index 8bd83a9..0000000 --- a/test/samples/encoder.expected +++ /dev/null @@ -1,65 +0,0 @@ -Encoders Demo: puts 17 + 4 - -Statistic: - -Code Statistics - -Tokens 8 - Non-Whitespace 4 -Bytes Total 12 - -Token Types (4): - type count ratio size (average) -------------------------------------------------------------- - TOTAL 8 100.00 % 1.5 - space 4 50.00 % 1.0 - integer 2 25.00 % 1.5 - ident 1 12.50 % 4.0 - operator 1 12.50 % 1.0 - - -Original text: -[{"type":"text","text":"puts","kind":"ident"},{"type":"text","text":" ","kind":"space"},{"type":"text","text":"17","kind":"integer"},{"type":"text","text":" ","kind":"space"},{"type":"text","text":"+","kind":"operator"},{"type":"text","text":" ","kind":"space"},{"type":"text","text":"4","kind":"integer"},{"type":"text","text":"\n","kind":"space"}] - -YAML: ---- -- - puts - - :ident -- - " " - - :space -- - "17" - - :integer -- - " " - - :space -- - + - - :operator -- - " " - - :space -- - "4" - - :integer -- - | - - - - :space - -Dump: -"x\234\355\3121\n\2000\f@\321\335StLp\022\204\236G0H\226\266\304\364\376\235\304K\374\365\361\374\266\2262f\276Z\274\245=\026rT-}X\\\331C\366\337O\335\234N\247\323\351t:\235N\247\323\351t:\235N\377\372\002\2613\031\257" -compressed: 79 byte < 1200 byte - -Undump: - -Code Statistics - -Tokens 800 - Non-Whitespace 400 -Bytes Total 1200 - -Token Types (4): - type count ratio size (average) -------------------------------------------------------------- - TOTAL 800 100.00 % 1.5 - space 400 50.00 % 1.0 - integer 200 25.00 % 1.5 - ident 100 12.50 % 4.0 - operator 100 12.50 % 1.0 - diff --git a/test/samples/encoder.rb b/test/samples/encoder.rb deleted file mode 100644 index bfcfbfa..0000000 --- a/test/samples/encoder.rb +++ /dev/null @@ -1,40 +0,0 @@ -require 'coderay' - -SAMPLE = "puts 17 + 4\n" -puts 'Encoders Demo: ' + SAMPLE -scanner = CodeRay::Scanners[:ruby].new SAMPLE -encoder = CodeRay::Encoders[:statistic].new - -tokens = scanner.tokenize -stats = encoder.encode_tokens tokens - -puts -puts 'Statistic:' -puts stats - -# alternative 1 -tokens = CodeRay.scan SAMPLE, :ruby -encoder = CodeRay.encoder(:json) -textual = encoder.encode_tokens tokens -puts -puts 'Original text:' -puts textual - -# alternative 2 -yaml = CodeRay.encoder(:yaml).encode SAMPLE, :ruby -puts -puts 'YAML:' -puts yaml - -# alternative 3 -require 'zlib' -BIGSAMPLE = SAMPLE * 100 -dump = Zlib::Deflate.deflate(CodeRay.scan(BIGSAMPLE, :ruby).debug) -puts -puts 'Dump:' -p dump -puts 'compressed: %d byte < %d byte' % [dump.size, BIGSAMPLE.size] - -puts -puts 'Undump:' -puts CodeRay.scan(Zlib::Inflate.inflate(dump), :debug).statistic diff --git a/test/samples/global_vars.actual b/test/samples/global_vars.actual deleted file mode 100644 index e69de29..0000000 diff --git a/test/samples/global_vars.diff b/test/samples/global_vars.diff deleted file mode 100644 index 2ed3382..0000000 --- a/test/samples/global_vars.diff +++ /dev/null @@ -1,5 +0,0 @@ -1,3d0 -< <--$IE-->.TEXT_FIELD(:NAME, "PANFRAGE OHNE $GV UND MIT #{<--$GV-->}").SET ARTIKEL -< ODER -< TEXT = <--$BLA-->.TEST(...) -\ No newline at end of file diff --git a/test/samples/global_vars.expected b/test/samples/global_vars.expected deleted file mode 100644 index 0dc13c8..0000000 --- a/test/samples/global_vars.expected +++ /dev/null @@ -1,3 +0,0 @@ -<--$IE-->.TEXT_FIELD(:NAME, "PANFRAGE OHNE $GV UND MIT #{<--$GV-->}").SET ARTIKEL -ODER -TEXT = <--$BLA-->.TEST(...) \ No newline at end of file diff --git a/test/samples/global_vars.rb b/test/samples/global_vars.rb deleted file mode 100644 index 8066d67..0000000 --- a/test/samples/global_vars.rb +++ /dev/null @@ -1,13 +0,0 @@ -code = <<'CODE' -$ie.text_field(:name, "pAnfrage ohne $gV und mit #{$gv}").set artikel -oder -text = $bla.test(...) -CODE - -require 'coderay' - -tokens = CodeRay.scan code, :ruby -tokens.each_text_token { |text, kind| text.upcase! } -tokens.each(:global_variable) { |text, kind| text.replace '<--%s-->' % text } - -print tokens diff --git a/test/samples/global_vars2.expected b/test/samples/global_vars2.expected deleted file mode 100644 index 964cf50..0000000 --- a/test/samples/global_vars2.expected +++ /dev/null @@ -1,10 +0,0 @@ - - - - - -$ie.text_field(:name, "pAnfrage ohne $gV und mit #{$gv}").set artikel -oder -text = $bla.test(...) - - diff --git a/test/samples/global_vars2.rb b/test/samples/global_vars2.rb deleted file mode 100644 index 7646890..0000000 --- a/test/samples/global_vars2.rb +++ /dev/null @@ -1,28 +0,0 @@ -require 'coderay' -require 'erb' -include ERB::Util - -code = <<'CODE' -$ie.text_field(:name, "pAnfrage ohne $gV und mit #{$gv}").set artikel -oder -text = $bla.test(...) -CODE -puts < - - - - -HTML - -CodeRay.scan_stream code, :ruby do |text, kind| - next if text.is_a? Symbol - text = h(text) - text = '%s' % text if kind == :global_variable - print text -end - -puts < - -HTML diff --git a/test/samples/highlight.expected b/test/samples/highlight.expected deleted file mode 100644 index 6a9b278..0000000 --- a/test/samples/highlight.expected +++ /dev/null @@ -1,175 +0,0 @@ -
- 
puts "Hello, World!"
-
- - - - - - - - -
1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-11
-12
-13
-14
-
require 'coderay'
-
-puts CodeRay.highlight('puts "Hello, World!"', :ruby)
-
-output = CodeRay.highlight_file($0, :line_numbers => :table)
-puts <<HTML
-<html>
-<head>
-#{output.stylesheet true}
-<body>
-#{output}
-</body>
-</html>
-HTML
-
- - - diff --git a/test/samples/highlight.rb b/test/samples/highlight.rb deleted file mode 100644 index 846efa4..0000000 --- a/test/samples/highlight.rb +++ /dev/null @@ -1,14 +0,0 @@ -require 'coderay' - -puts CodeRay.highlight('puts "Hello, World!"', :ruby) - -output = CodeRay.highlight_file($0, :line_numbers => :table) -puts < - -#{output.stylesheet true} - -#{output} - - -HTML diff --git a/test/samples/html.expected b/test/samples/html.expected deleted file mode 100644 index e98d589..0000000 --- a/test/samples/html.expected +++ /dev/null @@ -1,919 +0,0 @@ - - - - - CodeRay HTML Encoder Example - - - - - - - -
1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
-35
-36
-37
-38
-39
-40
-41
-42
-43
-44
-45
-46
-47
-48
-49
-50
-51
-52
-53
-54
-55
-56
-57
-58
-59
-60
-61
-62
-63
-64
-65
-66
-67
-68
-69
-70
-71
-72
-73
-74
-75
-76
-77
-78
-79
-80
-81
-82
-83
-84
-85
-86
-87
-88
-89
-90
-91
-92
-93
-94
-95
-96
-97
-98
-99
-100
-101
-102
-103
-104
-105
-106
-107
-108
-109
-110
-111
-112
-113
-114
-115
-116
-117
-118
-119
-120
-121
-122
-123
-124
-125
-126
-127
-128
-129
-130
-131
-132
-133
-134
-135
-136
-137
-138
-139
-140
-141
-142
-143
-144
-145
-146
-147
-148
-149
-150
-151
-152
-153
-154
-155
-156
-157
-158
-159
-160
-161
-162
-163
-164
-165
-166
-167
-168
-169
-170
-171
-172
-173
-174
-175
-176
-177
-178
-179
-180
-181
-182
-183
-184
-185
-186
-187
-188
-189
-190
-191
-192
-193
-194
-195
-196
-197
-198
-199
-200
-201
-202
-203
-204
-205
-206
-207
-208
-209
-210
-211
-212
-213
-214
-215
-216
-217
-218
-219
-220
-221
-222
-223
-224
-225
-226
-227
-228
-229
-230
-231
-232
-233
-234
-235
-236
-237
-238
-239
-240
-241
-242
-243
-244
-245
-246
-247
-248
-249
-250
-251
-252
-253
-254
-255
-256
-257
-258
-259
-260
-261
-262
-263
-264
-265
-266
-267
-268
-269
-270
-271
-272
-273
-274
-275
-276
-277
-278
-279
-280
-281
-282
-283
-284
-285
-286
-287
-288
-289
-290
-291
-292
-293
-294
-295
-296
-297
-298
-299
-300
-301
-302
-303
-304
-305
-306
-307
-308
-309
-310
-311
-312
-313
-314
-315
-316
-317
-318
-319
-320
-321
-322
-323
-324
-325
-326
-327
-328
-329
-330
-331
-332
-333
-334
-335
-336
-337
-338
-339
-340
-341
-342
-343
-344
-345
-346
-347
-348
-349
-350
-351
-352
-353
-354
-355
-356
-357
-358
-359
-360
-361
-362
-363
-364
-365
-366
-367
-368
-369
-370
-371
-372
-373
-374
-375
-376
-377
-378
-379
-380
-381
-382
-383
-384
-385
-
require 'scanner'
-
-module CodeRay
-  
-  class RubyScanner < Scanner
-    
-    RESERVED_WORDS = [
-      'and', 'def', 'end', 'in', 'or', 'unless', 'begin',
-      'defined?', 'ensure', 'module', 'redo', 'super', 'until',
-      'BEGIN', 'break', 'do', 'next', 'rescue', 'then',
-      'when', 'END', 'case', 'else', 'for', 'retry',
-      'while', 'alias', 'class', 'elsif', 'if', 'not', 'return',
-      'undef', 'yield',
-    ]
-
-    DEF_KEYWORDS = ['def']
-    MODULE_KEYWORDS = ['class', 'module']
-    DEF_NEW_STATE = WordList.new(:initial).
-      add(DEF_KEYWORDS, :def_expected).
-      add(MODULE_KEYWORDS, :module_expected)
-
-    WORDS_ALLOWING_REGEXP = [
-      'and', 'or', 'not', 'while', 'until', 'unless', 'if', 'elsif', 'when'
-    ]
-    REGEXP_ALLOWED = WordList.new(false).
-      add(WORDS_ALLOWING_REGEXP, :set)
-    
-    PREDEFINED_CONSTANTS = [
-      'nil', 'true', 'false', 'self',
-      'DATA', 'ARGV', 'ARGF', '__FILE__', '__LINE__',
-    ]
-
-    IDENT_KIND = WordList.new(:ident).
-      add(RESERVED_WORDS, :reserved).
-      add(PREDEFINED_CONSTANTS, :pre_constant)
-
-    METHOD_NAME = / #{IDENT} [?!]? /xo
-    METHOD_NAME_EX = /
-     #{METHOD_NAME}  # common methods: split, foo=, empty?, gsub!
-     | \*\*?         # multiplication and power
-     | [-+~]@?       # plus, minus
-     | [\/%&|^`]     # division, modulo or format strings, &and, |or, ^xor, `system`
-     | \[\]=?        # array getter and setter
-     | <=?>? | >=?   # comparison, rocket operator
-     | << | >>       # append or shift left, shift right
-     | ===?          # simple equality and case equality
-    /ox
-    GLOBAL_VARIABLE = / \$ (?: #{IDENT} | \d+ | [~&+`'=\/,;_.<>!@0$?*":F\\] | -[a-zA-Z_0-9] ) /ox
-
-    DOUBLEQ = / "  [^"\#\\]*  (?: (?: \#\{.*?\} | \#(?:$")?  | \\. ) [^"\#\\]*  )* "?  /ox
-    SINGLEQ = / '  [^'\\]*    (?:                              \\.   [^'\\]*    )* '?  /ox
-    STRING  = / #{SINGLEQ} | #{DOUBLEQ} /ox
-    SHELL   = / `  [^`\#\\]*  (?: (?: \#\{.*?\} | \#(?:$`)?  | \\. ) [^`\#\\]*  )* `?  /ox
-    REGEXP  = / \/ [^\/\#\\]* (?: (?: \#\{.*?\} | \#(?:$\/)? | \\. ) [^\/\#\\]* )* \/? /ox
-    
-    DECIMAL = /\d+(?:_\d+)*/  # doesn't recognize 09 as octal error
-    OCTAL = /0_?[0-7]+(?:_[0-7]+)*/
-    HEXADECIMAL = /0x[0-9A-Fa-f]+(?:_[0-9A-Fa-f]+)*/
-    BINARY = /0b[01]+(?:_[01]+)*/
-
-    EXPONENT = / [eE] [+-]? #{DECIMAL} /ox
-    FLOAT = / #{DECIMAL} (?: #{EXPONENT} | \. #{DECIMAL} #{EXPONENT}? ) /
-    INTEGER = /#{OCTAL}|#{HEXADECIMAL}|#{BINARY}|#{DECIMAL}/
-    
-    def reset
-      super
-      @regexp_allowed = false
-    end
-    
-    def next_token
-      return if @scanner.eos?
-
-      kind = :error
-      if @scanner.scan(/\s+/)  # in every state
-        kind = :space
-        @regexp_allowed = :set if @regexp_allowed or @scanner.matched.index(?\n)  # delayed flag setting
-
-      elsif @state == :def_expected
-        if @scanner.scan(/ (?: (?:#{IDENT}(?:\.|::))* | (?:@@?|$)? #{IDENT}(?:\.|::) ) #{METHOD_NAME_EX} /ox)
-          kind = :method
-          @state = :initial
-        else
-          @scanner.scan(/./)
-          kind = :error
-        end
-        @state = :initial
-        
-      elsif @state == :module_expected
-        if @scanner.scan(/<</)
-          kind = :operator
-        else
-          if @scanner.scan(/ (?: #{IDENT} (?:\.|::))* #{IDENT} /ox)
-            kind = :method
-          else
-            @scanner.scan(/./)
-            kind = :error
-          end
-          @state = :initial
-        end
-        
-      elsif # state == :initial
-        # IDENTIFIERS, KEYWORDS
-        if @scanner.scan(GLOBAL_VARIABLE)
-          kind = :global_variable
-        elsif @scanner.scan(/ @@ #{IDENT} /ox)
-          kind = :class_variable
-        elsif @scanner.scan(/ @ #{IDENT} /ox)
-          kind = :instance_variable
-        elsif @scanner.scan(/ __END__\n ( (?!\#CODE\#) .* )? | \#[^\n]* | =begin(?=\s).*? \n=end(?=\s|\z)(?:[^\n]*)? /x)
-          kind = :comment
-        elsif @scanner.scan(METHOD_NAME)
-          if @last_token_dot
-            kind = :ident
-          else
-            matched = @scanner.matched
-            kind = IDENT_KIND[matched]
-            if kind == :ident and matched =~ /^[A-Z]/
-              kind = :constant
-            elsif kind == :reserved
-              @state = DEF_NEW_STATE[matched]
-              @regexp_allowed = REGEXP_ALLOWED[matched]
-            end
-          end
-          
-        elsif @scanner.scan(STRING)
-          kind = :string
-        elsif @scanner.scan(SHELL)
-          kind = :shell
-        ## HEREDOCS
-        elsif @scanner.scan(/\//) and @regexp_allowed
-           @scanner.unscan
-           @scanner.scan(REGEXP)
-          kind = :regexp
-        ## %strings
-        elsif @scanner.scan(/:(?:#{GLOBAL_VARIABLE}|#{METHOD_NAME_EX}|#{STRING})/ox)
-          kind = :global_variable
-        elsif @scanner.scan(/
-          \? (?:
-            [^\s\\]
-          | 
-            \\ (?:M-\\C-|C-\\M-|M-\\c|c\\M-|c|C-|M-))? (?: \\ (?: . | [0-7]{3} | x[0-9A-Fa-f][0-9A-Fa-f] )
-          )
-        /ox)
-          kind = :integer
-          
-        elsif @scanner.scan(/ [-+*\/%=<>;,|&!()\[\]{}~?] | \.\.?\.? | ::? /x)
-          kind = :operator
-          @regexp_allowed = :set if @scanner.matched[-1,1] =~ /[~=!<>|&^,\(\[+\-\/\*%]\z/
-        elsif @scanner.scan(FLOAT)
-          kind = :float
-        elsif @scanner.scan(INTEGER)
-          kind = :integer
-        elsif @scanner.scan(/:(?:#{GLOBAL_VARIABLE}|#{METHOD_NAME_EX}|#{STRING})/ox)
-          kind = :global_variable
-        else
-          @scanner.scan(/./m)
-        end
-      end
-      
-      token = Token.new @scanner.matched, kind
-
-      if kind == :regexp
-        token.text << @scanner.scan(/[eimnosux]*/)
-      end
-      
-      @regexp_allowed = (@regexp_allowed == :set)  # delayed flag setting
-
-      token
-    end
-  end
-  
-  ScannerList.register RubyScanner, 'ruby'
-
-end
-
-module CodeRay
-  require 'scanner'
-
-  class Highlighter
-
-    def initialize lang
-      @scanner = Scanner[lang].new
-    end
-
-    def highlight code
-      @scanner.feed code
-      @scanner.all_tokens.map { |t| t.inspect }.join "\n"
-    end
-
-  end
-
-  class HTMLHighlighter < Highlighter
-    
-    ClassOfKind = {
-      :attribute_name => 'an',
-      :attribute_name_fat => 'af',
-      :attribute_value => 'av',
-      :attribute_value_fat => 'aw',
-      :bin => 'bi',
-       :char => 'ch',
-      :class => 'cl',
-      :class_variable => 'cv',
-      :color => 'cr',
-      :comment => 'c',
-      :constant => 'co',
-      :definition => 'df',
-      :directive => 'di',
-      :doc => 'do',
-      :doc_string => 'ds',
-      :exception => 'ex',
-      :error => 'er',
-      :float => 'fl',
-      :function => 'fu',
-      :global_variable => 'gv',
-      :hex => 'hx',
-      :include => 'ic',
-      :instance_variable => 'iv',
-      :integer => 'i',
-      :interpreted => 'in',
-      :label => 'la',
-      :local_variable => 'lv',
-      :oct => 'oc',
-      :operator_name => 'on',
-      :pre_constant => 'pc',
-      :pre_type => 'pt',
-      :predefined => 'pd',
-      :preprocessor => 'pp',
-      :regexp => 'rx',
-      :reserved => 'r',
-      :shell => 'sh',
-      :string => 's',
-      :symbol => 'sy',
-      :tag => 'ta',
-      :tag_fat => 'tf',
-      :tag_special => 'ts',
-      :type => 'ty',
-      :variable => 'v',
-      :xml_text => 'xt',
-
-      :ident => :NO_HIGHLIGHT,
-      :operator => :NO_HIGHLIGHT,
-      :space => :NO_HIGHLIGHT,
-    }
-    ClassOfKind[:procedure] = ClassOfKind[:method] = ClassOfKind[:function]
-    ClassOfKind.default = ClassOfKind[:error] or raise 'no class found for :error!'
-    
-    def initialize lang, options = {}
-      super lang
-      
-      @HTML_TAB = ' ' * options.fetch(:tabs2space, 8)
-      case level = options.fetch(:level, 'xhtml')
-        when 'html'
-          @HTML_BR = "<BR>\n"
-        when 'xhtml'
-          @HTML_BR = "<br />\n"
-      else
-        raise "Unknown HTML level: #{level}"
-      end
-    end
-
-    def highlight code
-      @scanner.feed code
-      
-      out = ''
-      while t = @scanner.next_token
-        warn t.inspect if t.text.nil?
-        out << to_html(t)
-      end
-      TEMPLATE =~ /<%CONTENT%>/
-      $` + out + $'
-    end
-    
-  private
-    def to_html token
-      css_class = ClassOfKind[token.kind]
-      if defined? ::DEBUG and not ClassOfKind.has_key? token.kind
-        warn "no token class found for :#{token.kind}"
-      end
-        
-      text = text_to_html token.text
-      if css_class == :NO_HIGHLIGHT
-        text
-      else
-        "<span class=\"#{css_class}\">#{text}</span>"
-      end
-    end
-    
-    def text_to_html text
-      return '' if text.empty?
-      text = text.dup  # important
-      if text.index(/["><&]/)
-        text.gsub!('&', '&amp;')
-        text.gsub!('"', '&quot;')
-        text.gsub!('>', '&gt;')
-        text.gsub!('<', '&lt;')
-      end
-      if text.index(/\s/)
-        text.gsub!("\n", @HTML_BR)
-        text.gsub!("\t", @HTML_TAB)
-        text.gsub!(/^ /, '&nbsp;')
-        text.gsub!('  ', ' &nbsp;')
-      end
-      text
-    end
-    
-    TEMPLATE = <<-'TEMPLATE'
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
-<html dir="ltr">
-<head>
-<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
-<meta http-equiv="Content-Style-Type" content="text/css">
-
-<title>RubyBB BBCode</title>
-<style type="text/css">
-.code {
-  width: 100%;
-  background-color: #FAFAFA;
-  border: 1px solid #D1D7DC;
-  font-family: 'Courier New', 'Terminal', monospace;
-  font-size: 10pt;
-  color: black;
-  vertical-align: top;
-  text-align: left;
-}
-.code .af { color:#00C; }
-.code .an { color:#007; }
-.code .av { color:#700; }
-.code .aw { color:#C00; }
-.code .bi { color:#509; font-weight:bold; }
-.code .c  { color:#888; }
-.code .ch { color:#C28; font-weight:bold; }
-.code .cl { color:#B06; font-weight:bold; }
-.code .co { color:#036; font-weight:bold; }
-.code .cr { color:#0A0; }
-.code .cv { color:#369; }
-.code .df { color:#099; font-weight:bold; }
-.code .di { color:#088; font-weight:bold; }
-.code .do { color:#970; }
-.code .ds { color:#D42; font-weight:bold; }
-.code .er { color:#F00; background-color:#FAA; }
-.code .ex { color:#F00; font-weight:bold; }
-.code .fl { color:#60E; font-weight:bold; }
-.code .fu { color:#06B; font-weight:bold; }
-.code .gv { color:#800; font-weight:bold; }
-.code .hx { color:#058; font-weight:bold; }
-.code .i  { color:#00D; font-weight:bold; }
-.code .ic { color:#B44; font-weight:bold; }
-.code .in { color:#B2B; font-weight:bold; }
-.code .iv { color:#33B; }
-.code .la { color:#970; font-weight:bold; }
-.code .lv { color:#963; }
-.code .oc { color:#40E; font-weight:bold; }
-.code .on { color:#000; font-weight:bold; }
-.code .pc { color:#038; font-weight:bold; }
-.code .pd { color:#369; font-weight:bold; }
-.code .pp { color:#579; }
-.code .pt { color:#339; font-weight:bold; }
-.code .r  { color:#080; font-weight:bold; }
-.code .rx { color:#927; font-weight:bold; }
-.code .s  { color:#D42; font-weight:bold; }
-.code .sh { color:#B2B; font-weight:bold; }
-.code .sy { color:#A60; }
-.code .ta { color:#070; }
-.code .tf { color:#070; font-weight:bold; }
-.code .ts { color:#D70; font-weight:bold; }
-.code .ty { color:#339; font-weight:bold; }
-.code .v  { color:#036; }
-.code .xt { color:#444; }
-</style>
-</head>
-<body>
-<div class="code">
-<%CONTENT%>
-</div>
-<div class="validators">
-<a href="http://validator.w3.org/check?uri=referer"><img src="http://www.w3.org/Icons/valid-html401" alt="Valid HTML 4.01!" height="31" width="88" style="border:none;"></a>
-<img style="border:0" src="http://jigsaw.w3.org/css-validator/images/vcss" alt="Valid CSS!" >
-</div>    
-</body>
-</html>
-    TEMPLATE
-
-  end
-
-end
-
- - - diff --git a/test/samples/html.rb b/test/samples/html.rb deleted file mode 100644 index c18284a..0000000 --- a/test/samples/html.rb +++ /dev/null @@ -1,394 +0,0 @@ -$: << '..' -require 'coderay' - -tokens = CodeRay.scan DATA.read, :ruby -html = tokens.page(:tab_width => 2, :line_numbers => :table, :title => 'CodeRay HTML Encoder Example') - -puts html - -__END__ -require 'scanner' - -module CodeRay - - class RubyScanner < Scanner - - RESERVED_WORDS = [ - 'and', 'def', 'end', 'in', 'or', 'unless', 'begin', - 'defined?', 'ensure', 'module', 'redo', 'super', 'until', - 'BEGIN', 'break', 'do', 'next', 'rescue', 'then', - 'when', 'END', 'case', 'else', 'for', 'retry', - 'while', 'alias', 'class', 'elsif', 'if', 'not', 'return', - 'undef', 'yield', - ] - - DEF_KEYWORDS = ['def'] - MODULE_KEYWORDS = ['class', 'module'] - DEF_NEW_STATE = WordList.new(:initial). - add(DEF_KEYWORDS, :def_expected). - add(MODULE_KEYWORDS, :module_expected) - - WORDS_ALLOWING_REGEXP = [ - 'and', 'or', 'not', 'while', 'until', 'unless', 'if', 'elsif', 'when' - ] - REGEXP_ALLOWED = WordList.new(false). - add(WORDS_ALLOWING_REGEXP, :set) - - PREDEFINED_CONSTANTS = [ - 'nil', 'true', 'false', 'self', - 'DATA', 'ARGV', 'ARGF', '__FILE__', '__LINE__', - ] - - IDENT_KIND = WordList.new(:ident). - add(RESERVED_WORDS, :reserved). - add(PREDEFINED_CONSTANTS, :pre_constant) - - METHOD_NAME = / #{IDENT} [?!]? /xo - METHOD_NAME_EX = / - #{METHOD_NAME} # common methods: split, foo=, empty?, gsub! - | \*\*? # multiplication and power - | [-+~]@? # plus, minus - | [\/%&|^`] # division, modulo or format strings, &and, |or, ^xor, `system` - | \[\]=? # array getter and setter - | <=?>? | >=? # comparison, rocket operator - | << | >> # append or shift left, shift right - | ===? # simple equality and case equality - /ox - GLOBAL_VARIABLE = / \$ (?: #{IDENT} | \d+ | [~&+`'=\/,;_.<>!@0$?*":F\\] | -[a-zA-Z_0-9] ) /ox - - DOUBLEQ = / " [^"\#\\]* (?: (?: \#\{.*?\} | \#(?:$")? | \\. ) [^"\#\\]* )* "? /ox - SINGLEQ = / ' [^'\\]* (?: \\. [^'\\]* )* '? /ox - STRING = / #{SINGLEQ} | #{DOUBLEQ} /ox - SHELL = / ` [^`\#\\]* (?: (?: \#\{.*?\} | \#(?:$`)? | \\. ) [^`\#\\]* )* `? /ox - REGEXP = / \/ [^\/\#\\]* (?: (?: \#\{.*?\} | \#(?:$\/)? | \\. ) [^\/\#\\]* )* \/? /ox - - DECIMAL = /\d+(?:_\d+)*/ # doesn't recognize 09 as octal error - OCTAL = /0_?[0-7]+(?:_[0-7]+)*/ - HEXADECIMAL = /0x[0-9A-Fa-f]+(?:_[0-9A-Fa-f]+)*/ - BINARY = /0b[01]+(?:_[01]+)*/ - - EXPONENT = / [eE] [+-]? #{DECIMAL} /ox - FLOAT = / #{DECIMAL} (?: #{EXPONENT} | \. #{DECIMAL} #{EXPONENT}? ) / - INTEGER = /#{OCTAL}|#{HEXADECIMAL}|#{BINARY}|#{DECIMAL}/ - - def reset - super - @regexp_allowed = false - end - - def next_token - return if @scanner.eos? - - kind = :error - if @scanner.scan(/\s+/) # in every state - kind = :space - @regexp_allowed = :set if @regexp_allowed or @scanner.matched.index(?\n) # delayed flag setting - - elsif @state == :def_expected - if @scanner.scan(/ (?: (?:#{IDENT}(?:\.|::))* | (?:@@?|$)? #{IDENT}(?:\.|::) ) #{METHOD_NAME_EX} /ox) - kind = :method - @state = :initial - else - @scanner.scan(/./) - kind = :error - end - @state = :initial - - elsif @state == :module_expected - if @scanner.scan(/<;,|&!()\[\]{}~?] | \.\.?\.? | ::? /x) - kind = :operator - @regexp_allowed = :set if @scanner.matched[-1,1] =~ /[~=!<>|&^,\(\[+\-\/\*%]\z/ - elsif @scanner.scan(FLOAT) - kind = :float - elsif @scanner.scan(INTEGER) - kind = :integer - elsif @scanner.scan(/:(?:#{GLOBAL_VARIABLE}|#{METHOD_NAME_EX}|#{STRING})/ox) - kind = :global_variable - else - @scanner.scan(/./m) - end - end - - token = Token.new @scanner.matched, kind - - if kind == :regexp - token.text << @scanner.scan(/[eimnosux]*/) - end - - @regexp_allowed = (@regexp_allowed == :set) # delayed flag setting - - token - end - end - - ScannerList.register RubyScanner, 'ruby' - -end - -module CodeRay - require 'scanner' - - class Highlighter - - def initialize lang - @scanner = Scanner[lang].new - end - - def highlight code - @scanner.feed code - @scanner.all_tokens.map { |t| t.inspect }.join "\n" - end - - end - - class HTMLHighlighter < Highlighter - - ClassOfKind = { - :attribute_name => 'an', - :attribute_name_fat => 'af', - :attribute_value => 'av', - :attribute_value_fat => 'aw', - :bin => 'bi', - :char => 'ch', - :class => 'cl', - :class_variable => 'cv', - :color => 'cr', - :comment => 'c', - :constant => 'co', - :definition => 'df', - :directive => 'di', - :doc => 'do', - :doc_string => 'ds', - :exception => 'ex', - :error => 'er', - :float => 'fl', - :function => 'fu', - :global_variable => 'gv', - :hex => 'hx', - :include => 'ic', - :instance_variable => 'iv', - :integer => 'i', - :interpreted => 'in', - :label => 'la', - :local_variable => 'lv', - :oct => 'oc', - :operator_name => 'on', - :pre_constant => 'pc', - :pre_type => 'pt', - :predefined => 'pd', - :preprocessor => 'pp', - :regexp => 'rx', - :reserved => 'r', - :shell => 'sh', - :string => 's', - :symbol => 'sy', - :tag => 'ta', - :tag_fat => 'tf', - :tag_special => 'ts', - :type => 'ty', - :variable => 'v', - :xml_text => 'xt', - - :ident => :NO_HIGHLIGHT, - :operator => :NO_HIGHLIGHT, - :space => :NO_HIGHLIGHT, - } - ClassOfKind[:procedure] = ClassOfKind[:method] = ClassOfKind[:function] - ClassOfKind.default = ClassOfKind[:error] or raise 'no class found for :error!' - - def initialize lang, options = {} - super lang - - @HTML_TAB = ' ' * options.fetch(:tabs2space, 8) - case level = options.fetch(:level, 'xhtml') - when 'html' - @HTML_BR = "
\n" - when 'xhtml' - @HTML_BR = "
\n" - else - raise "Unknown HTML level: #{level}" - end - end - - def highlight code - @scanner.feed code - - out = '' - while t = @scanner.next_token - warn t.inspect if t.text.nil? - out << to_html(t) - end - TEMPLATE =~ /<%CONTENT%>/ - $` + out + $' - end - - private - def to_html token - css_class = ClassOfKind[token.kind] - if defined? ::DEBUG and not ClassOfKind.has_key? token.kind - warn "no token class found for :#{token.kind}" - end - - text = text_to_html token.text - if css_class == :NO_HIGHLIGHT - text - else - "#{text}" - end - end - - def text_to_html text - return '' if text.empty? - text = text.dup # important - if text.index(/["><&]/) - text.gsub!('&', '&') - text.gsub!('"', '"') - text.gsub!('>', '>') - text.gsub!('<', '<') - end - if text.index(/\s/) - text.gsub!("\n", @HTML_BR) - text.gsub!("\t", @HTML_TAB) - text.gsub!(/^ /, ' ') - text.gsub!(' ', '  ') - end - text - end - - TEMPLATE = <<-'TEMPLATE' - - - - - - -RubyBB BBCode - - - -
-<%CONTENT%> -
-
-Valid HTML 4.01! -Valid CSS! -
- - - TEMPLATE - - end - -end diff --git a/test/samples/html2.expected b/test/samples/html2.expected deleted file mode 100644 index c8ae56a..0000000 --- a/test/samples/html2.expected +++ /dev/null @@ -1,185 +0,0 @@ - - - - - CodeRay HTML Encoder Example - - - - - - - -
1
-2
-3
-4
-5
-6
-7
-8
-9
-10
-11
-
require 'coderay'
-
-# scan this file
-tokens = CodeRay.scan(File.read($0) * 1, :ruby)
-
-# output it with two styles of line numbers
-out = tokens.div(:line_numbers => :table)
-out << '<hr />'
-out << tokens.div(:line_numbers => :inline, :line_number_start => 8)
-
-puts out.page(:title => 'CodeRay HTML Encoder Example')
-
-
- 
 8 require 'coderay'
- 9 
-10 # scan this file
-11 tokens = CodeRay.scan(File.read($0) * 1, :ruby)
-12 
-13 # output it with two styles of line numbers
-14 out = tokens.div(:line_numbers => :table)
-15 out << '<hr />'
-16 out << tokens.div(:line_numbers => :inline, :line_number_start => 8)
-17 
-18 puts out.page(:title => 'CodeRay HTML Encoder Example')
-
-
- - - diff --git a/test/samples/html2.rb b/test/samples/html2.rb deleted file mode 100644 index 618d168..0000000 --- a/test/samples/html2.rb +++ /dev/null @@ -1,11 +0,0 @@ -require 'coderay' - -# scan this file -tokens = CodeRay.scan(File.read($0) * 1, :ruby) - -# output it with two styles of line numbers -out = tokens.div(:line_numbers => :table) -out << '
' -out << tokens.div(:line_numbers => :inline, :line_number_start => 8) - -puts out.page(:title => 'CodeRay HTML Encoder Example') diff --git a/test/samples/html_list.expected b/test/samples/html_list.expected deleted file mode 100644 index a4092c8..0000000 --- a/test/samples/html_list.expected +++ /dev/null @@ -1,160 +0,0 @@ - - - - - CodeRay HTML Encoder Example - - - - -
- 
-1 $: << '..'
- 0 require 'coderay'
- 1 
- 2 tokens = CodeRay.scan File.read(__FILE__), :ruby
- 3 html = tokens.html(:tab_width => 2, :line_numbers => :inline, :line_number_start => -1)
- 4 
- 5 puts html.page(:title => 'CodeRay HTML Encoder Example')
- 6 
- 7 commment = <<_
- 8 This code must be > 10 lines
- 9 because I want to test the correct adjustment of the line numbers.
-10 _
-
-
- - - diff --git a/test/samples/html_list.rb b/test/samples/html_list.rb deleted file mode 100644 index fdfa512..0000000 --- a/test/samples/html_list.rb +++ /dev/null @@ -1,12 +0,0 @@ -$: << '..' -require 'coderay' - -tokens = CodeRay.scan File.read(__FILE__), :ruby -html = tokens.html(:tab_width => 2, :line_numbers => :inline, :line_number_start => -1) - -puts html.page(:title => 'CodeRay HTML Encoder Example') - -commment = <<_ -This code must be > 10 lines -because I want to test the correct adjustment of the line numbers. -_ diff --git a/test/samples/load_encoder.expected b/test/samples/load_encoder.expected deleted file mode 100644 index 1cff356..0000000 --- a/test/samples/load_encoder.expected +++ /dev/null @@ -1,8 +0,0 @@ -CodeRay::Encoders::YAML is not defined; you must load it first. -Now it is loaded: CodeRay::Encoders::YAML -See? -Require is also possible: CodeRay::Encoders::Tokens -See? -Now load some mapped encoders: stats and plain. -Require all Encoders: -[[:count, CodeRay::Encoders::Count], [:debug, CodeRay::Encoders::Debug], [:div, CodeRay::Encoders::Div], [:html, CodeRay::Encoders::HTML], [:null, CodeRay::Encoders::Null], [:page, CodeRay::Encoders::Page], [:plain, :text], [:span, CodeRay::Encoders::Span], [:statistic, CodeRay::Encoders::Statistic], [:stats, CodeRay::Encoders::Statistic], [:text, CodeRay::Encoders::Text], [:tokens, CodeRay::Encoders::Tokens], [:xml, CodeRay::Encoders::XML], [:yaml, CodeRay::Encoders::YAML]] diff --git a/test/samples/load_encoder.rb b/test/samples/load_encoder.rb deleted file mode 100644 index 9594bfa..0000000 --- a/test/samples/load_encoder.rb +++ /dev/null @@ -1,25 +0,0 @@ -require 'coderay' - -begin - CodeRay::Encoders::YAML -rescue - puts 'CodeRay::Encoders::YAML is not defined; you must load it first.' -end - -yaml_encoder = CodeRay::Encoders[:yaml] -print 'Now it is loaded: ' -p yaml_encoder -puts 'See?' - -tokens_encoder = CodeRay.require_plugin 'CodeRay::Encoders/tokens' -print 'Require is also possible: ' -p tokens_encoder -puts 'See?' - -puts 'Now load some mapped encoders: stats and plain.' -CodeRay.require_plugin 'CodeRay::Encoders/stats' -CodeRay.require_plugin 'CodeRay::Encoders/plain' - -puts 'Require all Encoders:' -CodeRay::Encoders.load_all -p CodeRay::Encoders.plugin_hash.sort_by { |k,v| k.to_s } diff --git a/test/samples/load_scanner.expected b/test/samples/load_scanner.expected deleted file mode 100644 index a2d200d..0000000 --- a/test/samples/load_scanner.expected +++ /dev/null @@ -1,8 +0,0 @@ -CodeRay::Encoders::Ruby is not defined; you must load it first. -Now it is loaded: CodeRay::Scanners::Ruby -See? -Require is also possible: CodeRay::Scanners::C -See? -Now load some mapped scanners: cpp and plain. -Require all Scanners: -[[nil, :plain], [:c, CodeRay::Scanners::C], [:cpp, :c], [:delphi, CodeRay::Scanners::Delphi], [:html, CodeRay::Scanners::HTML], [:irb, :ruby], [:nitro, :nitro_xhtml], [:nitro_xhtml, CodeRay::Scanners::NitroXHTML], [:pascal, :delphi], [:plain, CodeRay::Scanners::Plaintext], [:plaintext, CodeRay::Scanners::Plaintext], [:rhtml, CodeRay::Scanners::RHTML], [:ruby, CodeRay::Scanners::Ruby], [:xhtml, :nitro_xhtml], [:xml, :html]] diff --git a/test/samples/load_scanner.rb b/test/samples/load_scanner.rb deleted file mode 100644 index 23be8a2..0000000 --- a/test/samples/load_scanner.rb +++ /dev/null @@ -1,25 +0,0 @@ -require 'coderay' - -begin - CodeRay::Scanners::Ruby -rescue - puts 'CodeRay::Encoders::Ruby is not defined; you must load it first.' -end - -ruby_scanner = CodeRay::Scanners[:ruby] -print 'Now it is loaded: ' -p ruby_scanner -puts 'See?' - -c_scanner = CodeRay.require_plugin 'CodeRay::Scanners/c' -print 'Require is also possible: ' -p c_scanner -puts 'See?' - -puts 'Now load some mapped scanners: cpp and plain.' -CodeRay.require_plugin 'CodeRay::Scanners/cpp' -CodeRay.require_plugin 'CodeRay::Scanners/plain' - -puts 'Require all Scanners:' -CodeRay::Scanners.load_all -p CodeRay::Scanners.plugin_hash.sort_by { |k,v| k.to_s } diff --git a/test/samples/more.expected b/test/samples/more.expected deleted file mode 100644 index 196904d..0000000 --- a/test/samples/more.expected +++ /dev/null @@ -1,2 +0,0 @@ -Input: 4983B, Output: 23484B -Take a look with your browser. diff --git a/test/samples/more.rb b/test/samples/more.rb deleted file mode 100644 index 0db7ba4..0000000 --- a/test/samples/more.rb +++ /dev/null @@ -1,205 +0,0 @@ -require 'coderay' - -c, ruby = DATA.read.split(/^---$/) -DATA.rewind -me = DATA.read[/.*^__END__$/m] -$input = c + ruby + me - -require 'benchmark' -time = Benchmark.realtime do - - # here CodeRay comes to play - hl = CodeRay.encoder(:html, :tab_width => 2, :line_numbers => :table, :wrap => :div) - c = hl.highlight c, :c - ruby = hl.highlight ruby, :ruby - me = hl.highlight me, :ruby - - body = %w[C Ruby Genereated\ by].zip([c, ruby, me]).map do |title, code| - "

#{title}

\n#{code}" - end.join - body = hl.class::Output.new(body, hl.css, :div).page! - - # CodeRay also provides a simple page generator - $output = body #hl.class.wrap_in_page body -end - -File.open('test.html', 'w') do |f| - f.write $output -end -puts 'Input: %dB, Output: %dB' % [$input.size, $output.size] -#puts 'Created "test.html" in %0.3f seconds (%d KB/s).' % [time, $input.size / 1024.0 / time] -puts 'Take a look with your browser.' - -__END__ -/********************************************************************** - - version.c - - - $Author: nobu $ - $Date: 2004/03/25 12:01:40 $ - created at: Thu Sep 30 20:08:01 JST 1993 - - Copyright (C) 1993-2003 Yukihiro Matsumoto - -**********************************************************************/ - -#include "ruby.h" -#include "version.h" -#include - -const char ruby_version[] = RUBY_VERSION; -const char ruby_release_date[] = RUBY_RELEASE_DATE; -const char ruby_platform[] = RUBY_PLATFORM; - -void -Init_version() -{ - VALUE v = rb_obj_freeze(rb_str_new2(ruby_version)); - VALUE d = rb_obj_freeze(rb_str_new2(ruby_release_date)); - VALUE p = rb_obj_freeze(rb_str_new2(ruby_platform)); - - rb_define_global_const("RUBY_VERSION", v); - rb_define_global_const("RUBY_RELEASE_DATE", d); - rb_define_global_const("RUBY_PLATFORM", p); -} - -void -ruby_show_version() -{ - printf("ruby %s (%s) [%s]\n", RUBY_VERSION, RUBY_RELEASE_DATE, RUBY_PLATFORM); -} - -void -ruby_show_copyright() -{ - printf("ruby - Copyright (C) 1993-%d Yukihiro Matsumoto\n", RUBY_RELEASE_YEAR); - exit(0); -} ---- -# -# = ostruct.rb: OpenStruct implementation -# -# Author:: Yukihiro Matsumoto -# Documentation:: Gavin Sinclair -# -# OpenStruct allows the creation of data objects with arbitrary attributes. -# See OpenStruct for an example. -# - -# -# OpenStruct allows you to create data objects and set arbitrary attributes. -# For example: -# -# require 'ostruct' -# -# record = OpenStruct.new -# record.name = "John Smith" -# record.age = 70 -# record.pension = 300 -# -# puts record.name # -> "John Smith" -# puts record.address # -> nil -# -# It is like a hash with a different way to access the data. In fact, it is -# implemented with a hash, and you can initialize it with one. -# -# hash = { "country" => "Australia", :population => 20_000_000 } -# data = OpenStruct.new(hash) -# -# p data # -> -# -class OpenStruct - # - # Create a new OpenStruct object. The optional +hash+, if given, will - # generate attributes and values. For example. - # - # require 'ostruct' - # hash = { "country" => "Australia", :population => 20_000_000 } - # data = OpenStruct.new(hash) - # - # p data # -> - # - # By default, the resulting OpenStruct object will have no attributes. - # - def initialize(hash=nil) - @table = {} - if hash - for k,v in hash - @table[k.to_sym] = v - new_ostruct_member(k) - end - end - end - - # Duplicate an OpenStruct object members. - def initialize_copy(orig) - super - @table = @table.dup - end - - def marshal_dump - @table - end - def marshal_load(x) - @table = x - @table.each_key{|key| new_ostruct_member(key)} - end - - def new_ostruct_member(name) - unless self.respond_to?(name) - self.instance_eval %{ - def #{name}; @table[:#{name}]; end - def #{name}=(x); @table[:#{name}] = x; end - } - end - end - - def method_missing(mid, *args) # :nodoc: - mname = mid.id2name - len = args.length - if mname =~ /=$/ - if len != 1 - raise ArgumentError, "wrong number of arguments (#{len} for 1)", caller(1) - end - if self.frozen? - raise TypeError, "can't modify frozen #{self.class}", caller(1) - end - mname.chop! - @table[mname.intern] = args[0] - self.new_ostruct_member(mname) - elsif len == 0 - @table[mid] - else - raise NoMethodError, "undefined method `#{mname}' for #{self}", caller(1) - end - end - - # - # Remove the named field from the object. - # - def delete_field(name) - @table.delete name.to_sym - end - - # - # Returns a string containing a detailed summary of the keys and values. - # - def inspect - str = "<#{self.class}" - for k,v in @table - str << " #{k}=#{v.inspect}" - end - str << ">" - end - - attr_reader :table # :nodoc: - protected :table - - # - # Compare this object and +other+ for equality. - # - def ==(other) - return false unless(other.kind_of?(OpenStruct)) - return @table == other.table - end -end diff --git a/test/samples/scanner.expected b/test/samples/scanner.expected deleted file mode 100644 index 5015168..0000000 --- a/test/samples/scanner.expected +++ /dev/null @@ -1,16 +0,0 @@ -C Code: if (*p == '{') nest++; - -> print only operators: -(*==)++; ------------------------------- - -Ruby Code: ruby_code(:can, BE, %r[q[ui]te #{ /comple/x },] => $-s, &?\xee) - -> has a string? -false - -> number of regexps? -2 - -> has a string? -"ruby_code" (ident), "(" (operator), ":can" (symbol), "," (operator), " " (space), "BE" (constant), "," (operator), " " (space), "%r[" (delimiter), "q" (content), "[" (nesting_delimiter), "ui" (content), "]" (nesting_delimiter), "te " (content), "#{" (inline_delimiter), " " (space), "/" (delimiter), "comple" (content), "/" (delimiter), "x" (modifier), " " (space), "}" (inline_delimiter), "," (content), "]" (delimiter), " " (space), "=" (operator), ">" (operator), " " (space), "$-s" (global_variable), "," (operator), " " (space), "&" (operator), "?\xee" (integer), ")" (operator) diff --git a/test/samples/scanner.rb b/test/samples/scanner.rb deleted file mode 100644 index 6a0245e..0000000 --- a/test/samples/scanner.rb +++ /dev/null @@ -1,36 +0,0 @@ -require 'coderay' - -c_code = "if (*p == '{') nest++;" -puts 'C Code: ' + c_code -puts - -c_scanner = CodeRay::Scanners[:c].new c_code - -puts '> print only operators:' -for text, kind in c_scanner - print text if kind == :operator -end -puts -puts '-' * 30 -puts - -ruby_code = %q!ruby_code(:can, BE, %r[q[ui]te #{ /comple/x },] => $-s, &?\xee)! -puts 'Ruby Code: ' + ruby_code -puts - -ruby_scanner = CodeRay::Scanners[:ruby].new ruby_code - -puts '> has a string?' -puts ruby_scanner. - any? { |text, kind| kind == :string } -puts - -puts '> number of regexps?' -puts ruby_scanner. - select { |token| token == [:open, :regexp] }.size -puts - -puts '> has a string?' -puts ruby_scanner. - reject { |text, kind| not text.is_a? String }. - map { |text, kind| %("#{text}" (#{kind})) }.join(', ') diff --git a/test/samples/server.rb b/test/samples/server.rb deleted file mode 100644 index ccdff32..0000000 --- a/test/samples/server.rb +++ /dev/null @@ -1,110 +0,0 @@ -# CodeRay dynamic highlighter - -unless ARGV.grep(/-[hv]|--(help|version)/).empty? - puts <<-USAGE -CodeRay Server 0.5 -$Id: demo_server.rb 113 2006-03-15 23:24:37Z murphy $ - -Usage: - 1) Start this and your browser. - 2) Go to http://localhost:2468/? - and you should get the highlighted version. - -Parameters: - -d Debug mode; reload CodeRay engine for every file. - (prepare for MANY "already initialized" and "method redefined" - messages - ingore it.) - - ... More to come. - USAGE - exit -end - -require 'webrick' -require 'pathname' - -class << File - alias dir? directory? -end - -require 'erb' -include ERB::Util -def url_decode s - s.to_s.gsub(/%([0-9a-f]{2})/i) { [$1.hex].pack 'C' } -end - -class String - def to_link name = File.basename(self) - "#{name}" - end -end - -require 'coderay' -class CodeRayServlet < WEBrick::HTTPServlet::AbstractServlet - - STYLE = 'style="font-family: sans-serif; color: navy;"' - BANNER = '

Highlighted by CodeRay

' - - def do_GET req, res - q = req.query_string || '' - args = Hash[*q.scan(/(.*?)=(.*?)(?:&|$)/).flatten].each_value { |v| v.replace url_decode(v) } - path = args.fetch 'path', '.' - - backlinks = '

current path: %s
' % html_escape(path) + - (Pathname.new(path) + '..').cleanpath.to_s.to_link('up') + ' - ' + - '.'.to_link('current') + '

' - - res.body = - if File.dir? path - path = Pathname.new(path).cleanpath.to_s - dirs, files = Dir[File.join(path, '*')].sort.partition { |p| File.dir? p } - - page = "" - page << backlinks - - page << '
' - page << "
Directories
\n" + dirs.map do |p| - "
#{p.to_link}
\n" - end.join << "\n" - page << "
Files
\n" + files.map do |p| - "
#{p.to_link}
\n" - end.join << "\n" - page << "
\n" - page << "#{BANNER}" - - elsif File.exist? path - if $DEBUG - $".delete_if { |f| f =~ /coderay/ } - require 'coderay' - end - div = CodeRay.scan_file(path).html :tab_width => 8, :wrap => :div, :hint => :info - div.replace <<-DIV -
- #{backlinks} -#{div} -
- #{BANNER} - DIV - div.page - end - - res['Content-Type'] = 'text/html' - end -end - -# This port is taken by "qip_msgd" - I don't know that. Do you? -module CodeRay - PORT = 0xC0DE / 20 -end - -server = WEBrick::HTTPServer.new :Port => CodeRay::PORT - -server.mount '/', CodeRayServlet - -server.mount_proc '/version' do |req, res| - res.body = 'CodeRay::Version = ' + CodeRay::Version - res['Content-Type'] = "text/plain" -end - -trap("INT") { server.shutdown } -server.start diff --git a/test/samples/simple.expected b/test/samples/simple.expected deleted file mode 100644 index b3d7875..0000000 --- a/test/samples/simple.expected +++ /dev/null @@ -1 +0,0 @@ -puts 'Hello, world!' diff --git a/test/samples/simple.rb b/test/samples/simple.rb deleted file mode 100644 index a3129b0..0000000 --- a/test/samples/simple.rb +++ /dev/null @@ -1,10 +0,0 @@ - -# Load CodeRay -# If this doesn't work, try ruby -rubygems. -require 'coderay' - -# Generate HTML page for Ruby code. -page = CodeRay.scan("puts 'Hello, world!'", :ruby).span - -# Print it -puts page diff --git a/test/samples/stream.rb b/test/samples/stream.rb deleted file mode 100644 index 7ed8a22..0000000 --- a/test/samples/stream.rb +++ /dev/null @@ -1,25 +0,0 @@ -require 'coderay' - -code = File.read($0) * 500 -puts "Size of code: %d KB" % [code.size / 1024] - -puts "Use your system's memory tracker to see how much RAM this takes." -print 'Press some key to continue...'; gets - -require 'benchmark' -e = CodeRay.encoder(:div) -for do_stream in [true, false] - puts "Scanning and encoding in %s mode, please wait..." % - [do_stream ? 'streaming' : 'normal'] - output = '' - time = Benchmark.realtime do - if do_stream - output = e.encode_stream(code, :ruby) - else - output = e.encode_tokens(t = CodeRay.scan(code, :ruby)) - end - end - puts 'Finished after %4.2f seconds.' % time - puts "Size of output: %d KB" % [output.size / 1024] - print 'Press some key to continue...'; gets -end diff --git a/test/samples/stream2.expected b/test/samples/stream2.expected deleted file mode 100644 index 83aee98..0000000 --- a/test/samples/stream2.expected +++ /dev/null @@ -1,2 +0,0 @@ -kind: regexp, text size: 5. -kind: space, text size: 1. diff --git a/test/samples/stream2.rb b/test/samples/stream2.rb deleted file mode 100644 index d43cc9a..0000000 --- a/test/samples/stream2.rb +++ /dev/null @@ -1,8 +0,0 @@ -require 'coderay' - -token_stream = CodeRay::TokenStream.new do |kind, text| - puts 'kind: %s, text size: %d.' % [kind, text.size] -end - -token_stream << [:regexp, '/\d+/'] << [:space, "\n"] -#-> kind: rexpexp, text size: 5. diff --git a/test/samples/suite.rb b/test/samples/suite.rb deleted file mode 100644 index cfe53c0..0000000 --- a/test/samples/suite.rb +++ /dev/null @@ -1,86 +0,0 @@ -mydir = File.dirname(__FILE__) -$:.unshift mydir + '/../../lib/' - -$VERBOSE = true - -require 'test/unit' -include Test::Unit - -class CodeRaySuite < TestCase - - def self.dir &block - @dir ||= File.dirname(__FILE__) - if block - Dir.chdir @dir, &block - end - @dir - end - - def dir &block - self.class.dir(&block) - end - - def test_ALL - dir do - for input in Dir["*.rb"] - %w(server.rb stream.rb suite.rb) - next if input[/^load_/] - puts "[ testing #{input}... ]" - name = File.basename(input, ".rb") - output = name + '.expected' - code = File.open(input, 'rb') { |f| break f.read } - - result = `ruby -wI../../lib #{input}` - - diff = output.sub '.expected', '.diff' - File.delete diff if File.exist? diff - computed = output.sub '.expected', '.actual' - if File.exist? output - expected = File.read output - ok = expected == result - unless ok - File.open(computed, 'w') { |f| f.write result } - `diff #{output} #{computed} > #{diff}` - puts "Test failed; output written to #{diff}." - end - assert(ok, "Output error: #{computed} != #{output}") - else - File.open(output, 'w') do |f| f.write result end - puts "New test: #{output}" - end - - end - end - end - -end - -require 'test/unit/testsuite' -$suite = TestSuite.new 'CodeRay Demos Test' -$suite << CodeRaySuite.suite - -def load_suite name - begin - require name + '/suite.rb' - rescue LoadError - $stderr.puts <<-ERR - -!! Folder #{File.split(__FILE__).first + '/' + name} not found - - ERR - false - end -end - -if subsuite = ARGV.find { |a| break $1 if a[/^([^-].*)/] } - load_suite(subsuite) or exit -else - Dir[mydir + '/*/'].each { |suite| load_suite suite } -end - -if ARGV.include? '-f' - require 'test/unit/ui/fox/testrunner' - UI::Fox::TestRunner.run $suite -else - require 'test/unit/ui/console/testrunner' - UI::Console::TestRunner.run $suite -end diff --git a/test/samples/tokens.expected b/test/samples/tokens.expected deleted file mode 100644 index 747904e..0000000 --- a/test/samples/tokens.expected +++ /dev/null @@ -1 +0,0 @@ -[["puts", :ident], [" ", :space], ["3", :integer], [" ", :space], ["+", :operator], [" ", :space], ["4", :integer], [",", :operator], [" ", :space], [:open, :string], ["'", :delimiter], ["3 + 4", :content], ["'", :delimiter], [:close, :string]] diff --git a/test/samples/tokens.rb b/test/samples/tokens.rb deleted file mode 100644 index 91b8abb..0000000 --- a/test/samples/tokens.rb +++ /dev/null @@ -1,3 +0,0 @@ -require 'coderay' - -p CodeRay.scan("puts 3 + 4, '3 + 4'", :ruby) -- cgit v1.2.1 From d3197be3f207f8fcf52954d8815a0ea1948d25a4 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 22 Feb 2014 00:33:54 +0100 Subject: fix for #163 (SQL scanner), declare 1.1.1 --- Changes.textile | 4 ++++ lib/coderay/scanners/sql.rb | 40 ++++++++++++++++------------------------ lib/coderay/version.rb | 2 +- 3 files changed, 21 insertions(+), 25 deletions(-) diff --git a/Changes.textile b/Changes.textile index 8e388e0..137460a 100644 --- a/Changes.textile +++ b/Changes.textile @@ -2,6 +2,10 @@ h1=. CodeRay Version History p=. _This files lists all changes in the CodeRay library since the 0.9.8 release._ +h2. Changes in 1.1.1 + +* SQL scanner: fix open strings [#163, thanks to Adam] + h2. Changes in 1.1 New scanners: diff --git a/lib/coderay/scanners/sql.rb b/lib/coderay/scanners/sql.rb index 93aeaf3..c25f6d2 100644 --- a/lib/coderay/scanners/sql.rb +++ b/lib/coderay/scanners/sql.rb @@ -57,6 +57,12 @@ module Scanners STRING_PREFIXES = /[xnb]|_\w+/i + STRING_CONTENT_PATTERN = { + '"' => / (?: [^\\"] | "" )+ /x, + "'" => / (?: [^\\'] | '' )+ /x, + '`' => / (?: [^\\`] | `` )+ /x, + } + def scan_tokens encoder, options state = :initial @@ -115,40 +121,26 @@ module Scanners end elsif state == :string - if match = scan(/[^\\"'`]+/) - string_content << match - next + if match = scan(STRING_CONTENT_PATTERN[string_type]) + encoder.text_token match, :content elsif match = scan(/["'`]/) if string_type == match if peek(1) == string_type # doubling means escape - string_content << string_type << getch - next - end - unless string_content.empty? - encoder.text_token string_content, :content - string_content = '' + encoder.text_token match + getch, :content + else + encoder.text_token match, :delimiter + encoder.end_group :string + state = :initial + string_type = nil end - encoder.text_token match, :delimiter - encoder.end_group :string - state = :initial - string_type = nil else - string_content << match + encoder.text_token match, :content end elsif match = scan(/ \\ (?: #{ESCAPE} | #{UNICODE_ESCAPE} ) /mox) - unless string_content.empty? - encoder.text_token string_content, :content - string_content = '' - end encoder.text_token match, :char elsif match = scan(/ \\ . /mox) - string_content << match - next + encoder.text_token match, :content elsif match = scan(/ \\ | $ /x) - unless string_content.empty? - encoder.text_token string_content, :content - string_content = '' - end encoder.text_token match, :error unless match.empty? encoder.end_group :string state = :initial diff --git a/lib/coderay/version.rb b/lib/coderay/version.rb index 4b4f085..7ea3f70 100644 --- a/lib/coderay/version.rb +++ b/lib/coderay/version.rb @@ -1,3 +1,3 @@ module CodeRay - VERSION = '1.1.0' + VERSION = '1.1.1' end -- cgit v1.2.1 From f1d1e5bd49bc862be3872928e9b7652051bf2cef Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 22 Feb 2014 00:57:54 +0100 Subject: update term-ansicolor dependency --- Gemfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Gemfile b/Gemfile index 15a71ae..9d46501 100644 --- a/Gemfile +++ b/Gemfile @@ -9,7 +9,7 @@ group :development do gem "bundler", ">= 1.0.0" gem "rake" gem "RedCloth", RUBY_PLATFORM == 'java' ? ">= 4.2.7" : ">= 4.0.3" - gem "term-ansicolor", '~> 1.2.2' + gem "term-ansicolor", '~> 1.3.0' gem "shoulda-context", "~> 1.1.2" gem "json" if RUBY_VERSION < '1.9' gem "rdoc" -- cgit v1.2.1 From da39961195a297293bfe274e4f60c607ad21eada Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 17 May 2014 21:16:38 +0200 Subject: HTML envoder keeps \t with tab_width: false Fixes #170 --- lib/coderay/encoders/html.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/encoders/html.rb b/lib/coderay/encoders/html.rb index d2ebb5a..c7c0c2d 100644 --- a/lib/coderay/encoders/html.rb +++ b/lib/coderay/encoders/html.rb @@ -180,7 +180,7 @@ module Encoders @break_lines = (options[:break_lines] == true) - @HTML_ESCAPE = HTML_ESCAPE.merge("\t" => ' ' * options[:tab_width]) + @HTML_ESCAPE = HTML_ESCAPE.merge("\t" => options[:tab_width] ? ' ' * options[:tab_width] : "\t") @opened = [] @last_opened = nil -- cgit v1.2.1 From 38e3338f2824ddc519097a7ab9d03790025f802c Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sun, 18 May 2014 00:08:48 +0200 Subject: relax gem version requirements --- Gemfile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Gemfile b/Gemfile index 9d46501..0977943 100644 --- a/Gemfile +++ b/Gemfile @@ -6,11 +6,11 @@ gemspec # Add dependencies to develop your gem here. # Include everything needed to run rake, tests, features, etc. group :development do - gem "bundler", ">= 1.0.0" + gem "bundler" gem "rake" gem "RedCloth", RUBY_PLATFORM == 'java' ? ">= 4.2.7" : ">= 4.0.3" - gem "term-ansicolor", '~> 1.3.0' - gem "shoulda-context", "~> 1.1.2" + gem "term-ansicolor" + gem "shoulda-context" gem "json" if RUBY_VERSION < '1.9' gem "rdoc" end -- cgit v1.2.1 From e1aa98e7386609fd4c84bdcd2c3ea4b26663c8b7 Mon Sep 17 00:00:00 2001 From: BenBasson Date: Wed, 11 Jun 2014 22:47:51 +0100 Subject: Allow $ in SQL object names. --- lib/coderay/scanners/sql.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/scanners/sql.rb b/lib/coderay/scanners/sql.rb index c25f6d2..7d57f77 100644 --- a/lib/coderay/scanners/sql.rb +++ b/lib/coderay/scanners/sql.rb @@ -96,7 +96,7 @@ module Scanners state = :string encoder.text_token match, :delimiter - elsif match = scan(/ @? [A-Za-z_][A-Za-z_0-9]* /x) + elsif match = scan(/ @? [A-Za-z_][A-Za-z_0-9\$]* /x) encoder.text_token match, name_expected ? :ident : (match[0] == ?@ ? :variable : IDENT_KIND[match]) name_expected = false -- cgit v1.2.1 From 20b921c6c94dfdf793b8031a2dafc0f7a3df5fb2 Mon Sep 17 00:00:00 2001 From: Ryunosuke SATO Date: Wed, 12 Nov 2014 23:18:36 +0900 Subject: Test against Ruby 2.1 and 2.2 on Travis CI --- .travis.yml | 2 ++ Gemfile | 1 + README.markdown | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 6d926f3..8e18c0a 100644 --- a/.travis.yml +++ b/.travis.yml @@ -3,6 +3,8 @@ rvm: - ree - 1.9.3 - 2.0.0 + - 2.1 + - 2.2 - ruby-head - jruby-18mode - jruby-19mode diff --git a/Gemfile b/Gemfile index 0977943..0fae04b 100644 --- a/Gemfile +++ b/Gemfile @@ -11,6 +11,7 @@ group :development do gem "RedCloth", RUBY_PLATFORM == 'java' ? ">= 4.2.7" : ">= 4.0.3" gem "term-ansicolor" gem "shoulda-context" + gem "test-unit" gem "json" if RUBY_VERSION < '1.9' gem "rdoc" end diff --git a/README.markdown b/README.markdown index e23f603..15b3447 100644 --- a/README.markdown +++ b/README.markdown @@ -16,7 +16,7 @@ You put your code in, and you get it back colored; Keywords, strings, floats, co ### Dependencies -CodeRay needs Ruby 1.8.7, 1.9.3 or 2.0. It also runs on JRuby. +CodeRay needs Ruby 1.8.7, 1.9.3 or 2.0+. It also runs on JRuby. ## Example Usage -- cgit v1.2.1 From e5624a07e95cc7a3c704a4d08cddea582adc7f31 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 21 Mar 2015 03:44:49 +0100 Subject: prevent running out of regexp stack --- lib/coderay/scanners/diff.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/scanners/diff.rb b/lib/coderay/scanners/diff.rb index fd1aed6..74a6c27 100644 --- a/lib/coderay/scanners/diff.rb +++ b/lib/coderay/scanners/diff.rb @@ -100,7 +100,7 @@ module Scanners next elsif match = scan(/-/) deleted_lines_count += 1 - if options[:inline_diff] && deleted_lines_count == 1 && (changed_lines_count = 1 + check(/.*(?:\n\-.*)*/).count("\n")) && match?(/(?>.*(?:\n\-.*){#{changed_lines_count - 1}}(?:\n\+.*){#{changed_lines_count}})$(?!\n\+)/) + if options[:inline_diff] && deleted_lines_count == 1 && (changed_lines_count = 1 + check(/.*(?:\n\-.*)*/).count("\n")) && changed_lines_count <= 100_000 && match?(/(?>.*(?:\n\-.*){#{changed_lines_count - 1}}(?:\n\+.*){#{changed_lines_count}})$(?!\n\+)/) deleted_lines = Array.new(changed_lines_count) { |i| skip(/\n\-/) if i > 0; scan(/.*/) } inserted_lines = Array.new(changed_lines_count) { |i| skip(/\n\+/) ; scan(/.*/) } -- cgit v1.2.1 From cb18c6af5f53cba503fb9704ce656596ae3db075 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Wed, 22 Apr 2015 00:57:21 +0200 Subject: don't lie in --help output; thanks @Quintus --- bin/coderay | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/coderay b/bin/coderay index 889ae72..130a50b 100755 --- a/bin/coderay +++ b/bin/coderay @@ -35,7 +35,7 @@ defaults: common: coderay file.rb # highlight file to terminal - coderay file.rb > file.html # highlight file to HTML page + coderay file.rb -page > file.html # highlight file to HTML page coderay file.rb -div > file.html # highlight file to HTML snippet configure output: -- cgit v1.2.1 From 080f8a8225cb911d037d1f6e58e581dec9558c58 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 11:40:13 +0100 Subject: add support for Ruby 2.1 number literal suffixes --- lib/coderay/scanners/ruby.rb | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/coderay/scanners/ruby.rb b/lib/coderay/scanners/ruby.rb index 80165ca..0492a55 100644 --- a/lib/coderay/scanners/ruby.rb +++ b/lib/coderay/scanners/ruby.rb @@ -191,7 +191,10 @@ module Scanners encoder.text_token match, :error method_call_expected = false else - encoder.text_token match, self[1] ? :float : :integer # TODO: send :hex/:octal/:binary + kind = self[1] ? :float : :integer # TODO: send :hex/:octal/:binary + match << 'r' if match !~ /e/i && scan(/r/) + match << 'i' if scan(/i/) + encoder.text_token match, kind end value_expected = false -- cgit v1.2.1 From 39cbd37815f65f21e0433f4da4cf5fbeda2e1e3f Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 12:06:26 +0100 Subject: add support for Ruby 2.2 quoted hash keys KNOWN ISSUE: string interpolation will not work! --- lib/coderay/scanners/ruby.rb | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/lib/coderay/scanners/ruby.rb b/lib/coderay/scanners/ruby.rb index 0492a55..165d66b 100644 --- a/lib/coderay/scanners/ruby.rb +++ b/lib/coderay/scanners/ruby.rb @@ -164,15 +164,18 @@ module Scanners end elsif match = scan(/ ' (?:(?>[^'\\]*) ')? | " (?:(?>[^"\\\#]*) ")? /mx) - encoder.begin_group :string if match.size == 1 + encoder.begin_group :string encoder.text_token match, :delimiter state = self.class::StringState.new :string, match == '"', match # important for streaming else + kind = value_expected == true && scan(/:/) ? :key : :string + encoder.begin_group kind encoder.text_token match[0,1], :delimiter encoder.text_token match[1..-2], :content if match.size > 2 encoder.text_token match[-1,1], :delimiter - encoder.end_group :string + encoder.end_group kind + encoder.text_token ':', :operator if kind == :key value_expected = false end -- cgit v1.2.1 From d9d1eedcb235b371683eed22a6e4217caef73ffa Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 12:08:21 +0100 Subject: add support for Ruby 2.3 safe navigation operator --- lib/coderay/scanners/ruby.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/scanners/ruby.rb b/lib/coderay/scanners/ruby.rb index 165d66b..24ab71f 100644 --- a/lib/coderay/scanners/ruby.rb +++ b/lib/coderay/scanners/ruby.rb @@ -201,7 +201,7 @@ module Scanners end value_expected = false - elsif match = scan(/ [-+!~^\/]=? | [:;] | [*|&]{1,2}=? | >>? /x) + elsif match = scan(/ [-+!~^\/]=? | [:;] | &\. | [*|&]{1,2}=? | >>? /x) value_expected = true encoder.text_token match, :operator -- cgit v1.2.1 From 376884d457ac7953914cc84b94fe6404cd904fe0 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 12:10:18 +0100 Subject: add support for Ruby 2.3 squiggly heredoc --- lib/coderay/scanners/ruby/patterns.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/scanners/ruby/patterns.rb b/lib/coderay/scanners/ruby/patterns.rb index 0b36e13..3dd6ad5 100644 --- a/lib/coderay/scanners/ruby/patterns.rb +++ b/lib/coderay/scanners/ruby/patterns.rb @@ -114,7 +114,7 @@ module Scanners # NOTE: This is not completely correct, but # nobody needs heredoc delimiters ending with \n. HEREDOC_OPEN = / - << (-)? # $1 = float + << ([-~])? # $1 = float (?: ( [A-Za-z_0-9]+ ) # $2 = delim | -- cgit v1.2.1 From 415498eaf9417cf30656c4a745eef0409b214afc Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 13:11:31 +0100 Subject: allow indentation of squiggly heredoc delimiter --- lib/coderay/scanners/ruby.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/scanners/ruby.rb b/lib/coderay/scanners/ruby.rb index 24ab71f..f7feb46 100644 --- a/lib/coderay/scanners/ruby.rb +++ b/lib/coderay/scanners/ruby.rb @@ -214,7 +214,7 @@ module Scanners encoder.end_group kind heredocs ||= [] # create heredocs if empty heredocs << self.class::StringState.new(kind, quote != "'", delim, - self[1] == '-' ? :indented : :linestart) + self[1] ? :indented : :linestart) value_expected = false elsif value_expected && match = scan(/#{patterns::FANCY_STRING_START}/o) -- cgit v1.2.1 From c33f3f5c43064f7b468a59e086dc4a9a4f949ff7 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 13:17:23 +0100 Subject: check for keys with escape sequences, too --- lib/coderay/scanners/ruby.rb | 5 +++-- lib/coderay/scanners/ruby/string_state.rb | 8 ++++++++ 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/lib/coderay/scanners/ruby.rb b/lib/coderay/scanners/ruby.rb index f7feb46..5b8de42 100644 --- a/lib/coderay/scanners/ruby.rb +++ b/lib/coderay/scanners/ruby.rb @@ -165,9 +165,10 @@ module Scanners elsif match = scan(/ ' (?:(?>[^'\\]*) ')? | " (?:(?>[^"\\\#]*) ")? /mx) if match.size == 1 - encoder.begin_group :string + kind = check(self.class::StringState.simple_key_pattern(match)) ? :key : :string + encoder.begin_group kind encoder.text_token match, :delimiter - state = self.class::StringState.new :string, match == '"', match # important for streaming + state = self.class::StringState.new kind, match == '"', match # important for streaming else kind = value_expected == true && scan(/:/) ? :key : :string encoder.begin_group kind diff --git a/lib/coderay/scanners/ruby/string_state.rb b/lib/coderay/scanners/ruby/string_state.rb index 28ddd6c..93e7208 100644 --- a/lib/coderay/scanners/ruby/string_state.rb +++ b/lib/coderay/scanners/ruby/string_state.rb @@ -37,6 +37,14 @@ module Scanners end end + def self.simple_key_pattern delim + if delim == "'" + / (?> (?: [^\\']+ | \\. )* ) ' : /mx + else + / (?> (?: [^\\"\#]+ | \\. | \#\$[\\"] | \#(?!\{) )* ) " : /mx + end + end + def initialize kind, interpreted, delim, heredoc = false if heredoc pattern = heredoc_pattern delim, interpreted, heredoc == :indented -- cgit v1.2.1 From 036fb3291274ed87f106bdbeb65bbd10b4c561f9 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 13:39:08 +0100 Subject: skip over interpolation if not nested --- lib/coderay/scanners/ruby/string_state.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/scanners/ruby/string_state.rb b/lib/coderay/scanners/ruby/string_state.rb index 93e7208..95f1e83 100644 --- a/lib/coderay/scanners/ruby/string_state.rb +++ b/lib/coderay/scanners/ruby/string_state.rb @@ -41,7 +41,7 @@ module Scanners if delim == "'" / (?> (?: [^\\']+ | \\. )* ) ' : /mx else - / (?> (?: [^\\"\#]+ | \\. | \#\$[\\"] | \#(?!\{) )* ) " : /mx + / (?> (?: [^\\"\#]+ | \\. | \#\$[\\"] | \#\{[^\{\}]+\} | \#(?!\{) )* ) " : /mx end end -- cgit v1.2.1 From 998d1fc874d28759f5b9e4376a0e82809c8fc828 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 13:43:09 +0100 Subject: changelog --- Changes.textile | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Changes.textile b/Changes.textile index 137460a..3c389ff 100644 --- a/Changes.textile +++ b/Changes.textile @@ -5,6 +5,10 @@ p=. _This files lists all changes in the CodeRay library since the 0.9.8 release h2. Changes in 1.1.1 * SQL scanner: fix open strings [#163, thanks to Adam] +* Ruby scanner: Accept number literal suffixes @r@ and @i@ (Ruby 2.1) +* Ruby scanner: Accept quoted hash keys like @{ "a": boss }@ (Ruby 2.2) +* Ruby scanner: Accept save navigation operator @&.@ (Ruby 2.3) +* Ruby scanner: Accept squiggly heredoc @<<~@ (Ruby 2.3) h2. Changes in 1.1 -- cgit v1.2.1 From a14639c31bbe33c23853a66d6feb817da4248e1a Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 13:44:18 +0100 Subject: don't ruin indentation --- lib/coderay/scanners/ruby/patterns.rb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/coderay/scanners/ruby/patterns.rb b/lib/coderay/scanners/ruby/patterns.rb index 3dd6ad5..e5a156d 100644 --- a/lib/coderay/scanners/ruby/patterns.rb +++ b/lib/coderay/scanners/ruby/patterns.rb @@ -114,7 +114,7 @@ module Scanners # NOTE: This is not completely correct, but # nobody needs heredoc delimiters ending with \n. HEREDOC_OPEN = / - << ([-~])? # $1 = float + << ([-~])? # $1 = float (?: ( [A-Za-z_0-9]+ ) # $2 = delim | -- cgit v1.2.1 From 57de8d31099fcf4004255cebacf6482b98d51340 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 14:32:04 +0100 Subject: try Travis new infrastructure --- .travis.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.travis.yml b/.travis.yml index 8e18c0a..9244ff4 100644 --- a/.travis.yml +++ b/.travis.yml @@ -21,3 +21,4 @@ matrix: - rvm: rbx-18mode - rvm: rbx-19mode script: "rake test" # test:scanners" +sudo: false -- cgit v1.2.1 From 153f9fb053e7e59af7ac34744265a608c1c90ff7 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 14:35:45 +0100 Subject: add Ruby 2.3 --- .travis.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.travis.yml b/.travis.yml index 9244ff4..f815698 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,9 +2,10 @@ rvm: - 1.8.7 - ree - 1.9.3 - - 2.0.0 + - 2.0 - 2.1 - 2.2 + - 2.3.0 - ruby-head - jruby-18mode - jruby-19mode -- cgit v1.2.1 From ae5d868a13ac722e49f0c83080ee2e05ab8d9aa8 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 14:40:46 +0100 Subject: fix issue with tins on older Ruby versions --- Gemfile | 1 + 1 file changed, 1 insertion(+) diff --git a/Gemfile b/Gemfile index 0fae04b..6d3a176 100644 --- a/Gemfile +++ b/Gemfile @@ -10,6 +10,7 @@ group :development do gem "rake" gem "RedCloth", RUBY_PLATFORM == 'java' ? ">= 4.2.7" : ">= 4.0.3" gem "term-ansicolor" + gem 'tins', '~> 1.6.0' gem "shoulda-context" gem "test-unit" gem "json" if RUBY_VERSION < '1.9' -- cgit v1.2.1 From 3e4bb6a660f8f341eca9a87a310d00170ec40872 Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 15:31:41 +0100 Subject: add changelog for #164 --- Changes.textile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Changes.textile b/Changes.textile index 3c389ff..bf319ef 100644 --- a/Changes.textile +++ b/Changes.textile @@ -4,7 +4,8 @@ p=. _This files lists all changes in the CodeRay library since the 0.9.8 release h2. Changes in 1.1.1 -* SQL scanner: fix open strings [#163, thanks to Adam] +* SQL scanner: Allow @$@ signs in SQL identifiers [#164, thanks to jasir and Ben Basson] +* SQL scanner: Fix open strings [#163, thanks to Adam] * Ruby scanner: Accept number literal suffixes @r@ and @i@ (Ruby 2.1) * Ruby scanner: Accept quoted hash keys like @{ "a": boss }@ (Ruby 2.2) * Ruby scanner: Accept save navigation operator @&.@ (Ruby 2.3) -- cgit v1.2.1 From 7f1f2287650c3f3da75ffe6d9e79793dfcc7a67d Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 15:39:51 +0100 Subject: document new option to keep tabs --- lib/coderay/encoders/html.rb | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/coderay/encoders/html.rb b/lib/coderay/encoders/html.rb index c7c0c2d..942b9c8 100644 --- a/lib/coderay/encoders/html.rb +++ b/lib/coderay/encoders/html.rb @@ -25,7 +25,8 @@ module Encoders # == Options # # === :tab_width - # Convert \t characters to +n+ spaces (a number.) + # Convert \t characters to +n+ spaces (a number or false.) + # false will keep tab characters untouched. # # Default: 8 # -- cgit v1.2.1 From 0a1f500d524ff0fb5eeafef051ccbb641954a87a Mon Sep 17 00:00:00 2001 From: Kornelius Kalnbach Date: Sat, 13 Feb 2016 15:41:25 +0100 Subject: add older changes to changelog --- Changes.textile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Changes.textile b/Changes.textile index bf319ef..1276a33 100644 --- a/Changes.textile +++ b/Changes.textile @@ -10,6 +10,8 @@ h2. Changes in 1.1.1 * Ruby scanner: Accept quoted hash keys like @{ "a": boss }@ (Ruby 2.2) * Ruby scanner: Accept save navigation operator @&.@ (Ruby 2.3) * Ruby scanner: Accept squiggly heredoc @<<~@ (Ruby 2.3) +* Diff scanner: Prevent running out of regexp stack. +* HTML encoder: You can keep tabs intact now by setting @tab_width: false@. h2. Changes in 1.1 -- cgit v1.2.1