path: root/Changes.textile

h1=. CodeRay Version History

p=. _This files lists all changes in the CodeRay library since the 0.9.8 release._

h2. Changes in 1.1.3

* Tokens: Ensure Ruby 2.6 compatibility. [#233, thanks to Jun Aruga]
* SQL scanner: Add @numeric@ data type. [#223, thanks to m16a1]
* Java scanner: Add @var@ as type. [#229, thanks to Davide Angelocola]
* Gem: Fix deprecation warning. [#246, thanks to David Rodríguez]

h2. Changes in 1.1.2

* Ruby future: Add support for frozen string literals. [#211, thanks to Pat Allan]
* C++ scanner: Add C++11 keywords. [#195, thanks to Johnny Willemsen]
* Haml scanner: Allow @-@ in tags.
* Java scanner: Allow Unicode characters in identifiers. [#212, thanks to t-gergely]

h2. Changes in 1.1.1

* SQL scanner: Allow @$@ signs in SQL identifiers [#164, thanks to jasir and Ben Basson]
* SQL scanner: Fix open strings [#163, thanks to Adam]
* Ruby scanner: Accept number literal suffixes @r@ and @i@ (Ruby 2.1)
* Ruby scanner: Accept quoted hash keys like @{ "a": boss }@ (Ruby 2.2)
* Ruby scanner: Accept save navigation operator @&.@ (Ruby 2.3)
* Ruby scanner: Accept squiggly heredoc @<<~@ (Ruby 2.3)
* Diff scanner: Prevent running out of regexp stack.
* HTML encoder: You can keep tabs intact now by setting @tab_width: false@.
* Alpha style: Tweaked colors for @:function@ group with @:content@.
* File structure: One module per file, autoload CodeRay::Version, paths follow namespace hierarchy.

h2. Changes in 1.1

New scanners:

* Go [#28, thanks to Eric Guo and Nathan Youngman]
* Lua [#21, #22, thanks to Quintus]
* Sass [#93]
* Taskpaper [#39, thanks to shimomura]

More new stuff:

* @.xaml@ file type [#121, thanks to Kozman Bálint]
* recognize @Guardfile@, @Vagrantfile@, and @Appraisals@ as Ruby files [#121, thanks to Kozman Bálint]
* new token kind @:id@ for CSS/Sass [#27]
* new token kind @:done@ for Taskpaper [#39]
* new token kind @:map@ for Lua, introducing a nice nested-shades trick [#22, thanks to Quintus and Nathan Youngman]
* new token kind @:unknown@ for Debug scanner
* new DebugLint encoder that checks for empty tokens and correct nesting

Improvements:

* CSS scanner uses @:id@ and @:tag@ now [#27]
* Diff scanner: Highlight inline changes in multi-line changes [#99]
* JavaScript scanner: Highlight multi-line comments in diff correctly
* JSON scanner: simplify key/value heuristic, using look-ahead instead of a stack
* HTML scanner displays style tags and attributes now [#145]
* Ruby scanner: Accept @%i(…)@ and @%I(…)@ symbol lists (Ruby 2.0) [thanks to Nathan Youngman]
* Ruby scanner: Accept keywords as Ruby hash keys [#126]
* performance improvements to several scanners and encoders, especially Terminal and HTML
* added @:keep_state@ functionality to more scanners so they work nicely with diff now [#116]
* refactoring and cleanup to achieve better "Code Climate" ratings (but I don't really care)
* updated and cleaned up the documentation,
* documented list of TokenKinds
* Alpha style: tweaked colors for @.binary@, @.local-variable@, and @.predefined-type@
* @rake generate@ supports Git now instead of Subversion

Removed:

* @Tokens#dump@, @Tokens.load@, @Tokens::Undumping@, and @zlib@ dependency
* double-click toggle handler from HTML table output
* @rake_helpers@, @sample@ directories and several other ancient garbage

Fixes:

* fixes to CSS scanner (floats, pseudoclasses, nth-child) [#143]
* fixed empty tokens and unclosed token groups in HTML, CSS, Diff, Goovy, PHP, Raydebug, Ruby, SQL, and YAML scanners [#144]
* fixed @:docstring@ token type style
* fixed several infinite Hash caches and dynamic Symbol creation that might have been exploited by an attacker [#148]
* fixed HTML encoder when output is a StringIO (eg. when using @-HTML@ as a command line parameter)
* TokenKinds should not be frozen [#130, thanks to Gavin Kistner]
* display line numbers in HTML @:table@ mode even for single-line code (remove special case) [#41, thanks to Ariejan de Vroom]
* override Bootstrap's @pre { word-break: break-all }@ styling for line numbers [#102, thanks to lightswitch05]
* HTML encoder will not warn about unclosed token groups at the end of the stream
* fixed problem with coderay/version.rb being loaded twice

Internals:

* The Debug scanner maps unknown token kinds to @:unknown@ (to avoid creating Symbols based on possibly unsafe input).
* The Raydebug scanner highlights unknown token kinds as @:plain@.
* The Debug encoder refactored; use DebugLint if you want strict checking now..
* The Debug encoder will not warn about errors in the token stream.
* Plugin does not warn about fallback when default is defined.
* PluginHost now works with Strings instead of Symbols internally (to avoid using @#to_sym@).

h2. Changes in 1.0.9

* Fix Ruby scanner: Ruby 1.9 hash syntax @{ key: value }@ is highlighted correctly. [GH #106, thanks to Seth Vargo]
* Fix HTML scanner: Accept DTDs. [GH #83]
* Fix PHP scanner: Accept Unicode. [GH #40, thanks to Lance Li]

h2. Changes in 1.0.8

* add @:string/:char@, remove @:regexp/:function@ color from Terminal encoder [GH #29, thanks to Kyrylo Silin]
* allow @-@ in line number anchor prefix for HTML encoder [GH #32, thanks to shurizzle]
* Fix HTML scanner: Don't crash if HTML in a diff contains a JavaScript tag.

h2. Changes in 1.0.7

* Changed license from LGPL to MIT. [GH-25, thanks to jessehu]
* Fix issue with plugin files not being loaded. [GH-20, thanks to Will Read]
* Fix HTML scanner bug: Don't choke on boolean attributes. [GH-26, thanks to jugglinmike]

h2. Changes in 1.0.6

* New option @:break_lines@ for the HTML encoder (splits tokens at line breaks). [GH-15, thanks to Etienne Massip]
* Improved speed of @:line_numbers => :inline@ option for the HTML encoder.
* Fixed wrong HTML file type. (was @:page@) [GH-16, thanks to Doug Hammond]
* The CSS Scanner now highlights tokens like @url(...)@ as @:function@ instead of @:string@. [GH-13, thanks to Joel Holdbrooks]

h2. Changes in 1.0.5

Fixes:

* @autoload@ calls do not depend on @coderay/lib@ being in the load path (GitHub issue #6; thanks to tvon, banister, envygeeks, and ConradIrwin)
* avoid dark blue as terminal color (GitHub issue #9; thanks to shevegen)

h2. Changes in 1.0.4

Fixes in the CSS scanner:

* understands the unit "s" (seconds)
* ignores unexpected curly braces
* code inside of diffs is highlighted correctly

h2. Changes in 1.0.3

New:

* .tmproj files are recognized as XML.

Fixes:

* Removed files are highlighted inside diffs generated by git.

h2. Changes in 1.0.2

Fixes:

* .erb files are recognized as ERB.

h2. Changes in 1.0.1

New:

* YAML scanner allows "-" and "/" in key names

Changes:

* HTML page output has no white border anymore (alpha style)

Fixes:

* fixed warning in the output of "coderay stylesheet"
* fixed additional scrollbar in code when last line contains an eyecatcher
* minor fixes in the tests (issue github-#4)

h2. Changes in 1.0

CodeRay 1.0 is a major rewrite of the library, and incompatible to earlier versions.

The command line and programmer interfaces are similar to 0.9, but the internals have completely changed.

h3. General changes

* *NEW*: The new Diff scanner colorizes code inside of the diff, and highlights inline changes.
* *NEW*: Extended support and usage of HTML5 and CSS 3 features.
* *NEW*: Direct Streaming
* *NEW* scanners: Clojure and HAML
* *CHANGED*: Token classes (used as CSS classes) are readable names; breaks you stylesheet!
* *IMPROVED* documentation
* *IMPROVED* speed: faster startup (using @autoload@), scanning, and encoding
* *IMPROVED* Ruby 1.9 encodings support
* *IMPROVED* Tests: There are more of them now!

h3. Direct Streaming

CodeRay 1.0 introduces _Direct Streaming_ as a faster and simpler alternative to Tokens. It means that all Scanners,
Encoders and Filters had to be rewritten, and that older scanners using the Tokens API are no longer compatible with
this version.

The main benefits of this change are:

* more speed (benchmarks show 10% to 50% more tokens per second compared to CodeRay 0.9)
* the ability to stream output into a pipe on the command line
* a simpler API
* less code

Changes related to the new tokens handling include:
* *CHANGED*: The Scanners now call Encoders directly; tokens are not added to a Tokens array, but are send to the
  Encoder as a method call. The Tokens representation (which can be seen as a cache now) is still present, but as a
  special case; Tokens just encodes the given tokens into an Array for later use.
* *CHANGED*: The token actions (@text_token@, @begin_group@ etc.) are now public methods of @Encoder@ and @Tokens@.
* *REWRITE* of all Scanners, Encoders, Filters, and Tokens.
* *RENAMED* @:open@ and @:close@ actions to @:begin_group@ and @:end_group@.
* *RENAMED* @open_token@ and @close_token@ methods to @begin_group@ and @end_group@.
* *NEW* method @#tokens@ allows to add several tokens to the stream. @Tokens@ and @Encoders::Encoder@ define this
  method.
* *CHANGED* The above name changes also affect the JSON, XML, and YAML encoders. CodeRay 1.0 output will be incompatible
  with earlier versions.
* *REMOVED* @TokenStream@ and the @Streamable@ API and all related features like @NotStreamableError@ are now obsolete
  and have been removed.

h3. Command Line

The @coderay@ executable was rewritten and has a few new features:

* *NEW* Ability to stream into a pipe; try @coderay file | more -r@
* *NEW* help
* *IMPROVED*: more consistent parameter handling
* *REMOVED* @coderay_stylesheet@ executable; use @coderay stylesheet [name]@.

h3. @Tokens@

* *NEW* methods @count@, @begin_group@, @end_group@, @begin_line@, and @end_line@.
* *REMOVED* methods @#stream?@, @#each_text_token@.
* *REMOVED* methods @#optimize@, @#fix@, @#split_into_lines@ along with their bang! variants.
* *REMOVED* @#text@ and @#text_size@ methods. Use the @Text@ encoder instead.
* *REMOVED* special implementation of @#each@ taking a filter parameter. Use @TokenKindFilter@ instead.

h3. *RENAMED*: @TokenKinds@

Renamed from @Tokens::ClassOfKind@ (was also @Tokens::AbbreviationForKind@ for a while).
The term "token class" is no longer used in CodeRay. Instead, tokens have _kinds_.
See "#122":http://odd-eyed-code.org/issues/122.

* *CHANGED* all token CSS classes to readable names.
* *ADDED* token kinds @:filename@, @:namespace@, and @:eyecatcher@.
* *RENAMED* @:pre_constant@ and @:pre_type@ to @:predefined_constant@ and @predefined_type@.
* *RENAMED* @:oct@ and @:bin@ to @:octal@ and @binary@.
* *REMOVED* token kinds @:attribute_name_fat@, @:attribute_value_fat@, @:operator_fat@, @interpreted@,
  @:tag_fat@, @tag_special@, @:xml_text@, @:nesting_delimiter@, @:open@, and @:close@.
* *CHANGED*: Don't raise error for unknown token kinds unless in @$CODERAY_DEBUG@ mode.
* *CHANGED* the value for a token kind that is not highlighted from
  @:NO_HIGHLIGHT@ to @false@.

h3. @Duo@

* *NEW* method @call@ for allowing code like @CodeRay::Duo[:python => :yaml].(code)@ in Ruby 1.9.

h3. @Encoders::CommentFilter@

* *NEW* alias @:remove_comments@

h3. @Encoders::Filter@

* *NEW* option @tokens@.
* *CHANGED*: Now it simply delegates to the output.
* *REMOVED* @include_text_token?@ and @include_block_token?@ methods.

h3. @Encoders::HTML@

The HTML encoder was cleaned up and simplified.

* *NEW*: HTML5 and CSS 3 compatible.
  See "#215":http://odd-eyed-code.org/issues/215.
* *ADDED* support for @:line_number_anchors@.
  See "#208":http://odd-eyed-code.org/issues/208.
* *CHANGED* the default style to @:alpha@.
* *CHANGED*: Use double click to toggle line numbers in table mode (as single
  click jumps to an anchor.)
* *REMOVED* support for @:line_numbers => :list@.
* *FIXED* splitting of lines for @:line_numbers => :inline@, so that the line
  numbers don't get colored, too.
* *RENAMED* @Output#numerize@ to @#number@, which is an actual English word.

h3. @Encoders::LinesOfCode@

* *CHANGED*: @compile@ and @finish@ methods are now protected.

h3. *Renamed*: @Encoders::Terminal@ (was @Encoders::Term@)

* *RENAMED* from @Encoders::Term@, added @:term@ alias.
* *CLEANUP*: Use @#setup@'s @super@, don't use @:procedure@ token class.
* *CHANGED*: @#token@'s second parameter is no longer optional.
* *REMOVED* colors for obsolete token kinds.
* *FIXED* handling of line tokens.

h3. @Encoders::Text@

* *FIXED* default behavior of stripping the trailing newline.

h3. *RENAMED*: @Encoders::TokenKindFilter@ (was @Encoders::TokenClassFilter@)

* *NEW*: Handles token groups.
  See "#223":http://odd-eyed-code.org/issues/223.
* *RENAMED* @include_block_token?@ to @include_group?@.

h3. @Encoders::Statistic@

* *CHANGED*: Tokens actions are counted separately.

h3. @Scanners::Scanner@

* *NEW* methods @#file_extension@ and @#encoding@.
* *NEW*: The @#tokenize@ method also takes an Array of Strings as source. The
  code is highlighted as one and split into parts of the input lengths
  after that using @Tokens#split_into_parts@.
* *NEW* method @#binary_string@
* *REMOVED* helper method @String#to_unix@.
* *REMOVED* method @#streamable?@.
* *REMOVED* @#marshal_load@ and @#marshal_dump@.
* *RENAMED* class method @normify@ to @normalize@; it also deals with encoding now.
* *CHANGED*: @#column@ starts counting with 1 instead of 0

h3. *NEW*: @Scanners::Clojure@

Thanks to Licenser, CodeRay now supports the Clojure language.

h3. @Scanners::CSS@

* *NEW*: Rudimentary support for the @attr@, @counter@, and @counters@ functions.
  See "#224":http://odd-eyed-code.org/issues/224.
* *NEW*: Rudimentary support for CSS 3 colors.
* *CHANGED*: Attribute selectors are highlighted as @:attribute_name@ instead of @:string@.
* *CHANGED*: Comments are scanned as one token instead of three.

h3. @Scanners::Debug@

* *NEW*: Support for line tokens (@begin_line@ and @end_line@ represented by @[@ and @]@.)
* *FIXED*: Don't send @:error@ and @nil@ tokens for buggy input any more.
* *FIXED*: Closes unclosed tokens at the end of @scan_tokens@.
* *IMPROVED*: Highlight unknown tokens as @:error@.
* *CHANGED*: Raises an error when trying to end an invalid token group.

h3. @Scanners::Delphi@

* *FIXED*: Closes open string groups.

h3. @Scanners::Diff@

* *NEW*: Highlighting of code based on file names.
  See ticket "#52":http://odd-eyed-code.org/issues/52.

  Use the @:highlight_code@ option to turn this feature off. It's enabled
  by default.

  This is a very original feature. It enables multi-language highlighting for
  diff files, which is especially helpful for CodeRay development itself. The
  updated version of the scanner test suite generated .debug.diff.html files
  using this.

  Note: This is still experimental. Tokens spanning more than one line
  may get highlighted incorrectly. CodeRay tries to keep scanner states
  between the lines and changes, but the quality of the results depend on
  the scanner.
* *NEW*: Inline change highlighting, as suggested by Eric Thomas.
  See ticket "#227":http://odd-eyed-code.org/issues/227 for details.

  Use the @:inline_diff@ option to turn this feature off. It's enabled by
  default.

  For single-line changes (that is, a single deleted line followed by a single
  inserted line), this feature surrounds the changed parts with an
  @:eyecatcher@ group which appears in a more saturated background color.
  The implementation is quite complex, and highly experimental. The problem
  with multi-layer tokenizing is that the tokens have to be split into parts.
  If the inline change starts, say, in the middle of a string, then additional
  @:end_group@ and @:begin_group@ tokens must be inserted to keep the group
  nesting intact. The extended @Scanner#tokenize@ method and the new
  @Tokens#split_into_parts@ method take care of this.
* *NEW*: Highlight the file name in the change headers as @:filename@.
* *CHANGED*: Highlight unknown lines as @:comment@ instead of @:head@.
* *IMPROVED*: Background colors for Diff output have been optimized.

h3. *RENAMED*: @Scanners::ERB@ (was @Scanners::RHTML@)

h3. *NEW*: @Scanners::HAML@

It uses the new :state options of the HTML and Ruby scanners.

Some rare cases are not considered (like @#{...}@ snippets inside of :javascript blocks),
but it highlights pretty well.

h3. @Scanners::HTML@

* *FIXED*: Closes open string groups.

h3. @Scanners::JavaScript@

* *IMPROVED*: Added @NaN@ and @Infinity@ to list of predefined constants.
* *IMPROVED* recognition of RegExp literals with leading spaces.

h3. @Scanners::Java@

* *NEW*: Package names are highlighted as @:namespace@.
  See "#210":http://odd-eyed-code.org/issues/210.

h3. *REMOVED*: @Scanners::NitroXHTML@

Nitro is "dead":http://www.nitrohq.com/.

h3. *RENAMED*: @Scanners::Text@ (was @Scanners::Plaintext@)

* *IMPROVED*: Just returns the string without scanning (faster).

 This is much faster than scanning until @/\z/@ in Ruby 1.8.

h3. @Scanners::Python@

* *CHANGED*: Docstrings are highlighted as @:comment@.
  See "#190":http://odd-eyed-code.org/issues/190.

h3. *NEW*: @Scanners::Raydebug@

Copied from @Scanners::Debug@, highlights the token dump instead of importing it. It also reacts to the @.raydebug@ file
name suffix now.

h3. @Scanners::Ruby@

* *ADDED* more predefined keywords (see http://murfy.de/ruby-constants).
* *IMPROVED* support for singleton method definitions.
  See "#147":http://odd-eyed-code.org/issues/147.
* *FIXED*: Don't highlight methods with a capital letter as constants
  (eg. @GL.PushMatrix@).
* *NEW*: Highlight buggy floats (like .5) as @:error@.
* *CLEANUP* of documentation, names of constants and variables, state handling.

  Moved @StringState@ class from @patterns.rb@ into a separate file.
* *NEW*: Complicated rule for recognition of @foo=@ style method names.
* *NEW*: Handles @:keep_state@ option (a bit; experimental).

  Actually, Ruby checks if there is @[~>=]@, but not @=>@ following the name.

* *REMOVED* @EncodingError@

h3. *REMOVED* @Scanners::Scheme@

* It is too buggy, and nobody was using it. To be added again when it's fixed.
  See "#59":http://odd-eyed-code.org/issues/59.

h3. @Scanners::SQL@

* *IMPROVED*: Extended list of keywords and functions (thanks to
  Joshua Galvez, Etienne Massip, and others).

  See "#221":http://odd-eyed-code.org/issues/221.
* *FIXED*: Closes open string groups.
* *FIXED*: Words after @.@ are always recognized as @:ident@.

h3. @Scanners::YAML@

* *FIXED*: Allow spaces before colon in mappings.

  See "#231":http://odd-eyed-code.org/issues/231.

h3. *NEW*: @Styles::Alpha@

A style that uses transparent HSLA colors as defined in CSS 3. See "#199":http://odd-eyed-code.org/issues/199.

It also uses the CSS 3 property @user-select: none@ to keep the user from selecting the line numbers. This is especially
nice for @:inline@ line numbers. See "#226":http://odd-eyed-code.org/issues/226.

h3. @WordList@

Stripped down to 19 LOC.

* *RENAMED* @CaseIgnoringWordList@ to @WordList::CaseIgnoring@.
* *REMOVED* caching option because it creates memory leaks.
* *REMOVED* block option.

h3. @FileType@

* *NEW*: Recognizes @.gemspec@, @.rjs@, @.rpdf@ extensions, @Gemfile@, and @Capfile@ as Ruby.

  Thanks to the authors of the TextMate Ruby bundle!
* *REMOVED* @FileType#shebang@ is a protected method now.

h3. @Plugin@

* *IMPROVED*: @register_for@ sets the @plugin_id@; it can now be a @Symbol@.
* *ADDED* @PluginHost#const_missing@ method: Plugins are loaded automatically.
  Using @Scanners::JavaScript@ in your code loads @scanners/java_script.rb@.
* *ADDED* @#all_plugins@ method to simplify getting 
  information about all available plugins (suggested by bnhymn).
* *CHANGED* the default plugin key from @nil@ to @:default@.

h3. @GZip@

* *MOVED* into @CodeRay@ namespace.
* *MOVED* file from @gzip_simple.rb@ to @gzip.rb@.
* *REMOVED* @String@ extensions.

h3. More API changes

* *FIXED* @Encoders::HTML#token@'s second parameter is no longer optional.
* *CHANGED* @Encoders::HTML::Output@'s API.
* *REMOVED* lots of unused methods.

The helper classes were cleaned up; see above for details.

* *CHANGED* @Plugin@ API was simplified and stripped of all unnecessary features.
* *CHANGED* Moved @GZip@ and @FileType@ libraries into @CodeRay@; cleaned them up.