Changes-pre-1.0.textile

CodeRay Version History

This files lists all changes in the CodeRay library since the 0.8.4 release.

{{toc}}

Changes in 0.9.8 “banister” [2011-05-01]

Fixes for JRuby’s 1.9 mode and minor issues.

Rake tasks

• REMOVED obsolete has_rdoc gem specification, fixing a warning.

Scanners::Scanner

• NEW method #scan_rest replaces scan_until(/\z/), which is broken in JRuby 1.6 —1.9 mode.
  See #297.

Scanners::CSS

• FIXED LOC counting (should be 0).
  See #296.

Scanners::Ruby

• FIXED the IDENT pattern not to use character properties, which are broken in JRuby 1.6 —1.9 mode.
  See #297, thanks to banister for reporting!

Scanners::SQL

• ADDED more keywords: between, databases, distinct, fields, full, having, is, prompt, tables.
  See #221, thanks to Etienne Massip again.

FileType

• NEW regonizes ColdFusion file type extensions .cfm and .cfc as XML.
  See #298, thanks to Emidio Stani.

Changes in 0.9.7 “Etienne” [2011-01-14]

Fixes a dangerous JavaScript scanner bug, and a testing problem with Ruby 1.9.1.

Tests

• FIXED The functional tests now load the lib directory (instead of the gem) in Ruby 1.9.1.

Scanners::JavaScript

• FIXED KEY_CHECK_PATTERN regexp
  See #264, thanks to Etienne Massip!

Changes in 0.9.6 “WoNáDo” [2010-11-25]

Minor improvements to the Ruby scanner and a fix for Ruby 1.9.

Scanners::Ruby

• IMPROVED handling of new hash syntax (keys are marked as :key now,
  colon is a separate :operator token, all idents can be used as keys)
  See #257, thanks to WoNáDo!
• ADDED __ENCODING__ magic constant (Ruby 1.9)
• FIXED: Scanner no longer tries to modify the input string on Ruby 1.9.
  See #260, thanks to Jan Lelis!

Changes in 0.9.5 “Germany.rb” [2010-09-28]

Support for Rubinius (#251), improved mutlibyte handling, Ruby 1.9 syntax, and valid HTML.

Encoders::HTML

• FIXED: Line tokens use span with display: block instead of div, which was invalid HTML (#255).

Scanner::Scanner

• IMPROVED handling of encodings in Ruby 1.9: UTF-8 and Windows-1252 are checked.
• NEW: Invalid chars will be converted to ? in Ruby 1.9.
• FIXED string= method for Rubinius. See issue 481 on their site.

Scanners::CSS

• FIXED: Don’t use non-ASCII regexps.

Scanners::Diff

• FIXED: Highlight unexpected lines as :comment.

Scanners::PHP

• FIXED: Use ASCII-8BIT encoding for now.

Scanners::Ruby

• ADDED support for some Ruby 1.9 syntax (#254):
  • the -> lambda shortcut
  • new Hash syntax using colons ({ a: b })
• FIXED: Use UTF-8 encoding.
• IMPROVED unicode support on Ruby 1.8 (#253).
• FIXED recognition of non-ASCII identifiers in Ruby 1.9, JRuby, and Rubinius (#253).
• CHANGED heredoc recognition to ignore delimiters starting with a digit. This is incorrect, but causes less false positives.

Scanners::SQL

• FIXED scanning of comments; nice catch, Rubinius!
  (#252)

Changes in 0.9.4 “Ramadan” [2010-08-31]

Updated command line interface and minor scanner fixes for the Diff, HTML, and RHTML scanners.

coderay executable

• FIXED: Partly rewritten, simplified, fixed.
  (#244)

Scanners::Diff

• FIXED handling of change headers with code on the same line as the @@ marker.
  (#247)

Scanners::HTML

• FIXED a missing regexp modifier that slowed down the scanning.
  (#245)

Scanners::RHTML

• FIXED highlighting of ERB comment blocks.
  (#246)

Changes in 0.9.3 “Eyjafjallajökull” [2010-04-18]

• FIXED: Documentation of Tokens.
  (#218)

coderay executable

• NEW: automatic TTY detection (uses Term encoder)
• NEW: optional 3rd parameter for the filename
• FIXED: Converted to UNIX format.
• FIXED: Warn about generated files.
• FIXED: Ensure line break after the output (especially for LoC counter).

Scanners::JavaScript

• FIXED: Don’t keep state of XML scanner between calls for E4X literals.

Scanners::Java, Scanners::JSON

• FIXED: Close unfinished strings with the correct token kind.

Changes in 0.9.2 “Flameeyes” [2010-03-14]

• NEW Basic tests and a Rakefile are now included in the Gem. [Flameeyes]
  A doc task is also included.
• FIXED Use $CODERAY_DEBUG for debugging instead of $DEBUG. [Trans]
  (#192)
• REMOVED Term::Ansicolor was bundled under lib/, but not used. [Flameeyes]
  (#205)
• WORKAROUND for Ruby bug
  #2745

Encoders::Term

• FIXED strings are closed correctly
  (#138)
• FIXED several token kinds had no associated color
  (#139)
• NEW alias terminal

NOTE: This encoder will be renamed to Encoders::Terminal in the next release.

Scanners::Debug

• FIXED Don’t close tokens that are not open. Send :error token instead.

Scanners::Groovy

• FIXED token kind of closing brackets is :operator instead of nil
  (#148)

Scanners::PHP

• FIXED allow \ operator (namespace separator)
  (#209)

Scanners::YAML

• FIXED doesn’t send debug tokens when $DEBUG is true [Trans]
  (#149)

Changes in 0.9.1 [2009-12-31]

Token classes

• NEW token classes :complex, :decorator, :imaginary
  (all for Python)
• REMOVED token class :procedure
  – use :function or :method instead.

Tokens

• NEW method #scanner

Stores the scanner.

• REMOVED methods .write_token, .read_token, .escape, .unescape

They were only used by the Tokens encoder, which was removed also.

Encoders::Encoder

• REMOVED Don’t require the stringio library.
• NEW public methods #open_token, #close_token, #begin_line, #end_line
  These methods are called automatically, like #text_token.
• NEW proteced method #append_encoded_token_to_output

Encoders::Tokens

• REMOVED – use Tokens#dump and Tokens.load.

Encoders::Filter

• NEW
  A Filter encoder has another Tokens instance as output.

Encoders::TokenClassFilter

• NEW

It takes 2 options, :exclude and :include, that specify which token classes to include or exclude for the output. They can be a single token class, an Array of classes, or the value :all.

Encoders::CommentFilter

• NEW

Removes tokens of the :comment class.

Encoders::LinesOfCode

• NEW

Counts the lines of code according to the KINDS_NOT_LOC token class list defined by the scanner. It uses the new TokenClassFilter. Alias: :loc, as in tokens.loc.

Encoders::JSON

• NEW

Outputs tokens in a simple JSON format.

Encoders::Term

• NEW (beta, by Rob Aldred)

Outputs code highlighted for a color terminal.

Encoders::HTML

• NEW option :title (default value is CodeRay output)

Setting this changes the title of the HTML page.

• NEW option :highlight_lines (default: nil)

Highlights the given set of line numbers.

- REMOVED option :level

It didn’t do anything. CodeRay always outputs XHTML.

Encoders::Text

• Uses Encoder interface with super and #text_token.

Encoders::XML

• FIXED (#94)

It didn’t work at all.

Scanners

• NEW Mapped :h to :c, :cplusplus and :'c++' to :cpp,
  :ecma, :ecmascript, :ecma_script to :java_script,
  :pascal to :delphi, and :plain to :plaintext.

Scanners::Scanner

• NEW constant KINDS_NOT_LOC

A list of all token classes not considered in LOC count. Added appropriate values for scanners.

• NEW method #lang returns the scanner’s lang, which is its plugin_id.
• FIXED automatic, safe UTF-8 detection [Ruby 1.9]
• FIXED column takes care of multibyte encodings [Ruby 1.9]
• FIXED is dumpable (Tokens store their scanner in an @scanner variable)

Scanners::Cpp

• NEW (C++)

Scanners::Groovy

• NEW (beta)

Scanners::Python

• NEW

Scanners::PHP

• NEW (based on Stefan Walk’s work)

Scanners::SQL

• NEW (based on code by Josh Goebel)

Scanners::C

• IMPROVED added a list of :directive tokens that were :reserved before
• IMPROVED detection of labels
• IMPROVED allow 1L and 1LL style literals

Scanners::CSS

• IMPROVED element selectors are highlighted as :type instead of :keyword

Scanners::Delphi

• IMPROVED Don’t cache tokens in CaseIgnoringWordList.

Scanners::Java

• IMPROVED assert is highlighted as a :keyword now
• IMPROVED const and goto are highlighted as :reserved
• IMPROVED false, true, and null are highlighted as :pre_constant
• IMPROVED threadsafe is no longer a :directive
• IMPROVED String is highlighted as a :pre_type
• IMPROVED built-in classes ending with Error or Exception are
  highlighted as a :exception instead of :pre_type

Scanners::JavaScript

• NEW a list of PREDEFINED_CONSTANTS to be highlighted as :pre_constant
• NEW XML literals are recognized and highlighted
• NEW function name highlighting
• IMPROVED .1 is highlighted a number
• FIXED strings close with the correct kind when terminated unexpectedly

Scanners::JSON

• IMPROVED constants (true, false, nil) are highlighted as :value

Scanners::Ruby

• IMPROVED Patterns::KEYWORDS_EXPECTING_VALUE for more accurate
  value_expected detection
• IMPROVED handling of \ as a string delimiter
• IMPROVED handling of unicode strings; automatic switching to unicode
• IMPROVED highlighting of self.method definitions
• REMOVED Patterns::FANCY_START_SAVE (obsolete)
• FIXED encoding issues [Ruby 1.9]
• FIXED a problem in early Ruby 1.8.6 patch versions with Regexp.escape

Scanners::YAML

• IMPROVED indentation detection

Styles::Cycnus

• changed a few colors (exceptions, inline strings, predefined types)

Plugin

• NEW method #title

Set and get the plugin’s title. Titles can be arbitrary strings.

• NEW method #helper loads helpers from different plugins

Use this syntax: helper 'other_plugin/helper_name'

FileType

• NEW FileType[] takes Pathname instances
• NEW regonizes .cc, .cpp, .cp, .cxx, .c++, .C, .hh, .hpp, .h++, .cu extensions (C++)

Thanks to Sander Cox and the TextMate C bundle.

• NEW regonizes .pas, .dpr extensions (Delphi)
• NEW regonizes .gvy, .groovy extensions (Groovy)
• NEW regonizes .php, .php3, .php4, .php5 extensions (PHP)
• NEW regonizes .py, .py3, .pyw extensions (Python)
• NEW regonizes .rxml extension (Ruby)
• NEW regonizes .sql extension (SQL)
• File types list was sorted alphabetically.

CaseIgnoringWordList

• FIXED (#97)

The default value is no longer ignored.

ForRedCloth

• FIXED for RedCloth versions 4.2.0+ (#119)

Cleanups

• warnings about character classes [Ruby 1.9]
• encoding issues [Ruby 1.9]
• documentation, code