diff options
author | Aaron Patterson <aaron.patterson@gmail.com> | 2015-10-19 10:53:16 -0700 |
---|---|---|
committer | Florian Frank <flori@ping.de> | 2016-06-21 10:56:58 +0200 |
commit | 797ce5386e3919b33831c01561b381a475757f8a (patch) | |
tree | 93b34fcabfff623a0967462695b619f7cfaaf502 | |
parent | e16dab51937894c674c07660cc0916a08af2e165 (diff) | |
download | json-797ce5386e3919b33831c01561b381a475757f8a.tar.gz |
resize strings after parsing
The parser uses `rb_str_buf_new` to allocate new strings.
`rb_str_buf_new` [has a minimum size of 128 and is not an embedded
string](https://github.com/ruby/ruby/blob/9949407fd90c1c5bfe332141c75db995a9b867aa/string.c#L1119-L1135). This causes applications that parse JS to allocate extra memory when parsing short strings.
For a real-world example, we can use the mime-types gem. The mime-types
gem stores all mime types inside a JSON file and parses them when you
require the gem.
Here is a sample program:
```ruby
require 'objspace'
require 'mime-types'
GC.start
GC.start
p ObjectSpace.memsize_of_all String
```
The example program loads the mime-types gem and outputs the total space
used by all strings. Here are the results of the program before and
after this patch:
** Before **
```
[aaron@TC json (memuse)]$ ruby test.rb
5497494
[aaron@TC json (memuse)]$
```
** After **
```
[aaron@TC json (memuse)]$ ruby -I lib:ext test.rb
3335862
[aaron@TC json (memuse)]$
```
This change results in a ~40% reduction of memory use for strings in the
mime-types gem.
Thanks @matthewd for finding the problem, and @nobu for the patch!
-rw-r--r-- | ext/json/ext/parser/parser.rl | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/ext/json/ext/parser/parser.rl b/ext/json/ext/parser/parser.rl index 04d1b47..f479281 100644 --- a/ext/json/ext/parser/parser.rl +++ b/ext/json/ext/parser/parser.rl @@ -542,6 +542,8 @@ static char *JSON_parse_string(JSON_Parser *json, char *p, char *pe, VALUE *resu if (json->symbolize_names && json->parsing_name) { *result = rb_str_intern(*result); + } else { + rb_str_resize(*result, RSTRING_LEN(*result)); } if (cs >= JSON_string_first_final) { return p + 1; |