diff options
author | Aaron Patterson <aaron.patterson@gmail.com> | 2015-10-19 10:53:16 -0700 |
---|---|---|
committer | Aaron Patterson <aaron.patterson@gmail.com> | 2016-06-11 09:32:35 -0700 |
commit | 391edb01f8122b4e229a2d7c78529a4e080abc1a (patch) | |
tree | fb10f55a5c176e359377937a805cca78616d91eb | |
parent | 6327cbb815226cbdfe24b7ed41e015c6c1f2ac20 (diff) | |
download | json-391edb01f8122b4e229a2d7c78529a4e080abc1a.tar.gz |
resize strings after parsing
The parser uses `rb_str_buf_new` to allocate new strings.
`rb_str_buf_new` [has a minimum size of 128 and is not an embedded
string](https://github.com/ruby/ruby/blob/9949407fd90c1c5bfe332141c75db995a9b867aa/string.c#L1119-L1135). This causes applications that parse JS to allocate extra memory when parsing short strings.
For a real-world example, we can use the mime-types gem. The mime-types
gem stores all mime types inside a JSON file and parses them when you
require the gem.
Here is a sample program:
```ruby
require 'objspace'
require 'mime-types'
GC.start
GC.start
p ObjectSpace.memsize_of_all String
```
The example program loads the mime-types gem and outputs the total space
used by all strings. Here are the results of the program before and
after this patch:
** Before **
```
[aaron@TC json (memuse)]$ ruby test.rb
5497494
[aaron@TC json (memuse)]$
```
** After **
```
[aaron@TC json (memuse)]$ ruby -I lib:ext test.rb
3335862
[aaron@TC json (memuse)]$
```
This change results in a ~40% reduction of memory use for strings in the
mime-types gem.
Thanks @matthewd for finding the problem, and @nobu for the patch!
-rw-r--r-- | ext/json/ext/parser/parser.rl | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/ext/json/ext/parser/parser.rl b/ext/json/ext/parser/parser.rl index f3933cb..157b001 100644 --- a/ext/json/ext/parser/parser.rl +++ b/ext/json/ext/parser/parser.rl @@ -527,6 +527,8 @@ static char *JSON_parse_string(JSON_Parser *json, char *p, char *pe, VALUE *resu if (json->symbolize_names && json->parsing_name) { *result = rb_str_intern(*result); + } else { + rb_str_resize(*result, RSTRING_LEN(*result)); } if (cs >= JSON_string_first_final) { return p + 1; |