summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorAaron Patterson <aaron.patterson@gmail.com>2015-10-19 10:53:16 -0700
committerAaron Patterson <aaron.patterson@gmail.com>2016-06-11 09:32:35 -0700
commit391edb01f8122b4e229a2d7c78529a4e080abc1a (patch)
treefb10f55a5c176e359377937a805cca78616d91eb
parent6327cbb815226cbdfe24b7ed41e015c6c1f2ac20 (diff)
downloadjson-391edb01f8122b4e229a2d7c78529a4e080abc1a.tar.gz
resize strings after parsing
The parser uses `rb_str_buf_new` to allocate new strings. `rb_str_buf_new` [has a minimum size of 128 and is not an embedded string](https://github.com/ruby/ruby/blob/9949407fd90c1c5bfe332141c75db995a9b867aa/string.c#L1119-L1135). This causes applications that parse JS to allocate extra memory when parsing short strings. For a real-world example, we can use the mime-types gem. The mime-types gem stores all mime types inside a JSON file and parses them when you require the gem. Here is a sample program: ```ruby require 'objspace' require 'mime-types' GC.start GC.start p ObjectSpace.memsize_of_all String ``` The example program loads the mime-types gem and outputs the total space used by all strings. Here are the results of the program before and after this patch: ** Before ** ``` [aaron@TC json (memuse)]$ ruby test.rb 5497494 [aaron@TC json (memuse)]$ ``` ** After ** ``` [aaron@TC json (memuse)]$ ruby -I lib:ext test.rb 3335862 [aaron@TC json (memuse)]$ ``` This change results in a ~40% reduction of memory use for strings in the mime-types gem. Thanks @matthewd for finding the problem, and @nobu for the patch!
-rw-r--r--ext/json/ext/parser/parser.rl2
1 files changed, 2 insertions, 0 deletions
diff --git a/ext/json/ext/parser/parser.rl b/ext/json/ext/parser/parser.rl
index f3933cb..157b001 100644
--- a/ext/json/ext/parser/parser.rl
+++ b/ext/json/ext/parser/parser.rl
@@ -527,6 +527,8 @@ static char *JSON_parse_string(JSON_Parser *json, char *p, char *pe, VALUE *resu
if (json->symbolize_names && json->parsing_name) {
*result = rb_str_intern(*result);
+ } else {
+ rb_str_resize(*result, RSTRING_LEN(*result));
}
if (cs >= JSON_string_first_final) {
return p + 1;