/
notes
98 lines (75 loc) · 3.09 KB
/
notes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
;; -*- mode: outline; outline-regexp: " *[-\+]"; indent-tabs-mode: nil; fill-column: 120 -*-
^c^d hide subtree
^c^s show subtree
- skip_null_byte => omit_null_byte
- and move to dump parameters
- stream writes
- dump.c:685
- stream_writer.c
- use io C calls instead
- future
- object mode delegate
- future
- usual delegate
- detect_time (maybe)
- parser
- read file in separate thread
- object delegate
- new parser reasons or design notes
- parser itself is faster due to reduced branching
- bulk array create - 15x
- bulk hash - 3x
- reuse of memory
- better control over options
- auto rehash of cache
- memory use when reading from a file
- add default_options[]= method
- big decimal
- just in custom mode, maybe in strict?
- encode_big_decimal_as_string
- add as fix for rails change
- rails
- Process::Status
- ActiveSupport::TimeWithZone
- ActiveModel::Errors
- ActiveSupport::Multibyte::Chars
- ActiveRecord::Relation
- future
- option to allow invalid unicode through
- unit tests for 32 bit - test_float_parse
- look at RUBY_PLATFORM maybe?
- look at env var for OJ_DEBUG when doing ruby extconf.rb
- add printf for keys spots in parse and dump
- invalid character option
- don't check options
- replace with best attempt such as \ud83d => \xED\xA0\xBD
- that would be the normal replacement
- allow_invalid_unicode
---------------------------
Tried a separate thread for the parser and the results were poor. The parsing is
10% to 15% of the total so splitting ruby calls and c does not help much and the
overhead of swapping data was too high. It was not possible to split ruby calls
into both threads due to not getting a lock on the ruby environment for object
creation.
----------------
todo
- #570 - complain in strict mode about leading + or trailing .
- #569 - < and > are not escaped in compat mode
- ruby -r oj -e 'puts "#{Oj::VERSION} -> #{Oj.dump("<", {mode: :compat, escape_mode: :unicode_xss})}"'
3.7.5 -> "<"
3.7.4 -> "\u003c"
- #568 - We're still at Rails 4.2 so maybe all of this is outdated, but still.
It's documented that :bigdecimal_as_decimal isn't available for mode: :rails. However, if I run it like this:
Oj.optimize_rails
Oj.default_options = { mode: :rails, bigdecimal_as_decimal: true }
It does the job - Oj dumps BigDecimal as number.
If I later add bigdecimal_as_decimal: true explicitly in the dump it works for single number dump, but not for Hash.
Oj.dump(BigDecimal('1.2')) # => "1.2"
Oj.dump({a: BigDecimal('1.2')}) # => "{\"a\":1.2}"
Oj.dump(BigDecimal('1.2'), bigdecimal_as_decimal: true) # => "1.2"
Oj.dump({a: BigDecimal('1.2')}, bigdecimal_as_decimal: true) # => "{\"a\":\"1.2\"}" - unexpected
Also when I remove running Oj.optimize_rails, Oj ignores bigdecimal_as_decimal and dumps BigDecimal as string.
Currently, it works OK for me, just wanted to check what's the expected behavior and if it should be aware when upgrading Rails or Oj.
- #567 - irb(main):001:0> Oj.dump(Time.utc(2000, 1, 1, 1, 0, 0, 0), mode: :compat)
=> "\"2000-01-01 01:00:00 UTC\""
- missing fractional part in custom with second_precision id 0 fraction