Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changes to Mash initialization key string conversion. #521

Merged
merged 27 commits into from
May 5, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
9ce7b38
Adds to existing specs so that failures occur for symbolizing non str…
carolineartz Apr 23, 2020
94d0237
Fix converts keys to symbols only when respond to :to_sym
carolineartz Apr 23, 2020
bc50847
Only convert symbolizable mash keys to strings on initialization.
carolineartz Apr 23, 2020
44c2aad
Use respond_to instead of rescue NoMethodError.
carolineartz Apr 24, 2020
6386222
Mash#invert returns a properly inverted mash.
carolineartz Apr 24, 2020
fca3482
Adds spec for Mash initialization key type conversion.
carolineartz Apr 24, 2020
1e57f3b
Update method documentation.
carolineartz Apr 24, 2020
f372cdc
Make test assertion for stringifying keys more specific.
carolineartz Apr 24, 2020
c2bb200
Adds Mash spec for preserving key types which cannot be symbolized.
carolineartz Apr 24, 2020
4d2c20c
Remove moot spec for dig access with numeric key.
carolineartz Apr 24, 2020
aecd9a6
Add Mash initialization spec with integer, symbol and string types.
carolineartz Apr 24, 2020
3fdf130
Fix typo in Mash SymbolizeKeys extension docs.
carolineartz Apr 24, 2020
bfcf4e6
Add Misc entry in changelog for docs typo fix.
carolineartz Apr 24, 2020
f8a5ac1
Add TODO for change description.
carolineartz Apr 24, 2020
da9f81c
Remove non-symbolizable keys from duoble splat specs.
carolineartz Apr 27, 2020
a8bc220
Revert unnecessary formatting change.
carolineartz Apr 27, 2020
2fae965
Change Hashie::Hash for consistent symbolize keys behavior.
carolineartz Apr 30, 2020
09e0f25
Refactor Mash#custom_writer for clarity.
carolineartz Apr 30, 2020
8cea5be
Update README.
carolineartz Apr 30, 2020
851e7d0
Remove the article 'a' from Mash reference.
carolineartz Apr 30, 2020
7782e0c
Update Changelog.
carolineartz Apr 30, 2020
90c8764
Revert removal of 'Your contribution here.' from Changelog.
carolineartz May 4, 2020
e4fd4e0
Format types as code in Changelog.
carolineartz May 4, 2020
5a29ed4
Add entries to Upgrading.
carolineartz May 4, 2020
54284ee
Add entry in changelog for fix for issue #516.
carolineartz May 4, 2020
d816177
Bump version to 5.0.0.
carolineartz May 4, 2020
536c3fa
Remove from Changelog.
carolineartz May 4, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ scheme are considered to be bugs.

### Changed

* [#521](https://github.com/hashie/hashie/pull/499): Do not convert keys that cannot be represented as symbols to `String` in `Mash` initialization - [@carolineartz](https://github.com/carolineartz).
* Your contribution here.

### Deprecated
Expand All @@ -28,6 +29,7 @@ scheme are considered to be bugs.

### Fixed

* [#516](https://github.com/hashie/hashie/issues/516): Fixed `NoMethodError` raised when including `Hashie::Extensions::Mash::SymbolizeKeys` and `Hashie::Extensions::SymbolizeKeys` in mashes/hashes with non string or symbol keys - [@carolineartz](https://github.com/carolineartz).
* Your contribution here.

### Security
Expand Down
23 changes: 18 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,13 +191,13 @@ end

### KeyConversion

The KeyConversion extension gives you the convenience methods of `symbolize_keys` and `stringify_keys` along with their bang counterparts. You can also include just stringify or just symbolize with `Hashie::Extensions::StringifyKeys` or [`Hashie::Extensions::SymbolizeKeys`](#mash-extension-symbolizekeys).
The KeyConversion extension gives you the convenience methods of `symbolize_keys` and `stringify_keys` along with their bang counterparts. You can also include just stringify or just symbolize with `Hashie::Extensions::StringifyKeys` or `Hashie::Extensions::SymbolizeKeys`.
carolineartz marked this conversation as resolved.
Show resolved Hide resolved

Hashie also has a utility method for converting keys on a Hash without a mixin:

```ruby
Hashie.symbolize_keys! hash # => Symbolizes keys of hash.
Hashie.symbolize_keys hash # => Returns a copy of hash with keys symbolized.
Hashie.symbolize_keys! hash # => Symbolizes all string keys of hash.
Hashie.symbolize_keys hash # => Returns a copy of hash with string keys symbolized.
Hashie.stringify_keys! hash # => Stringifies keys of hash.
Hashie.stringify_keys hash # => Returns a copy of hash with keys stringified.
```
Expand Down Expand Up @@ -580,6 +580,19 @@ my_gem = MyGem.new(name: "Hashie", dependencies: { rake: "< 11", rspec: "~> 3.0"
my_gem.dependencies.class #=> MyGem
```

### How does Mash handle key types which cannot be symbolized?

Mash preserves keys which cannot be converted *directly* to both a string and a symbol, such as numeric keys. Since Mash is conceived to provide psuedo-object functionality, handling keys which cannot represent a method call falls outside its scope of value.

#### Example

```ruby
Hashie::Mash.new('1' => 'one string', :'1' => 'one sym', 1 => 'one num')
# => {"1"=>"one sym", 1=>"one num"}
```

The symbol key `:'1'` is converted the string `'1'` to support indifferent access and consequently its value `'one sym'` will override the previously set `'one string'`. However, the subsequent key of `1` cannot directly convert to a symbol and therefore **not** converted to the string `'1'` that would otherwise override the previously set value of `'one sym'`.

### What else can Mash do?

Mash allows you also to transform any files into a Mash objects.
Expand Down Expand Up @@ -642,7 +655,7 @@ Mash.load('data/user.csv', permitted_classes: [Symbol], permitted_symbols: [], a

### Mash Extension: KeepOriginalKeys

This extension can be mixed into a Mash to keep the form of any keys passed directly into the Mash. By default, Mash converts keys to strings to give indifferent access. This extension still allows indifferent access, but keeps the form of the keys to eliminate confusion when you're not expecting the keys to change.
This extension can be mixed into a Mash to keep the form of any keys passed directly into the Mash. By default, Mash converts symbol keys to strings to give indifferent access. This extension still allows indifferent access, but keeps the form of the keys to eliminate confusion when you're not expecting the keys to change.

```ruby
class KeepingMash < ::Hashie::Mash
Expand Down Expand Up @@ -701,7 +714,7 @@ safe_mash[:zip] = 'test' # => still ArgumentError

### Mash Extension: SymbolizeKeys

This extension can be mixed into a Mash to change the default behavior of converting keys to strings. After mixing this extension into a Mash, the Mash will convert all keys to symbols. It can be useful to use with keywords argument, which required symbol keys.
This extension can be mixed into a Mash to change the default behavior of converting keys to strings. After mixing this extension into a Mash, the Mash will convert all string keys to symbols. It can be useful to use with keywords argument, which required symbol keys.

```ruby
class SymbolizedMash < ::Hashie::Mash
Expand Down
38 changes: 38 additions & 0 deletions UPGRADING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,44 @@
Upgrading Hashie
================

### Upgrading to 5.0.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether we should do 4.1.0 for this. Some elements of the behavior change, but not major enough for API? @michaelherold what do you prefer?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was wavering on suggesting this...but it is not technically inconsistent with Semver to have entries in changed that don't correspond with a major version bump?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do I know? :) We've always treated the second digit as "major". We'd be at 24.0 otherwise and that feels like a lot. I'm open to this debate, but for now I say let's do 4.2.0. Will you make this change here @carolineartz, please?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually .. re-reading what @michaelherold added to CHANGELOG I'll change my mind. Merging as is.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being strict with semver is always hard, especially when folks expect new features in the Major versions. I can agree with the bump to 5.0.0 if we've changed expected behavior in a way that could be breaking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We did. Amen.


#### Mash initialization key conversion

Mash initialization now only converts to string keys which can be represented as symbols.

```ruby
Hashie::Mash.new(
{foo: "bar"} => "baz",
"1" => "one string",
:"1" => "one sym",
1 => "one num"
)

# Before
{"{:foo=>\"bar\"}"=>"baz", "1"=>"one num"}

# After
{{:foo=>"bar"}=>"baz", "1"=>"one sym", 1=>"one num"}
```

#### Mash#dig with numeric keys

`Hashie::Mash#dig` no longer considers numeric keys for indifferent access.

```ruby
my_mash = Hashie::Mash.new("1" => "a") # => {"1"=>"a"}

my_mash.dig("1") # => "a"
my_mash.dig(:"1") # => "a"

# Before
my_mash.dig(1) # => "a"

# After
my_mash.dig(1) # => nil
```

### Upgrading to 4.0.0

#### Non-destructive Hash methods called on Mash
Expand Down
10 changes: 5 additions & 5 deletions lib/hashie/extensions/mash/symbolize_keys.rb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ module Mash
#
# @example
# class LazyResponse < Hashie::Mash
# include Hashie::Extensions::Mash::SymbolizedKeys
# include Hashie::Extensions::Mash::SymbolizeKeys
# end
#
# response = LazyResponse.new("id" => 123, "name" => "Rey").to_h
Expand All @@ -24,13 +24,13 @@ def self.included(base)

private

# Converts a key to a symbol
# Converts a key to a symbol, if possible
#
# @api private
# @param [String, Symbol] key the key to convert to a symbol
# @return [void]
# @param [<K>] key the key to attempt convert to a symbol
# @return [Symbol, K]
carolineartz marked this conversation as resolved.
Show resolved Hide resolved
def convert_key(key)
key.to_sym
key.respond_to?(:to_sym) ? key.to_sym : key
end
end
end
Expand Down
13 changes: 12 additions & 1 deletion lib/hashie/extensions/symbolize_keys.rb
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def symbolize_keys!(hash)
hash.extend(Hashie::Extensions::SymbolizeKeys) unless hash.respond_to?(:symbolize_keys!)
hash.keys.each do |k| # rubocop:disable Performance/HashEachMethods
symbolize_keys_recursively!(hash[k])
hash[k.to_sym] = hash.delete(k)
hash[convert_key(k)] = hash.delete(k)
end
hash
end
Expand All @@ -61,6 +61,17 @@ def symbolize_keys(hash)
symbolize_keys!(new_hash)
end
end

private

# Converts a key to a symbol, if possible
#
# @api private
# @param [<K>] key the key to attempt convert to a symbol
# @return [Symbol, K]
def convert_key(key)
key.respond_to?(:to_sym) ? key.to_sym : key
end
end

class << self
Expand Down
4 changes: 2 additions & 2 deletions lib/hashie/hash.rb
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@ def to_hash(options = {})
assignment_key =
if options[:stringify_keys]
k.to_s
elsif options[:symbolize_keys]
k.to_s.to_sym
elsif options[:symbolize_keys] && k.respond_to?(:to_sym)
k.to_sym
else
k
end
Expand Down
19 changes: 9 additions & 10 deletions lib/hashie/mash.rb
Original file line number Diff line number Diff line change
Expand Up @@ -121,23 +121,21 @@ class << self; alias [] new; end
alias regular_reader []
alias regular_writer []=

# Retrieves an attribute set in the Mash. Will convert
# any key passed in to a string before retrieving.
# Retrieves an attribute set in the Mash. Will convert a key passed in
# as a symbol to a string before retrieving.
def custom_reader(key)
default_proc.call(self, key) if default_proc && !key?(key)
value = regular_reader(convert_key(key))
yield value if block_given?
value
end

# Sets an attribute in the Mash. Key will be converted to
# a string before it is set, and Hashes will be converted
# into Mashes for nesting purposes.
# Sets an attribute in the Mash. Symbol keys will be converted to
# strings before being set, and Hashes will be converted into Mashes
# for nesting purposes.
def custom_writer(key, value, convert = true) #:nodoc:
key_as_symbol = (key = convert_key(key)).to_sym

log_built_in_message(key_as_symbol) if log_collision?(key_as_symbol)
regular_writer(key, convert ? convert_value(value) : value)
log_built_in_message(key) if key.respond_to?(:to_sym) && log_collision?(key.to_sym)
regular_writer(convert_key(key), convert ? convert_value(value) : value)
end

alias [] custom_reader
Expand Down Expand Up @@ -371,7 +369,7 @@ def method_suffix(method_name)
end

def convert_key(key) #:nodoc:
key.to_s
key.respond_to?(:to_sym) ? key.to_s : key
end

def convert_value(val, duping = false) #:nodoc:
Expand Down Expand Up @@ -406,6 +404,7 @@ def log_built_in_message(method_key)
end

def log_collision?(method_key)
return unless method_key.is_a?(String) || method_key.is_a?(Symbol)
return unless respond_to?(method_key)

_, suffix = method_name_and_suffix(method_key)
Expand Down
2 changes: 1 addition & 1 deletion lib/hashie/version.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module Hashie
VERSION = '4.1.0'.freeze
VERSION = '5.0.0'.freeze
end
24 changes: 21 additions & 3 deletions spec/hashie/extensions/mash/symbolize_keys_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,30 @@
end.to raise_error(ArgumentError)
end

it 'symbolizes all keys in the Mash' do
my_mash = Class.new(Hashie::Mash) do
context 'when included in a Mash' do
class SymbolizedMash < Hashie::Mash
include Hashie::Extensions::Mash::SymbolizeKeys
end

expect(my_mash.new('test' => 'value').to_h).to eq(test: 'value')
it 'symbolizes string keys in the Mash' do
my_mash = SymbolizedMash.new('test' => 'value')
expect(my_mash.to_h).to eq(test: 'value')
end

it 'preserves keys which cannot be symbolized' do
my_mash = SymbolizedMash.new(
'1' => 'symbolizable one',
1 => 'one',
[1, 2, 3] => 'testing',
{ 'test' => 'value' } => 'value'
)
expect(my_mash.to_h).to eq(
:'1' => 'symbolizable one',
1 => 'one',
[1, 2, 3] => 'testing',
{ 'test' => 'value' } => 'value'
)
end
end

context 'implicit to_hash on double splat' do
Expand Down
5 changes: 3 additions & 2 deletions spec/hashie/extensions/symbolize_keys_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -89,9 +89,10 @@ def invoke(method)
context 'singleton methods' do
subject { Hash }
let(:object) do
subject.new.merge('a' => 1, 'b' => { 'c' => 2 }).extend(Hashie::Extensions::SymbolizeKeys)
subject.new.merge('a' => 1, 'b' => { 'c' => 2 }, 1 => 'numeric key')
.extend(Hashie::Extensions::SymbolizeKeys)
end
let(:expected_hash) { { a: 1, b: { c: 2 } } }
let(:expected_hash) { { a: 1, b: { c: 2 }, 1 => 'numeric key' } }

describe '.symbolize_keys' do
it 'does not raise error' do
Expand Down
6 changes: 3 additions & 3 deletions spec/hashie/hash_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
it '#to_hash with symbolize_keys set to true returns a hash with symbolized keys' do
hash = Hashie::Hash['a' => 'hey', 123 => 'bob', 'array' => [1, 2, 3]]
symbolized_hash = hash.to_hash(symbolize_keys: true)
expect(symbolized_hash).to eq(a: 'hey', :"123" => 'bob', array: [1, 2, 3])
expect(symbolized_hash).to eq(a: 'hey', 123 => 'bob', array: [1, 2, 3])
end

it "#to_hash should not blow up when #to_hash doesn't accept arguments" do
Expand Down Expand Up @@ -112,9 +112,9 @@ def to_hash(options = {})

expected = {
a: 'hey',
:"123" => 'bob',
123 => 'bob',
array: [1, 2, 3],
subhash: { a: 'hey', b: 'bar', :'123' => 'bob', array: [1, 2, 3] }
subhash: { a: 'hey', b: 'bar', 123 => 'bob', array: [1, 2, 3] }
}

expect(symbolized_hash).to eq(expected)
Expand Down
22 changes: 13 additions & 9 deletions spec/hashie/mash_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -613,6 +613,18 @@ class SubMash < Hashie::Mash
expect(converted.to_hash['a'].first.is_a?(Hash)).to be_truthy
expect(converted.to_hash['a'].first['c'].first.is_a?(Hashie::Mash)).to be_falsy
end

it 'only stringifies keys which can be converted to symbols' do
initial_hash = { 1 => 'a', ['b'] => 2, 'c' => 3, d: 4 }
converted = Hashie::Mash.new(initial_hash)
expect(converted).to eq(1 => 'a', ['b'] => 2, 'c' => 3, 'd' => 4)
end

it 'preserves keys which cannot be converted to symbols' do
initial_hash = { 1 => 'a', '1' => 'b', :'1' => 'c' }
converted = Hashie::Mash.new(initial_hash)
expect(converted).to eq(1 => 'a', '1' => 'c')
end
end

describe '#fetch' do
Expand Down Expand Up @@ -900,7 +912,7 @@ class SubMash < Hashie::Mash
end

it 'returns a mash with the keys and values inverted' do
expect(mash.invert).to eq('apple' => 'a', '4' => 'b')
expect(mash.invert).to eq('apple' => 'a', 4 => 'b')
end

context 'when using with subclass' do
Expand Down Expand Up @@ -977,14 +989,6 @@ class SubMash < Hashie::Mash
expect(subject.dig('a', 'b')).to eq(1)
end

context 'with numeric key' do
carolineartz marked this conversation as resolved.
Show resolved Hide resolved
subject { described_class.new('1' => { b: 1 }) }
it 'accepts a numeric value as key' do
expect(subject.dig(1, :b)).to eq(1)
expect(subject.dig('1', :b)).to eq(1)
end
end

context 'when the Mash wraps a Hashie::Array' do
it 'handles digging into an array' do
mash = described_class.new(alphabet: { first_three: Hashie::Array['a', 'b', 'c'] })
Expand Down