Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate yaml.load and add FullLoader and UnsafeLoader classes #257

Closed
wants to merge 2 commits into from

Conversation

ingydotnet
Copy link
Member

@ingydotnet ingydotnet commented Feb 23, 2019

This addresses CVE-2017-18342

YAML is cross-programming language serialization language. YAML happens to be
used quite often as a configuration format. PyYAML is an implementation of YAML
that provides human friendly (plain text) data serialization for Python.

The CVE asserts that yaml.load() can execute arbitrary code. This is true.
Here are some trivial examples:

python -c 'import yaml; yaml.load("!!python/object/new:os.system [echo EXPLOIT!]")'`
python -c 'import yaml; print(yaml.load("!!python/object/new:abs [-5]"))'
python -c 'import yaml; yaml.load("!!python/object/new:eval [exit(5)]")' ; echo $?
python -c 'import yaml; yaml.load("!!python/object/new:exit [5]")' ; echo $?

This PR prevents arbitrary code execution for yaml_load().

YAML has always had a safe_load method that can load a subset of YAML
without the risk of code execution. CVE-2017-18342 seems to suggest that load
should call safe_load by default. This is not feasible, because it will break
code that is using PyYAML as a full serialization language, not just for simple
config.

Since PyYAML was created as serialization module for Python, it was heavily
influenced by Python's primary (but not human readable) serialization
format/module, Pickle. Pickle has the same problems of code execution on
untrusted data, yet there isn't a similar CVE for Pickle.

The PyYAML maintainers have decided that the plain usage of yaml.load() has
to be deprecated. We can't make it call safe_load and it is unsafe by
default. Even though it has been loudly documented as being unsafe from the
very beginning (just like Pickle), the Python community is raising red flags.


Here is how things will work in 5.1:

yaml.load() will issue a warning unless you call it with the Loader=
parameter. The available Loaders are:

  • BaseLoader
  • FullLoader
  • SafeLoader
  • UnsafeLoader
  • Loader (old name for UnsafeLoader)

You can also use the sugar methods:

  • full_load() -- for load(..., Loader=FullLoader)
  • safe_load()
  • unsafe_load()

There are methods to disable the warning when you use modules that use
yaml.load() that you can't change.

The end result is that you need to declare which Loader you want to use.


We added a new loader class called FullLoader, and we made it the default for
load(). This class is almost as complete for serialization as
UnsafeLoader/Loader, but it avoids arbitrary code execution. We don't expect it
will break any code in the wild.

We still recommend that people choose SafeLoader for untrusted data, but
aribitrary code execution will no longer be possible using yaml.load() with
the default loader (FullLoader). FullLoader will instantiate objects of classes
that you have imported. Since object instantiation runs the class's constructor
code, that may be exploitable.

In a future release (after 5.1) yaml.load() will raise an exception if you don't
explicitly choose the Loader to use.

@ingydotnet ingydotnet added this to To Do in 5.1 Release Feb 23, 2019
@ingydotnet ingydotnet moved this from Possible PRs and To Do items to Added to release/5/1 branch in 5.1 Release Feb 23, 2019
@ingydotnet
Copy link
Member Author

The warning issued by load() contains this url: https://msg.pyyaml.org/load
Which has full current details about the warning and how to respond to it.

ingydotnet and others added 2 commits March 8, 2019 09:06
The `load` and `load_all` methods will issue a warning when they are
called without the 'Loader=' parameter. The warning will point to a URL
that is always up to date with the latest information on the usage of
`load`.

There are several ways to stop the warning:

* Use `full_load(input)` - sugar for `yaml.load(input, FullLoader)`
  * FullLoader is the new safe but complete loader class
* Use `safe_load(input)` - sugar for `yaml.load(input, SafeLoader)`
  * Make sure your input YAML consists of the 'safe' subset
* Use `unsafe_load(input)` - sugar for `yaml.load(input, UnsafeLoader)`
  * Make sure your input YAML consists of the 'safe' subset
* Use `yaml.load(input, Loader=yaml.<loader>)`
  * Or shorter `yaml.load(input, yaml.<loader>)`
  * Where '<loader>' can be:
    * FullLoader - safe, complete Python YAML loading
    * SafeLoader - safe, partial Python YAML loading
    * UnsafeLoader - more explicit name for the old, unsafe 'Loader' class
* yaml.warnings({'YAMLLoadWarning': False})
  * Use this when you use third party modules that use `yaml.load(input)`
  * Only do this if input is trusted

The above `load()` expressions all have `load_all()` counterparts.

You can get the original unsafe behavior with:
* `yaml.unsafe_load(input)`
* `yaml.load(input, Loader=yaml.UnsafeLoader)`

In a future release, `yaml.load(input)` will raise an exception.

The new loader called FullLoader is almost entirely complete as
Loader/UnsafeLoader but it does it avoids all known code execution
paths. It is the preferred YAML loader, and the current default for
`yaml.load(input)` when you get the warning.

Here are some of the exploits that can be triggered with UnsafeLoader
but not with FullLoader:
```
python -c 'import os, yaml; yaml.full_load("!!python/object/new:os.system [echo EXPLOIT!]")'`
python -c 'import yaml; print yaml.full_load("!!python/object/new:abs [-5]")'
python -c 'import yaml; yaml.full_load("!!python/object/new:eval [exit(5)]")' ; echo $?
python -c 'import yaml; yaml.full_load("!!python/object/new:exit [5]")' ; echo $?
@ingydotnet ingydotnet force-pushed the maintainers/load-deprecation branch from 8de78b6 to 2869cea Compare March 8, 2019 17:07
@perlpunk
Copy link
Member

Merged in 0cedb2a and 857dff1

@perlpunk perlpunk closed this Mar 14, 2019
fblackburn1 added a commit to wazo-platform/wazo-auth that referenced this pull request Apr 16, 2019
reason: with pyyaml >= 5, the load function without Loader is deprecated.
Even if we do not use version 5 for now, we should use good practices.
https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml
yaml/pyyaml#257
fblackburn1 added a commit to wazo-platform/wazo-auth-keys that referenced this pull request Apr 16, 2019
reason: with pyyaml >= 5, the load function without Loader is deprecated.
Even if we do not use version 5 for now, we should use good practices.
https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml
yaml/pyyaml#257
fblackburn1 added a commit to wazo-platform/wazo-confd that referenced this pull request Apr 16, 2019
reason: with pyyaml >= 5, the load function without Loader is deprecated.
Even if we do not use version 5 for now, we should use good practices.
https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml
yaml/pyyaml#257
fblackburn1 added a commit to wazo-platform/wazo-confgend that referenced this pull request Apr 16, 2019
reason: with pyyaml >= 5, the load function without Loader is deprecated.
Even if we do not use version 5 for now, we should use good practices.
https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml
yaml/pyyaml#257
fblackburn1 added a commit to wazo-platform/wazo-dird that referenced this pull request Apr 16, 2019
reason: with pyyaml >= 5, the load function without Loader is deprecated.
Even if we do not use version 5 for now, we should use good practices.
https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml
yaml/pyyaml#257
fblackburn1 added a commit to wazo-platform/wazo-plugind that referenced this pull request Apr 16, 2019
reason: with pyyaml >= 5, the load function without Loader is deprecated.
Even if we do not use version 5 for now, we should use good practices.
https://pyyaml.org/wiki/PyYAMLDocumentation#loading-yaml
yaml/pyyaml#257
@perlpunk perlpunk deleted the maintainers/load-deprecation branch December 2, 2019 22:55
mtremer pushed a commit to ipfire/ipfire-2.x that referenced this pull request Feb 14, 2022
- Update from 3.13 to 6.0
- Update of rootfile
- Changelog
6.0 (2021-10-13)
* yaml/pyyaml#327 -- Change README format to Markdown
* yaml/pyyaml#483 -- Add a test for YAML 1.1 types
* yaml/pyyaml#497 -- fix float resolver to ignore `.` and `._`
* yaml/pyyaml#550 -- drop Python 2.7
* yaml/pyyaml#553 -- Fix spelling of “hexadecimal”
* yaml/pyyaml#556 -- fix representation of Enum subclasses
* yaml/pyyaml#557 -- fix libyaml extension compiler warnings
* yaml/pyyaml#560 -- fix ResourceWarning on leaked file descriptors
* yaml/pyyaml#561 -- always require `Loader` arg to `yaml.load()`
* yaml/pyyaml#564 -- remove remaining direct distutils usage
5.4.1 (2021-01-20)
* yaml/pyyaml#480 -- Fix stub compat with older pyyaml versions that may unwittingly load it
5.4 (2021-01-19)
* yaml/pyyaml#407 -- Build modernization, remove distutils, fix metadata, build wheels, CI to GHA
* yaml/pyyaml#472 -- Fix for CVE-2020-14343, moves arbitrary python tags to UnsafeLoader
* yaml/pyyaml#441 -- Fix memory leak in implicit resolver setup
* yaml/pyyaml#392 -- Fix py2 copy support for timezone objects
* yaml/pyyaml#378 -- Fix compatibility with Jython
5.3.1 (2020-03-18)
* yaml/pyyaml#386 -- Prevents arbitrary code execution during python/object/new constructor
5.3 (2020-01-06)
* yaml/pyyaml#290 -- Use `is` instead of equality for comparing with `None`
* yaml/pyyaml#270 -- Fix typos and stylistic nit
* yaml/pyyaml#309 -- Fix up small typo
* yaml/pyyaml#161 -- Fix handling of __slots__
* yaml/pyyaml#358 -- Allow calling add_multi_constructor with None
* yaml/pyyaml#285 -- Add use of safe_load() function in README
* yaml/pyyaml#351 -- Fix reader for Unicode code points over 0xFFFF
* yaml/pyyaml#360 -- Enable certain unicode tests when maxunicode not > 0xffff
* yaml/pyyaml#359 -- Use full_load in yaml-highlight example
* yaml/pyyaml#244 -- Document that PyYAML is implemented with Cython
* yaml/pyyaml#329 -- Fix for Python 3.10
* yaml/pyyaml#310 -- Increase size of index, line, and column fields
* yaml/pyyaml#260 -- Remove some unused imports
* yaml/pyyaml#163 -- Create timezone-aware datetimes when parsed as such
* yaml/pyyaml#363 -- Add tests for timezone
5.2 (2019-12-02)
* Repair incompatibilities introduced with 5.1. The default Loader was changed,
  but several methods like add_constructor still used the old default
  yaml/pyyaml#279 -- A more flexible fix for custom tag constructors
  yaml/pyyaml#287 -- Change default loader for yaml.add_constructor
  yaml/pyyaml#305 -- Change default loader for add_implicit_resolver, add_path_resolver
* Make FullLoader safer by removing python/object/apply from the default FullLoader
  yaml/pyyaml#347 -- Move constructor for object/apply to UnsafeConstructor
* Fix bug introduced in 5.1 where quoting went wrong on systems with sys.maxunicode <= 0xffff
  yaml/pyyaml#276 -- Fix logic for quoting special characters
* Other PRs:
  yaml/pyyaml#280 -- Update CHANGES for 5.1
5.1.2 (2019-07-30)
* Re-release of 5.1 with regenerated Cython sources to build properly for Python 3.8b2+
5.1.1 (2019-06-05)
* Re-release of 5.1 with regenerated Cython sources to build properly for Python 3.8b1
5.1 (2019-03-13)
* yaml/pyyaml#35 -- Some modernization of the test running
* yaml/pyyaml#42 -- Install tox in a virtualenv
* yaml/pyyaml#45 -- Allow colon in a plain scalar in a flow context
* yaml/pyyaml#48 -- Fix typos
* yaml/pyyaml#55 -- Improve RepresenterError creation
* yaml/pyyaml#59 -- Resolves #57, update readme issues link
* yaml/pyyaml#60 -- Document and test Python 3.6 support
* yaml/pyyaml#61 -- Use Travis CI built in pip cache support
* yaml/pyyaml#62 -- Remove tox workaround for Travis CI
* yaml/pyyaml#63 -- Adding support to Unicode characters over codepoint 0xffff
* yaml/pyyaml#75 -- add 3.12 changelog
* yaml/pyyaml#76 -- Fallback to Pure Python if Compilation fails
* yaml/pyyaml#84 -- Drop unsupported Python 3.3
* yaml/pyyaml#102 -- Include license file in the generated wheel package
* yaml/pyyaml#105 -- Removed Python 2.6 & 3.3 support
* yaml/pyyaml#111 -- Remove commented out Psyco code
* yaml/pyyaml#129 -- Remove call to `ord` in lib3 emitter code
* yaml/pyyaml#149 -- Test on Python 3.7-dev
* yaml/pyyaml#158 -- Support escaped slash in double quotes "\/"
* yaml/pyyaml#175 -- Updated link to pypi in release announcement
* yaml/pyyaml#181 -- Import Hashable from collections.abc
* yaml/pyyaml#194 -- Reverting yaml/pyyaml#74
* yaml/pyyaml#195 -- Build libyaml on travis
* yaml/pyyaml#196 -- Force cython when building sdist
* yaml/pyyaml#254 -- Allow to turn off sorting keys in Dumper (2)
* yaml/pyyaml#256 -- Make default_flow_style=False
* yaml/pyyaml#257 -- Deprecate yaml.load and add FullLoader and UnsafeLoader classes
* yaml/pyyaml#261 -- Skip certain unicode tests when maxunicode not > 0xffff
* yaml/pyyaml#263 -- Windows Appveyor build

Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>

 --git a/config/rootfiles/packages/python3-yaml b/config/rootfiles/packages/python3-yaml
x 0870a2346..bd4009a08 100644
* yaml/pyyaml#195 -- Build libyaml on travis
* yaml/pyyaml#196 -- Force cython when building sdist
* yaml/pyyaml#254 -- Allow to turn off sorting keys in Dumper (2)
* yaml/pyyaml#256 -- Make default_flow_style=False
* yaml/pyyaml#257 -- Deprecate yaml.load and add FullLoader and Uns
oader classes
* yaml/pyyaml#261 -- Skip certain unicode tests when maxunicode not
xffff
* yaml/pyyaml#263 -- Windows Appveyor build

Signed-off-by: Adolf Belka <adolf.belka@ipfire.org>
Reviewed-by: Peter Müller <peter.mueller@ipfire.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
5.1 Release
Added to release/5/1 branch
Development

Successfully merging this pull request may close these issues.

None yet

2 participants