Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Azure name cleanup #752

Merged
merged 2 commits into from Sep 9, 2019
Merged

Fix Azure name cleanup #752

merged 2 commits into from Sep 9, 2019

Conversation

nitely
Copy link
Contributor

@nitely nitely commented Sep 9, 2019

fixes #609

Changes:

  • remove stripping special characters from file name
  • escape URL reserved and unsafe characters on url method

Notes:

  • This is consistent with the other storages
  • Django does the same name cleaning as the previous code, anyway
  • Django name cleaning can be bypassed by overriding get_valid_name or calling default_storage.save directly. This was not possible with the old code
  • Since this is a superset of what was previously allowed, it's a minor breaking change
  • Files uploaded with the previous code will still work

CC @jschneier

@nitely nitely mentioned this pull request Sep 9, 2019
@jschneier jschneier merged commit 2fbaa70 into jschneier:master Sep 9, 2019
@jschneier
Copy link
Owner

Thanks as usual @nitely.

bors bot added a commit to mozilla/normandy that referenced this pull request Sep 17, 2019
1984: Scheduled weekly dependency update for week 37 r=rehandalal a=pyup-bot






### Update [botocore](https://pypi.org/project/botocore) from **1.12.224** to **1.12.228**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.12.228
   ```
   ========

* api-change:``elbv2``: Update elbv2 client to latest version
* api-change:``ec2``: Update ec2 client to latest version
* api-change:``workmailmessageflow``: Update workmailmessageflow client to latest version
* api-change:``medialive``: Update medialive client to latest version
   ```
   
  
  
   ### 1.12.227
   ```
   ========

* api-change:``stepfunctions``: Update stepfunctions client to latest version
* api-change:``rds``: Update rds client to latest version
* api-change:``ec2``: Update ec2 client to latest version
* api-change:``mediaconnect``: Update mediaconnect client to latest version
* api-change:``ses``: Update ses client to latest version
* api-change:``config``: Update config client to latest version
   ```
   
  
  
   ### 1.12.226
   ```
   ========

* api-change:``storagegateway``: Update storagegateway client to latest version
   ```
   
  
  
   ### 1.12.225
   ```
   ========

* api-change:``qldb``: Update qldb client to latest version
* api-change:``marketplacecommerceanalytics``: Update marketplacecommerceanalytics client to latest version
* api-change:``appstream``: Update appstream client to latest version
* api-change:``ec2``: Update ec2 client to latest version
* api-change:``robomaker``: Update robomaker client to latest version
* api-change:``appmesh``: Update appmesh client to latest version
* api-change:``qldb-session``: Update qldb-session client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/botocore
  - Changelog: https://pyup.io/changelogs/botocore/
  - Repo: https://github.com/boto/botocore
</details>





### Update [certifi](https://pypi.org/project/certifi) from **2019.6.16** to **2019.9.11**.


*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/certifi
  - Homepage: https://certifi.io/
</details>





### Update [datadog](https://pypi.org/project/datadog) from **0.29.3** to **0.30.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 0.30.0
   ```
   * [BUGFIX] Treat `API_HOST` as URL, not as string [411][]
* [FEATURE] Add `return_raw_response` option to `initialize` to enable adding raw responses to return values [414][]
* [IMPROVEMENT] Add project URLs to package metadata [413][] (thanks [Tenzer][])
* [IMPROVEMENT] Add support for handling a 401 status as an API error [418][]
* [IMPROVEMENT] Allow configuring proxy in `~/.dogrc` for usage with dogshell [415][]
* [IMPROVEMENT] Update `user` resource name to `users` to match new plural endpoints [421][]
* [OTHER] Add deprecation warning to old aws lambda threadstats integration [417][]
* [OTHER] Removed functionality to delete events and comments, as it&#39;s no longer supported by API [420][]
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/datadog
  - Changelog: https://pyup.io/changelogs/datadog/
  - Homepage: https://www.datadoghq.com
</details>





### Update [importlib-metadata](https://pypi.org/project/importlib-metadata) from **0.20** to **0.23**.


*The bot wasn't able to find a changelog for this release. [Got an idea?](https://github.com/pyupio/changelogs/issues/new)*

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/importlib-metadata
  - Docs: http://importlib-metadata.readthedocs.io/
</details>





### Update [pluggy](https://pypi.org/project/pluggy) from **0.12.0** to **0.13.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 0.13.0
   ```
   ==========================

Trivial/Internal Changes
------------------------

- `222 &lt;https://github.com/pytest-dev/pluggy/issues/222&gt;`_: Replace ``importlib_metadata`` backport with ``importlib.metadata`` from the
  standard library on Python 3.8+.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/pluggy
  - Changelog: https://pyup.io/changelogs/pluggy/
  - Repo: https://github.com/pytest-dev/pluggy
</details>





### Update [boto3](https://pypi.org/project/boto3) from **1.9.224** to **1.9.228**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.9.228
   ```
   =======

* api-change:``elbv2``: [``botocore``] Update elbv2 client to latest version
* api-change:``ec2``: [``botocore``] Update ec2 client to latest version
* api-change:``workmailmessageflow``: [``botocore``] Update workmailmessageflow client to latest version
* api-change:``medialive``: [``botocore``] Update medialive client to latest version
   ```
   
  
  
   ### 1.9.227
   ```
   =======

* api-change:``stepfunctions``: [``botocore``] Update stepfunctions client to latest version
* api-change:``rds``: [``botocore``] Update rds client to latest version
* api-change:``ec2``: [``botocore``] Update ec2 client to latest version
* api-change:``mediaconnect``: [``botocore``] Update mediaconnect client to latest version
* api-change:``ses``: [``botocore``] Update ses client to latest version
* api-change:``config``: [``botocore``] Update config client to latest version
   ```
   
  
  
   ### 1.9.226
   ```
   =======

* api-change:``storagegateway``: [``botocore``] Update storagegateway client to latest version
   ```
   
  
  
   ### 1.9.225
   ```
   =======

* api-change:``qldb``: [``botocore``] Update qldb client to latest version
* api-change:``marketplacecommerceanalytics``: [``botocore``] Update marketplacecommerceanalytics client to latest version
* api-change:``appstream``: [``botocore``] Update appstream client to latest version
* api-change:``ec2``: [``botocore``] Update ec2 client to latest version
* api-change:``robomaker``: [``botocore``] Update robomaker client to latest version
* api-change:``appmesh``: [``botocore``] Update appmesh client to latest version
* api-change:``qldb-session``: [``botocore``] Update qldb-session client to latest version
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/boto3
  - Changelog: https://pyup.io/changelogs/boto3/
  - Repo: https://github.com/boto/boto3
</details>





### Update [django-countries](https://pypi.org/project/django-countries) from **5.4** to **5.5**.


<details>
  <summary>Changelog</summary>
  
  
   ### 5.5
   ```
   =======================

- Django 3.0 compatibility.

- Plugin system for extending the ``Country`` object.
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/django-countries
  - Changelog: https://pyup.io/changelogs/django-countries/
  - Repo: https://github.com/SmileyChris/django-countries/
</details>





### Update [django-storages](https://pypi.org/project/django-storages) from **1.7.1** to **1.7.2**.


<details>
  <summary>Changelog</summary>
  
  
   ### 1.7.2
   ```
   ******************

S3
--

- Avoid misleading ``AWS_DEFAULT_ACL`` warning for insecure ``default_acl`` when
  overridden as a class variable (`591_`)
- Propagate file deletion to cache when ``preload_metadata`` is ``True``,
  (not the default) (`743`_, `749`_)
- Fix exception raised on closed file (common if using ``ManifestFilesMixin`` or
  ``collectstatic``. (`382`_, `754`_)

Azure
-----

- Pare down the required packages in ``extra_requires`` when installing the ``azure`` extra to only
  ``azure-storage-blob`` (`680`_, `684`_)
- Fix compatability with ``generate_blob_shared_access_signature`` updated signature (`705`_, `723`_)
- Fetching a file now uses the configured timeout rather than hardcoding one (`727`_)
- Add support for configuring all blobservice options: ``AZURE_EMULATED_MODE``, ``AZURE_ENDPOINT_SUFFIX``,
  ``AZURE_CUSTOM_DOMAIN``, ``AZURE_CONNECTION_STRING``, ``AZURE_CUSTOM_CONNECTION_STRING``,
  ``AZURE_TOKEN_CREDENTIAL``. See the docs for more info. Huge thanks once again to nitely. (`750`_)
- Fix filename handling to not strip special characters (`609`_, `752`_)


Google Cloud
------------

- Set the file acl in the same call that uploads it (`698`_)
- Reduce the number of queries and required permissions when ``GS_AUTO_CREATE_BUCKET`` is
  ``False`` (the default) (`412`_, `718`_)
- Set the ``predefined_acl`` when creating a ``GoogleCloudFile`` using ``.write``
  (`640`_, `756`_)
- Add ``GS_BLOB_CHUNK_SIZE`` setting to enable efficient uploading of large files (`757`_)

Dropbox
-------

- Complete migration to v2 api with file fetching and metadata fixes (`724`_)
- Add ``DROPBOX_TIMEOUT`` to configure client timeout defaulting to 100 seconds
  to match the underlying sdk. (`419`_, `747`_)

SFTP
----

- Fix reopening a file (`746`_)

.. _591: jschneier/django-storages#591
.. _680: jschneier/django-storages#680
.. _684: jschneier/django-storages#684
.. _698: jschneier/django-storages#698
.. _705: jschneier/django-storages#705
.. _723: jschneier/django-storages#723
.. _727: jschneier/django-storages#727
.. _746: jschneier/django-storages#746
.. _724: jschneier/django-storages#724
.. _412: jschneier/django-storages#412
.. _718: jschneier/django-storages#718
.. _743: jschneier/django-storages#743
.. _749: jschneier/django-storages#749
.. _750: jschneier/django-storages#750
.. _609: jschneier/django-storages#609
.. _752: jschneier/django-storages#752
.. _382: jschneier/django-storages#382
.. _754: jschneier/django-storages#754
.. _419: jschneier/django-storages#419
.. _747: jschneier/django-storages#747
.. _640: jschneier/django-storages#640
.. _756: jschneier/django-storages#756
.. _757: jschneier/django-storages#757
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/django-storages
  - Changelog: https://pyup.io/changelogs/django-storages/
  - Repo: https://github.com/jschneier/django-storages
</details>





### Update [kinto-http](https://pypi.org/project/kinto-http) from **10.4.1** to **10.5.0**.


<details>
  <summary>Changelog</summary>
  
  
   ### 10.5.0
   ```
   ===================

**New features**

- Add history support (fixes 112), Thanks FlorianKuckelkorn!
   ```
   
  
</details>


 

<details>
  <summary>Links</summary>
  
  - PyPI: https://pypi.org/project/kinto-http
  - Changelog: https://pyup.io/changelogs/kinto-http/
  - Repo: https://github.com/Kinto/kinto-http.py/
</details>







Co-authored-by: pyup-bot <github-bot@pyup.io>
@nitely nitely deleted the bugfix_remove_name_cleanup branch September 22, 2020 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Azure backend strips some non-alphabetic characters from the file name
2 participants