Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preserve xml file encoding to prevent non-ascii chars from disappearing #82

Open
carlossanlop opened this issue Aug 9, 2021 · 2 comments
Labels
port-to-docs Issues related to the PortToDocs tool.

Comments

@carlossanlop
Copy link
Member

carlossanlop commented Aug 9, 2021

For example, System/Math.xml and MathF.xml have several unicode characters that get lost after the tool runs.

@danmoseley
Copy link
Member

Previously the tool loaded in 1252
https://github.com/carlossanlop/DocsPortingTool/blob/bcbb8d36dc3c4e264b2d37b063cdb737ba0e1796/Libraries/Docs/DocsCommentsContainer.cs#L179-L182
so I changed the save code to save in 1252 based on that comment
https://github.com/carlossanlop/DocsPortingTool/blob/bcbb8d36dc3c4e264b2d37b063cdb737ba0e1796/Libraries/Docs/DocsCommentsContainer.cs#L50-L51

I'm now unsure what is correct. @gewarren is asking. (UTF-8 everywhere would be wonderful)

@carlossanlop
Copy link
Member Author

Thanks for that. I chose that encoding because 99% of the xml files had that particular encoding.

Ideally (and I tried to do this in many ways) we would extract the original encoding of the file, and use that when loading the file, then when saving the file. Unfortunately, it wasn't as straighforward as I hoped: whenever a non-ascii character showed up, the XML APIs would either throw an unexpected exception, or the characters would get unexpectedly replaced with the square character □.

@carlossanlop carlossanlop added the port-to-docs Issues related to the PortToDocs tool. label Aug 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
port-to-docs Issues related to the PortToDocs tool.
Projects
None yet
Development

No branches or pull requests

2 participants