- Sponsor
-
Notifications
You must be signed in to change notification settings - Fork 646
Shell string encoder #1526
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
I figured (duh), the basic functionality can be already done within yq "\"'\" + sub(\"'\", \"'\''\") + \"'\" |
I'd love to put something like this into |
Hi @mikefarah. Yes, the library would surely fit the functionality. Its pretty much only interesting piece of code is this simple line, where the author replaces What And to alleviate your concerns - the shell quoting is really that dead simple. Once in a single quoted string block, it gobbles everything as-is. The only concern is the All in all, it's always better to have a feature, even though potentially not performance optimal 🙂 So feel free to use |
Thanks for your help - I ended up doing the Will be available in the next release. I didn't end up doing the newline logic - happy to take an MR for it though |
Hey @mikefarah , I love you got to it this quickly, thank you, however there's a bug in your implementation (at least in the commit referenced above). Let me break this down real quick in an example, but pls read this carefully once more 🙂
TL;DR, the replace logic (line 41) should be: value = "'" + strings.ReplaceAll(value, "'", "'\\''") + "'" and the rest accommodated accordingly. In more detail - having the string
I think there's actually one missing piece to the puzzle - the way how shell concatenates strings. That is by sticking things right after each other, eventually taking out the quotes, e.g.:
FYI, I just realized while this rather trivial approach is sound and will produce a valid string, it's a bit fuzzy around two edge cases that are by-producing quoted empty strings
To tackle this, the replacing logic would have to track whether it's inside a quote or not and might look like this: # pseudo python :D (sorry, not yet a go dev)
inQuote = false
for ic in input_characters:
if ic == "'": # ic is quote
if inQuote:
output "'" # to get out of a quote block
inQuote = not inQuote
output "\'" # print the escaped quote
else: # not a quote
if not inQuote:
output "'" # get into a quote block
inQuote = not inQuote
output ic
if inQuote:
output "'" # to print the last quote if required This would work IMHO the most predictable - non-quote chunks wrapped in quotes, literal quotes backslash escaped, e.g.:
This doesn't take the "safe chars" else: # not a quote
if not is_safe(ic) and not inQuote:
output "'" # get into a quote block In that case the quoted blocks would start with the first unsafe character and end on a quote or the very end. This would be the minimal option (regarding extra quotes):
|
Hi @mikefarah, I thought it might be fun to code some actual Go, so here's a PR built on top of what you did. |
Working implementation (hopefully) merged to |
Thanks for the PR!
…________________________________
From: Vít Zikmund ***@***.***>
Sent: Thursday, February 9, 2023 10:10:07 AM
To: mikefarah/yq ***@***.***>
Cc: Mike Farah ***@***.***>; Mention ***@***.***>
Subject: Re: [mikefarah/yq] Shell string encoder (Issue #1526)
Closed #1526<#1526> as completed.
—
Reply to this email directly, view it on GitHub<#1526 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAIZHNNRHRTQHHHOWUA2Q7LWWSX67ANCNFSM6AAAAAAT66MGOU>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Release in v4.31.1 :) |
Please describe your feature request.
Hi Mike (thanks for this great tool and remarkable development effort), it would be great to have something as
jq
's@sh
operator that encodes a string into a literal shell string representation, so one could safely generate things like.env
files and otherwise combineyq
with shell scripts.Describe the solution you'd like
In general, one should be able to wrap a string with single quotes and escape any that are in the payload. The handling differs from the yaml
"single"
quoting style, though.Let's say we have this little
test.yaml
:So the result from
yq '.a | @sh' test.yaml
would do the same as:Shell considers any consecutive string of character as a single token as long as it doesn't contain unescaped/unquoted characters from the special
IFS
env variable, which by default contains<space><tab><newline>
. These are the default (and usually only) separators. I would safely assume this to be the case to not make the code overly complicated. As there's no way how to escape a single quote within a single-quoted string, one needs to exit the current quoted block, print a literal escaped quote and start another quoted block, which is exactly what can be seen in the output above.Naturally, this encoder would only apply to the string type and should bail out on another type. Also I think there's no need to have a decoder thanks to all the environment variable handling operators
yq
already has.Describe alternatives you've considered
I've imagined that some folks would also appreciate this to be a single-line output. Although this is not supported in lightweight shells, the widespread BASH can use literal escapes in a "dollar single quoted string" like
$'\n'
, which is a simple newline. So in addition to the above, the encode could escape newlines with\n
and wrapping that part with$'...'
instead of plain'...'
.In that case the output from
yq '.a | @sh("oneline")' test.yaml
could look like this:I don't have any preference of the above, each gets the job done, however the first and last can be implemented as a single pass without holding a context. Also while this alternative is a fun-to-have, it's way beyond the original desire ;)
The text was updated successfully, but these errors were encountered: