Skip to content

replacer #339

Closed
Closed
@sjpt

Description

@sjpt

I needed a yaml dumper that included a replacer option similar to JSON.stringify.
This turned out to be a simple modification to js-yaml; there may be details not covered.
Its use changes the handling of undefined fields so that they are omitted from the yaml (similar to JSON).

If there is any interest in this change I will make it available.

I will probably add a reviver option for parsing.

Activity

puzrin

puzrin commented on Apr 7, 2017

@puzrin
Member

I'm not sure this is needed here, but leave this open to collect feedback.

sjpt

sjpt commented on Apr 7, 2017

@sjpt
Author

Thanks for the comment.

For background information: The reason we need the replacer is to prevent some objects getting placed in the output. We have an application using three.js, and many of my objects have associated THREE objects. However, when we dump the state we don't want to dump most of those, especially not the gory details of large vertex buffers, but rather recreate them after load.

We used to use JSON, but it was very noisy, especially for multiline strings with double quotes,;so awkward when we needed to edit by hand. (At least it wasn't nearly as bad as XML)

puzrin

puzrin commented on Apr 7, 2017

@puzrin
Member

You could just create filtering function before calling yaml dump: yaml.safe_dump(your_filter(data))

sjpt

sjpt commented on Apr 7, 2017

@sjpt
Author

I don't want to modify my data structure or clone the major part of it; which if I understand you right is what you expect your_filter(data) to do? Rather, just to omit certain objects from the dump.

zakhenry

zakhenry commented on Jun 23, 2017

@zakhenry

In the same boat (wanting to discard undefined field); it is odd that the behaviour should differ from json as yaml is supposed to be a superset of json.

graphicore

graphicore commented on Jan 23, 2018

@graphicore

See #391 for more discussion about this topic, includes also examples. Closed because it is a duplicate.

graphicore

graphicore commented on Jan 23, 2018

@graphicore

Can we have this issue renamed to something like:

Feature Request: add optional replacer API like toJSON of JSON.stringify

@sjpt I'd suggest an option like {replacer: 'toJSON'}to turn on the feature and to configure it.

I'm still not sure if this feature is welcome.

If there is any interest in this change I will make it available.

If possible, could you please make it available as a PR (or as a branch in your fork), I'm interested in it anyways :-)

sjpt

sjpt commented on Jan 23, 2018

@sjpt
Author

I'll try to make a PR soon. My replacer is like the global replacer of JSON, not like the toJSON object function. Both have their place. How about options {replacer: globalFunction, localReplacer: 'toJSON'}

Actually, I specifically want to avoid 'toJSON' discussed in #391 as it is misused in three.js so for example
JSON.stringify(new THREE.Texture())
JSON.stringify({ x: new THREE.Texture()})
throws an exception, as does serializing anything that indirectly contains a texture unless you explicitly avoid it with a replacer. (I raised an issue there years ago and never followed it up.)

sjpt

sjpt commented on Jan 23, 2018

@sjpt
Author

Pull request created, I apologize if I have not done this correctly.

graphicore

graphicore commented on Jan 24, 2018

@graphicore

Pull request created

PR is here: #392

I had a look and added some comments.

My replacer is like the global replacer of JSON

I see, you are talking about the replacer parameter of JSON.stringify. Awesome, I didn't know that :-)

JSON.stringify(value[, replacer[, space]])

I specifically want to avoid 'toJSON' discussed in #391 as it is misused in three.js

In this case, should I re-open #391 ?

Is that a problem of the 'toJSON' or of three.js. You can't avoid misuse of any API, I don't get how a bad usage of an API makes it undesirable. It's different if the API itself leads to bad design, that would be clearly a flaw. I'd like to see what's the case here, misuse of API or inherent bad API design/concept.

I'm looking for good arguments not to want//have/use/pursue toJSON, as I'm quite in favor of it. A good argument can change my mind.

I raised an issue there years ago and never followed it up.

Do you have a link to this issue. I'm interested in the discussion.

graphicore

graphicore commented on Jan 24, 2018

@graphicore

I raised an issue there years ago and never followed it up.

Do you have a link to this issue. I'm interested in the discussion.

Found it: mrdoob/three.js#8428

With having an option to configure localReplacer this is a none-issue in here anyways. And, for three.js using a method toJSON with different semantics than JSON.stringify expects, this just means that something like JSON.stringify(new THREE.Texture()) is not a scenario intended by the three.js developers. Just don't do it.

sjpt

sjpt commented on Jan 24, 2018

@sjpt
Author

Thanks for finding that three.js issue of mine, I was about to hunt for it.

I agree that configurable localReplacer makes this a non-issue now we are using yaml rather than JSON. In fact, for us the localReplacer is not needed at all, replacer does what we need.

Also, it is easy just not to do JSON.stringify(new THREE.Texture()) However, it is not easy to avoid an indirect equivalent, when the THREE.Texture is referenced deep down inside my own structures which I did want to JSON serialize (leaving out the references to THREE objects which we reconstruct after load).

I could make a deep copy of our structure without the THREE references and then serialize that, but that is much more work

graphicore

graphicore commented on Jan 24, 2018

@graphicore

In fact, for us the localReplacer is not needed at all, replacer does what we need.

Not in my case, that is when the objects you serialize actually implement toJSON for use with JSON.stringify.

24 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @puzrin@graphicore@zakhenry@rlidwka@sjpt

        Issue actions

          replacer · Issue #339 · nodeca/js-yaml