Example has invalid JSON

[visch]

https://sdk.meltano.com/en/latest/stream_maps.html has invalid JSON

Specifically

{
    "stream_maps": {
        "customers": {
            // exclude these since we're capturing them in the pii stream
            "email": null,
            "full_name": null
        },
        "customers_pii": {
            "__source__": "customers",
            // include just the PII and the customer_id
            "customer_id": "customer_id",
            "email": "email",
            "full_name": "full_name",
            // exclude anything not declared
            "__else__": null,
        },
    },
}

Valid json would be

{
	"stream_maps": {
		"customers": {
			"email": null,
			"full_name": null
		},
		"customers_pii": {
			"__source__": "customers",
			"customer_id": "customer_id",
			"email": "email",
			"full_name": "full_name",
			"__else__": null
		}
	}
}

[visch]

Looked a fixing this an it's a slightly bigger question https://sdk.meltano.com/en/latest/stream_maps.html

On this page instead of the examples being JSON they are python dicts. What about using meltano.yml instead? Then we could add comments and be compliant. Seems like a much better place than we are. Issue is users that don't use meltano may be confused, so maybe for each example we have a meltano.yml , and a json equivalent?

That seems to make a lot of sense to me. Maybe a ui element like this https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#:~:text=The%20tables%20below%20include%20the%20known%20endpoints%20for%20supported%20operating%20systems%3A . Default to meltano.yml ,and offer the json option so folks can copy paste into their config.json

[edgarrmondragon]

@visch I like the idea of having config examples both in meltano YAML format and raw JSON. I might give sphinx-tabs a try.

[edgarrmondragon]

I might give sphinx-tabs a try.

Bad luck, sphinx-tabs depends on docutils>=0.18 but sphinx-rtd-theme depends on docutils<0.18: readthedocs/sphinx_rtd_theme#1302

[edgarrmondragon]

We moved to Furo, so we should now be able to support sphinx-tabs, if anyone wants to tackle this

[echen805]

@edgarrmondragon if no one has called it internally, would love to try this over the weekend!

[edgarrmondragon]

@edgarrmondragon if no one has called it internally, would love to try this over the weekend!

@echen805 By all means, go for it!

[edgarrmondragon]

@echen805 Let me know if you're still interested in contributing 🙂

[echen805]

Ooof sorry work has been keeping me real busy. You can un-assign me and I can help out when I have time again

[mjsqu]

I made a start on this here: https://github.com/mjsqu/sdk/tree/feat/sphinx_tabs

First time doing anything with sphinx so I thought I'd just make one update first to check it all looks ok on one of the code samples. I've shifted comments out of the JSON and into the meltano.yml tab.

[mjsqu]

Added tabs for all stream map samples to the same branch ⬆️

When using stream maps in meltano.yml is it ok to leave null out? I prefer specifying it to aid readability, so in the above example I think I corrected - e.g.:

# Exclude email
email: 

# to the following for more readability

# Explicitly use null to show email is not mapped
email: null