Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Union together different channels for Fenix views #1077

Open
jklukas opened this issue Jan 21, 2020 · 2 comments
Open

Union together different channels for Fenix views #1077

jklukas opened this issue Jan 21, 2020 · 2 comments

Comments

@jklukas
Copy link
Contributor

jklukas commented Jan 21, 2020

The GUD datasets currently include only the release version of Fenix and they ignore any data in the org_mozilla_fenix_nightly_stable dataset. We should probably build in that support.

But it brings up a bigger question of how we want to present Fenix data to users. Should we have separate ETL pathways for the different source tables, unioning together the final results? Or should we union together these different channels as early as possible?

We could alter the org_mozilla_fenix.baseline view to be a union of the release and nightly tables, setting the normalized_channel field to "release" for rows coming from the one tables and "nightly" for rows coming from the other table. That approach would be vulnerable if there's schema drift between the two tables; it's not clear to me whether the probes are sourced independently for the different fenix channels or if we should always expect the schemas to match exactly. If the schemas ever didn't match, the view would return errors, which would be a bad user experience.

It would certainly be possible to union the two tables at the clients_daily level and let rows from nightly flow through that way.

Or we could duplicate all the queries from Fenix release to Fenix nightly. This is the purest solution, but leads to code duplication and proliferation of tasks in Airflow.

cc @fbertsch @relud

@relud
Copy link
Contributor

relud commented Jan 21, 2020

It would certainly be possible to union the two tables at the clients_daily level and let rows from nightly flow through that way.

seems like a nice compromize

@jklukas
Copy link
Contributor Author

jklukas commented May 24, 2021

union views for clients_daily-like tables have been deployed as part of https://bugzilla.mozilla.org/show_bug.cgi?id=1708166

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants