Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery export: Add option to include path_params in schema views #1248

Open
jorroll opened this issue Sep 29, 2022 · 6 comments · May be fixed by #1790
Open

BigQuery export: Add option to include path_params in schema views #1248

jorroll opened this issue Sep 29, 2022 · 6 comments · May be fixed by #1790
Assignees
Labels
extension: firestore-bigquery-export Related to firestore-bigquery-export extension type: feature request New feature or request

Comments

@jorroll
Copy link

jorroll commented Sep 29, 2022

[REQUIRED] Step 2: Extension name

This feature request is for extension: firestore-bigquery-export

  • Specifically it's for the @firebaseextensions/fs-bq-schema-views script

What feature would you like to see?

I'd like for the @firebaseextensions/fs-bq-schema-views script to have an option to include the path_params in the generated schema view.

How would you use it?

Using the path_params in SQL queries. The value of this feels self evident, but let me know if it's not.

@yamankatby
Copy link
Member

yamankatby commented Sep 29, 2022

Hey @jorroll, thank you for raising this issue, I'll add it to the project tracker for further investigation.

@yamankatby yamankatby added the type: feature request New feature or request label Sep 29, 2022
@yamankatby yamankatby added this to Under consideration in Extension Update Tracker via automation Sep 29, 2022
@BenjaminKlatt
Copy link

We got a similar need and it would be great to have this option

@cabljac cabljac added the extension: firestore-bigquery-export Related to firestore-bigquery-export extension label Oct 25, 2022
@otopba
Copy link

otopba commented Nov 11, 2022

Yes, this is super important.

@rahulsekar
Copy link

Workaround: The generated view query can be manually edited to get fields from 'path_params' instead of 'data'.

...
FIRST_VALUE(JSON_EXTRACT_SCALAR(data path_params, '$.classId')) OVER(
PARTITION BY document_name
ORDER BY
timestamp DESC
) AS classId,
...

Having this in the script would be great. My suggestion for the schema json:

{ "name": "classId", "type": "string", "source": "path_params"}

@conceptualben
Copy link

This is much needed indeed. I do have a Firestore structure with sub-collections and need this for an easier join between master and child documents.

I did add my path wildcard as a field manually to the view by just like @rahulsekar mentioned pretty easily.

@luksfarris
Copy link

Seems like this feature doesn't exist yet. Would be great to have it.
I'm thinking of an alternative approach, where I create another extra view that maps parent_id (parent collection) to child_id (child collection). This obviously assumes child collections have unique identifiers. Then I can join both tables to get the "path_params" id

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
extension: firestore-bigquery-export Related to firestore-bigquery-export extension type: feature request New feature or request
Projects
Extension Update Tracker
  
Under consideration
Development

Successfully merging a pull request may close this issue.

8 participants