-
Notifications
You must be signed in to change notification settings - Fork 169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add catalog indexer worker #4330
base: main
Are you sure you want to change the base?
Conversation
@@ -0,0 +1,71 @@ | |||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied from ingestion_server
without changes to logic.
@@ -0,0 +1,58 @@ | |||
import logging as log |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied from ingestion_server
without changes to logic.
@@ -0,0 +1,335 @@ | |||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied from ingestion_server
without changes to logic. This provides the mappings from db record to ES document.
@@ -0,0 +1,51 @@ | |||
import logging as log |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied from ingestion_server
without changes to logic.
Fixes
Fixes #4147 by @stacimc
Description
TODO:
Testing Instructions
just build && just up
Use elasticvue to
Delete All Documents
from one of your existing indices. I usedaudio-init-filtered
. The index should now have 0 documents.Now run
just catalog/shell
, and you should be able to curl the indexer worker.You should get back a response like:
Now curl the status_check endpoint that was just returned:
And you should see something like:
In elasticvue, you should now see 100 docs in your target index.
If you used a filtered index, you can now query the API (http://localhost:50281/v1/audio) and see that the results are returned as expected.⚠️ Important: In Elasticvue you must first refresh the index to make the documents available. Then you can inspect them in Elasticvue, or check the API. This will be handled by the DAG later.
Checklist
Update index.md
).main
) or a parent feature branch.just catalog/generate-docs
for catalogPRs) or the media properties generator (
just catalog/generate-docs media-props
for the catalog or
just api/generate-docs
for the API) where applicable.Developer Certificate of Origin
Developer Certificate of Origin