Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

problem with delete or update existing records on WeaviateIndex store #1636

Open
2 of 6 tasks
vincetrep opened this issue Jun 9, 2023 · 0 comments
Open
2 of 6 tasks
Assignees

Comments

@vincetrep
Copy link

vincetrep commented Jun 9, 2023

Initial Checks

  • I have read and followed the docs and still think this is a bug

Description

I'm seeing an odd behavior which may be a bug on the WeaviateIndex implementation:
I have a document with subindices.
Expected Behavior
When that document is saved in the Weaviate store and i then try to save it again, should I expect the record to be updated for the same docarrayid?
Actual Behavior
What happens is that the parent index e.g. RecipeIndex doesn't get duplicated. However, all the subindices get created again and they all have the same parent_id. My expectation would be to update the children classes like a RecipeIndex_name that have the same parent_id as the RecipeIndex class I am updating get updated and not duplicated.

###Update
There is the same issue when using store.delitem => only the parent record is deleted. Then subindices are still there after the delete operation.

Example Code

import docarray
from docarray.documents import TextDoc
class RecipeRecord(BaseDoc):
    recipe_id: str
    name: DocList[TextDoc]

from docarray.index.backends.weaviate import WeaviateDocumentIndex
import weaviate

host="http://localhost:9999"
dbconfig = WeaviateDocumentIndex.DBConfig(
    host=host,
    index_name='RecipeIndex'
)  # Replace with your endpoint)

store = WeaviateDocumentIndex[RecipeRecord](db_config=dbconfig)

doc = RecipeRecord(
id='id123'
recipe_id='id123',
name = DocList[TextDoc]([TextDoc(text='test name',embedding=None)])
)

docList = DocList[RecipeRecord]()
docList.append(doc)

store.index(docList)
#upon running the index again, there will be partial duplication
store.index(docList)

Python, Pydantic & OS Version

docarray 0.33

Affected Components

@vincetrep vincetrep changed the title problem with updating existing records on WeaviateIndex store problem with delete or update existing records on WeaviateIndex store Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo/Bug
Development

No branches or pull requests

3 participants