Class-typed fields are required when inserting a new document

[oytuntez]

Initial Checks

• I have read and followed the docs and still think this is a bug

Description

I have this dataclass:

class QuoteFile(BaseDoc):
    quote_file_id: int = None
    extracted_data: Optional[ExtractedDocumentData] = None

When indexing a new document, extracted_data is always required no matter what I do (type it with Optional etc). This is especially caused by BaseDocIndex._get_values_by_column().

I have finally forked docarray, will send a PR for various minor fixes like this one:

You can see my changes here, the highlighted line returns None when the field object is optional and not given for a new document during indexing.

Example Code

No response

Python, DocArray & OS Version

Python 3.8.12
docarray==0.40.0

Affected Components

• Vector Database / Index
• Representing
• Sending
• storing
• multi modal data type

[JoanFM]

Hey @oytuntez ,

Can you please provide a code snippet of the code that showcases the problem, this turns out to be extremely helpful

[JoanFM]

Thanks a lot, we are looking forward the PRs from you.

[ai-naymul]

Hey @oytuntez ,

Can you please provide a code snippet of the code that showcases the problem, this turns out to be extremely helpful

Hi @oytuntez,

Do you have a reproducible code to find out the main problem, that would be great to solve it asap ;)

Looking forward