test: crud api

[florian-hoenicke]

Besides the tests, this pr contains fixes for the REST API as well.

• changed from starlette.responses import StreamingResponse to from fastapi.responses import StreamingResponse
• added 'text', 'embedding' and 'buffer' to the pydantic model.
• remove none values from the requests

[codecov]

Codecov Report

Merging #2104 (7d6f6d0) into master (f0b6a44) will increase coverage by 0.42%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master    #2104      +/-   ##
==========================================
+ Coverage   89.90%   90.33%   +0.42%     
==========================================
  Files         211      211              
  Lines       11209    11223      +14     
==========================================
+ Hits        10078    10138      +60     
+ Misses       1131     1085      -46     

Flag	Coverage Δ
daemon	50.41% <25.92%> (-0.05%)	⬇️
jina	90.82% <88.88%> (+0.44%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
jina/clients/sugary_io.py	92.10% <76.92%> (ø)
jina/peapods/runtimes/asyncio/rest/app.py	94.07% <100.00%> (+16.38%)	⬆️
jina/peapods/runtimes/asyncio/rest/models.py	100.00% <100.00%> (ø)
jina/executors/encoders/frameworks.py	63.33% <0.00%> (+8.33%)	⬆️
jina/executors/encoders/numeric/__init__.py	100.00% <0.00%> (+57.57%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f0b6a44...90e750b. Read the comment docs.

[github-actions]

Latency summary

Current PR yields:

• 😶 index QPS at 1125, delta to last 3 avg.: -3%
• 😶 query QPS at 19, delta to last 3 avg.: +0%

Breakdown

Version	Index QPS	Query QPS
current	1125	19
1.0.8	1202	19
1.0.7	1117	18

Backed by latency-tracking. Further commits will update this comment.

[nan-wang]

BTW, I just found executors.requests.CompoundIndexer.yml is out-dated. The UpdateRequest part is wrong.

[JoanFM]

Hey @florian-hoenicke,

would you provide some information and context related to the changes, since according to the title it seems to be just a test, but some changes are seen

[florian-hoenicke]

Hey @florian-hoenicke,

would you provide some information and context related to the changes, since according to the title it seems to be just a test, but some changes are seen

Yes, you are right. I added the required information to the description

[florian-hoenicke]

BTW, I just found executors.requests.CompoundIndexer.yml is out-dated. The UpdateRequest part is wrong.

I had a look and could not find what would be the issue with the UpdateRequest.
We use the VectorIndexDriver for updates when setting mothod: update. Is that what you mean? In that case it is as expected.

on:
UpdateRequest:
- !VectorIndexDriver
with:
method: update
executor: vecidx
traversal_paths: [ 'r' ]
- !KVIndexDriver
with:
method: update
executor: docidx
traversal_paths: [ 'r' ]

[nan-wang]

BTW, I just found executors.requests.CompoundIndexer.yml is out-dated. The UpdateRequest part is wrong.

I had a look and could not find what would be the issue with the UpdateRequest.
We use the VectorIndexDriver for updates when setting mothod: update. Is that what you mean? In that case it is as expected.

on:
UpdateRequest:

• !VectorIndexDriver
  with:
  method: update
  executor: vecidx
  traversal_paths: [ 'r' ]
• !KVIndexDriver
  with:
  method: update
  executor: docidx
  traversal_paths: [ 'r' ]

Yes, this is what I mean

[deepankarm]

@florian-hoenicke I have fixed the issues with the response and the pydantic models in #2121. We can focus only on tests in this PR.

[florian-hoenicke]

@deepankarm really cool. Let's merge your pr first.

[nan-wang]

LGTM👍

[florian-hoenicke]

I fixed an issue with the oneof type in the protobuf definition.
The problem was that always all optional fields were initialized in the pydantic model.
Providing more than one of the optional fields in the document creation makes it fail.
As discussed with @deepankarm, there are feature requests to add the support for a more elegant solution.
pydantic/pydantic#656
pydantic/pydantic#619

Validating the request to check that the client obeys the oneof rules should be done in a separate ticket.
#2132