Skip to content

Commit

Permalink
Users/akataria/non streaming order by (#40156)
Browse files Browse the repository at this point in the history
* Initial changes

* Initial changes

* [Cosmos][VectorIndex]Adding changes for vectorIndex and vectorEmbeddingPolicy (#40004)

* Adding changes for vectorIndex and vectorEmbeddingPolicy

* Adding some necessary comments

* Adding test case

* updating enum values

* Updating test case

* Updating test case

* Updating test case

* updating changelog

* Updating test case

* Resolving comments

* Resolving comments

* Fixing test case

* Resolving comments

* Resolving Comments

* Fixing build issues

* Resolving comments

* Resolving Comments

* Initial changes

* Initial changes

* Initial changes

* Resolving comments

* Fixing build issues

* [Cosmos][VectorSearch] Non Streaming Order By Query (#40085)

* Increment versions for core releases (#40003)

Increment package versions for core releases

* Ensure ServiceBus session idle timeout fall back to retry-options::try-timeout (#39994)

* Added Alpha3 Java Media Streaming Events (#40002)

* Added Alpha3 Java Media Streaming Events

* updating readme to add the media streaming events to remove model

---------

Co-authored-by: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>

* Update version of github-event-processor to 1.0.0-dev.20240502.2 (#40012)

Co-authored-by: James Suplizio <jasupliz@microsoft.com>

* Prepare May 2024 Identity Release (#40006)

* Prepare Identity Broker May 2024 Release (#40014)

* Increment package versions for identity releases (#40015)

* [JobRouter] SDK Review updates (#40011)

* SDK Review updates

* Update auto-generated models

* Add customization

* Fix customization

* Update package

* Update tests

* Linting

* FixFaultInjectionRuleFailedToApplyPerPartitionInGatewayMode (#40005)

* fix fault injection rule failed to apply per partition in gateway mode

---------

Co-authored-by: annie-mac <xinlian@microsoft.com>

* azure-cosmos-test_1.0.0.beta.7Release (#40021)

* release azure-cosmos-test 1.0.0.beta.7
---------

Co-authored-by: annie-mac <xinlian@microsoft.com>

---------

Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com>
Co-authored-by: v-durgeshs <146056835+v-durgeshs@users.noreply.github.com>
Co-authored-by: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
Co-authored-by: Bill Wert <billwert@microsoft.com>
Co-authored-by: williamzhao87 <williamzhao87@users.noreply.github.com>
Co-authored-by: Annie Liang <64233642+xinlian12@users.noreply.github.com>
Co-authored-by: annie-mac <xinlian@microsoft.com>

* [Cosmos][VectorSearch] Non Streaming Order By Query (#40096)

* Initial changes

* Initial changes

* Increment versions for core releases (#40003)

Increment package versions for core releases

* Ensure ServiceBus session idle timeout fall back to retry-options::try-timeout (#39994)

* Added Alpha3 Java Media Streaming Events (#40002)

* Added Alpha3 Java Media Streaming Events

* updating readme to add the media streaming events to remove model

---------

Co-authored-by: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>

* Update version of github-event-processor to 1.0.0-dev.20240502.2 (#40012)

Co-authored-by: James Suplizio <jasupliz@microsoft.com>

* Prepare May 2024 Identity Release (#40006)

* Prepare Identity Broker May 2024 Release (#40014)

* Increment package versions for identity releases (#40015)

* [JobRouter] SDK Review updates (#40011)

* SDK Review updates

* Update auto-generated models

* Add customization

* Fix customization

* Update package

* Update tests

* Linting

* FixFaultInjectionRuleFailedToApplyPerPartitionInGatewayMode (#40005)

* fix fault injection rule failed to apply per partition in gateway mode

---------

Co-authored-by: annie-mac <xinlian@microsoft.com>

* azure-cosmos-test_1.0.0.beta.7Release (#40021)

* release azure-cosmos-test 1.0.0.beta.7
---------

Co-authored-by: annie-mac <xinlian@microsoft.com>

* Fixed existsById API in ReactiveCosmosTemplate (#40022)

* Fixed existsById API in ReactiveCosmosTemplate

* Added changelog

* Initial changes

* Initial changes

* Skip Recorded test and delete Event record until test proxy to work with Event recordings (#40029)

Co-authored-by: Min Woo Lee 🧊 <77083090+minwoolee-ms@users.noreply.github.com>

* Fix invalid CODEOWNERS (#40032)

* Initial changes

* ServiceBus: fix session tracing (#39962)

* remove additional matrix

* Fix session processing and disposition instrumentation

* return matrix config

* review suggestions

* [Automation] Generate SDK based on TypeSpec 0.15.15 (#40048)

* [CODEOWNERS] Updates for org changes (#40049)

* [CODEOWNERS] Updates for org changes

The focus of these changes is to remove an individual who no longer is responsible for the products which their GH account is associated to.

* Move from using the docker image to java2docfx for docs validation (#39744)

* Move from using the docker image to java2docfx for docs validation

* Temporarily turn on docs processing for template libraries for testing

* Actually install the rex validation tool

* Fix the if not Test-Path statement

* Update java2docfx version and add a couple of diagnostics output lines

* Add missing close paren

* Ensure that Sort-Object always returns an array even if there's only one item

* add another piece of diagnostics output

* trying one more thing

* remove some diag, add other

* Remove the additional diagnostics, add permanent output message

* Invoke java -jar on java2docfx to show the help command to ensure the install is okay

* fiddling with the java -jar command

* Set the working directory to the java2docfx directory before executing the mvn dependency download

* Actually create the directory before trying to set location...oops

* Update rex validation to verify MAVEN_HOME is set

* Updates for Java PR 39875 which had changes from this PR that were more immediate

* Update java2docfx version

* remove check for MAVEN_HOME which was only for testing

* Update the version of java2docfx to test a fix

* Update version of java2docfx to 1.0.4

* revert template's ci.yml changes that were only necessary to test java2docfx

* owners (#39686)

* Use ClientLogger in testing output (#40010)

Use ClientLogger in testing output

* Fix null pointer exception and context usage (#40053)

* Rename AML to AzureMachineLearning (#40056)

* Fixed the Key Vault `test-resources.json` file to properly configure a deployment script for certificate creation. (#40037)

* Close response body in bearer policy (#40052)

* Running Prepare-Release for azure-messaging-servicebus 7.17.0 (#40058)

* mgmt, TypeSpec code generation pipeline (#39963)

* typespec generation pipeline

echo command

PR_TITLE

* generation typespec

Update generation.yml for Azure Pipelines

Update generation.yml for Azure Pipelines

Update generation.yml for Azure Pipelines

* remove typespec pipeline file

* fix pr title

* address comments

* Add codeowner linter owners (#39997)

* Update to ESRP task version that supports federated auth (#40059)

* Increment package versions for cosmos releases (#40031)

* Update azure-sdk-build-tools Repository Resource Refs in Yaml files (#39627)

* Add reduced embeddings sample to azure-search-documents (#40069)

* Add reduced embeddings sample

* Fix cspell

* Fix link

* Search May Preview Regen Updates (#40057)

* Search May Preview Regeneration
- Still need to add varargs convenience

* Removing ovveride statements from `setFields` for `VectorizableImageUrlQuery` and `VectorizableImageBinaryQuery`

* Removing ovveride statements from `setFields` for `VectorizableImageUrlQuery` and `VectorizableImageBinaryQuery`

* adding varargs

* Additional adjustments to FieldBuilder and Search Index Customizations

* Updating cspell.json

* Adjust `SearchScoreThreshold` customization
Re-enable code generation in CI

* Updates:

- Updated Cspell
- Rename `maxStoragePerIndex` property to `maxStoragePerIndexInBytes` in SearchServiceLimits
- Set `hybridSearch` property to be type `HybridSearch` in SearchRequest
- Add `hybridSearch` to SearchOptions and `SearchAsyncClient.createSearchRequest()`

* Adding Support and testing byte[] and List<byte> within field builder

* Fix linting

---------

Co-authored-by: alzimmermsft <48699787+alzimmermsft@users.noreply.github.com>

* Preparing Search May 2024 Beta Release (#40071)

* Preparing Search May 2024 Beta Release

* Preparing Search May 2024 Beta Release

* Resolving comments

* Fixing build issues

* eng, update autorest.java, improve error output in sdk automation (#40073)

* improve error output

* autorest.java 4.1.29

* Merge to main after spring cloud azure 4.18.0 released (#40075)

* Prepare for Spring Cloud Azure 4.18.0 release (#40063)

* update version client

* update version/changelog/readme

* update changelog

* Increment versions for spring releases (#40074)

* Increment package versions for spring releases

* Update version_client.txt

* Update pom.xml

---------

Co-authored-by: Muyao Feng <92105726+Netyyyy@users.noreply.github.com>

---------

Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>

* Miscellaneous Core performance improvements (#39552)

Miscellaneous Core performance improvements

* Increment package versions for search releases (#40072)

* Update io.fabric8:kubernetes-client (#40086)

5.12.3 -> 6.12.1

* Increment package versions for servicebus releases (#40094)

* Emit stable auto-instrumented otel metrics (#39960)

* Update otel metrics logic

* add runtime metrics

* adding a few metrics I forgot

* small correction

* Update

* Fix

* Update

* Delete pre-stable metrics

---------

Co-authored-by: Harsimar Kaur (from Dev Box) <harskaur@microsoft.com>

---------

Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com>
Co-authored-by: v-durgeshs <146056835+v-durgeshs@users.noreply.github.com>
Co-authored-by: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
Co-authored-by: Bill Wert <billwert@microsoft.com>
Co-authored-by: williamzhao87 <williamzhao87@users.noreply.github.com>
Co-authored-by: Annie Liang <64233642+xinlian12@users.noreply.github.com>
Co-authored-by: annie-mac <xinlian@microsoft.com>
Co-authored-by: Kushagra Thapar <kuthapar@microsoft.com>
Co-authored-by: minwoolee-msft <77083090+minwoolee-msft@users.noreply.github.com>
Co-authored-by: Min Woo Lee 🧊 <77083090+minwoolee-ms@users.noreply.github.com>
Co-authored-by: Alan Zimmer <48699787+alzimmermsft@users.noreply.github.com>
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
Co-authored-by: Jesse Squire <jsquire@microsoft.com>
Co-authored-by: Harsimar Kaur <skaur21@gmail.com>
Co-authored-by: vcolin7 <vicolina@microsoft.com>
Co-authored-by: Xiaofei Cao <92354331+XiaofeiCao@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
Co-authored-by: Jair Myree <jairmyree@microsoft.com>
Co-authored-by: Weidong Xu <weidxu@microsoft.com>
Co-authored-by: Muyao Feng <92105726+Netyyyy@users.noreply.github.com>
Co-authored-by: Helen <56097766+heyams@users.noreply.github.com>
Co-authored-by: Harsimar Kaur (from Dev Box) <harskaur@microsoft.com>

* Initial changes

* Initial changes

* Fixes

* Fixing some merge issues

* Fixing some merge issues

* Fixing some merge issues

* Resolving comments

---------

Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com>
Co-authored-by: v-durgeshs <146056835+v-durgeshs@users.noreply.github.com>
Co-authored-by: Vinothini Dharmaraj <v-vdharmaraj@microsoft.com>
Co-authored-by: James Suplizio <jasupliz@microsoft.com>
Co-authored-by: Bill Wert <billwert@microsoft.com>
Co-authored-by: williamzhao87 <williamzhao87@users.noreply.github.com>
Co-authored-by: Annie Liang <64233642+xinlian12@users.noreply.github.com>
Co-authored-by: annie-mac <xinlian@microsoft.com>
Co-authored-by: Kushagra Thapar <kuthapar@microsoft.com>
Co-authored-by: minwoolee-msft <77083090+minwoolee-msft@users.noreply.github.com>
Co-authored-by: Min Woo Lee 🧊 <77083090+minwoolee-ms@users.noreply.github.com>
Co-authored-by: Alan Zimmer <48699787+alzimmermsft@users.noreply.github.com>
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
Co-authored-by: Jesse Squire <jsquire@microsoft.com>
Co-authored-by: Harsimar Kaur <skaur21@gmail.com>
Co-authored-by: vcolin7 <vicolina@microsoft.com>
Co-authored-by: Xiaofei Cao <92354331+XiaofeiCao@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Patrick Hallisey <pahallis@microsoft.com>
Co-authored-by: Jair Myree <jairmyree@microsoft.com>
Co-authored-by: Weidong Xu <weidxu@microsoft.com>
Co-authored-by: Muyao Feng <92105726+Netyyyy@users.noreply.github.com>
Co-authored-by: Helen <56097766+heyams@users.noreply.github.com>
Co-authored-by: Harsimar Kaur (from Dev Box) <harskaur@microsoft.com>
  • Loading branch information
1 parent 0f1be0c commit ba2ab5b
Show file tree
Hide file tree
Showing 22 changed files with 74 additions and 911 deletions.
4 changes: 4 additions & 0 deletions sdk/cosmos/azure-cosmos-test/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@

#### Breaking Changes

#### Bugs Fixed

#### Other Changes

### 1.0.0-beta.7 (2024-05-03)

#### Bugs Fixed
Expand Down

This file was deleted.

6 changes: 3 additions & 3 deletions sdk/cosmos/azure-cosmos/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,8 @@
### 4.60.0-beta.1 (Unreleased)

#### Features Added
* Added `cosmosVectorEmbeddingPolicy` in `cosmosContainerProperties` and `vectorIndexes` in `indexPolicy` to support vector search in CosmosDB - See[39379](https://github.com/Azure/azure-sdk-for-java/pull/39379)

* Added support for non-streaming OrderBy query and a query feature `NonStreamingOrderBy` to support Vector Search queries. - See [PR 39897](https://github.com/Azure/azure-sdk-for-java/pull/39897/)
* Added support for non-streaming OrderBy query and a query feature `NonStreamingOrderBy` to support Vector Search queries. - See [PR 39897](https://github.com/Azure/azure-sdk-for-java/pull/39897/)

* Added support for non-streaming OrderBy query and a query feature `NonStreamingOrderBy` to support Vector Search queries. - See [PR 39897](https://github.com/Azure/azure-sdk-for-java/pull/39897/)

Expand All @@ -16,9 +15,10 @@
#### Other Changes

### 4.59.0 (2024-04-27)

#### Features Added
* Added public APIs `getCustomItemSerializer` and `setCustomItemSerializer` to allow customers to specify custom payload transformations or serialization settings. - See [PR 38997](https://github.com/Azure/azure-sdk-for-java/pull/38997) and [PR 39933](https://github.com/Azure/azure-sdk-for-java/pull/39933)

#### Other Changes
* Load Blackbird or Afterburner into the ObjectMapper depending upon Java version and presence of modules in classpath. Make Afterburner and Blackbird optional maven dependencies. See - [PR 39689](https://github.com/Azure/azure-sdk-for-java/pull/39689)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -166,12 +166,12 @@ public class Configs {
public static final String MAX_RETRIES_IN_LOCAL_REGION_WHEN_REMOTE_REGION_PREFERRED = "COSMOS.MAX_RETRIES_IN_LOCAL_REGION_WHEN_REMOTE_REGION_PREFERRED";
private static final int DEFAULT_MAX_RETRIES_IN_LOCAL_REGION_WHEN_REMOTE_REGION_PREFERRED = 1;

private static final String MAX_ITEM_SIZE_FOR_VECTOR_SEARCH = "COSMOS.MAX_ITEM_SIZE_FOR_VECTOR_SEARCH";
private static final int DEFAULT_MAX_ITEM_SIZE_FOR_VECTOR_SEARCH = 50000;
private static final String MAX_ITEM_COUNT_FOR_VECTOR_SEARCH = "COSMOS.MAX_ITEM_SIZE_FOR_VECTOR_SEARCH";
private static final int DEFAULT_MAX_ITEM_COUNT_FOR_VECTOR_SEARCH = 50000;

private static final String MAX_ITEM_SIZE_FOR_VECTOR_SEARCH_ENABLED = "COSMOS.MAX_ITEM_SIZE_FOR_VECTOR_SEARCH_ENABLED";
private static final String AZURE_COSMOS_DISABLE_NON_STREAMING_ORDER_BY = "COSMOS.AZURE_COSMOS_DISABLE_NON_STREAMING_ORDER_BY";

private static final boolean DEFAULT_MAX_ITEM_SIZE_FOR_VECTOR_SEARCH_ENABLED = true;
private static final boolean DEFAULT_AZURE_COSMOS_DISABLE_NON_STREAMING_ORDER_BY = false;

public static final int MIN_MAX_RETRIES_IN_LOCAL_REGION_WHEN_REMOTE_REGION_PREFERRED = 1;

Expand Down Expand Up @@ -491,12 +491,18 @@ public static int getMaxRetriesInLocalRegionWhenRemoteRegionPreferred() {
MIN_MAX_RETRIES_IN_LOCAL_REGION_WHEN_REMOTE_REGION_PREFERRED);
}

public static int getMaxItemSizeForVectorSearch() {
return getJVMConfigAsInt(MAX_ITEM_SIZE_FOR_VECTOR_SEARCH, DEFAULT_MAX_ITEM_SIZE_FOR_VECTOR_SEARCH);
public static int getMaxItemCountForVectorSearch() {
return Integer.parseInt(System.getProperty(MAX_ITEM_COUNT_FOR_VECTOR_SEARCH,
firstNonNull(
emptyToNull(System.getenv().get(MAX_ITEM_COUNT_FOR_VECTOR_SEARCH)),
String.valueOf(DEFAULT_MAX_ITEM_COUNT_FOR_VECTOR_SEARCH))));
}

public static boolean getMaxItemSizeForVectorSearchEnabled() {
return getJVMConfigAsBoolean(MAX_ITEM_SIZE_FOR_VECTOR_SEARCH_ENABLED, DEFAULT_MAX_ITEM_SIZE_FOR_VECTOR_SEARCH_ENABLED);
public static boolean getAzureCosmosNonStreamingOrderByDisabled() {
return Boolean.parseBoolean(System.getProperty(AZURE_COSMOS_DISABLE_NON_STREAMING_ORDER_BY,
firstNonNull(
emptyToNull(System.getenv().get(AZURE_COSMOS_DISABLE_NON_STREAMING_ORDER_BY)),
String.valueOf(DEFAULT_AZURE_COSMOS_DISABLE_NON_STREAMING_ORDER_BY))));
}

public static Duration getMinRetryTimeInLocalRegionWhenRemoteRegionPreferred() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -120,15 +120,6 @@ public static final class Properties {
public static final String SPATIAL_INDEXES = "spatialIndexes";
public static final String TYPES = "types";

// Vector Embedding Policy
public static final String VECTOR_EMBEDDING_POLICY = "vectorEmbeddingPolicy";
public static final String VECTOR_INDEXES = "vectorIndexes";
public static final String VECTOR_EMBEDDINGS = "vectorEmbeddings";
public static final String VECTOR_INDEX_TYPE = "type";
public static final String VECTOR_DATA_TYPE = "dataType";
public static final String VECTOR_DIMENSIONS = "dimensions";
public static final String DISTANCE_FUNCTION = "distanceFunction";

// Unique index.
public static final String UNIQUE_KEY_POLICY = "uniqueKeyPolicy";
public static final String UNIQUE_KEYS = "uniqueKeys";
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ public final class CosmosQueryRequestOptionsImpl extends CosmosQueryRequestOptio
private boolean queryPlanRetrievalDisallowed;
private boolean emptyPageDiagnosticsEnabled;
private String queryName;
private Integer maxItemSizeForVectorSearch;
private Integer maxItemCountForVectorSearch;
private List<CosmosDiagnostics> cancelledRequestDiagnosticsTracker = new ArrayList<>();

/**
Expand Down Expand Up @@ -63,7 +63,7 @@ public CosmosQueryRequestOptionsImpl(CosmosQueryRequestOptionsImpl options) {
this.queryName = options.queryName;
this.feedRange = options.feedRange;
this.cancelledRequestDiagnosticsTracker = options.cancelledRequestDiagnosticsTracker;
this.maxItemSizeForVectorSearch = options.maxItemSizeForVectorSearch;
this.maxItemCountForVectorSearch = options.maxItemCountForVectorSearch;
}

/**
Expand Down Expand Up @@ -203,18 +203,18 @@ public CosmosQueryRequestOptionsImpl setMaxItemCount(Integer maxItemCount) {
*
* @return the max number of items for vector search.
*/
public Integer getMaxItemSizeForVectorSearch() {
return this.maxItemSizeForVectorSearch;
public Integer getMaxItemCountForVectorSearch() {
return this.maxItemCountForVectorSearch;
}

/**
* Sets the maximum item size to fetch during non-streaming order by queries.
*
* @param maxItemSizeForVectorSearch the max number of items for vector search.
* @param maxItemCountForVectorSearch the max number of items for vector search.
* return the CosmosQueryRequestOptions.
*/
public CosmosQueryRequestOptionsImpl setMaxItemSizeForVectorSearch(Integer maxItemSizeForVectorSearch) {
this.maxItemSizeForVectorSearch = maxItemSizeForVectorSearch;
public CosmosQueryRequestOptionsImpl setMaxItemCountForVectorSearch(Integer maxItemCountForVectorSearch) {
this.maxItemCountForVectorSearch = maxItemCountForVectorSearch;
return this;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,10 @@
import com.azure.cosmos.CosmosItemSerializer;
import com.azure.cosmos.implementation.apachecommons.lang.StringUtils;
import com.azure.cosmos.implementation.caches.SerializableWrapper;
import com.azure.cosmos.models.ChangeFeedPolicy;
import com.azure.cosmos.models.ClientEncryptionPolicy;
import com.azure.cosmos.models.ChangeFeedPolicy;
import com.azure.cosmos.models.ComputedProperty;
import com.azure.cosmos.models.ConflictResolutionPolicy;
import com.azure.cosmos.models.CosmosVectorEmbeddingPolicy;
import com.azure.cosmos.models.IndexingPolicy;
import com.azure.cosmos.models.ModelBridgeInternal;
import com.azure.cosmos.models.PartitionKeyDefinition;
Expand All @@ -25,8 +24,6 @@
import java.util.Collection;
import java.util.Collections;

import static com.azure.cosmos.implementation.guava25.base.Preconditions.checkNotNull;

/**
* Represents a document collection in the Azure Cosmos DB database service. A collection is a named logical container
* for documents.
Expand All @@ -43,7 +40,6 @@ public final class DocumentCollection extends Resource {
private UniqueKeyPolicy uniqueKeyPolicy;
private PartitionKeyDefinition partitionKeyDefinition;
private ClientEncryptionPolicy clientEncryptionPolicyInternal;
private CosmosVectorEmbeddingPolicy cosmosVectorEmbeddingPolicy;

/**
* Constructor.
Expand Down Expand Up @@ -414,33 +410,6 @@ public void setClientEncryptionPolicy(ClientEncryptionPolicy value) {
this.set(Constants.Properties.CLIENT_ENCRYPTION_POLICY, value, CosmosItemSerializer.DEFAULT_SERIALIZER);
}

/**
* Gets the Vector Embedding Policy containing paths for embeddings along with path-specific settings for the item
* used in performing vector search on the items in a collection in the Azure CosmosDB database service.
*
* @return the Vector Embedding Policy.
*/
public CosmosVectorEmbeddingPolicy getVectorEmbeddingPolicy() {
if (this.cosmosVectorEmbeddingPolicy == null) {
if (super.has(Constants.Properties.VECTOR_EMBEDDING_POLICY)) {
this.cosmosVectorEmbeddingPolicy = super.getObject(Constants.Properties.VECTOR_EMBEDDING_POLICY,
CosmosVectorEmbeddingPolicy.class);
}
}
return this.cosmosVectorEmbeddingPolicy;
}

/**
* Sets the Vector Embedding Policy containing paths for embeddings along with path-specific settings for the item
* used in performing vector search on the items in a collection in the Azure CosmosDB database service.
*
* @param value the Vector Embedding Policy.
*/
public void setVectorEmbeddingPolicy(CosmosVectorEmbeddingPolicy value) {
checkNotNull(value, "cosmosVectorEmbeddingPolicy cannot be null");
this.set(Constants.Properties.VECTOR_EMBEDDING_POLICY, value, CosmosItemSerializer.DEFAULT_SERIALIZER);
}

public void populatePropertyBag() {
super.populatePropertyBag();
if (this.indexingPolicy == null) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -281,6 +281,8 @@ void setCancelledRequestDiagnosticsTracker(
Integer getMaxItemCount(CosmosQueryRequestOptions options);

String getRequestContinuation(CosmosQueryRequestOptions options);

Integer getMaxItemCountForVectorSearch(CosmosQueryRequestOptions options);
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,12 @@
*/
public class DocumentQueryExecutionContextFactory {

private static final ImplementationBridgeHelpers
.CosmosQueryRequestOptionsHelper
.CosmosQueryRequestOptionsAccessor qryOptAccessor = ImplementationBridgeHelpers
.CosmosQueryRequestOptionsHelper
.getCosmosQueryRequestOptionsAccessor();

private final static int PageSizeFactorForTop = 5;
private static final Logger logger = LoggerFactory.getLogger(DocumentQueryExecutionContextFactory.class);
private static Mono<Utils.ValueHolder<DocumentCollection>> resolveCollection(DiagnosticsClientContext diagnosticsClientContext,
Expand Down Expand Up @@ -362,7 +368,7 @@ public static <T> Flux<? extends IDocumentQueryExecutionContext<T>> createSpecia
boolean getLazyFeedResponse = queryInfo.hasTop();

// We need to compute the optimal initial age size for non-streaming order-by queries
if (queryInfo.hasNonStreamingOrderBy() && Configs.getMaxItemSizeForVectorSearchEnabled()) {
if (queryInfo.hasNonStreamingOrderBy()) {
// Validate the TOP or LIMIT for non-streaming order-by queries
if (!queryInfo.hasTop() && !queryInfo.hasLimit() && queryInfo.getTop() < 0 && queryInfo.getLimit() < 0) {
throw new NonStreamingOrderByBadRequestException(HttpConstants.StatusCodes.BADREQUEST,
Expand All @@ -373,8 +379,8 @@ public static <T> Flux<? extends IDocumentQueryExecutionContext<T>> createSpecia
// Validate the size of TOP or LIMIT against MaxItemSizeForVectorSearch
int maxLimit = Math.max(queryInfo.hasTop() ? queryInfo.getTop() : 0,
queryInfo.hasLimit() ? queryInfo.getLimit() : 0);
int maxItemSizeForVectorSearch = Math.max(Configs.getMaxItemSizeForVectorSearch(),
ModelBridgeInternal.getMaxItemSizeForVectorSearchFromQueryRequestOptions(cosmosQueryRequestOptions));
int maxItemSizeForVectorSearch = Math.max(Configs.getMaxItemCountForVectorSearch(),
qryOptAccessor.getMaxItemCountForVectorSearch(cosmosQueryRequestOptions));
if (maxLimit > maxItemSizeForVectorSearch) {
throw new NonStreamingOrderByBadRequestException(HttpConstants.StatusCodes.BADREQUEST,
"Executing a vector search query with TOP or LIMIT larger than the maxItemSizeForVectorSearch " +
Expand All @@ -383,7 +389,8 @@ public static <T> Flux<? extends IDocumentQueryExecutionContext<T>> createSpecia
// Set initialPageSize based on the smallest of TOP or LIMIT
if (queryInfo.hasTop() || queryInfo.hasLimit()) {
int pageSizeWithTopOrLimit = Math.min(queryInfo.hasTop() ? queryInfo.getTop() : Integer.MAX_VALUE,
queryInfo.hasLimit() ? queryInfo.getLimit() : Integer.MAX_VALUE);
queryInfo.hasLimit() && queryInfo.hasOffset() ?
queryInfo.getLimit() + queryInfo.getOffset() : Integer.MAX_VALUE);
if (initialPageSize > 0) {
initialPageSize = Math.min(pageSizeWithTopOrLimit, initialPageSize);
} else {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import com.azure.cosmos.BridgeInternal;
import com.azure.cosmos.CosmosEndToEndOperationLatencyPolicyConfig;
import com.azure.cosmos.implementation.Configs;
import com.azure.cosmos.implementation.DiagnosticsClientContext;
import com.azure.cosmos.implementation.ImplementationBridgeHelpers;
import com.azure.cosmos.implementation.routing.PartitionKeyInternal;
Expand Down Expand Up @@ -46,6 +47,18 @@ class QueryPlanRetriever {
QueryFeature.NonValueAggregate.name() + ", " +
QueryFeature.NonStreamingOrderBy.name();

private static final String OLD_SUPPORTED_QUERY_FEATURES = QueryFeature.Aggregate.name() + ", " +
QueryFeature.CompositeAggregate.name() + ", " +
QueryFeature.MultipleOrderBy.name() + ", " +
QueryFeature.MultipleAggregates.name() + ", " +
QueryFeature.OrderBy.name() + ", " +
QueryFeature.OffsetAndLimit.name() + ", " +
QueryFeature.Distinct.name() + ", " +
QueryFeature.GroupBy.name() + ", " +
QueryFeature.Top.name() + ", " +
QueryFeature.DCount.name() + ", " +
QueryFeature.NonValueAggregate.name();

static Mono<PartitionedQueryExecutionInfo> getQueryPlanThroughGatewayAsync(DiagnosticsClientContext diagnosticsClientContext,
IDocumentQueryClient queryClient,
SqlQuerySpec sqlQuerySpec,
Expand All @@ -62,7 +75,8 @@ static Mono<PartitionedQueryExecutionInfo> getQueryPlanThroughGatewayAsync(Diagn
final Map<String, String> requestHeaders = new HashMap<>();
requestHeaders.put(HttpConstants.HttpHeaders.CONTENT_TYPE, RuntimeConstants.MediaTypes.JSON);
requestHeaders.put(HttpConstants.HttpHeaders.IS_QUERY_PLAN_REQUEST, TRUE);
requestHeaders.put(HttpConstants.HttpHeaders.SUPPORTED_QUERY_FEATURES, SUPPORTED_QUERY_FEATURES);
requestHeaders.put(HttpConstants.HttpHeaders.SUPPORTED_QUERY_FEATURES,
Configs.getAzureCosmosNonStreamingOrderByDisabled() ? OLD_SUPPORTED_QUERY_FEATURES : SUPPORTED_QUERY_FEATURES);
requestHeaders.put(HttpConstants.HttpHeaders.QUERY_VERSION, HttpConstants.Versions.QUERY_VERSION);

if (partitionKey != null && partitionKey != PartitionKey.NONE) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ public CompositePathSortOrder getOrder() {
}

/**
* Sets the sort order for the composite path.
* Gets the sort order for the composite path.
* <p>
* For example if you want to run the query "SELECT * FROM c ORDER BY c.age asc, c.height desc",
* then you need to make the order for "/age" "ascending" and the order for "/height" "descending".
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -347,28 +347,6 @@ public CosmosContainerProperties setClientEncryptionPolicy(ClientEncryptionPolic
return this;
}

/**
* Gets the Vector Embedding Policy containing paths for embeddings along with path-specific settings for the item
* used in performing vector search on the items in a collection in the Azure CosmosDB database service.
*
* @return the Vector Embedding Policy.
*/
public CosmosVectorEmbeddingPolicy getVectorEmbeddingPolicy() {
return this.documentCollection.getVectorEmbeddingPolicy();
}

/**
* Sets the Vector Embedding Policy containing paths for embeddings along with path-specific settings for the item
* used in performing vector search on the items in a collection in the Azure CosmosDB database service.
*
* @param value the Vector Embedding Policy.
* @return the CosmosContainerProperties.
*/
public CosmosContainerProperties setVectorEmbeddingPolicy(CosmosVectorEmbeddingPolicy value) {
this.documentCollection.setVectorEmbeddingPolicy(value);
return this;
}

Resource getResource() {
return this.documentCollection;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -261,18 +261,18 @@ CosmosQueryRequestOptions setMaxItemCount(Integer maxItemCount) {
*
* @return the max number of items for vector search.
*/
public Integer getMaxItemSizeForVectorSearch() {
return this.actualRequestOptions.getMaxItemSizeForVectorSearch();
Integer getMaxItemCountForVectorSearch() {
return this.actualRequestOptions.getMaxItemCountForVectorSearch();
}

/**
* Sets the maximum item size to fetch during non-streaming order by queries.
*
* @param maxItemSizeForVectorSearch the max number of items for vector search.
* @param maxItemCountForVectorSearch the max number of items for vector search.
* @return the CosmosQueryRequestOptions.
*/
public CosmosQueryRequestOptions setMaxItemSizeForVectorSearch(Integer maxItemSizeForVectorSearch) {
this.actualRequestOptions.setMaxItemSizeForVectorSearch(maxItemSizeForVectorSearch);
CosmosQueryRequestOptions setMaxItemCountForVectorSearch(Integer maxItemCountForVectorSearch) {
this.actualRequestOptions.setMaxItemCountForVectorSearch(maxItemCountForVectorSearch);
return this;
}

Expand Down Expand Up @@ -623,6 +623,11 @@ public Integer getMaxItemCount(CosmosQueryRequestOptions options) {
public String getRequestContinuation(CosmosQueryRequestOptions options) {
return options.getRequestContinuation();
}

@Override
public Integer getMaxItemCountForVectorSearch(CosmosQueryRequestOptions options) {
return options.getMaxItemCountForVectorSearch();
}
});
}

Expand Down

0 comments on commit ba2ab5b

Please sign in to comment.