[Enhancement]: Improve Vcpkg Installation way

[samhuang-z]

Is there an existing issue for this?

• I have searched the existing issues

Background

Milvus, a highly flexible and scalable vector database, currently utilizes CMake and Conan for managing most of its dependencies. Additionally, Milvus relies on specific libraries such as azure-identity-cpp, azure-storage-blobs-cpp, and gtest. The optimal method to install these libraries is through Vcpkg, a versatile C++ library manager that supports multiple platforms.

Motivation

The primary motivation for this proposal is to enhance the isolation between different dependency management systems used within the Milvus project. By improving dependency management, we can ensure more reliable builds and clearer separation of concerns within the project infrastructure. Utilizing Docker’s multi-stage build feature offers a robust solution to achieve this isolation.

Implementation Plan

Stage 1: azure-storage Dependency Installation

• Objective: Isolate dependency management using Vcpkg within a dedicated Docker build stage.
• Actions:
  1. Create a Docker stage specifically for managing dependencies with Vcpkg.
  2. Install azure-identity-cpp, azure-storage-blobs-cpp, and gtest using Vcpkg in this stage.
  3. Compile these libraries and ensure all necessary binaries, libraries, and headers are ready for the next stage.

Stage 2: Main Dependency Installation

• Actions:

1. install Main Dependency
2. Copy the artifacts (libraries and headers) generated in Stage 1 for the further use.

Additional Benefits

Enhanced Build Efficiency

• Caching Mechanisms: Docker's multi-stage build process is friendly to Docker’s build cache. If there are no changes to the dependency versions, there is no need to re-download or recompile them, significantly speeding up the build process.
• Build Time Reduction: By caching earlier stages and only rebuilding them when necessary, we reduce the overall time spent on builds during continuous integration (CI) cycles.

Improved Maintainability

• Clear Separation: Separating dependency management into different stages enhances the maintainability and scalability of the build system, making it easier to update and manage over time.
• Version Control: Managing dependencies in isolated stages allows for more precise control over version updates and dependency configurations.

Conclusion

Implementing Docker multi-stage builds in Milvus provides a more isolated and efficient way to manage project dependencies. This approach not only optimizes the build process but also improves the maintainability and scalability of the project infrastructure.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.