Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor the database metadata to use Protobuf as the serialization protocol #3935

Open
WenyXu opened this issue May 14, 2024 · 0 comments
Open
Milestone

Comments

@WenyXu
Copy link
Member

WenyXu commented May 14, 2024

What type of enhancement is this?

Refactor, Tech debt reduction

What does the enhancement do?

Currently, our database metadata have following problems:
High level:

  • The JSON serialization protocol is not optimal for efficiently storing Vec<u8> data.
  • Poor backward compatibility.

Detail:

  • TableRoute redundantly stores Peer information multiple times.
  • ColumnSchema does not store the SemanticType.

BTW, during the design of the metadata, we should also:

  • Consider how to effectively retrieve and cache the metadata.
  • Treat structured data as first-class citizens (i.g., Avoid the use of Vec<u8> and HashMap<String, String> as possible).

Implementation challenges

No response

@WenyXu WenyXu added this to the v0.9 milestone May 14, 2024
@killme2008 killme2008 mentioned this issue May 14, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant