Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: refactor lib/module document #136

Merged
merged 3 commits into from May 14, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
8 changes: 1 addition & 7 deletions springql-core/src/expression.rs
@@ -1,12 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! Expression has two forms:
//!
//! 1. Value expression, which is evaluated into an SqlValue from a row.
//! 2. Aggregate expression, which is evaluated into an SqlValue from set of rows.
//!
//! Since SQL parser cannot distinguish column reference and value expression,
//! `ValueExprOrAlias` is used for value expressions excluding select_list.
#![doc = include_str!("expression/module.md")]
kazuk marked this conversation as resolved.
Show resolved Hide resolved

pub(crate) mod boolean_expression;
pub(crate) mod function_call;
Expand Down
9 changes: 9 additions & 0 deletions springql-core/src/expression/module.md
@@ -0,0 +1,9 @@
# module expression
kazuk marked this conversation as resolved.
Show resolved Hide resolved

Expression has two forms:

1. Value expression, which is evaluated into an SqlValue from a row.
2. Aggregate expression, which is evaluated into an SqlValue from set of rows.

Since SQL parser cannot distinguish column reference and value expression,
`ValueExprOrAlias` is used for value expressions excluding select_list.
7 changes: 7 additions & 0 deletions springql-core/src/lib.md
@@ -0,0 +1,7 @@
# crate springql-core

SpringQL-core implementation.

## High-level architecture diagram

![High-level architecture diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/springql-architecture.svg)
7 changes: 1 addition & 6 deletions springql-core/src/lib.rs
@@ -1,11 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! SpringQL-core implementation.
//!
//! # High-level architecture diagram
//!
//! ![High-level architecture diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/springql-architecture.svg)

#![doc = include_str!("lib.md")]
#![deny(missing_debug_implementations, missing_docs)]

#[macro_use]
Expand Down
13 changes: 1 addition & 12 deletions springql-core/src/pipeline.rs
@@ -1,17 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! # Pipeline concept diagram
//!
//! Omit _Task Graph_ here. A pipeline has:
//!
//! - Stream
//! - (native) Stream
//! - Source/Sink Stream
//! - Source Reader
//! - Sink Writer
//! - Pump
//!
//! ![Pipeline concept diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/pipeline-and-task-graph.svg)
#![doc = include_str!("pipeline/module.md")]

pub(crate) mod field;
pub(crate) mod name;
Expand Down
12 changes: 12 additions & 0 deletions springql-core/src/pipeline/module.md
@@ -0,0 +1,12 @@
# Pipeline concept diagram

Omit _Task Graph_ here. A pipeline has:

- Stream
- (native) Stream
- Source/Sink Stream
- Source Reader
- Sink Writer
- Pump

![Pipeline concept diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/pipeline-and-task-graph.svg)
26 changes: 1 addition & 25 deletions springql-core/src/stream_engine.rs
@@ -1,30 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! Stream Engine component.
//!
//! Responsible for pipeline management and execution.
//!
//! Stream engine has 2 executors:
//!
//! 1. SQL executor
//! 2. Autonomous executor
//!
//! SQL executor receives commands from user interface to quickly change some status of a pipeline.
//! It does not deal with stream data (Row, to be precise).
//!
//! Autonomous executor deals with stream data.
//!
//! Both SQL executor and autonomous executor instance run at a main thread, while autonomous executor has workers which run at different worker threads.
//!
//! # Entities inside Stream Engine
//!
//! ![Entities inside Stream Engine](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-entity.svg)
//!
//! # Communication between entities
//!
//! Workers in AutonomousExecutor interact via EventQueue (Choreography-based Saga pattern).
//!
//! ![Communication between entities](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-communication.svg)
#![doc = include_str!("stream_engine/module.md")]

pub use autonomous_executor::SpringValue;

Expand Down
27 changes: 27 additions & 0 deletions springql-core/src/stream_engine/module.md
@@ -0,0 +1,27 @@
# Module stream_engine

Stream Engine component.

Responsible for pipeline management and execution.

Stream engine has 2 executors:

1. SQL executor
2. Autonomous executor

SQL executor receives commands from user interface to quickly change some status of a pipeline.
It does not deal with stream data (Row, to be precise).

Autonomous executor deals with stream data.

Both SQL executor and autonomous executor instance run at a main thread, while autonomous executor has workers which run at different worker threads.

## Entities inside Stream Engine

![Entities inside Stream Engine](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-entity.svg)

## Communication between entities

Workers in AutonomousExecutor interact via EventQueue (Choreography-based Saga pattern).

![Communication between entities](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-communication.svg)