Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: refactor lib/module document #136

Merged
merged 3 commits into from May 14, 2022
Merged

docs: refactor lib/module document #136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
608c478
docs: refactor lib/module document
kazuk May 11, 2022
9da84d2
move module document to same as code
kazuk May 14, 2022
b3a6ec5
add to CHANGELOG
kazuk May 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Expand Up @@ -4,8 +4,13 @@ All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog][Keep a Changelog] and this project adheres to [Semantic Versioning][Semantic Versioning].

<!-- markdownlint-disable MD024 -->
## [Unreleased]

### Changed

- Refactor : crate/module document move to pure Markdown ([#136](https://github.com/SpringQL/SpringQL/pull/136))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nits] I interpreted from "keep a changelog" and semantic versioning that Changed sections include "breaking changes", which bump up major versions.

But "keep a changelog" is a human readable format (not a machine readable one) and you noted Refactor:, it should be OK to add this line in CHANGELOG.


## [v0.9.0]

### Added
Expand Down
9 changes: 9 additions & 0 deletions springql-core/src/expression.md
View file
Open in desktop
@@ -0,0 +1,9 @@
<!-- markdownlint-disable MD041 -->

Expression has two forms:

1. Value expression, which is evaluated into an SqlValue from a row.
2. Aggregate expression, which is evaluated into an SqlValue from set of rows.

Since SQL parser cannot distinguish column reference and value expression,
`ValueExprOrAlias` is used for value expressions excluding select_list.
8 changes: 1 addition & 7 deletions springql-core/src/expression.rs
View file
Open in desktop
@@ -1,12 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! Expression has two forms:
//!
//! 1. Value expression, which is evaluated into an SqlValue from a row.
//! 2. Aggregate expression, which is evaluated into an SqlValue from set of rows.
//!
//! Since SQL parser cannot distinguish column reference and value expression,
//! `ValueExprOrAlias` is used for value expressions excluding select_list.
#![doc = include_str!("expression.md")]

pub(crate) mod boolean_expression;
pub(crate) mod function_call;
Expand Down
6 changes: 6 additions & 0 deletions springql-core/src/lib.md
View file
Open in desktop
@@ -0,0 +1,6 @@
<!-- markdownlint-disable MD041 -->
SpringQL-core implementation.

## High-level architecture diagram

![High-level architecture diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/springql-architecture.svg)
7 changes: 1 addition & 6 deletions springql-core/src/lib.rs
View file
Open in desktop
@@ -1,11 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! SpringQL-core implementation.
//!
//! # High-level architecture diagram
//!
//! ![High-level architecture diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/springql-architecture.svg)

#![doc = include_str!("lib.md")]
#![deny(missing_debug_implementations, missing_docs)]

#[macro_use]
Expand Down
12 changes: 12 additions & 0 deletions springql-core/src/pipeline.md
View file
Open in desktop
@@ -0,0 +1,12 @@
# Pipeline concept diagram

Omit _Task Graph_ here. A pipeline has:

- Stream
- (native) Stream
- Source/Sink Stream
- Source Reader
- Sink Writer
- Pump

![Pipeline concept diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/pipeline-and-task-graph.svg)
13 changes: 1 addition & 12 deletions springql-core/src/pipeline.rs
View file
Open in desktop
@@ -1,17 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! # Pipeline concept diagram
//!
//! Omit _Task Graph_ here. A pipeline has:
//!
//! - Stream
//! - (native) Stream
//! - Source/Sink Stream
//! - Source Reader
//! - Sink Writer
//! - Pump
//!
//! ![Pipeline concept diagram](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/pipeline-and-task-graph.svg)
#![doc = include_str!("pipeline.md")]

pub(crate) mod field;
pub(crate) mod name;
Expand Down
27 changes: 27 additions & 0 deletions springql-core/src/stream_engine.md
View file
Open in desktop
@@ -0,0 +1,27 @@
<!-- markdownlint-disable MD041 -->

Stream Engine component.

Responsible for pipeline management and execution.

Stream engine has 2 executors:

1. SQL executor
2. Autonomous executor

SQL executor receives commands from user interface to quickly change some status of a pipeline.
It does not deal with stream data (Row, to be precise).

Autonomous executor deals with stream data.

Both SQL executor and autonomous executor instance run at a main thread, while autonomous executor has workers which run at different worker threads.

## Entities inside Stream Engine

![Entities inside Stream Engine](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-entity.svg)

## Communication between entities

Workers in AutonomousExecutor interact via EventQueue (Choreography-based Saga pattern).

![Communication between entities](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-communication.svg)
26 changes: 1 addition & 25 deletions springql-core/src/stream_engine.rs
View file
Open in desktop
@@ -1,30 +1,6 @@
// This file is part of https://github.com/SpringQL/SpringQL which is licensed under MIT OR Apache-2.0. See file LICENSE-MIT or LICENSE-APACHE for full license details.

//! Stream Engine component.
//!
//! Responsible for pipeline management and execution.
//!
//! Stream engine has 2 executors:
//!
//! 1. SQL executor
//! 2. Autonomous executor
//!
//! SQL executor receives commands from user interface to quickly change some status of a pipeline.
//! It does not deal with stream data (Row, to be precise).
//!
//! Autonomous executor deals with stream data.
//!
//! Both SQL executor and autonomous executor instance run at a main thread, while autonomous executor has workers which run at different worker threads.
//!
//! # Entities inside Stream Engine
//!
//! ![Entities inside Stream Engine](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-entity.svg)
//!
//! # Communication between entities
//!
//! Workers in AutonomousExecutor interact via EventQueue (Choreography-based Saga pattern).
//!
//! ![Communication between entities](https://raw.githubusercontent.com/SpringQL/SpringQL.github.io/main/static/img/stream-engine-architecture-communication.svg)
#![doc = include_str!("stream_engine.md")]

pub use autonomous_executor::SpringValue;

Expand Down