Document crate topology (#2594) (#2913)

* Document crate topology (#2594) * Review feedback
apache · Oct 24, 2022 · bca8445 · bca8445
1 parent 28d6152
commit bca8445
Show file tree

Hide file tree

Showing 4 changed files with 39 additions and 6 deletions.
diff --git a/arrow-array/src/lib.rs b/arrow-array/src/lib.rs
@@ -16,7 +16,7 @@
 // under the License.
 
 //! The central type in Apache Arrow are arrays, which are a known-length sequence of values
-//! all having the same type. This module provides concrete implementations of each type, as
+//! all having the same type. This crate provides concrete implementations of each type, as
 //! well as an [`Array`] trait that can be used for type-erasure.
 //!
 //! # Downcasting an Array

diff --git a/arrow-data/src/lib.rs b/arrow-data/src/lib.rs
@@ -15,7 +15,7 @@
 // specific language governing permissions and limitations
 // under the License.
 
-//! Buffer abstractions for [Apache Arrow](https://docs.rs/arrow)
+//! Array data abstractions for [Apache Arrow](https://docs.rs/arrow)
 
 mod bitmap;
 pub use bitmap::Bitmap;

diff --git a/arrow/src/array/mod.rs b/arrow/src/array/mod.rs
@@ -15,7 +15,9 @@
 // specific language governing permissions and limitations
 // under the License.
 
-//! Re-exports APIs from [arrow_array]
+//! Statically typed implementations of Arrow Arrays
+//!
+//! **See [arrow_array] for examples and usage instructions**
 
 #[cfg(feature = "ffi")]
 mod ffi;

diff --git a/arrow/src/lib.rs b/arrow/src/lib.rs
@@ -21,10 +21,34 @@
 //! Please see the [arrow crates.io](https://crates.io/crates/arrow)
 //! page for feature flags and tips to improve performance.
 //!
+//! # Crate Topology
+//!
+//! The [`arrow`] project is implemented as multiple sub-crates, which are then re-exported by
+//! this top-level crate.
+//!
+//! Crate authors can choose to depend on this top-level crate, or just
+//! the sub-crates they need.
+//!
+//! The current list of sub-crates is:
+//!
+//! * [`arrow-array`][arrow_array] - type-safe arrow array abstractions
+//! * [`arrow-buffer`][arrow_buffer] - buffer abstractions for arrow arrays
+//! * [`arrow-data`][arrow_data] - the underlying data of arrow arrays
+//! * [`arrow-schema`][arrow_schema] - the logical types for arrow arrays
+//! * [`arrow-select`][arrow_select] - selection kernels for arrow arrays
+//!
+//! _This list is likely to grow as further functionality is split out from the top-level crate_
+//!
+//! Some functionality is also distributed independently of this crate:
+//!
+//! * [`arrow-flight`] - support for [Arrow Flight RPC]
+//! * [`arrow-integration-test`] - support for [Arrow JSON Test Format]
+//! * [`parquet`](https://docs.rs/parquet/latest/parquet/) - support for [Apache Parquet]
+//!
 //! # Columnar Format
 //!
-//! The [`array`] module provides statically typed implementations of all the array
-//! types as defined by the [Arrow Columnar Format](https://arrow.apache.org/docs/format/Columnar.html).
+//! The [`array`] module provides statically typed implementations of all the array types as defined
+//! by the [Arrow Columnar Format](https://arrow.apache.org/docs/format/Columnar.html)
 //!
 //! For example, an [`Int32Array`](array::Int32Array) represents a nullable array of `i32`
 //!
@@ -77,7 +101,7 @@
 //! assert_eq!(min(&StringArray::from(vec!["b", "a", "c"])), Some("a"));
 //! ```
 //!
-//! For more examples, consult the [`array`] docs.
+//! For more examples, consult the [arrow_array] docs.
 //!
 //! # Type Erasure / Trait Objects
 //!
@@ -235,13 +259,20 @@
 //! orchestrates the primitives exported by this crate into an embeddable query engine, with
 //! SQL and DataFrame frontends, and heavily influences this crate's roadmap.
 //!
+//! [`arrow`]: https://github.com/apache/arrow-rs
 //! [`array`]: mod@array
 //! [`Array`]: array::Array
 //! [`ArrayRef`]: array::ArrayRef
 //! [`ArrayData`]: array::ArrayData
 //! [`make_array`]: array::make_array
 //! [`Buffer`]: buffer::Buffer
 //! [`RecordBatch`]: record_batch::RecordBatch
+//! [`arrow-flight`]: https://docs.rs/arrow-flight/latest/arrow_flight/
+//! [`arrow-integration-test`]: https://docs.rs/arrow-integration-test/latest/arrow_integration_test/
+//! [`parquet`]: https://docs.rs/parquet/latest/parquet/
+//! [Arrow Flight RPC]: https://arrow.apache.org/docs/format/Flight.html
+//! [Arrow JSON Test Format]: https://github.com/apache/arrow/blob/master/docs/source/format/Integration.rst#json-test-data-format
+//! [Apache Parquet]: https://parquet.apache.org/
 //! [DataFusion]: https://github.com/apache/arrow-datafusion
 //! [issue tracker]: https://github.com/apache/arrow-rs/issues
 //!