Skip to content

Commit

Permalink
Restructure reference information and remove some duplicated headers
Browse files Browse the repository at this point in the history
  • Loading branch information
astrojuanlu committed Jul 22, 2022
1 parent 2162255 commit 442765d
Show file tree
Hide file tree
Showing 7 changed files with 205 additions and 223 deletions.
67 changes: 67 additions & 0 deletions docs/source/fundamentals/data_passing.rst
Expand Up @@ -147,3 +147,70 @@ Having the above order of connections, step 3's ``input_data`` becomes:
}
Top-to-bottom in the visual editor corresponds to left-to-right in ``unnamed``.

.. _r:

R example
---------

.. tip::
👉 Import the `example project
<https://github.com/orchest-examples/orchest-pipeline-r-python-mix>`_ showcasing **R** straight
in Orchest (:ref:`how to import a project <how to import a project>`).

The Orchest SDK in **R** works through the `reticulate <https://rstudio.github.io/reticulate/>`_
package. To explain its usage, an example project is provided below.

First, create an Orchest environment which uses the ``orchest/base-kernel-r`` base image (you can
find more details :ref:`here <environments>`). Next you want to install ``reticulate`` and configure
access to Python and the Orchest SDK. You can do so by having a script (let's say ``Install.r``) in
your project with the following content:

.. code-block:: r
install.packages("reticulate", repos = "http://cran.us.r-project.org")
library(reticulate)
# Dynamically find system Python
python_path <- system("which python", intern=TRUE)
use_python(python_path)
# Pre compile orchest deps
orchest <- import("orchest")
print(orchest)
and having the environment set-up script perform ``Rscript Install.r``. You will then be able to
access the Orchest SDK through R **in every step that makes use of this environment** . To do data
passing, for example, you would do the following:

.. code-block:: r
library(reticulate);
python_path <- system("which python", intern=TRUE);
use_python(python_path);
orchest <- import("orchest");
orchest$transfer$output(2, name="Test");
In a child step you will be able to retrieve the output:

.. code-block:: r
library(reticulate);
python_path <- system("which python", intern=TRUE);
use_python(python_path);
orchest <- import("orchest")
step_inputs = orchest$transfer$get_inputs()
step_inputs$Test
Julia example
-------------

Refer to the `Julia example project <https://github.com/orchest-examples/julia-orchest-sdk>`_
showcasing **Julia** in Orchest (:ref:`how to import a project <how to import a project>`).

JavaScript example
------------------

Refer to the `JavaScript example project <https://github.com/orchest-examples/javascript-orchest-sdk>`_
showcasing **JavaScript** in Orchest (:ref:`how to import a project <how to import a project>`).
114 changes: 0 additions & 114 deletions docs/source/fundamentals/sdk/index.rst

This file was deleted.

@@ -1,10 +1,6 @@
# How to...
# Frequently Asked Questions (FAQ)

## Pass data between pipeline steps

Please refer to the dedicated section on {ref}`data passing <data passing>`.

## Install new packages
## How to install new packages?

```{tip}
👉 Would you rather watch a short video tutorial? Check it our here: [installing additional
Expand All @@ -27,17 +23,9 @@ Do **not** install new packages by running bash commands inside the Notebooks. T
packages to be installed every time you do a pipeline run, since the state of the kernel environment
is ephemeral.

## Use `git` inside Orchest

Please refer to the dedicated section on {ref}`using git inside Orchest <git inside Orchest>`.

(how-to-import-a-project)=

## Import a project

Check out our video: [importing a project](https://www.tella.tv/video/cknr7of9c000409jr5gx4efjy/view).

## Share code between steps
## How to share code between steps?

```{note}
💡 This approach also works to share code between pipelines.
Expand All @@ -57,7 +45,7 @@ import utils
utils.transform(...)
```

## Minimize Orchest's disk size
## How to minimize Orchest's disk size?

To keep Orchest's disk footprint to a minimal you can use the following best practices:

Expand All @@ -70,101 +58,13 @@ To keep Orchest's disk footprint to a minimal you can use the following best pra
number of pipeline runs and automatically delete the older ones. Steps: (1) edit an existing job
or create a new one, (2) go to _pipeline runs_, and (3) select _auto clean-up_.

## Use a GPU in Orchest
## How to use a GPU in Orchest?

Currently GPU support is not yet available. Coming soon!

## Use the Orchest CLI

Below you will find the most important `orchest-cli` commands that you need to know (you can also get all this
information by running `orchest -h`):

```sh
orchest start

# Stop Orchest (shuts down Orchest completely).
orchest stop

# Install Orchest (check out the dedicated `Installation` guide in
# the `Getting started` section).
orchest install

# Update Orchest to a newer version (NOTE: this can also be done
# through the settings in the UI).
orchest update

# Get extensive version information. Useful to see whether the
# installation was successful.
orchest version
```

## Use Orchest shortcuts like a pro

### Command palette

```{eval-rst}
.. list-table::
:widths: 25 25
:header-rows: 1
:align: left
* - Key(s)
- Action
* - :kbd:`Control`/:kbd:`Command` + :kbd:`K`
- Open command palette
* - :kbd:`↑`/:kbd:`↓`
- Navigate command palette commands
* - :kbd:`PageUp`/:kbd:`PageDown`
- Navigate command palette commands
* - :kbd:`Escape`
- Dismiss command palette
```

### Pipeline editor

```{eval-rst}
.. list-table::
:widths: 25 25
:header-rows: 1
:align: left
* - Key(s)
- Action
* - :kbd:`Space` + click + drag
- Pan canvas*
* - :kbd:`Ctrl` + click
- Select multiple steps
* - :kbd:`Ctrl` + :kbd:`A`
- Select all steps*
* - :kbd:`Ctrl` + :kbd:`Enter`
- Run selected steps*
* - :kbd:`H`
- Center view and reset zoom
* - :kbd:`Escape`
- Deselect steps
* - :kbd:`Delete`/:kbd:`Backspace`
- Delete selected step(s)
* - Double click a step
- Open file in JupyterLab
```

\* Requires mouse to hover the canvas

(skip-notebook-cells)=

## Skip notebook cells
## How to skip notebook cells?

Notebooks facilitate an experimental workflow, meaning that there will be cells that should not be
run when executing the notebook (from top to bottom). Since {term}`pipeline runs <pipeline run>`
Expand All @@ -180,7 +80,7 @@ To skip a cell during pipeline runs:
The cells with the _skip_ tag are still runnable through JupyterLab, but when executing these
notebooks as part of pipelines in Orchest they will not be run.

## Migrate to Kubernetes
## How to migrate to Kubernetes?

The moment we have moved to a Kubernetes backed Orchest version (and deprecated the Docker based
version), we will update this section of the documentation to include steps on how to migrate your
Expand Down
13 changes: 11 additions & 2 deletions docs/source/index.md
Expand Up @@ -38,7 +38,6 @@
getting_started/installation
getting_started/quickstart
getting_started/when_to_use_orchest
getting_started/how_to
```

```{toctree}
Expand All @@ -54,9 +53,19 @@ fundamentals/notifications
fundamentals/environments
fundamentals/environment_variables
fundamentals/services
fundamentals/sdk/index
fundamentals/settings
fundamentals/glossary
getting_started/faq
```

```{toctree}
:caption: Reference
:hidden: true
:maxdepth: 1
reference/shortcuts
reference/sdk/index
reference/cli/index
```

```{toctree}
Expand Down
23 changes: 23 additions & 0 deletions docs/source/reference/cli/index.md
@@ -0,0 +1,23 @@
# Orchest CLI reference

Below you will find the most important `orchest-cli` commands that you need to know (you can also get all this
information by running `orchest -h`):

```sh
orchest start

# Stop Orchest (shuts down Orchest completely).
orchest stop

# Install Orchest (check out the dedicated `Installation` guide in
# the `Getting started` section).
orchest install

# Update Orchest to a newer version (NOTE: this can also be done
# through the settings in the UI).
orchest update

# Get extensive version information. Useful to see whether the
# installation was successful.
orchest version
```

0 comments on commit 442765d

Please sign in to comment.