Skip to content

sciety/data-pipeline

Repository files navigation

How to run

Software Dependencies

  • python3
  • python3-venv (might be included in python3)
  • kubectl
  • docker
  • awscli
  • make
  • jq
  • bash

Access Requirements

No secrets are stored in this repo. Credentials live on your machine.

  • AWS access to the 540790251273 account. Ensure the AWS credentials point to the Sciety profile, e.g. by setting AWS_PROFILE (if needed).
  • Access to the k8s cluster: libero-eks--franklin
  • GCloud access to elife-data-pipeline project

Gcloud login

Run:

make gcloud-login

Follow the prompt to open the browser and authenticate. This stores authentication into a docker volume, pruning of volumes might affect this.

Update Data Studio

The export might take a few minutes.

make update-datastudio

Prototype dashboard using this data.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published