Wastewater Outlier Detection Investigation Using PPL

This project investigated the use of probabilistic programming techniques to model and detect outliers in Covid-19 wastewater data.

Table of Contents

• About the Project
  • Built With
  • Project Steps
• Getting Started
  • Prerequisites
  • Installation
• Usage
• Contributing
• License
• Contact
• Acknowledgements

About The Project

This project uses the Julia programming language and the Gen probabilistic modeling and inferencing framework to model the data and to identify outliers among the sample data points.

Sample Results

Project Steps

For ease of understanding, this project is broken up into a set of steps:

1. Importing Data

In this step, we import our data from a file using Julia utilities such as dataframes.

2. Linear Model

Next, we use a linear probabilistic model to fit a line through the data points along with identifying outliers.

3. Linear Spline

Rather than trying to approximate our data with a single line, we break the data into groups and fit a set of linear segments through the sequence of groups.

4. Linear Log Spline

Since our data points encompass a large range of values and because epidemic trends frequently follow exponential trends, we transform the values to a log scale and fit linear segments to the data in the logarithmic space.

5. Quadratic Spline

In order to overcome the limitations of using a linear model to fit varying data, we use a quadratic model instead, fitting the data to a series of quadratic spline segments.

6. Smooth Quadratic Spline

In order to smooth the segmented quadratic spline from the previous step, we match the derivatives at the endpoints of the quadratic spline segments.

Built With

• Julia - The Julia Programming Language
• Gen - PPL Framework for Generative Modeling and Probabilistic Inference
• Jupyter - Notebooks for Interactive Computing

Getting Started

To get a local copy up and running follow these simple steps.

Prerequisites

• Julia

To install Julia, download and run the appropriate installer: Julia Downloads

• Jupyter

To run Jupyter notebooks, follow the instructions provided: Jupyter Installation

Installation

1. Clone the repo

git clone https://github.com/AFIDSI/PPL_OutlierPrediction.git

Usage

The project examples are provided in two different forms (1) as Julia script files that may be executed from the command line and (2) as Jupyter notebook files that may be viewed in a Jupyter editor.

1. Go to the desired project step

cd src/step01-importing-data

1. Run the Julia code from the command line OR open the notebook file in a Jupyter editor.

julia step01.jl

jupyter notebook step01.ipynb

Contributing

We are not accepting contributions to this project at this time.

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Steve Goldstein - (mailto:sgoldstein@wisc.edu) - email

Project Link: https://github.com/AFIDSI/PPL_OutlierPrediction

Acknowledgements

• Data Science Institute @ University of Wisconsin-Madison

=======

PPL_OutlierPrediction

This project looks to analyze Covid-19 outliers using probabilistic programing models in Gen. https://drive.google.com/drive/folders/1m5wJr5PzOk0qeckA_VT_dturEy3F8h4B?usp=sharing