Release
Quality
Repository
Dependencies

---

Rating Tracker

A web service fetching and providing financial and ESG ratings for stocks.

Features

Stock List with sorting and filtering

Stocks and their information are presented in a paginated table which offers comprehensive and in-depth sorting and filtering by many of the available attributes.

Automatic and scheduled data fetching from several providers

By providing identifiers for stocks from Morningstar, MarketScreener, MSCI, LSEG Data & Analytics, Standard & Poor’s and Sustainalytics in the “Add Stock” dialog, Rating Tracker can automatically fetch financial data as well as financial and ESG ratings. The identifiers to use can be found in the provider’s URL for the stock as shown in the following examples:

• Morningstar: https://tools.morningstar.it/it/stockreport/default.aspx?Site=it&id=0P000000GY&LanguageId=it-IT&SecurityToken=0P000000GY]3]0]E0WWE$$ALL
• MarketScreener: https://www.marketscreener.com/quote/stock/APPLE-INC-4849
• MSCI: https://www.msci.com/our-solutions/esg-investing/esg-ratings-climate-search-tool/issuer/apple-inc/IID000000002157615
• LSEG Data & Analytics: https://www.lseg.com/bin/esg/esgsearchresult?ricCode=AAPL.O (see also Refinitiv Identification Code)
• Standard & Poor’s: https://www.spglobal.com/esg/scores/results?cid=4004205
• Sustainalytics: https://www.sustainalytics.com/esg-rating/apple-inc/1007903183

The fetching can be scheduled by providing a Cron-like specifier in an environment variable. See below for details.

Stock Logos

When providing an ISIN for a stock, its logo is automatically fetched and cached from TradeRepublic broker.

Rating Scores

The fetched ratings of a stock are aggregated to both a financial and ESG score using the average values of all ratings, such that a score of 0 is assigned to an average stock and a score of 100 is assigned to a stock with perfect scores in all underlying ratings.

The financial and ESG score are used to compute a total score using the harmonic mean of both numbers, so that a stock has to perform well in both financial and ESG ratings to obtain a good total score.

Watchlists

Stocks noteworthy to a user can be organized in watchlists. A dedicated Favorites watchlist is provided by default and can easily be maintained by clicking the star icon for a stock.

Users can subscribe to a watchlist, so they receive notifications when a stock’s rating is updated.

Portfolios

Stocks weighted by a given amount of currency can be aggretated in portfolios. The average rating scores of the portfolio can be displayed, as well as the distribution of regions, industry sectors, company sizes and styles of the stocks in the portfolio.

Portfolio Builder

This tool can provide optimal weights of stocks in a portfolio based on the preferred proportions of regions, sectors and other factors.

Stocks can be taken from an existing portfolio, a watchlist, or searched for manually:

Constraints for regions, sectors, sizes and styles can then be provided, as well as a currency for the portfolio and other settings, such as the total amount to invest, the minimum currency amount per stock or the algorithm to use for proportional representation:

The resulting stock distribution is presented in a histogram-like chart, and differences between target and actual percentages are highlighted:

The dialog to save the weighted stocks to a new or existing portfolio transparently lists all updates to be made as well as the progress in transferring them to the server:

User Management

The Rating Tracker supports multiple users, who can self-register via WebAuthn and access the application after being granted fine-grained access by an administrator, for whom a “User Management” web interface is provided.

Notification Messages via Signal

Based on their access rights, users can subscribe to updates of stock ratings, fetch error reports, or new user registrations by providing a phone number capable of receiving messages via the instant messenger Signal.

Error reports with screenshots

When fetching a stock fails, a screenshot of the page the fetch was attempted from is stored and a link to them is sent to stock maintainers who subscribed to error reports, so they can analyze and fix the issue.

Logging

Logs are printed to stdout as well as rotating log files with pino-pretty. While the stdout log output is already rendered with beautiful colors and icons in a p10k-like fashion (for which a font supporting all characters may be required), the log files are JSON-formatted and can be pretty-printed using pino-pretty. A minimal containerized setup can be added to your .zshrc or .bashrc like this:

# ~/.zshrc
alias pino-pretty="{ (echo -e \"FROM node:alpine\nWORKDIR /home/node\nRUN npm install -g pino-pretty\nUSER node\nCOPY --from=marvinruder/rating-tracker /app/pino-pretty-config.cjs pino-pretty-config.cjs\nENTRYPOINT [ \\\"pino-pretty\\\", \\\"--config\\\", \\\"/home/node/pino-pretty-config.cjs\\\" ]\" | docker build -q - -t pino-pretty > /dev/null) && docker run -i --rm -e FORCE_COLOR=1 pino-pretty -c; }"

# To view (and follow) a log file:
tail -n +1 -f logs/rating-tracker.log | pino-pretty | less -r

If you prefer a plain log viewer, you can use pino-pretty without the configuration file and set the PLAIN_LOG environment variable as documented below.

…and more to come!

Planned features are documented here. If you feel that something is missing, feel free to request a feature!

Demo

An instance of the Rating Tracker is publicly available at https://ratingtracker.mruder.dev, for which access is granted at request.

Deployment

Rating Tracker is built to be deployed using Docker or a similar container platform.

Prerequisites

To run Rating Tracker, the following services must be available:

• PostgreSQL, storing information related to stocks and users
• Redis, caching session IDs, stock logos and other resources
• Signal Messenger REST API, sending notifications via the Signal messenger
• nginx, set up as a reverse proxy to provide SSL encryption (required for most WebAuthn clients)

Minimal Example Setup using Docker Compose

Docker Compose is the preferred way to run Rating Tracker together with all the services it depends on. The following configuration file shows an exemplary setup.

View Docker Compose configuration

version: "3.8"

services:
  postgres:
    image: postgres:alpine
    ports:
      - "127.0.0.1:5432:5432"
    environment:
      POSTGRES_DB: "rating-tracker"
      POSTGRES_USER: "rating-tracker"
      POSTGRES_PASSWORD: "********"
      PGDATA: /var/lib/postgresql/data
    volumes:
      - ./postgresql/data:/var/lib/postgresql/data
    shm_size: '256mb'

  redis:
    image: redis:alpine
    ports:
      - "127.0.0.1:6379:6379"
    command: redis-server --save 60 1 --activedefrag yes --aclfile /etc/redis/users.acl
    volumes:
      - ./redis/data:/data
      - ./redis/config:/etc/redis # the ACL file with the user and password must be created in this folder

  signal:
    image: bbernhard/signal-cli-rest-api
    environment:
      MODE: json-rpc
    ports:
      - "127.0.0.1:8080:8080"
    volumes:
      - ./signal-cli:/home/.local/share/signal-cli

  rating-tracker:
    image: marvinruder/rating-tracker
    tty: true # required for colored output to stdout
    init: true # required for graceful shutdown
    environment:
      PORT: 21076
      DOMAIN: "example.com"
      SUBDOMAIN: "ratingtracker"
      LOG_FILE: "/app/logs/rating-tracker-log-(DATE).log" # (DATE) is replaced by the current date to support log rotation
      DATABASE_URL: "postgresql://rating-tracker:********@postgres:5432/rating-tracker?schema=rating-tracker"
      REDIS_URL: "redis://redis:6379"
      REDIS_USER: "rating-tracker"
      REDIS_PASS: "********"
      MAX_FETCH_CONCURRENCY: 4
      AUTO_FETCH_SCHEDULE: "0 0 0 * * *" # this format includes seconds
      SIGNAL_URL: "http://signal:8080"
      SIGNAL_SENDER: "+12345678900"
    ports:
      - "127.0.0.1:443:21076" # optional if nginx runs in same Docker Compose setup
    volumes:
      - ./logs/rating-tracker:/app/logs
    depends_on:
      - postgres
      - redis
      - signal
    restart: unless-stopped

The port bindings are optional but helpful to connect to the services from the host, e.g. for debugging purposes.

Setup steps

Initialize database setup

Rating Tracker uses Prisma to interact with the PostgreSQL database. At first startup, Prisma Migrate will automatically create the required tables and indexes.

Create Redis user and password

Create the file users.acl with the following content:

user default off
user rating-tracker allcommands allkeys allchannels on >********

Refer to the exemplary Docker Compose setup for information on where to place the ACL file.

To use a password hash, first create the file above and start up Redis, then connect, authenticate and run ACL GETUSER rating-tracker. The output shows the hash of the password, which can then be used in the ACL file:

user default off
user rating-tracker allcommands allkeys allchannels on #07ab59f4[…]072e07fb

More info on ACL files in Redis can be found here.

Create Signal account

Run a shell in the Signal REST API container and proceed with this excellent documentation.

Configure webserver as reverse proxy

After setting up NGINX as a webserver with SSL, the following virtual host configuration can be used to run a reverse proxy which also adds security- and privacy-related HTTP headers compatible with Rating Tracker.

View NGINX configuration

add_header "Content-Security-Policy" "default-src 'self' 'wasm-unsafe-eval'; worker-src 'self' blob:; img-src 'self' data:; style-src-elem 'self' 'unsafe-inline' rsms.me; style-src 'self' 'unsafe-inline' rsms.me; font-src rsms.me; frame-ancestors 'none'; form-action 'self'; base-uri 'none';";add_header "Strict-Transport-Security" "max-age=31536000; includeSubDomains" always;
add_header "X-Frame-Options" "DENY";
add_header "X-Content-Type-Options" "nosniff";
add_header "Referrer-Policy" "same-origin";
add_header "Cross-Origin-Opener-Policy" "same-origin";
add_header "Cross-Origin-Resource-Policy" "same-site";
add_header "Cross-Origin-Embedder-Policy" "credentialless";
add_header "Permissions-Policy" "interest-cohort=();";

resolver 127.0.0.11 valid=15s; # DNS resolver from Docker to resolve Docker Compose container names

location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
    set $target_host rating-tracker; # use 127.0.0.1 here if nginx runs outside of Docker Compose setup
    proxy_pass http://$target_host:21076;
}

Initial admin registration and activation

After setting up your Rating Tracker instance, navigate to its URL and register, creating WebAuthn credentials. Then manually connect to the database and set the new user’s accessRights value to 255, granting ultimate access. After that, you can log in using your credentials. This is a one-time setup step, all other users can be granted access via the User Management web interface.

Supported environment variables

Variables in bold font are mandatory.

View complete list of environment variables

Variable	Example Value	Explanation
PORT	21076	The TCP port Rating Tracker is served on.
DOMAIN	example.com	The domain Rating Tracker will be available at. This is especially important for WebAuthn, since credentials will only be offered to the user by their client when the domain provided as part of the registration or authentication challence matches the domain of the URL the user navigated to.
SUBDOMAIN	ratingtracker	An optional subdomain. Credentials created for one domain can be used to authenticate to different Rating Tracker instances served on all subdomains of that domain, making it easy to use multiple deployment stages, development servers etc.
DATABASE_URL	postgresql://rating-tracker:********@127.0.0.1:5432/rating-tracker?schema=rating-tracker	The connection URL of the PostgreSQL instance, specifying username, password, host, port, database and schema. Can also use the PostgreSQL service name (e.g. postgres in this configuration) as hostname if set up within the same Docker Compose file.
REDIS_URL	redis://127.0.0.1:6379	The URL of the Redis instance. Can also use the Redis service name (e.g. redis in this configuration) as hostname if set up within the same Docker Compose file.
REDIS_USER, REDIS_PASS,	rating-tracker, ********	The username and password to connect to the Redis instance. Read more here on how to set up a password-protected Redis user. If unset, the Redis instance must grant write access to the default user.
LOG_FILE	/var/log/rating-tracker-(DATE).log	A file path for storing Rating Tracker log files. The string (DATE) will be replaced by the current date. If unset, logs are stored in the /tmp directory.
LOG_LEVEL	debug	The level for the log output to stdout. Can be one of fatal, error, warn, info, debug, trace. If unset, info will be used.
PLAIN_LOG	1	If set to a truthy value, the log output to stdout will not be rendered with colors and icons.
AUTO_FETCH_SCHEDULE	0 30 2 * * *	A Cron-like specification of a schedule for when to fetch all stocks from all providers. The format in use includes seconds, so the example value resolves to “every day at 2:30:00 AM”. If unset, no automatic fetching will happen.
MAX_FETCH_CONCURRENCY	4	The number of fetcher instances used concurrently when fetching information for multiple stocks. If unset, no concurrent fetches will be performed.
SIGNAL_URL	http://127.0.0.1:8080	The URL of the Signal REST API. Can also use the Signal REST API service name (e.g. signal in this configuration) as hostname if set up within the same Docker Compose file. If unset, no Signal notification messages will be sent.
SIGNAL_SENDER	+12345678900	The phone number of the Signal account registered with the Signal CLI service, which will be used to send notification messages. Read more here on how to register a Signal account. If unset, no Signal notification messages will be sent.

API Reference

Any Rating Tracker instance’s API is self-documented, its OpenAPI web interface is hosted at /api-docs. The complete OpenAPI specification document can be downloaded at /api-spec/v3.

Development

Create an environment for developing

An environment with all tools required for developing Rating Tracker and the services it depends on can quickly be created using the VS Code development container configuration included in this repository. The scripts section in the package.json provides helpful commands:

• Clone the repository and open it in Visual Studio Code. When prompted, select “Reopen in Container”. This will create a Docker container with all required tools, recommended extensions and dependencies installed.
• Check your environment. SSL Certificates and the Redis ACL file must be provided beforehand, and a Signal account must be created before starting the server (see section Setup steps for details). The NGINX configuration might require adjustment to your situation.
• Run yarn prisma:migrate:deploy to initialize the PostgreSQL database.
• Run yarn dev to start the backend server as well as the Vite frontend development server.

Environment variables not included in the development container configuration can easily be defined in an .env file:

See recommended development environment variables

# .env
# # Those variables are already defined in the development container configuration. You only need to define them here if you want to override them.
# DATABASE_URL="postgresql://rating-tracker:rating-tracker@postgres:5432/rating-tracker?schema=rating-tracker"
# REDIS_URL="redis://redis:6379"
# SIGNAL_URL="http://signal:8080"
# NODE_ENV="development"
# PORT="3001"
# MAX_FETCH_CONCURRENCY="4"
# REDIS_USER="rating-tracker"
# REDIS_PASS="rating-tracker"
# POSTGRES_USER="rating-tracker"
# POSTGRES_PASS="rating-tracker"
# LOG_LEVEL="trace"

DOMAIN=example.com
SUBDOMAIN=ratingtracker
SIGNAL_SENDER="+12345678900"
# AUTO_FETCH_SCHEDULE=" 0 * * * * *" # This would run every minute, activate for debugging only.

Run tests

The VS Code development container configuration includes additional PostgreSQL and Redis instances for running tests. To use them, configure your development environment as described above and run yarn test to run all tests from all packages. Additionally, the packages’ package.json configurations contain a test:watch script to run tests in watch mode.

Contribute

Contributions are welcome!

Disclaimer

This software is provided under the conditions of the MIT License.

Important

Use this tool at your own risk. Excessive data fetching from providers, publishing or selling the information obtained by fetching is not recommended. Your actions may have consequences… 🦋

Authors

• Marvin A. Ruder (he/him)