Skip to content

Commit

Permalink
Merge pull request #294 from covidcanidoit/function-to-load-covidexit…
Browse files Browse the repository at this point in the history
…strategy

Function to load covidexitstrategy
  • Loading branch information
awwaiid committed Dec 4, 2020
2 parents 754688b + c560eb5 commit 9f361a1
Show file tree
Hide file tree
Showing 5 changed files with 1,124 additions and 57 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ jobs:
# Install NPM dependencies, cache them correctly
# and run all Cypress tests
- name: Cypress run
uses: cypress-io/github-action@v2
# uses: cypress-io/github-action@v2
uses: cypress-io/github-action@v2.3.8
with:
start: yarn serve
wait-on: 'http://localhost:8080'
Expand Down
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Check out [CONTRIBUTING.md](./CONTRIBUTING.md) for more details!
- [Vuetify](https://vuetifyjs.com/)
- [Firebase Hosting](https://firebase.google.com/docs/hosting)
- [Firebase Database](https://firebase.google.com/docs/database)
- [Firebase Functions](https://firebase.google.com/docs/functions)

## Dev workflow

Expand Down Expand Up @@ -123,3 +124,19 @@ database and other dynamic content).
- Based on project needs, sometimes we do a bulk-data-import using tools in the `utils/` directory
- We have a separate firebase project named `ccidi-staging` that does all of the same things, which is nice for testing infrastructure and major database changes
- The Google Places API doesn't provide busyness data directly, so there is also an API service that uses the [populartimes](https://github.com/m-wrzr/populartimes) python library to scrape this data. Right now this is hosted on @awwaiid's server, but we'll move it into a Google Cloud function at some point

## Functions

### Daily Covidexitstrategy Data Import

This is a firebase function that automatically loads each region (in the form of US States including Puerto Rico and D.C.) provided by CovidExitStrategy along with their current risk level.

- The code will run every midnight EST.
- Each region will have a timestamp node `updated` with the time it was last updated. Viewing this node will let you know if it ran properly.
![image](https://user-images.githubusercontent.com/61799449/99341945-afa7ea00-283f-11eb-8b40-f3049ae85b1c.png)
- To run manually, visit the [cloud scheduler dashboard](https://console.cloud.google.com/cloudscheduler?_ga=2.94024345.543360165.1605582414-1181068750.1594143574&project=ccidi-staging&folder=&organizationId=). Click RUN NOW.
![image](https://user-images.githubusercontent.com/61799449/99342307-7623ae80-2840-11eb-80e3-521eba12d327.png)
- You can also see information about the run by going to the [Firebase console](https://console.firebase.google.com/u/0/project/ccidi-staging/functions/logs?search=&&severity=DEBUG). Click Functions. Click Logs.
![image](https://user-images.githubusercontent.com/61799449/99341819-73748980-283f-11eb-9d7f-05ef04df95d8.png)

To deploy changes, run `firebase deploy --only functions`
152 changes: 149 additions & 3 deletions functions/index.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,154 @@
const firebase = require("firebase");
const functions = require("firebase-functions");
const auth = require("firebase/auth");
const axios = require("axios");
const { firebaseConfig } = require("firebase-functions");
const csv = require("csvtojson");

// // Create and Deploy Your First Cloud Functions
// // https://firebase.google.com/docs/functions/write-firebase-functions
//
// exports.helloWorld = functions.https.onRequest((request, response) => {
// response.send("Hello from Firebase!");
// });
exports.helloWorld = functions.https.onRequest((request, response) => {
response.send("Hello from Firebase!");
});

const abbrv = {
Arizona: "AZ",
Alabama: "AL",
Alaska:"AK",
Arkansas: "AR",
California: "CA",
Colorado: "CO",
Connecticut: "CT",
Delaware: "DE",
'District of Columbia': "DC",
Florida: "FL",
Georgia: "GA",
Hawaii: "HI",
Idaho: "ID",
Illinois: "IL",
Indiana: "IN",
Iowa: "IA",
Kansas: "KS",
Kentucky: "KY",
Louisiana: "LA",
Maine: "ME",
Maryland: "MD",
Massachusetts: "MA",
Michigan: "MI",
Minnesota: "MN",
Mississippi: "MS",
Missouri: "MO",
Montana: "MT",
Nebraska: "NE",
Nevada: "NV",
'New Hampshire': "NH",
'New Jersey': "NJ",
'New Mexico': "NM",
'New York': "NY",
'North Carolina': "NC",
'North Dakota': "ND",
Ohio: "OH",
Oklahoma: "OK",
Oregon: "OR",
Pennsylvania: "PA",
'Rhode Island': "RI",
'South Carolina': "SC",
'South Dakota': "SD",
Tennessee: "TN",
Texas: "TX",
Utah: "UT",
Vermont: "VT",
Virginia: "VA",
Washington: "WA",
'West Virginia': "WV",
Wisconsin: "WI",
Wyoming: "WY",
'Puerto Rico': "PR"
};

const trendNames = {
"0": "better",
"1": "caution",
"2": "poor",
"3": "poor"
};

exports.autoPullCovidExitStrategyData = functions.pubsub
.schedule("every day 00:00")
.onRun(context => {
const covidExitStrategyData =
"https://docs.google.com/spreadsheets/d/e/2PACX-1vStD_EMR9El7agVp-Oi6d1c5EMAOYgoYOsSc2xhwzht1ae4Fku7F6zSmF4PB9J_aHA1DAb2PpAelomO/pub?output=csv&gid=237779988";
// COMM W/ DB
var admin = require("firebase-admin");
//var functions = require("firebase-functions");

console.log(`admin.name: ${admin.name}`);
try {
admin.initializeApp();
}
catch (err) {
console.error("firebase likely already exists", err);
}

const database = admin.database();

// FETCH DATA FROM COVID EXIT STRATEGY
axios
.get(covidExitStrategyData)
.then((response) => {

csv()
.fromString(response.data)
.then(regionsArray => {
console.log(`${regionsArray.length} number of regions to update`)
regionsArray.forEach((region) => {
const longName = region["STATE"];
const shortName = abbrv[region["STATE"]];
if (longName && shortName) {
const slug = shortName.toLowerCase();
const trending = trendNames[region["GATING SCORE"]];

try {
database.ref(`content/US/regions/${slug}`).set({
longName,
shortName,
slug,
trending,
updated: new Date().toString()
});
}
catch (err) {
console.error("error accessing content", err);
}
}
else {
console.error(
`row has no region name: `,
longName,
shortName,
region
);
}

});

console.log('outside array foreach');
return null;

})
.catch((err) => {
console.error('csv error',err);
});

console.log('outside csv')
return response;
})
.catch((err) => {
console.log('axios error',err);
});

console.log('outside axios');
return true;
//response.send("checking for nonexistent region?");
});

0 comments on commit 9f361a1

Please sign in to comment.