PaperWatch

A tool for automatically forwarding CloudWatch logs from your lambda functions to Papertrail.

Setup

Follow these steps to set up and deploy PaperWatch to AWS:

1. Configure

The two lambda functions reads the following configuration information from config/papertrail.json

Key	Required	Type	Description
host	Yes	String	The paper trail endpoint's address
port	Yes	Number	The paper trail endpoint's port
consumer	Yes	String	The name of the consumer function. Used by the Subscribe function to avoid subscribing the consumer to itself. Do not change unless you are modifying the CFN template.
retentionPeriod	No	Number	If included, updates the CloudWatch log group's retention period (in days)
sources	Yes	Array of Objects	List of log sources. A log source is identified by the prefix property. Log's generated by a source are passed to it's consumer function. Default configuration includes log sources for lambda functions and api gateway
exclude	No	Array of Strings	List of lambda function names that the Subscribe function should ignore
logLevelExtractor	Yes	RegExp	Regular expression that matches the incoming CloudWatch logs and extracts the log level from them. The extracted log level will be used in the PaperTrail log. Wrap the log level part in a regex group and you are good to go
defaultLogLevel	Yes	String	If logLevelExtractor is unable to match, use the level provided here.
Example config:

{
  "host": "samplehost.papertrailapp.com",
  "port": 12345,
  "retentionPeriod": 3
  "sources": [
    {
      "prefix": "/aws/lambda/",
      "consumer": "LambdaLogConsumer"
    },
    {
      "prefix": "API-Gateway-Execution-Logs_",
      "consumer": "APIGatewayLogConsumer"
    }
  ]
  "exclude": [
    "TestFunction",
    "LoudFunction"
  ],
  "logLevelExtractor": ".* - (error|warn|info|verbose|debug|emerg|alert|crit|notice): ",
  "defaultLogLevel": "info"
}

2. Build

Running the build script will install dependencies and create the .zip deployment package (as lambda.zip)

paperwatch $ npm run build

3. Deploy

First, upload the deployment package to an s3 bucket. Take note of the bucket name, you will need this later. This can be done with the aws-cli as follows:

paperwatch $ aws s3 cp ./lambda.zip s3://<BUCKET_NAME>

Next, create a new cloudformation stack using from the template at cfn/deploy.cfn.json.
Enter a stack name. You will also be prompted to enter the following parameters:

• SourceBucket - the name of the s3 bucket you uploaded the deployment package to
• SourceKey - name of the deployment package in the s3 bucket (lambda.zip), including any prefixes if applicable

Once deployed, logs will be forwarded to papertrail for all new lambda functions.
Existing functions must be subscribed manually.

Manual Subscription

To manually subscribe a function, navigate to the CloudWatch console and follow these steps:

1. Locate the log group for the function you wish to subscribe, selecting it by checking the box on the left
2. Open the Actions dropdown and choose Subscription filters -> Create lambda subscription filter
3. From the Lambda Function dropdown, select the consumer function (LambdaLogConsumer or APIGatewayLogConsumer by default)
4. From the Log Format dropdown, select Other and click next
5. For the Subscription filter name enter Papertrail
6. Click Start Streaming

PaperWatch will now begin forwarding the functions logs to Papertrail