Skip to content

Commit

Permalink
Make the code work,
Browse files Browse the repository at this point in the history
Document it,
Use what I did for EventSource 1.0.1, EventSource/eventsource#74
Use new name,
Publish :)
  • Loading branch information
ScalaWilliam committed May 10, 2017
1 parent caee430 commit 942823f
Show file tree
Hide file tree
Showing 13 changed files with 206 additions and 132 deletions.
3 changes: 2 additions & 1 deletion .gitignore
@@ -1,4 +1,5 @@
node_modules/
target/
lib/main.js
*.log
*.log
*.tsv
3 changes: 2 additions & 1 deletion .npmignore
Expand Up @@ -2,4 +2,5 @@ node_modules
.idea
target
*.log
target
target
*.tsv
72 changes: 46 additions & 26 deletions README.md
@@ -1,49 +1,69 @@
# Scala.js CLI demo [![Build Status](https://travis-ci.org/ScalaWilliam/scalajs-cli-demo.svg?branch=master)](https://travis-ci.org/ScalaWilliam/scalajs-cli-demo)
# actionfps-clone-logs

> Publish Scala.js apps through NPM
[![NPM](https://nodei.co/npm/scalajs-cli-demo.png?compact=true)](https://nodei.co/npm/scalajs-cli-demo/)
> Clone live ActionFPS logs into a file.
[![NPM](https://nodei.co/npm/actionfps-clone-logs.png?compact=true)](https://nodei.co/npm/actionfps-clone-logs/)

## Rationale

Most likely, you've never ran a Scala CLI app, let alone a Scala.js CLI app.

But more likely you have ran Node.js CLI apps. This is because it is super
easy to publish your own CLI application through the NPM repository.
In order to turn ActionFPS into a platform, we need to share
raw logs with the potential users, and we'd also like to do it live
because there are some use cases where only live data is useful.

The simplest solution with features like authentication that is available
is [EventSource](https://www.w3.org/TR/2015/REC-eventsource-20150203/)
over HTTP/S.

On the other hand. publishing Scala apps is not the easiest thing in the world.
I want to get the best of both worlds: an excellent programming language
and an excellent lightweight distribution channel.
EventSource is supported by [Node.js](https://github.com/EventSource/eventsource/)
and HTML5/Google Chrome, and ActionFPS supplies these events live using an HTTPS
endpoint

Here's a demo to show you that it is possible.
The project is written in [Scala.js](https://www.scala-js.org/) and is based on
[scalajs-cli-demo](https://github.com/ScalaWilliam/scalajs-cli-demo).

It includes the use of the [Scala.js Node.js strong-typed API](https://github.com/scalajs-io/nodejs)
by [Lawrence Daniels](https://github.com/ldaniels528).
Node/JavaScript platform is chosen because it is a good distribution platform with easy access.

Scala is chosen as this is what ActionFPS is built on and provides great testing
and refactoring capabilities.

## Usage
Use the pre-built npmjs package.

```
$ npm install -g scalajs-cli-demo
$ scalajs-cli-demo
Hello world from Scala!
----
To demonstrate we're in Node, here's some of your environment:
PATH = /home/...
$ npm install -g actionfps-clone-logs
$ touch actionfps.tsv
$ actionfps-clone-logs actionfps.tsv
Reading file actionfps.tsv...
Resuming from time 2016-01-02T03:04:05Z, with 0 lines at this time
```

## Development
I recommend IntelliJ IDEA.
### Authorization

To iterate, inside SBT run:
An authorization token can be specified via an environment variable:

```
> ~run
$ AUTHORIZATION="Bearer xyz..." actionfps-clone-logs actionfps.tsv
```

or:
This may let you see full IP addresses for example.

### Default start time

`DEFAULT_START_TIME` (ISO8601 datetime) can be specified
for a default start of the stream.

This may be useful if you aren't interested in very historical data.

## Development
I recommend IntelliJ IDEA.

To continuously test inside SBT, run: `~test`.

To test app locally, run:

```
> ~test
$ sbt publishLocal
$ ./bin/actionfps-clone-logs
```

## Publishing
Expand Down
File renamed without changes.
10 changes: 7 additions & 3 deletions build.sbt
Expand Up @@ -4,8 +4,12 @@ name := "Scala.js CLI Demo"
scalaVersion := "2.12.2"
scalaJSUseMainModuleInitializer := true
scalaJSModuleKind := ModuleKind.CommonJSModule
mainClass in Compile := Some("HelloWorldApp")
moduleName in fullOptJS := "scalajs-cli-demo"
mainClass in Compile := Some("AfCloneLogsApp")
moduleName in fullOptJS := "actionfps-clone-logs"
libraryDependencies += "io.scalajs" %%% "nodejs" % "0.4.0-pre5"
libraryDependencies += "org.scalatest" %%% "scalatest" % "3.0.3" % "test"
cancelable in Global := true
cancelable in Global := true

publishLocal := {
IO.copyFile((fullOptJS in Compile).value.data, file("lib/main.js"))
}
19 changes: 11 additions & 8 deletions package.json
@@ -1,17 +1,17 @@
{
"name": "scalajs-cli-demo",
"version": "1.0.4",
"description": "POC to see if we can publish ScalaJS apps on npmjs.",
"name": "actionfps-clone-logs",
"version": "1.0.0",
"description": "Clone logs from ActionFPS to local file",
"bin": {
"scalajs-cli-demo": "bin/scalajs-cli-demo"
"actionfps-clone-logs": "bin/actionfps-clone-logs"
},
"scripts": {
"prepublish": "sbt 'show fullOptJS' && cp target/scala-2.12/scalajs-cli-demo-opt.js ./lib/main.js",
"prepublish": "sbt publishLocal",
"test": "sbt test"
},
"repository": {
"type": "git",
"url": "git+https://github.com/ScalaWilliam/scalajs-cli-demo.git"
"url": "git+https://github.com/ScalaWilliam/actionfps-clone-logs.git"
},
"keywords": [
"scalajs",
Expand All @@ -20,7 +20,10 @@
"author": "ScalaWilliam",
"license": "ISC",
"bugs": {
"url": "https://github.com/ScalaWilliam/scalajs-cli-demo/issues"
"url": "https://github.com/ScalaWilliam/actionfps-clone-logs/issues"
},
"homepage": "https://github.com/ScalaWilliam/scalajs-cli-demo#readme"
"homepage": "https://github.com/ScalaWilliam/actionfps-clone-logs#readme",
"dependencies": {
"eventsource": "^1.0.1"
}
}
62 changes: 62 additions & 0 deletions src/main/scala/AfCloneLogsApp.scala
@@ -0,0 +1,62 @@
import scala.scalajs.js
import js._
import io.scalajs.nodejs._

object AfCloneLogsApp extends JSApp {
import scala.scalajs.concurrent.JSExecutionContext.Implicits.queue

private val DefaultStartTimeEnv = "DEFAULT_START_TIME"

private val AuthorizationEnv = "AUTHORIZATION"

private val defaultStartTime: String =
process.env.get(DefaultStartTimeEnv).getOrElse("2016-01-02T03:04:05Z")

private val authorization: Option[String] =
process.env.get(AuthorizationEnv)

def main(): Unit = {

val atFile = {
val targetFile = process.argv.toList.drop(2).lastOption.getOrElse {
console.error("Target file not specified.")
process.exit(1)
null
}
console.info(s"Reading file ${targetFile}...")
AtEventFile(targetFile)
}

for {
optionalLastTime <- atFile.getLastLogTime
} {
val fromTime = optionalLastTime.map(_.timeString).getOrElse(defaultStartTime)
var remainingIgnore = optionalLastTime.map(_.recordsAtTime).getOrElse(0)

console.info(s"Resuming from time ${fromTime}, with ${remainingIgnore} lines at this time")
val headers = Dictionary("Last-Event-Id" -> fromTime)
authorization.foreach { authorization =>
headers += "Authorization" -> authorization
}
val eventSourceInitDict =
Dictionary("headers" -> headers)
val es = new EventSource("https://actionfps.com/logs", eventSourceInitDict)
es.on("open", { _: js.Dynamic =>
console.log("EventSource connection opened.")
})
es.on("error", { e: js.Dynamic =>
console.log("EventSource error: ", JSON.stringify(e))
})
es.on(
"log", { e: js.Dynamic =>
val line = e.data.toString
if (remainingIgnore == 0) {
atFile.appendLine(line)
} else {
remainingIgnore = remainingIgnore - 1
}
}
)
}
}
}
16 changes: 16 additions & 0 deletions src/main/scala/AtEventFile.scala
@@ -0,0 +1,16 @@
import io.scalajs.nodejs.fs

import scala.concurrent.Future

/**
* Created by william on 10/5/17.
*/
case class AtEventFile(targetFile: String) {

def appendLine(line: String): Unit = {
fs.Fs.appendFileSync(targetFile, line + "\n")
}

def getLastLogTime: Future[Option[LastLogTime]] = LastLogTime.fromFile(targetFile)

}
17 changes: 17 additions & 0 deletions src/main/scala/EventSource.scala
@@ -0,0 +1,17 @@
import io.scalajs.nodejs.events.IEventEmitter

import scala.scalajs.js
import scala.scalajs.js.Dictionary
import scala.scalajs.js.annotation.JSImport

/**
* Created by william on 10/5/17.
*
* Mapping to <a href="https://github.com/EventSource/eventsource/">EventSource library</a>
*/
@js.native
@JSImport("eventsource", JSImport.Namespace)
class EventSource() extends IEventEmitter {
def this(url: String) = this()
def this(url: String, eventSourceInitDict: Dictionary[_]) = this()
}
53 changes: 0 additions & 53 deletions src/main/scala/HelloWorldApp.scala

This file was deleted.

38 changes: 38 additions & 0 deletions src/main/scala/LastLogTime.scala
@@ -0,0 +1,38 @@
import io.scalajs.nodejs
import io.scalajs.nodejs.readline.ReadlineOptions

import scala.concurrent.{Future, Promise}

/**
* Read the log to determine the last timestamp and number of times it appears.
*
* The stream may be terminated at any point. If we didn't do this,
* we'd potentially have duplicate messages, and the chance that these
* duplicates happen is quite high.
*/
case class LastLogTime(timeString: String, recordsAtTime: Int) {
def accept(newTimeString: String): LastLogTime = {
if (timeString == newTimeString) {
copy(recordsAtTime = recordsAtTime + 1)
} else LastLogTime(timeString = newTimeString, recordsAtTime = 1)
}
}

object LastLogTime {
def fromFile(file: String): Future[Option[LastLogTime]] = {
val readInterface = nodejs.readline.Readline.createInterface(
new ReadlineOptions(
input = nodejs.fs.Fs.createReadStream(file)
))
val promise = Promise[Option[LastLogTime]]
var haveLogTime = Option.empty[LastLogTime]
readInterface.onLine { line =>
val timeString = line.split("\t").head
haveLogTime = haveLogTime.map(_.accept(timeString)).orElse(Some(LastLogTime(timeString, 1)))
}
readInterface.onClose(() => {
promise.success(haveLogTime)
})
promise.future
}
}
35 changes: 0 additions & 35 deletions src/main/scala/ReadLastTime.scala

This file was deleted.

0 comments on commit 942823f

Please sign in to comment.