Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Docker pull i/o timeout with v2 registry #878

Closed
kyleburnett opened this issue Aug 14, 2015 · 20 comments
Closed

Docker pull i/o timeout with v2 registry #878

kyleburnett opened this issue Aug 14, 2015 · 20 comments

Comments

@kyleburnett
Copy link

I'm running a private registry (v2) with the following configuration.

version: 0.1
storage:
    cache:
        layerinfo: inmemory
    filesystem:
        rootdirectory: /tmp/registry-dev
    maintenance:
        uploadpurging:
            enabled: false
http:
    addr: :5000
    debug:
        addr: localhost:5001
auth:
  token:
    realm: https://thinknode.dev/v2/token/
    service: registry.thinknode.dev
    issuer: thinknode.dev
    rootcertbundle: /usr/local/share/ca-certificates/thinknode.dev/thinknode.dev.crt
redis:
  addr: localhost:6379
  pool:
    maxidle: 16
    maxactive: 64
    idletimeout: 300s
  dialtimeout: 10ms
  readtimeout: 10ms
  writetimeout: 10ms

The registry sits behind nginx which is configured as follows:

upstream docker {
  server localhost:5000;
}

server {
  listen       443;
  server_name  registry.thinknode.dev;

  add_header Docker-Distribution-API-Version registry/2.0 always;

  proxy_set_header Host       $http_host;    # Required for Docker client
  proxy_set_header X-Real-IP  $remote_addr;  # Passthrough real client IP

  client_max_body_size 0;        # Disable any limits to avoid HTTP 413 for large image uploads
  chunked_transfer_encoding on;  # Required to avoid HTTP 411

  ssl on;
  ssl_certificate /usr/local/share/ca-certificates/registry.thinknode.dev/registry.thinknode.dev.crt;
  ssl_certificate_key /etc/ssl/private/registry.thinknode.dev.key;

  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;
  proxy_set_header X-Original-URI $request_uri;
  proxy_set_header Docker-Distribution-Api-Version registry/2.0;

  location / {
    proxy_pass http://docker;
  }
}

Occasionally (about 15% of the time), pulling an image with the npm module dockerode fails with the following logs from the docker daemon.

ERRO[10608] Error from V2 registry: Get https://registry.thinknode.dev/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8: read tcp 127.0.0.1:443: i/o timeout 
ERRO[10608] Handler for POST /images/create returned error: pulling with digest reference failed from v2 registry 
ERRO[10608] HTTP Error                                    err=pulling with digest reference failed from v2 registry statusCode=500

Here is my docker version information:

Client version: 1.7.0
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 0baf609
OS/Arch (client): linux/amd64
Server version: 1.7.0
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 0baf609
OS/Arch (server): linux/amd64

Not sure where to go from here. Any suggestions?
@dmp42
Copy link
Contributor

dmp42 commented Aug 14, 2015

read tcp 127.0.0.1:443: i/o timeout points to network communication failure between your client and your registry.

I suggest you start with fetching your nginx error log and registry log as well, to validate the request never reach them - then you will have to investigate why your client fails to communicate with the service.

@stevvooe
Copy link
Collaborator

This seems related to moby/moby#13337. Let's eliminate registry configuration as the root cause and work from there.

@kyleburnett
Copy link
Author

I'm getting this log entry over an over again in the nginx log:

2015/08/14 17:34:57 [error] 1054#0: *24820 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: registry.thinknode.dev, request: "GET /v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8 HTTP/1.1", upstream: "http://127.0.0.1:5000/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8", host: "registry.thinknode.dev"

@stevvooe
Copy link
Collaborator

@kyleburnett Looks like nginx is timing out on fetching the manifest. Looking at the default value for a proxy_read_timeout of 60s, there might be an upstream problem.

I noticed you have what looks like a random id for the repository name, "035088a9398b/math". Is this intended? Do you have the registry logs?

@kyleburnett
Copy link
Author

The "random id" is intentional.

Here are the logs

172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=545091f5-89d0-4dca-8650-50874b18168f http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=778635ec-760e-40a0-a869-9804218bd601 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=f7f31ab2-8777-43b5-8708-6e6a50443a67 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.694914ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=d74fa32d-289b-4a62-bcf8-4571287fdbb9 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.320824ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=c9b17606-4358-4fe7-9b17-73a26b3b09e7 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:9e69af933faaf1a3e95880c922dae255b645ed9e9e661079b28a89d9f02eaf80" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.677149ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:9e69af933faaf1a3e95880c922dae255b645ed9e9e661079b28a89d9f02eaf80 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=574a0075-10c3-45a5-b1f5-9050e99c3c0e http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.777835ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=0f0ea373-554d-4287-95e6-eb2ab3bc85bf http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.519308ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=c4e2056f-b98e-438b-bb89-2fd7f9d971ea http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.092025ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=cf20c156-a8c9-45f7-81c8-f7a7db6910c9 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.575922ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=739a5ace-94f7-400d-ba33-a30d79ff373c http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.469501ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=79f32466-952d-4e23-b738-1990deca611b http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.607046ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "HEAD /v2/035088a9398b/math/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:38Z" level=warning msg="the ResponseWriter does not implement CloseNotifier" auth.user.name="035088a9398b/admin" http.request.host=registry.thinknode.dev http.request.id=7fd1d803-f407-44c2-bf04-f098074196a3 http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/1.0.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d vars.name="035088a9398b/math" vars.reference=1.0.0 version=v2.1.1 
time="2015-08-14T19:41:38Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=7fd1d803-f407-44c2-bf04-f098074196a3 http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/1.0.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.duration=10.809194ms http.response.status=201 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:38 +0000] "PUT /v2/035088a9398b/math/manifests/1.0.0 HTTP/1.0" 201 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=573d4fb6-0076-4b33-8089-3a4e7e72a3ec http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/1.0.0" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=100.861071ms http.response.status=200 http.response.written=96999 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/035088a9398b/math/manifests/1.0.0 HTTP/1.0" 200 96999 "" ""
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=dbe960cd-713d-4d8b-a15b-ab061235a125 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=4e70149d-043a-4b8f-a50e-dd25885cfc95 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=987ec456-9362-4977-8fa4-b2978555de17 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=8a34b129-e31a-4e08-a73a-f5b86b7cf806 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.226044ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=a5768983-804f-4f33-915b-ebb33a236682 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.68087ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=93dc8540-4a63-4170-b49d-b66e72fe82fb http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:cb76b4e2cf1f70df92e73a92fa316d346cd512b8dfc223f381e8678a9af223db" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.359568ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:cb76b4e2cf1f70df92e73a92fa316d346cd512b8dfc223f381e8678a9af223db HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=7f265850-785a-4a92-90d1-841318919eba http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.048767ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=14bd06f3-9011-452f-b355-5f8952770bc9 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.481554ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=78974458-7670-416d-b525-6ca9912edaaf http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.613557ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=c722768e-66f1-4bb2-bfb9-3ea2493aad2e http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.851351ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=9cd01839-7158-41f5-a430-fda4a356e046 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.30774ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=3c9adf9e-fd3d-45d8-93a0-40584a6a3c87 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.481449ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="the ResponseWriter does not implement CloseNotifier" auth.user.name="035088a9398b/admin" http.request.host=registry.thinknode.dev http.request.id=7ef23170-7e61-4dbd-9831-fac133886c42 http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.1.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d vars.name="035088a9398b/anomaly" vars.reference=0.1.0 version=v2.1.1 
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=7ef23170-7e61-4dbd-9831-fac133886c42 http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.1.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.duration=9.914972ms http.response.status=201 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "PUT /v2/035088a9398b/anomaly/manifests/0.1.0 HTTP/1.0" 201 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=610310d4-b057-4c15-befb-1d085862b11c http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=ed40aaff-4f40-4216-ae4e-8c8a85763a2c http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=c8b26f65-018a-49af-9292-7c2e4bfecc71 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=1275f458-579f-4240-9ecb-e25aa76c0295 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.687424ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=096b3a46-87cd-4fbb-8be2-2aa6e145939e http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.195718ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=db8c560b-6ac6-461e-9c5e-a73d6f098d33 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:9242f0cda51c5f1fccf96360f7492b0be0e376de7afa7639be0a63c137032441" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.889534ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:9242f0cda51c5f1fccf96360f7492b0be0e376de7afa7639be0a63c137032441 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=f23ac3ff-87b4-4828-9c79-7ce405584ed5 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.211515ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=1bfd7eab-5ad3-4b5d-9e90-42217de5685f http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.866571ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=263660f3-f62b-492c-92b1-f0a2bbc5a566 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.834113ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=d676bf39-f329-4cb9-bd75-464b7b65982c http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.839131ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=7b1f73b3-6fdf-4d9f-90e3-9d2af0f9accb http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.792546ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=51f30128-3be1-4997-ba85-90f15dcf544d http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.051222ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="the ResponseWriter does not implement CloseNotifier" auth.user.name="035088a9398b/admin" http.request.host=registry.thinknode.dev http.request.id=400f5dd2-3e4f-4d9a-8de2-3301debeb25e http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.2.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d vars.name="035088a9398b/anomaly" vars.reference=0.2.0 version=v2.1.1 
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=400f5dd2-3e4f-4d9a-8de2-3301debeb25e http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.2.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.duration=10.118276ms http.response.status=201 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "PUT /v2/035088a9398b/anomaly/manifests/0.2.0 HTTP/1.0" 201 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=e210ebee-97cf-4c17-b3f5-600a551bd395 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=3ac5e0cd-c037-4175-bce2-7a1563d40b3d http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=3257b55b-45c3-4edb-aa5b-a1b3095909ec http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=06083780-e7e3-465b-a843-f96f6ac20bf4 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.165859ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=a4aa8e84-a52a-4c6d-87b1-d8e5ed4d67f5 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.905454ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=3803751b-d099-45c2-bca2-fcf0465ea932 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:42f95e55aa126b4baf0630873e3c68d761acb09ef09695cc151d681918415810" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.234667ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:42f95e55aa126b4baf0630873e3c68d761acb09ef09695cc151d681918415810 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=4186333f-f6f1-4e16-ba54-257a70a54940 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=1.590668ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:c8aec311c67437b08a05dd8b3dbfc4a54751ecc6d5d006c1e5b0a053e8abfa8e HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=fdd25c3a-3880-478e-8e09-3d0b7c8023f1 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.567495ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=a101142c-aba1-4f2f-88a6-7ef9dd55daa9 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.181974ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=da00720e-74a9-4e5a-be19-aac69ae0d4e3 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.3151ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:1db09adb5ddd7f1a07b6d585a7db747a51c7bd17418d47e91f901bdf420abd66 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=7ebfcd91-7906-4bb0-9b4b-b1c574dd8ca9 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.841347ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=8c0e9bb8-5d17-48b7-accc-a60d12802ec2 http.request.method=HEAD http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/octet-stream" http.response.duration=2.473159ms http.response.status=200 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "HEAD /v2/035088a9398b/anomaly/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 HTTP/1.0" 200 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=warning msg="the ResponseWriter does not implement CloseNotifier" auth.user.name="035088a9398b/admin" http.request.host=registry.thinknode.dev http.request.id=1658f271-41ac-4ba9-97e6-221bb923edf9 http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.3.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d vars.name="035088a9398b/anomaly" vars.reference=0.3.0 version=v2.1.1 
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=1658f271-41ac-4ba9-97e6-221bb923edf9 http.request.method=PUT http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.3.0" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.duration=9.906968ms http.response.status=201 http.response.written=0 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "PUT /v2/035088a9398b/anomaly/manifests/0.3.0 HTTP/1.0" 201 0 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:39Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=ee32df38-6f40-45e7-8843-710793fc27fd http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.1.0" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=101.353177ms http.response.status=200 http.response.written=97002 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/035088a9398b/anomaly/manifests/0.1.0 HTTP/1.0" 200 97002 "" ""
time="2015-08-14T19:41:40Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=a7e82526-da00-49d7-a916-29f909894902 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=90.949557ms http.response.status=200 http.response.written=97002 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:39 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8 HTTP/1.0" 200 97002 "" ""
time="2015-08-14T19:41:40Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=5f53fcef-5041-47c3-a6ce-40c28d971bed http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.2.0" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=112.259763ms http.response.status=200 http.response.written=97002 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/035088a9398b/anomaly/manifests/0.2.0 HTTP/1.0" 200 97002 "" ""
time="2015-08-14T19:41:40Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=5af34372-2d68-4bbf-b115-fadb5ec31b42 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=90.839912ms http.response.status=200 http.response.written=97002 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884 HTTP/1.0" 200 97002 "" ""
time="2015-08-14T19:41:40Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=cedb58d9-9188-4dda-abc7-5c435a387134 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/0.3.0" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=96.269504ms http.response.status=200 http.response.written=97002 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/035088a9398b/anomaly/manifests/0.3.0 HTTP/1.0" 200 97002 "" ""
time="2015-08-14T19:41:40Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=6685d8cf-b35c-4d55-9204-fa4d26f7f291 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent= http.response.contenttype="application/json; charset=utf-8" http.response.duration=93.243922ms http.response.status=200 http.response.written=97002 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f HTTP/1.0" 200 97002 "" ""
time="2015-08-14T19:41:40Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=61607179-648b-44fd-9c64-392d86e68b06 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:40Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=65e3ddb9-7697-4b4c-b7e4-123169c2672b http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:40Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=c0e96d75-d4fe-40a1-9364-99cb4346d09f http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:41Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=b4ba4fa4-14e5-4b7f-b8c9-d7a111386a3d http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=89.426191ms http.response.status=200 http.response.written=96999 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:40 +0000] "GET /v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8 HTTP/1.0" 200 96999 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:42Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=ec56d976-cafc-4b07-9064-8bc154a6823d http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:42 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:42Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=8f5499c9-d9a1-491c-9a2d-099f80af9577 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:42 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:42Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=835b3e87-84ee-4870-9559-041497fcab2e http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:42 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:42Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=d57aeeb0-f13b-4a6c-88f2-f59c0832fa3c http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=90.469629ms http.response.status=200 http.response.written=96999 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:42 +0000] "GET /v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8 HTTP/1.0" 200 96999 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:43Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=f7397e68-a9df-441a-86ff-af904e99fe1b http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:43 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:43Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=99d387f7-aa4c-4441-ac07-4bcf4f19a60d http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:43 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:43Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=d12741bc-8d8b-4de5-8084-085ce3d3dfdb http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:43 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:43Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=705b29df-2236-4daf-8bbe-b0568d9aba3b http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=96.344074ms http.response.status=200 http.response.written=96999 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:43 +0000] "GET /v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8 HTTP/1.0" 200 96999 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:44Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=903f799f-b5f9-4fcb-befa-ee633998fb49 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:44Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=d9b41995-3911-422f-8fd3-b8a91ddfded4 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:41:44Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=57683639-0387-42a7-b0ae-d2435d3d68eb http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T19:43:28Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=ee814b0d-58d3-4814-8922-778b4c5f9907 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1m44.529400608s http.response.status=200 http.response.written=86282 instance.id=2e703210-9626-4977-aae9-0677ae87691d version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:19:41:44 +0000] "GET /v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8 HTTP/1.0" 200 86282 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"

@stevvooe
Copy link
Collaborator

@kyleburnett So, it looks like the manifest requests are taking up to 100 seconds or more. This explains the nginx timeouts. Increasing the timeout will avoid the errors, but there seems to be a performance problem.

Would it be possible to change to the debug logging level and try the pull again?

@kyleburnett
Copy link
Author

Here are the debug level logs

172.17.42.1 - - [14/Aug/2015:20:14:39 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884 HTTP/1.0" 200 16142 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:14:39.728653922Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=c8bcaa54-b585-41b3-b31e-19293a1090de http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a version=v2.1.1 
time="2015-08-14T20:14:39.728734811Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=c8bcaa54-b585-41b3-b31e-19293a1090de http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:14:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:14:39.737395324Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=682c4d7b-940b-462e-a66f-dbb9c107fdd6 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a version=v2.1.1 
time="2015-08-14T20:14:39.738037511Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=682c4d7b-940b-462e-a66f-dbb9c107fdd6 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:14:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:14:39.747223243Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=fd975429-94e9-4b24-90b4-51e5c813e152 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a version=v2.1.1 
time="2015-08-14T20:14:39.747319792Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=fd975429-94e9-4b24-90b4-51e5c813e152 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:14:39 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:14:39.782472765Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.783056882Z" level=debug msg=GetImageManifest auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.783285332Z" level=debug msg="(*manifestStore).Get" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.783564309Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=142.615µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=0716d290-73d4-4397-b134-034fcb6aa94c trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.791596679Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=160.362µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=2f255e0f-134f-43be-94f2-4da1f194c605 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.792865993Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/4c/4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=1.063212ms trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=0fa37e67-147b-44d5-b00b-f8fcf183d7e0 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.793151451Z" level=debug msg="filesystem.List(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f/signatures/sha256\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=77.74µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=9c5f8172-af1c-43fe-887a-0db92ef8b037 trace.line=123 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.793248654Z" level=debug msg="fetching signature \"sha256:4f639ad86b1b9e07438b42b57ff019e899dcf74b89fe06ca4747b226108acd8d\"" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.793432436Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f/signatures/sha256/4f639ad86b1b9e07438b42b57ff019e899dcf74b89fe06ca4747b226108acd8d/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=74.056µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=92336518-e992-4e3f-bdc9-b59a53e5fa24 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.793615559Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/4f/4f639ad86b1b9e07438b42b57ff019e899dcf74b89fe06ca4747b226108acd8d/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=68.443µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=c5695d94-945e-49c6-b0cc-2ef7fe40e2c6 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:14:39.797695023Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=40e846e4-6d39-43f3-9dad-4eada50d99f2 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=17.53543ms http.response.status=200 http.response.written=16142 instance.id=a197b530-9700-458f-aa00-77c77009768a version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:14:39 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f HTTP/1.0" 200 16142 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:14:42.521504747Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=62.6µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=f8a5a9d0-93b6-4fc2-b3cb-f01af6899944 trace.line=123 version=v2.1.1 
time="2015-08-14T20:14:52.572092759Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=152.627µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=9f479499-7fee-45a8-b545-231f686e6df1 trace.line=123 version=v2.1.1 
time="2015-08-14T20:15:02.521573237Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=99.087µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=01c376a6-3fa0-45c2-a02d-f43497470f1f trace.line=123 version=v2.1.1 
time="2015-08-14T20:15:12.568551231Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=58.385µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=ac6fed2f-9f77-43c7-bd91-2e6ef5f5ec7b trace.line=123 version=v2.1.1 
time="2015-08-14T20:15:22.571828292Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=152.74µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=78b7989a-6d2c-4c78-839a-045dbfee7145 trace.line=123 version=v2.1.1 
time="2015-08-14T20:15:32.52158621Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=119.793µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=fb163028-c70d-424d-9d45-101e028175c5 trace.line=123 version=v2.1.1 
time="2015-08-14T20:15:42.515133072Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=125.312µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=6b2d7f8c-3cd4-42a8-8035-5825eaa8f10f trace.line=123 version=v2.1.1 
time="2015-08-14T20:15:52.521566454Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=127.752µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=807d57ba-d971-40b8-9a42-b1f3b027265f trace.line=123 version=v2.1.1 
time="2015-08-14T20:16:02.573460067Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=685.94µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=3822159b-86d1-427b-916b-f6a68324c02d trace.line=123 version=v2.1.1 
time="2015-08-14T20:16:12.521570113Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=118.917µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=d2874048-a63b-460d-95e1-2557cdf6877f trace.line=123 version=v2.1.1 
time="2015-08-14T20:16:22.521542844Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=133.228µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=406355af-c556-4914-a7a3-acb56f49448e trace.line=123 version=v2.1.1 
time="2015-08-14T20:16:32.511903519Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=72.986µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=c5d3489a-4068-472b-bad2-653d02611b5a trace.line=123 version=v2.1.1 
time="2015-08-14T20:16:42.572893635Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=137.68µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=5863110c-6493-4dbe-a753-7bf08638239b trace.line=123 version=v2.1.1 
time="2015-08-14T20:16:52.521554574Z" level=debug msg="filesystem.List(\"/\")" instance.id=a197b530-9700-458f-aa00-77c77009768a trace.duration=118.434µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=bc10390b-4388-4b78-9d6c-535e1c689b4f trace.line=123 version=v2.1.1

@kyleburnett
Copy link
Author

Could this issue stem from the way our storage backend is configured? Specifically the location /tmp/registry-dev within the docker container?

@kyleburnett
Copy link
Author

We were just looking at more of the logs and thought that this was interesting:

172.17.42.1 - - [14/Aug/2015:20:34:40 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:40.687984075Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.688202798Z" level=debug msg=GetImageManifest auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.688973879Z" level=debug msg="(*manifestStore).Get" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.689120755Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=85.347µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=3ee9742f-b276-4464-a6b6-ea3c51398cf3 trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.689184631Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=40.838µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=5aeb38da-1814-4116-8c0f-6c4a8c759bcc trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.689344329Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/10/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=99.116µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=534a1725-537f-4053-ae95-387a7c94d3a0 trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.689451992Z" level=debug msg="filesystem.List(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/signatures/sha256\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=46.7µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=ec10f5cf-b51d-40e7-a162-55cabf680608 trace.line=123 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.689526119Z" level=debug msg="fetching signature \"sha256:a44a852364a56bce84793fbafac5f8ae308f4aab8ffc17675c4063417455834b\"" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.689658626Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/signatures/sha256/a44a852364a56bce84793fbafac5f8ae308f4aab8ffc17675c4063417455834b/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=38.941µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=999e9c89-e589-47d8-a332-4f89e837e5a7 trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.689724191Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/a4/a44a852364a56bce84793fbafac5f8ae308f4aab8ffc17675c4063417455834b/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=43.477µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=38c0b4ae-1f11-4a7b-b158-3b6c6ff3b4a0 trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:40.693514235Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=28dad185-8590-4387-a9cb-4391ecb201f3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=6.938938ms http.response.status=200 http.response.written=16139 instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:40 +0000] "GET /v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8 HTTP/1.0" 200 16139 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:41.582538356Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=b68e6a5c-3186-4879-9eff-8863c97f893e http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:41.582620098Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=b68e6a5c-3186-4879-9eff-8863c97f893e http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:41 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:41.592615563Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=0759636e-eaaa-4040-b6f6-4c35b2cfccbe http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:41.593525266Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=0759636e-eaaa-4040-b6f6-4c35b2cfccbe http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:41 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:41.602950956Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=c55c29c9-9763-4a80-b63d-4fc71d4e8b6e http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:41.604020647Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=c55c29c9-9763-4a80-b63d-4fc71d4e8b6e http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:41 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:41.63617135Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.637103731Z" level=debug msg=GetImageManifest auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.637165653Z" level=debug msg="(*manifestStore).Get" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.637334728Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=99.677µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=82b110b2-3569-4bfe-b04b-c52bb0fe380d trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.637480104Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=16.959µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=cccd31b3-315f-45a7-abb9-112db13d225d trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.637734131Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/10/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=146.322µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=471f2c12-a0f4-4cee-b5e0-5430bbc6672a trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.637861783Z" level=debug msg="filesystem.List(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/signatures/sha256\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=40.728µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=5c380c9e-f33b-4149-b949-f08a5b0c1cb8 trace.line=123 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.637916815Z" level=debug msg="fetching signature \"sha256:a44a852364a56bce84793fbafac5f8ae308f4aab8ffc17675c4063417455834b\"" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.638128413Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/math/_manifests/revisions/sha256/10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8/signatures/sha256/a44a852364a56bce84793fbafac5f8ae308f4aab8ffc17675c4063417455834b/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=6.693µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=786ecf16-d136-49e0-9b14-c633df5a2b85 trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.638252937Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/a4/a44a852364a56bce84793fbafac5f8ae308f4aab8ffc17675c4063417455834b/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=27.462µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=6bc19ad7-3270-4589-82cc-3f87d0d07712 trace.line=55 vars.name="035088a9398b/math" vars.reference="sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" version=v2.1.1 
time="2015-08-14T20:34:41.643447252Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=5f74e827-a699-4a69-ac32-2acbce49ad80 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=9.348021ms http.response.status=200 http.response.written=16139 instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:41 +0000] "GET /v2/035088a9398b/math/manifests/sha256:10af39c70b7b7f3bc6d9539c38180d67a626c033487e706e5396d3c5af6c58a8 HTTP/1.0" 200 16139 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:43.570995097Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=d1987ab0-ef81-4a8f-881b-ab7b830e3f98 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:43.571072643Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=d1987ab0-ef81-4a8f-881b-ab7b830e3f98 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:43 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:43.578981201Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=5034b949-94b1-4c60-b442-72bd6058b8c8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:43.580005116Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=5034b949-94b1-4c60-b442-72bd6058b8c8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:43 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:43.588266915Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=e1b19a70-1ca7-457c-b461-00569b10b769 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:43.588368993Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=e1b19a70-1ca7-457c-b461-00569b10b769 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:43 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:43.632561885Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.632980964Z" level=debug msg=GetImageManifest auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.633080354Z" level=debug msg="(*manifestStore).Get" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.634262739Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=983.052µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=fbad4132-1219-4819-b19c-df7bc3843c83 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.634539137Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=61.455µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=7936ef28-1bd1-493e-9904-6233cc623c31 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.634656345Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/98/98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=45.614µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=aaedc37a-1187-4bef-ae0f-6316f410c396 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.634754055Z" level=debug msg="filesystem.List(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8/signatures/sha256\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=28.527µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=9e6fd097-a507-4b08-bbbd-615e9cc8649e trace.line=123 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.634877923Z" level=debug msg="fetching signature \"sha256:2603572d303ee04b910adad2b83b4473d2d0c0ec8ec7503e160e5dda9d6a632f\"" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.635081593Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8/signatures/sha256/2603572d303ee04b910adad2b83b4473d2d0c0ec8ec7503e160e5dda9d6a632f/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=62.204µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=beb05113-020e-4528-b36b-99b37c974c23 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.635326469Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/26/2603572d303ee04b910adad2b83b4473d2d0c0ec8ec7503e160e5dda9d6a632f/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=128.048µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=83e87b49-e7df-4465-a425-9c2fc7f7007f trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" version=v2.1.1 
time="2015-08-14T20:34:43.638263756Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=27e1ceb0-4301-4130-8e9a-47b36fa27528 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=8.119008ms http.response.status=200 http.response.written=16142 instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:43 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:98b442e66160824b7945e0a78b74bc82ead0595bcf05409837e8236bc2a04aa8 HTTP/1.0" 200 16142 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:44.251717876Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=52bdfb0f-9bed-4092-aa78-70a31bf9e2d1 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:44.251932787Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=52bdfb0f-9bed-4092-aa78-70a31bf9e2d1 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:44.263720538Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=7f3c8f13-1716-46af-88ab-162bd58f2083 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:44.263960262Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=7f3c8f13-1716-46af-88ab-162bd58f2083 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:44.278713852Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=328d162f-d4c7-4af3-9158-f26ffaa492c8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:44.279765833Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=328d162f-d4c7-4af3-9158-f26ffaa492c8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:44.318491314Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.31904595Z" level=debug msg=GetImageManifest auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.319152781Z" level=debug msg="(*manifestStore).Get" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.319368327Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=111.052µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=fc3d80c4-f0c6-4802-9732-ed6cdd52cb40 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.319593717Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=51.833µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=28dae867-8569-4b42-8723-2910e23771af trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.319883223Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/fc/fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=166.428µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=d3e6b4b1-dbf2-45b9-a07a-d7974c03d879 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.320132539Z" level=debug msg="filesystem.List(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884/signatures/sha256\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=83.849µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=737e0183-9d79-4763-be8f-37bf58904a0d trace.line=123 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.32032223Z" level=debug msg="fetching signature \"sha256:3d58179f4ae7a9855e7a43e7320ea9e1d37faab43ebc6512986b3ecfabdca74e\"" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.320556595Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/repositories/035088a9398b/anomaly/_manifests/revisions/sha256/fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884/signatures/sha256/3d58179f4ae7a9855e7a43e7320ea9e1d37faab43ebc6512986b3ecfabdca74e/link\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=84.503µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=3097fc1e-9106-4da4-b602-e27d221f0613 trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.320782457Z" level=debug msg="filesystem.GetContent(\"/docker/registry/v2/blobs/sha256/3d/3d58179f4ae7a9855e7a43e7320ea9e1d37faab43ebc6512986b3ecfabdca74e/data\")" auth.user.name="system/supervisor" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=63.484µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).GetContent" trace.id=254f6f41-ab4b-48f0-bfd5-8483693a157e trace.line=55 vars.name="035088a9398b/anomaly" vars.reference="sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" version=v2.1.1 
time="2015-08-14T20:34:44.324881252Z" level=info msg="response completed" http.request.host=registry.thinknode.dev http.request.id=d962fbfb-23bd-48c5-b961-d1b7532bfbb8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" http.response.contenttype="application/json; charset=utf-8" http.response.duration=9.437413ms http.response.status=200 http.response.written=16142 instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:44 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:fc19b71e00034ca88a473d32d437b25e0049bae075b2cac4685ecf55f9a4d884 HTTP/1.0" 200 16142 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:44.968307407Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=cd087805-cc13-4cf9-834f-03cbe08c0ae8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:44.968390257Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=cd087805-cc13-4cf9-834f-03cbe08c0ae8 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:44.978023596Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=ca18329f-b42d-4038-8ac8-3bc2ac7eed88 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:44.978149602Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=ca18329f-b42d-4038-8ac8-3bc2ac7eed88 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:44.990016393Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=a3713a18-75a2-4b27-9069-fe5fc09810b3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
time="2015-08-14T20:34:44.990561645Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=a3713a18-75a2-4b27-9069-fe5fc09810b3 http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:44 +0000] "GET /v2/ HTTP/1.0" 401 114 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:50.351270817Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=83.252µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=56f7f19f-1ec6-4d8c-b7ec-58adcd3d36b7 trace.line=123 version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:54 +0000] "GET /v1/_ping HTTP/1.0" 404 19 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:34:55.002477804Z" level=debug msg="authorizing request" http.request.host=registry.thinknode.dev http.request.id=7ba11607-cccc-45a8-a08f-3f9e118c139c http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
time="2015-08-14T20:34:55.002574972Z" level=warning msg="error authorizing context: authorization token required" http.request.host=registry.thinknode.dev http.request.id=7ba11607-cccc-45a8-a08f-3f9e118c139c http.request.method=GET http.request.remoteaddr=127.0.0.1 http.request.uri="/v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" http.request.useragent="docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 vars.name="035088a9398b/anomaly" vars.reference="sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f" version=v2.1.1 
172.17.42.1 - - [14/Aug/2015:20:34:55 +0000] "GET /v2/035088a9398b/anomaly/manifests/sha256:4cac9a7dc1992cee6db848575ab11771e93d0759f73cdf63aacd93e21e2f869f HTTP/1.0" 401 179 "" "docker/1.7.0 go/go1.4.2 git-commit/0baf609 kernel/3.13.0-55-generic os/linux arch/amd64"
time="2015-08-14T20:35:00.346352596Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=144.85µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=e2659dcc-82ed-46fa-865f-65cfc4a73642 trace.line=123 version=v2.1.1 
time="2015-08-14T20:35:10.351305028Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=142.779µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=4b5ad2b6-1b8a-4cbe-886a-bda992532bcb trace.line=123 version=v2.1.1 
time="2015-08-14T20:35:20.351274924Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=79.764µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=ce41f4e2-1efd-4b03-801f-f4f57c360c4c trace.line=123 version=v2.1.1 
time="2015-08-14T20:35:30.351274836Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=85.364µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=8c451917-3cfd-4a8c-b37e-e328e018cc7d trace.line=123 version=v2.1.1 
time="2015-08-14T20:35:40.351255028Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=114.258µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=337ca348-c96f-4a4a-8314-1176dba6152a trace.line=123 version=v2.1.1 
time="2015-08-14T20:35:50.344535828Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=121.559µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=f9497c4f-9428-4409-8d50-0965f04ce1c0 trace.line=123 version=v2.1.1 
time="2015-08-14T20:36:00.351251854Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=166.978µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=6fff6e1a-a9a5-4567-9644-a1bda443bcaa trace.line=123 version=v2.1.1 
time="2015-08-14T20:36:10.348380658Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=216.774µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=f7e7b3cc-b27b-489c-b7fd-4e57acc09239 trace.line=123 version=v2.1.1 
time="2015-08-14T20:36:20.351253376Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=71.254µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=a60711d8-cf76-46c0-b7fa-b288fa1ab618 trace.line=123 version=v2.1.1 
time="2015-08-14T20:36:30.345599358Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=230.719µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=d0052d5e-dbd0-4ace-af9e-0452455b13eb trace.line=123 version=v2.1.1 
time="2015-08-14T20:36:40.35119562Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=78.744µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=9f2f316e-794e-4727-b050-6022c5031562 trace.line=123 version=v2.1.1 
time="2015-08-14T20:36:50.344492276Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=84.104µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=0c92857c-e342-4d95-8d84-52eaebc074ce trace.line=123 version=v2.1.1 
time="2015-08-14T20:37:00.351237252Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=78.215µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=d11dee39-4d46-42de-89f5-9aa5094a7212 trace.line=123 version=v2.1.1 
time="2015-08-14T20:37:10.351251299Z" level=debug msg="filesystem.List(\"/\")" instance.id=5406aeac-b05a-42a9-b80c-1e357a6e1eb1 trace.duration=68.71µs trace.file="/go/src/github.com/docker/distribution/registry/storage/driver/base/base.go" trace.func="github.com/docker/distribution/registry/storage/driver/base.(*Base).List" trace.id=27653ab6-b547-41da-b107-cdef01d97264 trace.line=123 version=v2.1.1

It tries to do a GET /v2/ request, receives a 401 and then falls back to GET /v1/_ping

@dmp42
Copy link
Contributor

dmp42 commented Aug 14, 2015

@kyleburnett this last behavior is expected (fallback to v1 on v2 failure).
Failure to authenticate might be the culprit in this specific case.

@kyleburnett
Copy link
Author

Trying some new things this morning. We upgraded docker to the latest version. Here is our new docker version information:

Client:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:35:49 UTC 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:        Thu Aug 13 02:35:49 UTC 2015
 OS/Arch:      linux/amd64

We also changed the registry to use silly authentication.

The errors are a bit different, but they still seem to be related to timeouts. Again, the errors happen intermittently. Running docker using sudo docker -d -D, we get the following errors:

DEBU[1380] Error trying v2 registry: Received unexpected HTTP status: 504 Gateway Time-out 
DEBU[1380] Trying to pull registry.thinknode.dev/035088a9398b/anomaly from https://registry.thinknode.dev v1 
DEBU[1380] Trying to pull registry.thinknode.dev/035088a9398b/anomaly from http://registry.thinknode.dev v2 
DEBU[1380] Error getting v2 registry: endpoint does not support v2 API 
DEBU[1380] Trying to pull registry.thinknode.dev/035088a9398b/anomaly from http://registry.thinknode.dev v1 
ERRO[1380] Handler for POST /images/create returned error: endpoint does not support v2 API 
ERRO[1380] HTTP Error                                    err=endpoint does not support v2 API statusCode=500

@dmp42
Copy link
Contributor

dmp42 commented Aug 17, 2015

@kyleburnett: 504 Gateway Time-out indicates your (nginx) proxy is misconfigured and cannot talk to your registry (or your registry is down).
Start there.

Also, disable all sort of authentication, and validate that your registry is working that way.

Let me know.

@stevvooe
Copy link
Collaborator

The "random id" is intentional.

@kyleburnett I really don't recommend random repository names. The registry is not designed to cope with a high cardinality of repositories. How many random ids are you generating per day?

@kyleburnett
Copy link
Author

@dmp42 Ok, here is where I'm at. I have upgraded docker from 1.7.0 to 1.8.1 (the latest). I added these lines in the nginx configuration.

  proxy_connect_timeout       60;
  proxy_send_timeout          60;
  proxy_read_timeout          60;
  send_timeout                60;

It still works sometimes. Other times, it times out with the following error in the nginx error.log file:

2015/08/17 19:51:41 [error] 20342#0: *279 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: registry.thinknode.dev, request: "GET /v2/035088a9398b/anomaly/manifests/sha256:2840976d932a9578d25225c27afeeb4af3e37d67ab95ecdd8fb1f5d2935466bd HTTP/1.1", upstream: "http://127.0.0.1:5000/v2/035088a9398b/anomaly/manifests/sha256:2840976d932a9578d25225c27afeeb4af3e37d67ab95ecdd8fb1f5d2935466bd", host: "registry.thinknode.dev"

When the timeout hits, I see the the following error in the docker log:

ERRO[3523] Handler for POST /images/create returned error: endpoint does not support v2 API 
ERRO[3523] HTTP Error                                    err=endpoint does not support v2 API statusCode=500

because nginx has returned a 404.

127.0.0.1 - - [17/Aug/2015:20:06:22 +0000] "GET /v2/ HTTP/1.1" 404 142 "-" "docker/1.8.1 go/go1.4.2 git-commit/d12ea79 kernel/3.13.0-55-generic os/linux arch/amd64"

Seems like the registry just stops responding.

@stevvooe We're running a test suite, so we set up an account with a random name (the "random id" that you see). In a production environment, accounts would correspond to real entities that would be created very infrequently.

@kyleburnett
Copy link
Author

Also I have removed the auth section from the config.yml.

@kyleburnett
Copy link
Author

@stevvooe and @dmp42
So, after quite a bit of trial-and-error (and a little bit of luck), it seems as though the issue is fixed if I change the storage: cache: layerinfo setting in the config.yml from inmemory to redis (specifically, I also changed layerinfo to blobdescriptor, though I believe those two are synonymous). I'm not sure why this fixes the issue, but after making this change, our tests are reliably passing without incident. Perhaps the inmemory caching was having an issue under heavy load during our automated test suite?

Either way, I'm closing this issue since there is really no reason to not use redis for caching (that is obviously what we were planning to use in production, so it isn't a big deal to use it in development as well).

Thanks!

@depay
Copy link

depay commented Aug 3, 2016

@kyleburnett i think there may be something wrong with inmemory cache..i'm also using imemory on my registry proxy. I found a problem that the proxy's scheduler removed the expired blob, but its info still existed in cache, which leading client pulling blob with failure. Then i remove cache settings in config.yml to solve it.

i''m using distribution v2.2.1.

@stevvooe
Copy link
Collaborator

stevvooe commented Aug 3, 2016

@depay There may be a cache invalidation bug.

@RichardScothern Do we have an issue open for that?

@RichardScothern
Copy link
Contributor

There was a bug, it was fixed and I replied in the other thread which @depay raised this issue in.

@depay
Copy link

depay commented Aug 4, 2016

@stevvooe @RichardScothern thanks and i saw the reply there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@stevvooe @dmp42 @RichardScothern @kyleburnett @depay