FEATURE: Snapshots (#459)

* Commit 1 * Limit the number of snapshots groups to 300 by default * Add test case for snapshots custom fields * Display custom fields on snapshot page * Simplify/change snapshots storage * Correct key instance var name * Pass arguments in correct order to method * Some changes to snapshot page * Minor changes * Remove unnecessary string sub call * Cache counter script * Documnet snapshots in README * Add some assertions * Document more things
MiniProfiler · Aug 31, 2020 · 8fbc5bf · 8fbc5bf
1 parent 79ec6c4
commit 8fbc5bf
Show file tree

Hide file tree

Showing 20 changed files with 1,224 additions and 62 deletions.
diff --git a/README.md b/README.md
@@ -186,6 +186,18 @@ There are two additional `pp` options that can be used to analyze memory which d
 * Use `?pp=profile-gc` to report on Garbage Collection statistics
 * Use `?pp=analyze-memory` to report on ObjectSpace statistics
 
+### Snapshots Sampling
+
+In a complex web application, it's possible for a request to trigger rare conditions that result in poor performance. Mini Profiler ships with a feature to help detect those rare conditions and fix them. It works by enabling invisible profiling on one request every N requests, and saving the performance metrics that are collected during the request (a.k.a snapshot of the request) so that they can be viewed later. To turn this feature on, set the `snapshot_every_n_requests` config to a value larger than 0. The larger the value is, the less frequently requests are profiled.
+
+Mini Profiler will exclude requests that are made to skippd paths (see `skip_paths` config below) from being sampled. Additionally, if profiling is enabled for a request that later finishes with a non-2xx status code, Mini Profiler will discard the snapshot and not save it (this behavior may change in the future).
+
+After enabling snapshots sampling, you can see the snapshots that have been collected at `/mini-profiler-resources/snapshots` (or if you changed the `base_url_path` config, substitute `mini-profiler-resources` with your value of the config). You'll see on that page a table where each row represents a group of snapshots with the duration of the worst snapshot in that group. The worst snapshot in a group is defined as the snapshot whose request took longer than all of the snapshots in the same group. Snapshots grouped by HTTP method and path of the request, and if your application is a Rails app, Mini Profiler will try to convert the path to `controller#action` and group by that instead of request path. Clicking on a group will display the snapshots of that group sorted from worst to best. From there, you can click on a snapshot's ID to see the snapshot with all the performance metrics that were collected.
+
+Access to the snapshots page is restricted to only those who can see the speed badge on their own requests, see the section below this one about access control.
+
+Mini Profiler will keep a maximum of 1000 snapshots by default, and you can change that via the `snapshots_limit` config. When snapshots reach the configured limit, Mini Profiler will save a new snapshot only if it's worse than at least one of the existing snapshots and delete the best one (i.e. the snapshot whose request took the least time compared to other snapshots).
+
 ## Access control in non-development environments
 
 rack-mini-profiler is designed with production profiling in mind. To enable that run `Rack::MiniProfiler.authorize_request` once you know a request is allowed to profile.
@@ -370,6 +382,8 @@ html_container|`body`|The HTML container (as a jQuery selector) to inject the mi
 show_total_sql_count|`false`|Displays the total number of SQL executions.
 enable_advanced_debugging_tools|`false`|Enables sensitive debugging tools that can be used via the UI. In production we recommend keeping this disabled as memory and environment debugging tools can expose contents of memory that may contain passwords.
 assets_url|`nil`|See the "Register MiniProfiler's assets in the Rails assets pipeline" section above.
+snapshot_every_n_requests|`-1`|Determines how frequently snapshots are taken. See the "Snapshots Sampling" above for more details.
+snapshots_limit|`1000`|Determines how many snapshots Mini Profiler is allowed to keep.
 
 ### Using MiniProfiler with `Rack::Deflate` middleware
 

diff --git a/Rakefile b/Rakefile
@@ -16,8 +16,10 @@ require 'rspec/core'
 require 'rspec/core/rake_task'
 RSpec::Core::RakeTask.new(:spec) do |spec|
   pattern = ARGV[1] || 'spec/**/*_spec.rb'
-  spec.pattern = FileList[pattern]
+  excluded = 'spec/support/*.rb'
+  spec.pattern = FileList[pattern] - FileList[excluded]
   spec.verbose = false
+  # spec.rspec_opts = ["-p"] # turns on profiling
 end
 
 desc "builds a gem"
@@ -26,7 +28,7 @@ task build: :update_asset_version do
 end
 
 desc "compile sass"
-task compile_sass: :copy_files do
+task :compile_sass do
   require "sassc"
   scss = File.read("lib/html/includes.scss")
   css = SassC::Engine.new(scss).render
@@ -120,8 +122,3 @@ task :client_dev do
 rescue Interrupt
   listener.stop
 end
-
-desc "copy files from other parts of the tree"
-task :copy_files do
-  # TODO grab files from MiniProfiler/UI
-end
diff --git a/lib/html/includes.css b/lib/html/includes.css
@@ -1,9 +1,20 @@
 @charset "UTF-8";
+.mp-snapshots,
 .profiler-result,
 .profiler-queries {
   color: #555;
   line-height: 1;
   font-size: 12px; }
+  .mp-snapshots pre,
+  .mp-snapshots code,
+  .mp-snapshots label,
+  .mp-snapshots table,
+  .mp-snapshots tbody,
+  .mp-snapshots thead,
+  .mp-snapshots tfoot,
+  .mp-snapshots tr,
+  .mp-snapshots th,
+  .mp-snapshots td,
   .profiler-result pre,
   .profiler-result code,
   .profiler-result label,
@@ -33,22 +44,33 @@
     background-color: transparent;
     overflow: visible;
     max-height: none; }
+  .mp-snapshots table,
   .profiler-result table,
   .profiler-queries table {
     border-collapse: collapse;
     border-spacing: 0; }
+  .mp-snapshots a,
+  .mp-snapshots a:hover,
   .profiler-result a,
   .profiler-result a:hover,
   .profiler-queries a,
   .profiler-queries a:hover {
     cursor: pointer;
     color: #0077cc; }
+  .mp-snapshots a,
   .profiler-result a,
   .profiler-queries a {
     text-decoration: none; }
+    .mp-snapshots a:hover,
     .profiler-result a:hover,
     .profiler-queries a:hover {
       text-decoration: underline; }
+  .mp-snapshots .custom-fields-title,
+  .profiler-result .custom-fields-title,
+  .profiler-queries .custom-fields-title {
+    color: #555;
+    font: Helvetica, Arial, sans-serif;
+    font-size: 14px; }
 
 .profiler-result {
   font-family: Helvetica, Arial, sans-serif; }
@@ -407,3 +429,15 @@
     background: #ffffbb; }
   100% {
     background: #fff; } }
+
+.mp-snapshots {
+  font-family: Helvetica, Arial, sans-serif;
+  font-size: 16px; }
+  .mp-snapshots .snapshots-table thead {
+    background: #6a737c;
+    color: #ffffff; }
+  .mp-snapshots .snapshots-table th, .mp-snapshots .snapshots-table td {
+    padding: 5px;
+    box-sizing: border-box; }
+  .mp-snapshots .snapshots-table th {
+    border-right: 1px solid #ffffff; }
diff --git a/lib/html/includes.js b/lib/html/includes.js
@@ -673,6 +673,19 @@ var MiniProfiler = (function() {
     });
   };
 
+  var initSnapshots = function initSnapshots(dataElement) {
+    var data = JSON.parse(dataElement.textContent);
+    var temp = document.createElement("DIV");
+    if (data.page === "overview") {
+      temp.innerHTML = MiniProfiler.templates.snapshotsGroupsList(data);
+    } else if (data.group_name) {
+      temp.innerHTML = MiniProfiler.templates.snapshotsList(data);
+    }
+    Array.from(temp.children).forEach(function (child) {
+      document.body.appendChild(child);
+    });
+  };
+
   var initControls = function initControls(container) {
     if (options.showControls) {
       var _controls = document.createElement("div");
@@ -1003,6 +1016,12 @@ var MiniProfiler = (function() {
       })();
 
       var doInit = function doInit() {
+        var snapshotsElement = document.getElementById("snapshots-data");
+        if (snapshotsElement != null) {
+          initSnapshots(snapshotsElement);
+          return;
+        }
+
         // when rendering a shared, full page, this div will exist
         container = document.querySelectorAll(".profiler-result-full");
 
@@ -1396,6 +1415,31 @@ var MiniProfiler = (function() {
     },
     showTotalSqlCount: function showTotalSqlCount() {
       return options.showTotalSqlCount;
+    },
+    timestampToRelative: function timestampToRelative(timestamp) {
+      var now = Math.round((new Date()).getTime() / 1000);
+      timestamp = Math.round(timestamp / 1000);
+      var diff = now - timestamp;
+      if (diff < 60) {
+        return "< 1 minute";
+      }
+      var buildDisplayTime = function buildDisplayTime(num, unit) {
+        var res = num + " " + unit;
+        if (num !== 1) {
+          res += "s";
+        }
+        return res;
+      }
+      diff = Math.round(diff / 60);
+      if (diff <= 60) {
+        return buildDisplayTime(diff, "minute");
+      }
+      diff = Math.round(diff / 60);
+      if (diff <= 24) {
+        return buildDisplayTime(diff, "hour");
+      }
+      diff = Math.round(diff / 24);
+      return buildDisplayTime(diff, "day");
     }
   };
 })();

diff --git a/lib/html/includes.scss b/lib/html/includes.scss
@@ -14,9 +14,10 @@ $codeFonts: Consolas, monospace, serif;
 $zindex: 2147483640; // near 32bit max 2147483647
 
 // do some resets
+.mp-snapshots,
 .profiler-result,
 .profiler-queries {
-  color: #555;
+  color: $textColor;
   line-height: 1;
   font-size: 12px;
 
@@ -55,6 +56,11 @@ $zindex: 2147483640; // near 32bit max 2147483647
       text-decoration: underline;
     }
   }
+  .custom-fields-title {
+    color: $textColor;
+    font: $normalFonts;
+    font-size: 14px;
+  }
 }
 
 // styles shared between popup view and full view
@@ -199,7 +205,7 @@ $zindex: 2147483640; // near 32bit max 2147483647
 
     th {
       background-color: #fff;
-      border-bottom: 1px solid #555;
+      border-bottom: 1px solid $textColor;
       font-weight: bold;
       padding: 15px;
       white-space: nowrap;
@@ -452,7 +458,7 @@ $zindex: 2147483640; // near 32bit max 2147483647
     line-height: 18px;
     overflow: auto;
 
-    @include box-shadow(0px, 1px, 15px, #555);
+    @include box-shadow(0px, 1px, 15px, $textColor);
 
     .profiler-info {
       margin-bottom: 3px;
@@ -592,7 +598,7 @@ $zindex: 2147483640; // near 32bit max 2147483647
       }
       th {
         font-size: 16px;
-        color: #555;
+        color: $textColor;
         line-height: 20px;
       }
 
@@ -617,3 +623,22 @@ $zindex: 2147483640; // near 32bit max 2147483647
     background: #fff;
   }
 }
+
+.mp-snapshots {
+  font-family: $normalFonts;
+  font-size: 16px;
+
+  .snapshots-table {
+    thead {
+      background: #6a737c;
+      color: #ffffff;
+    }
+    th, td {
+      padding: 5px;
+      box-sizing: border-box;
+    }
+    th {
+      border-right: 1px solid #ffffff;
+    }
+  }
+}
diff --git a/lib/html/includes.tmpl b/lib/html/includes.tmpl
@@ -92,6 +92,19 @@
             </tfoot>
           </table>
         {{?}}
+        {{? it.custom_fields && Object.keys(it.custom_fields).length > 0 }}
+          <p class="custom-fields-title">Snapshot custom fields</p>
+          <table class="profiler-timings">
+            <tbody>
+              {{~ Object.keys(it.custom_fields) :key }}
+                <tr>
+                  <td class="profiler-label">{{= key }}</td>
+                  <td class="profiler-label">{{= it.custom_fields[key] }}</td>
+                </tr>
+              {{~}}
+            </tbody>
+          </table>
+        {{?}}
       </div>
     </div>
 
@@ -216,3 +229,58 @@
     </td>
   </tr>
 </script>
+
+<script id="snapshotsGroupsList" type="text/x-dot-tmpl">
+  {{? it.list && it.list.length }}
+    <table class="snapshots-table">
+      <thead>
+        <tr>
+          <th>Requests Group</th>
+          <th>Worst Time (ms)</th>
+        </tr>
+      </thead>
+      <tbody>
+        {{~ it.list :row}}
+          <tr>
+            <td><a href="{{= row.url }}">{{= row.name }}</a></td>
+            <td>{{= MiniProfiler.formatDuration(row.worst_score) }}</td>
+          </tr>
+        {{~}}
+      </tbody>
+    </table>
+  {{??}}
+    <h2>No snapshots exist</h2>
+  {{?}}
+</script>
+
+<script id="snapshotsList" type="text/x-dot-tmpl">
+  {{? it.list && it.list.length }}
+    <h2>Snapshots for {{= it.group_name }}</h2>
+    <table class="snapshots-table">
+      <thead>
+        <tr>
+          <th>ID</th>
+          <th>Duration (ms)</th>
+          <th>Age</th>
+        </tr>
+      </thead>
+      <tbody>
+        {{~ it.list :row}}
+          <tr>
+            <td><a href="{{= row.url }}">
+              {{= row.id }}
+            </a></td>
+            <td>{{= MiniProfiler.formatDuration(row.duration) }}</td>
+            <td>
+              {{? row.timestamp }}
+                {{= MiniProfiler.timestampToRelative(row.timestamp) }}
+              {{?}}
+            </td>
+          </tr>
+        {{~}}
+      </tbody>
+    </table>
+  {{??}}
+    <h2>No snapshots for {{= it.group_name }}</h2>
+  {{?}}
+</script>