Shorter update for diffs in huge snapshots

[matheus23]

What you're trying to do

I'm writing snapshot tests for a code generator. These snapshots are just strings.

Why you can't use AVA for this

The snapshots are ~50k lines of code. If there's some line diff in there, AVA prints the whole snapshot. This fills my terminal buffer so much, I can't even scroll to the actual diff anymore.

And maybe how you think AVA could handle this

I've switched to AVA from Jest (yay!). If the file was big, jest would only print parts of the file with some lines of context lines above and below differences, similar to the unix diff command.

I don't think everyone prefers this, so ideally it'd become a setting in the .snapshot function.

[novemberborn]

I think this is an issue for https://github.com/concordancejs/concordance/ which generates the diffs. Would you mind opening it there?

Some questions we'd need to resolve:

• How would we indicate that lines are elided? Would we indicate how many?
• What if there are multiple sections that contain changes, but other sections are the same?
• Would you want the diff to "give up" when there are too many changes, even after eliding? After how many lines should that kick in?

Perhaps some of those things could be configurable on a a per-assertion basis but we'd need to support it in Concordance first.

(I'm closing this issue for housekeeping purposes, but let's keep the conversation going.)

[matheus23]

How would we indicate that lines are elided? Would we indicate how many?
What if there are multiple sections that contain changes, but other sections are the same?

Here's an example of how jest outputs diffs:

- Snapshot  -  2
+ Received  + 34

@@ -1,7 +1,38 @@
- "module Tailwind.WithAutoprefixer exposing (..)
+ Object {
+   "breakpointsModule": "module Tailwind.WithAutoprefixer.Breakpoints exposing (..)
+
+ import Css
+ import Css.Media
+
+
+ sm : List Css.Style -> Css.Style
+ sm =
+     Css.Media.withMediaQuery [ \"(min-width: 640px)\" ]
+
+
+ md : List Css.Style -> Css.Style
+ md =
+     Css.Media.withMediaQuery [ \"(min-width: 768px)\" ]
+
+
+ lg : List Css.Style -> Css.Style
+ lg =
+     Css.Media.withMediaQuery [ \"(min-width: 1024px)\" ]
+
+
+ xl : List Css.Style -> Css.Style
+ xl =
+     Css.Media.withMediaQuery [ \"(min-width: 1280px)\" ]
+

+ xxl : List Css.Style -> Css.Style
+ xxl =
+     Css.Media.withMediaQuery [ \"(min-width: 1536px)\" ]
+ ",
+   "utilitiesModule": "module Tailwind.WithAutoprefixer.Utilities exposing (..)
+
  import Css
  import Css.Global
  import Css.Media


@@ -32865,6 +32896,7 @@


  animate_bounce : Css.Style
  animate_bounce =
      Css.property \"animation\" \"bounce 1s infinite\"
- "
+ ",
+ }

As you can see, it uses a standard syntax for eliding unchanged lines from a diff: @@ -32865,6 +32896,7 @@ (even highlight.js understands this format!).

This format seems to be called unified diff format.

Would you want the diff to "give up" when there are too many changes, even after eliding?

No. While it's true that the biggest use-case for this is when two huge files are mostly similar, except for one or two small differences, I still want the output to contain all information for the difference. If there's so much diff that the output becomes long, so be it. I feel it would be confusing/something unpredicted if it wouldn't output the whole diff, even with a message.

After how many lines should that kick in?

It looks like by default, jest prints 5 lines above and below each diff block, which is a default setting I never thought of changing :)

I took a look at the GNU diff tool (man diff). Its default is 3 lines of "context", but personally I like having 5 lines of context. (I think this could/should become a setting anyway.)

I think this is an issue for https://github.com/concordancejs/concordance/ which generates the diffs. Would you mind opening it there?
(I'm closing this issue for housekeeping purposes, but let's keep the conversation going.)

Should we keep the conversation going here and create an issue in concordance once we've found a good solution, or should I create an issue right away?

[novemberborn]

I've converted this to a discussion, so we can hash it out here. Thanks for your input so far.

The headers make sense to me, though what would that look like if the string is inside some other structure? Indent the headers as well?

Five lines of context seems reasonable. I don't expect we'd need it to be configurable.

If, say, 10 lines are shared before the diff kicks in, do you think we should print all those lines? Or always cut off at the context threshold? Should that threshold be somewhat variable?

[matheus23]

I'd say the algorithm should be as stupid as possible. If one wants more context, you'd re-run the test without context collapsing (or a different number of context lines, in case it is configurable).

[novemberborn]

Thanks. I think this covers it: concordancejs/concordance#68