Tests for New Lexers

[pyrmont]

As part of the effort to tidy up outstanding pull requests and merge them into a new branch, I've found myself coming across two issues relating to PRs for new lexers: one specific and one broad.

First, what should new lexers do if they define a filename that conflicts with a filename in an existing lexer? Should any PR that adds a lexer with a conflicting filename implement the self.detect? method on the new lexer? Does it need to implement the same method in the other lexer as well?

Second, is there a clear position about what tests new lexers should implement? I've looked through various lexers that are part of Rouge and there doesn't seem to be a consistent pattern. Most seem to test all filenames that are defined. Most seem to test all mimetypes that are defined. After that, it becomes a bit of a free for all.

/cc @jneen @dblessing @ashmaroli @vidarh

[pyrmont]

Based on discussion in this issue it sounds like the tests that should be implemented by a lexer are:

1. a spec setting our expectations about filename globs, mimetypes and (if applicable) sources;
2. a minimal demo file; and
3. a more extensive visual sample.

On this assumption, I've drafted up documentation for lexer development and submitted it as #1111.