Need clarification on data design / filtering

[halostatue]

I’m trying to understand the decisions made when building the content_type_mime.db file, because I’m embarking on a fairly major upgrade to the data in mime-types-data (https://github.com/mime-types/mime-types-data/tree/priority-extensions) where I’ve made it possible to specify relative extension priorities.

As far as I can tell, the ext_mime.db file looks to be mostly OK. It’s helping me find some issues (font/ttf should be the highest priority result for .ttf, which it isn’t).

However, it looks like the prioritization code I have put in place will break content_type_mime.db in unpleasant ways.

I’m using my translation of the conversion code from your own Rakefile (also in the same tree), but here’s some of the odder diffs:

+bin         application/java-vm                                                       base64          
+txt         application/winhlp                                                        base64          
+t           application/x-troff-man                                                   8bit            
+t           application/x-troff-me                                                    base64          
+t           application/x-troff-ms                                                    base64          

The first should be class, but the code at https://github.com/discourse/mini_mime/blob/main/Rakefile#L77-L80 forces the first extension from application/octet-stream (because application/java-vm no longer sorts first for class).

There are a couple of other examples like this, but I’m trying to figure out what the intent for this particular file is in order to provide (a) recommendations for updates to your own code and (b) convert it correctly myself.

[halostatue]

I have made changes to the referenced branch and ruby-mime-types (the two changes go hand-in-hand) that have made this less of a problem, but there are changes that would be generated from this reconstruction.

Here’s the simplified difference for ext_mime.db:

-ai          application/pdf                                                           base64          
+ai          application/postscript                                                    8bit            
-asc         application/pgp-signature                                                 base64          
+asc         text/plain                                                                quoted-printable
-bat         application/x-msdos-program                                               base64          
+bat         application/x-msdownload                                                  base64          
-bdm         application/vnd.syncml.dm+wbxml                                           base64          
+bdm         video/MP2T                                                                base64          
+cjs         application/javascript                                                    8bit            
-cmd         application/x-msdos-program                                               base64          
+cmd         application/x-msdownload                                                  base64          
-com         application/x-msdos-program                                               base64          
+com         application/x-msdownload                                                  base64          
-cu          application/cu-seeme                                                      base64          
+cu          application/x-cu-seeme                                                    base64          
-doc         application/msword                                                        base64          
+doc         text/plain                                                                quoted-printable
-dtd         application/xml-dtd                                                       8bit            
+dtd         application/xml                                                           8bit            
-exe         application/x-ms-dos-executable                                           base64          
+exe         application/octet-stream                                                  base64          
-mid         audio/midi                                                                base64          
-midi        audio/midi                                                                base64          
+mid         audio/sp-midi                                                             base64          
+midi        audio/sp-midi                                                             base64          
-mp2         audio/mpeg                                                                base64          
+mp2         video/mpeg                                                                base64          
-mp4         application/mp4                                                           base64          
+mp4         audio/mp4                                                                 base64          
-mpg4        application/mp4                                                           base64          
+mpg4        audio/mp4                                                                 base64          
-mts         model/vnd.mts                                                             base64          
+mts         video/MP2T                                                                base64          
-odc         application/vnd.oasis.opendocument.chart                                  base64          
-odf         application/vnd.oasis.opendocument.formula                                base64          
+odc         application/vnd.oasis.opendocument.chart-template                         base64          
+odf         application/vnd.oasis.opendocument.formula-template                       base64          
-odi         application/vnd.oasis.opendocument.image                                  base64          
+odi         application/vnd.oasis.opendocument.image-template                         base64          
+p7z         application/pkcs7-mime                                                    base64          
-ps1         application/x-msdos-program                                               base64          
+ps1         application/x-msdownload                                                  base64          
-reg         application/x-msdos-program                                               base64          
+reg         application/x-msdownload                                                  base64          
-rtx         text/richtext                                                             8bit            
+rtx         application/rtx                                                           base64          
-sgml        application/sgml                                                          base64          
+sgml        text/sgml                                                                 quoted-printable
+sml         application/smil+xml                                                      8bit            
-spp         application/scvp-vp-response                                              base64          
+spp         application/vnd.sealed.ppt                                                base64          
-vbs         application/x-msdos-program                                               base64          
+vbs         application/x-msdownload                                                  base64          
-wks         application/vnd.lotus-1-2-3                                               base64          
+wks         application/vnd.ms-works                                                  base64          
-wmz         application/x-ms-wmz                                                      base64          
+wmz         application/x-msmetafile                                                  base64          
-xlsm        application/vnd.ms-excel.sheet.macroEnabled.12                            base64          
+xlsm        application/vnd.openxmlformats-officedocument.spreadsheetml.sheet         base64          
-xsd         text/xml                                                                  8bit            
+xsd         application/xml                                                           8bit            

Here’s the difference for content_type_mime.db:

-cu          application/cu-seeme                                                      base64          
+rtx         application/rtx                                                           base64          
-spp         application/scvp-vp-response                                              base64          
-sgml        application/sgml                                                          base64          
-smi         application/smil+xml                                                      8bit            
+smil        application/smil+xml                                                      8bit            
-xlsm        application/vnd.ms-excel.sheet.macroEnabled.12                            base64          
-odc         application/vnd.oasis.opendocument.chart                                  base64          
-odc         application/vnd.oasis.opendocument.chart-template                         base64          
+otc         application/vnd.oasis.opendocument.chart-template                         base64          
-odf         application/vnd.oasis.opendocument.formula                                base64          
-odf         application/vnd.oasis.opendocument.formula-template                       base64          
+odft        application/vnd.oasis.opendocument.formula-template                       base64          
-odi         application/vnd.oasis.opendocument.image                                  base64          
-odi         application/vnd.oasis.opendocument.image-template                         base64          
+oti         application/vnd.oasis.opendocument.image-template                         base64          
-bdm         application/vnd.syncml.dm+wbxml                                           base64          
-exe         application/x-ms-dos-executable                                           base64          
-wmz         application/x-ms-wmz                                                      base64          
-cmd         application/x-msdos-program                                               base64          
-dtd         application/xml-dtd                                                       8bit            
-mp4         audio/mp4                                                                 base64          
+m4a         audio/mp4                                                                 base64          
+mid         audio/sp-midi                                                             base64          
-igs         model/iges                                                                base64          
-msh         model/mesh                                                                base64          
+iges        model/iges                                                                base64          
+mesh        model/mesh                                                                base64          
-mts         model/vnd.mts                                                             base64          
-x_b         model/vnd.parasolid.transmit.binary                                       base64          
-x_t         model/vnd.parasolid.transmit.text                                         quoted-printable
+xmt_bin     model/vnd.parasolid.transmit.binary                                       base64          
+xmt_txt     model/vnd.parasolid.transmit.text                                         quoted-printable
-wrl         model/vrml                                                                base64          
+vrml        model/vrml                                                                base64          
-rtx         text/richtext                                                             8bit            
-xml         text/xml                                                                  8bit            

I believe that all of these will result in better matches for users of these libraries.

[halostatue]

Bump. I’d like to make these changes, but I don’t want to produce something that will be negative for users.

[SamSaffron]

So sorry .. .this has dropped off my list.

My general thinking is that I just wanted 100% parity and was totally happy for mime-types to set the priority rules. The tests were designed so we simply walk through all the resolutions and make sure we get the exact same thing mime types.

The changes look mostly reasonable ... but there are some things there that feel a bit odd like treating mp4 as audio by default.

[halostatue]

Hmm. I thought that I had set video/mp4 as higher priority for .mp4 than audio/mp4 (which has .m4a as higher priority).