Check against non-mosquitto brokers

[andypiper]

Currently the code relies on the $SYS topic tree (only RSMB and mosquitto support it) for the display of the topic list http://mqtt.org/wiki/doku.php/conventions#sys

We could do a check for $SYS existing as a topic and warn that the bridge cannot list the topics otherwise.

[njh]

It doesn't depend upon the $SYS tree.

It just specifically subscribes to it because in mosquitto, when you subscribe to '#' it doesn't automatically give you '$SYS/#' too.

If the code is used with a broker that doesn't support $SYS with a special meaning, it shouldn't fail because it doesn't know anything about it?

Having said that, the bridge might work a lot better if there was a way of getting a list of topics without getting the payloads too.

[andypiper]

ahhh in that case I am confused about how this is working (I didn't look very hard, and jumped to conclusions)

I tried it with RabbitMQ and got no topics. Could be that it isn't responding in the expected manner. I'll have a play around with some other brokers too then.

[andypiper]

Weird behaviour with hivemq too (which does have $SYS) - it doesn't list any of the topics I manually publish retained messages on, but does list the $SYS tree and also lists # a number of times. I ran a bridge locally pointing at broker.mqtt-dashboard.com to test this out.

[andypiper]

Tested with Apache Apollo:

• run Apollo (listens on 61613)
• mosquitto_pub -h localhost -p 61613 -t blah -m "bloo" -r (several times, varying the topic and message)
• configure bridge with remote_port of 61613 and rackup
• lists # a number of times equivalent to the number of topics created.

So it seems that with non-mosquitto or RSMB brokers the topic list is not getting populated properly. I'm curious if this is a difference in the way they are sending packets compared to the more common brokers.

[andypiper]

OK so I tried this simple code snippet

#!/usr/bin/env ruby

require 'rubygems'
require 'mqtt'

MQTT::Client.connect('localhost',61613) do |client|
  client.get('#') do |topic,message|
    puts "#{topic}: #{message}"
  end
end

My original suspicion was that maybe Apollo etc were not properly returning the topics. That's not always true.
In fact, I see the following behaviours across a number of brokers:

• Apollo does not return all of the topics that already have retained messages on them. Publications "new" after the Ruby code starts are received, and the topic string is correctly populated
• RabbitMQ does not return all of the topics that already have retained messages on them. Publications "new" after the Ruby code starts are received, and the topic string is correctly populated
• the node-based MQTT.js broker (from the orig.js sample included in the project) does not return all of the topics that already have retained messages on them. Publications "new" after the Ruby code starts are received, and the topic string is correctly populated
• HiveMQ does return some data equivalent to the number of existing topics with retained publications on them, however the topic strings are all set to '#'
• mosquitto and rsmb perform as expected

I'm going to post about this on the mqtt protocol Google group since it affects a number of the server implementations.

[dobermai]

@andypiper Thanks for pointing out the issue that HiveMQ sets the topic for the published retained message on subscriptions to "#" instead of the actual topic. I just fixed that issue and the fix will be available soon on mqttdashboard.com and the downloadable HiveMQ version will be replaced with the new one. We also updated our MQTT compliance test suite, that case was not covered yet. Thanks again for that thorough analysis!

[njh]

Thank you very much for doing this work to test against various brokers @andypiper. It was a job that I wasn't looking forward to!

Would be really good to have a test suite than can be run against brokers to test conformance.

[andypiper]

Additional observations:

• RabbitMQ ignores the retained flag as the underlying AMQP broker does not have a concept of retained (this is documented in the MQTT adapter blog post)
• mqtt.js as a library is simplistic and the broker sample I was testing orig.js is equally simple. No retained publication support.

Still to test:

• IBM WebSphere MQ
• IBM MessageSight
• moquette (Java)
• eMQTT (Erlang)
• mosca (based on MQTT.js)
• ActiveMQ "classic"
• myChannels Nirvana

... plus various other hosted services like Xively, m2m.io etc.

[andypiper]

• moquette does return data equivalent to the number of existing topics with retained publications on them, however the topic strings are all set to '#'