Add support for automatic reconnect mode

[njh]

No description provided.

[ajays1991]

+1

[ajays1991]

I can see this issue opened since more than 2 years and none of the contributors. One of my colleague is constantly facing issue with dropped connections. I went through the src code and to avoid half-open connections , keep alive poll is implemented to send empty packed to broker with ping request to detect dropped connection on the client side.

I was wondering if some workaround can be done around keep alive exception raise to reconnect logic to reconnect the client to the broker. I can take up the initiative to add this , but i want some kind of acknowledgement from one of the contributors that i am heading in right direction.

[njh]

Hello @ajays871,

I would very much welcome a contribution to this - it is a long overdue enhancement. I have had to reject some Pull Requests; here are some things to look out for:

• don't do too many things in a single Pull Request - in the past people have done masses of changes and then submitted a single PR
• please write some tests - it is much easier to have confidence that the change works and is behaving correctly, if there are tests.

Some things to look out for related to this issue:

• The MQTT Server will close the connection (without any extra information) when it encounters some kind of protocol error - as such there should be some kind of exponential back-off as part of the re-connect code - so that the server isn't bombarded will loads of connections if something goes wrong
• Ideally QoS 1 publishes will be re-sent after a problem/reconnect

Would be great if there was a way of doing a small first step in the right direction...

[Xasin]

Heh...
I went ahead and wrote a full wrapping script around this MQTT library that handles receiving of packets asynchronously and with topic-bound callbacks (instead of firing a single block for all messages), as well as errors (by wrapping all the MQTT code inside rescue blocks).
It also queues up qos > 0 messages to be sent on reconnect, as well as subscriptions, and upon setting up, it connects to the server with cleansession enabled, then disconnects and reconnects with cleansession off, so that the server knows to queue up messages after a lengthy disconnect.

It can handle automatic reconnecting and actually has been running for a few days flawlessly, even surviving full internet outages.

I'm really, REALLY new to writing ruby gems, and I bet I am doing some quirky stuff, but ...
The MQTT gem has been extremely helpful in setting up my IoT environment.
Maybe, @njh, there would be some way for me to contribute?

There is a lot of things that I would love to see being handled in a smoother way (nonblocking code, etc).

[pgouv]

I am struggling to find a way to handle disconnects. Currently an error is raised on parent thread causing problems.

[Xasin]

@parhs, if you want I could put my MQTT Code in a separate repository.
It handles disconnects and reconnecting, and runs completely asynchronously with callbacks that can use the classical MQTT Topic-Wildcards.

The only way I found that allows for complete control over the thrown errors is by not using "MQTT::Client.connect()" directly, but instead creating a new MQTT Client class, and then wrapping every MQTT-Related statement including the connect in a begin rescue block.

[pgouv]

@Xasin if you could it could be helpful. The only workaround I found is to use blocking mode and rescue exceptions. However messages could be lost this way.
I don't know whether a pure EventMachine implementation would perform better.

[Xasin]

@parhs, my implementation is more or less "stand-alone-ish", and I could have probably used a lot more documentation etc., and possibly look at pre-existing event-based systems like EventMachine

For now, you could take the files MQTTSubscriber.rb and Waitpoint.rb from my general purpose library repo.
Put them in your source file directory, then require_relative "MQTTSubscriber.rb"

MQTT::SubHandler.new(<String with mqtt URL or MQTT::Client class with attributes set>) will create a new connection.
mqtt.subscribe_to(<topic>[, qos: 1]) do |topicTree, data| end will add a callback to a MQTT Topic. Wildcards can be used, and the topic "branches" that match the wildcard are left in the topicTree array. data is the transmitted data (as String format, like MQTT::Client.get outputs it)
You can add as many callbacks to any topics you want, they are all non-blocking.
The function also returns a MQTT::Subscription object, that you can use to remove the callback it belongs to again (by calling mqtt.unregister_subscription(<Subscription>))

You can also use mqtt.publish_to(<topic>[, qos: 1, retain: false]) to publish data, which will be queued and sent later if the client is disconnected.

Aside from the occasional ArgumentError, none of my functions should raise any exceptions and should be thread-safe.
I've been running some home-automation scripts with that library for days without failure or permanent disconnection ^^

[Xasin]

@parhs, I've finally made my code a gem.
mqtt-sub_handler
Documentation code is written.
It's fully asynchronous, should handle reconnecting properly, and doesn't throw any errors.

YMMV on unstable connections, but with a normal internet connection, I've managed to run my personal smart home code for weeks without needing a restart.

@njh, I hope this is something you find interesting and might link it on the Readme.
I'd greatly appreciate it ^^'

[Vincent14]

Is it dependent of #52 ?

[njh]

@Vincent14 no, I don't think so

[Xasin]

Just a heads-up that my MQTT code is still maintained, and has been rewritten to be a bit cleaner now.
I think I might spend a bit of extra time working on this gem too, if that's ok, to get QOS 1 and 2 handled asynchronously too :D

[njh]

There is also the Paho Ruby MQTT library:
https://github.com/eclipse/paho.mqtt.ruby

[rgaufman]

Paho is nice but it throws various exceptions in normal operation and seems to have some bugs, for example: PahoMqtt::WritingException: Broken pipe

Is there any way to detect a connection loss and reconnect with this gem?

[Xasin]

@rgaufman The way I handled it was by doing the subscribe/publishing from separate threads with appropriate begin-rescue wrapping, and adding a bit of manual reconnection.

You could try out my MQTT Gem, which just wraps this gem up a little more conveniently, at https://github.com/XasWorks/XasCode/tree/master/Ruby/MQTT

[rgaufman]

Interesting, will have a play with it, thank you!

[nlsrchtr]

Hi @njh,

maybe it's worth getting some inspiration from the implementation from the Locant MQTT gem as well? It looks promissing to me.

Regards,