Skip to content

NetCoreMQTTExampleCluster is a project to run a multi MQTT broker system based on MQTTnet, Dapper (for database connection) and Orleans.

License

Notifications You must be signed in to change notification settings

SeppPenner/NetCoreMQTTExampleCluster

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

77 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NetCoreMQTTExampleCluster

NetCoreMQTTExampleCluster is a project to run a multi MQTT broker system based on MQTTnet, Dapper (for database connection) and Orleans.

Build status GitHub issues GitHub forks GitHub stars License: MIT Known Vulnerabilities

Hints

The user mqtt-broker-sync must be present and must not be used for anything else than the broker synchronization!

Setup

  1. Create the database with the project NetCoreMQTTExampleCluster.DatabaseSetup, adapting the appsettings.json as required.
  2. Start the projects NetCoreMQTTExampleCluster.SiloHost and NetCoreMQTTExampleCluster.Cluster as several start projects.
  3. Connect with a MQTT client using Test as user, test as Password and any clientId. The user can only publish to the topics: a/b and a/d and subcribe to d/e and e.

Use

  • The broker can be reached at localhost: 1883 (locally, unencrypted).
  • If you add a valid certificate for a domain, it can be accessed under yourdomain:8883.
  • Users can be managed in the database mqtt. The blacklists and whitelists are also saved there.
  • The Orleans dashboard is available by default at http://localhost:4321.

Projects

Project Description
NetCoreMQTTExampleCluster.Cluster Contains an instance of a broker that connects to the SiloHost. "MQTT cluster client".
NetCoreMQTTExampleCluster.DatabaseSetup Contains a project to set up the database initially.
NetCoreMQTTExampleCluster.Grains Contains the implementations of the Orleans Grains.
NetCoreMQTTExampleCluster.Grains.Interfaces Contains the interfaces of the Orleans Grains.
NetCoreMQTTExampleCluster.Models Contains all data models.
NetCoreMQTTExampleCluster.PasswordHashGenerator Contains a program for generating a new password hash for user passwords.
NetCoreMQTTExampleCluster.SiloHost Contains the Orleans SiloHost.
NetCoreMQTTExampleCluster.Storage Contains the administration of the database access.
NetCoreMQTTExampleCluster.TopicCheck Contains the project to check whether MQTT topics are valid or not.
NetCoreMQTTExampleCluster.TopicCheck.Tests Contains the project to test the topic check project.
NetCoreMQTTExampleCluster.Validation Contains the project to validate the MQTT connections, subscriptions and publishings.
NetCoreMQTTExampleCluster.Validation.Tests Contains the project to test the validation project.

Requirements

The project requires at least NetCore 3.1 and NetStandard 2.1.

Settings

NetCoreMQTTExampleCluster.Cluster

To do this, change the file appsettings.json.

Setting Description Optional
LogFolderPath The folder the service logs to. No
Port The encrypted port of the MQTT cluster client. No
UnencryptedPort The unencrypted port of the MQTT cluster client. (This should only be used locally). Yes
HeartbeatIntervalInMilliseconds The heartbeat interval. No
BrokerConnectionSettings The connection that the SiloHost establishes to the broker if a message is published by another MQTT cluster client. No
BrokerConnectionSettings.ClientId The ClientId of the connection. No
BrokerConnectionSettings.HostName The host name of the connection. No
BrokerConnectionSettings.Port The port of the connection. No
BrokerConnectionSettings.UserName The user name of the connection. No
BrokerConnectionSettings.Password The password of the connection. No
BrokerConnectionSettings.UseTls Whether TLS should be used for the connection. No
BrokerConnectionSettings.UseCleanSession Whether the "CleanSession" flag should be used for the connection. No
DatabaseSettings The database connection settings. No
DatabaseSettings.Host The host of the database connection. No
DatabaseSettings.Database The database of the database connection. No
DatabaseSettings.Username The user name of the database connection. No
DatabaseSettings.Password The password of the database connection. No
DatabaseSettings.Port The port of the database connection. No
OrleansConfiguration.ClusterOptions The connection to the Orleans SiloHost. No
OrleansConfiguration.ClusterOptions.ClusterId The CluserId of the Orleans-SiloHost. No
OrleansConfiguration.ClusterOptions.ServiceId The ServiceId of the Orleans-SiloHost. No

Example:

{
  "NetCoreMQTTExampleCluster.Cluster": {
    "LogFolderPath": "C:\\log\\NetCoreMQTTExampleCluster.Cluster",
    "HeartbeatIntervalInMilliseconds": 30000,
    "Port": 8883,
    "UnencryptedPort": 1883,
    "BrokerConnectionSettings": {
      "ClientId": "mqtt-broker-sync-1",
      "HostName": "localhost",
      "Port": 8883,
      "Password": "Test",
      "UserName": "mqtt-broker-sync",
      "UseTls": true,
      "UseCleanSession": true
    },
    "DatabaseSettings": {
      "Host": "localhost",
      "Database": "mqtt",
      "Username": "postgres",
      "Password": "postgres",
      "Port": 5432
    },
    "OrleansConfiguration": {
      "ClusterOptions": {
        "ClusterId": "mqtt001",
        "ServiceId": "mqtt-silo-001-A"
      }
    }
  }
}

NetCoreMQTTExampleCluster.DatabaseSetup

To do this, change the file appsettings.json.

Setting Description Optional
Host The host of the database connection. No
Database The database of the database connection. No
Username The user name of the database connection. No
Password The password of the database connection. No
Port The port of the database connection. No
Pooling A value that indicates whether pooling should be used when connecting to the database. No

Example:

{
  "Host": "localhost",
  "Database": "mqtt",
  "Username": "postgres",
  "Password": "postgres",
  "Port": 5432,
  "Pooling" : false
}

NetCoreMQTTExampleCluster.SiloHost

To do this, change the file appsettings.json.

Setting Description Optional
LogFolderPath The folder the service logs to. No
HeartbeatIntervalInMilliseconds The heartbeat interval. No
Database The database connection settings. No
Database.Host The host of the database connection. No
Database.Database The database of the database connection. No
Database.Username The user name of the database connection. No
Database.Password The password of the database connection. No
Database.Port The port of the database connection. No
OrleansConfiguration The configuration of the Orleans SiloHost. No
OrleansConfiguration.ClusterOptions The cluster configuration of the Orleans-SiloHost. No
OrleansConfiguration.ClusterOptions.ClusterId The CluserId of the Orleans-SiloHost. No
OrleansConfiguration.ClusterOptions.ServiceId The ServiceId of the Orleans-SiloHost. No
OrleansConfiguration.DashboardOptions The configuration of the Orleans dashboard of the Orleans SiloHost. No
OrleansConfiguration.DashboardOptions.CounterUpdateIntervalMs The update interval of the Orleans dashboard. No
OrleansConfiguration.DashboardOptions.Port The port of the Orleans dashboard. No
OrleansConfiguration.EndpointOptions The endpoint configuration of the Orleans SiloHost. No
OrleansConfiguration.EndpointOptions.AdvertisedIPAddress The Orleans SiloHost IP address. No
OrleansConfiguration.EndpointOptions.SiloPort The Orleans SiloHost port. No
OrleansConfiguration.EndpointOptions.GatewayPort The port of the Orleans SiloHost gateway. No
OrleansConfiguration.EndpointOptions.SiloListeningEndpointAddress The Orleans SiloHost IP address. No
OrleansConfiguration.EndpointOptions.SiloListeningEndpointPort The Port des Orleans-SiloHost. No
OrleansConfiguration.EndpointOptions.GatewayListeningEndpointAddress The IP address of the Orleans SiloHost gateway. No
OrleansConfiguration.EndpointOptions.GatewayListeningEndpointPort The port of the Orleans SiloHost gateway. No

Example:

{
  "NetCoreMQTTExampleCluster.SiloHost": {
    "LogFolderPath": "C:\\log\\NetCoreMQTTExampleCluster.SiloHost",
    "HeartbeatIntervalInMilliseconds": 30000,
    "DatabaseSettings": {
      "Host": "localhost",
      "Database": "mqtt",
      "Username": "postgres",
      "Password": "postgres",
      "Port": 5432
    },
    "OrleansConfiguration": {
      "ClusterOptions": {
        "ClusterId": "mqtt001",
        "ServiceId": "mqtt-silo-001-A"
      },
      "DashboardOptions": {
        "CounterUpdateIntervalMs": 1000,
        "Port": 4321
      },
      "EndpointOptions": {
        "AdvertisedIPAddress": "127.0.0.1",
        "SiloPort": 7101,
        "GatewayPort": 7102,
        "SiloListeningEndpointAddress": "127.0.0.1",
        "SiloListeningEndpointPort": 7101,
        "GatewayListeningEndpointAddress": "127.0.0.1",
        "GatewayListeningEndpointPort": 7102
      }
    }
  }
}

Structure

The MQTT clients connect to any MQTT broker. All inquiries are forwarded to the SiloHost and synchronized there.

Schedule

General

Every action is saved by the SiloHost into the table eventlog.

Broker connect

The MQTT broker connects to the SiloHost by specifying the BrokerConnectionSettings and a random brokerId (GUUID). The SiloHost stores the BrokerConnectionSettings for the brokerId in a Dictionary.

Broker disonnect

The MQTT broker disconnects from the SiloHost by specifying its brokerId. The SiloHost removes the entry with the brokerId from its Dictionary.

Client connect

The MQTT client connects to the MQTT broker by specifying the MqttConnectionValidatorContext. The MQTT broker forwards the connection request to the SiloHost as SimpleMqttConnectionValidatorContext. This validates the data and returns the MQTT broker whether the MQTT client is allowed to connect or not.

Client disconnect

The MQTT client passes the MqttServerClientDisconnectedEventArgs to the MQTT broker. The broker passes this on to the SiloHost. This deletes all saved client data from its list.

Client subscribe

The MQTT client passes the MqttSubscriptionInterceptorContext to the MQTT broker. The broker passes this on to the SiloHost. This validates the data and returns the MQTT broker whether the MQTT client can subscribe or not.

Client unsubscribe

The MQTT client passes the MqttUnsubscriptionInterceptorContext to the MQTT broker. The broker passes this on to the SiloHost. In this case, the SiloHost does not have to do anything else.

Client publish

The MQTT client passes the MqttApplicationMessageInterceptorContext to the MQTT broker. The broker passes this on to the SiloHost and supplements his brokerId. The SiloHost validates the data and returns to the MQTT broker whether the MQTT client is allowed to publish or not. If this is the case, the SiloHost sends the packet to all other brokers except the one who initially sent the message.

Temporarily locking clients

Users can be blocked temporarily (until the end of the current month) if they exceed a certain data quota. Therefore, the parameter ThrottleUser in the User table must be set to true and MonthlyByteLimit to the desired threshold in bytes.

Attention

  • Only the following UTF-8 chars are supported for topics:
Unicode code point character UTF-8(hex.) Name
U+0021 ! 21 EXCLAMATION MARK
U+0022 " 22 QUOTATION MARK
U+0023 # 23 NUMBER SIGN
U+0024 $ 24 DOLLAR SIGN
U+0025 % 25 PERCENT SIGN
U+0026 & 26 AMPERSAND
U+0027 ' 27 APOSTROPHE
U+0028 ( 28 LEFT PARENTHESIS
U+0029 ) 29 RIGHT PARENTHESIS
U+002A * 2a ASTERISK
U+002B + 2b PLUS SIGN
U+002C , 2c COMMA
U+002D - 2d HYPHEN-MINUS
U+002E . 2e FULL STOP
U+002F / 2f SOLIDUS
U+0030 0 30 DIGIT ZERO
U+0031 1 31 DIGIT ONE
U+0032 2 32 DIGIT TWO
U+0033 3 33 DIGIT THREE
U+0034 4 34 DIGIT FOUR
U+0035 5 35 DIGIT FIVE
U+0036 6 36 DIGIT SIX
U+0037 7 37 DIGIT SEVEN
U+0038 8 38 DIGIT EIGHT
U+0039 9 39 DIGIT NINE
U+003A : 3a COLON
U+003B ; 3b SEMICOLON
U+003C < 3c LESS-THAN SIGN
U+003D = 3d EQUALS SIGN
U+003E > 3e GREATER-THAN SIGN
U+003F ? 3f QUESTION MARK
U+0040 @ 40 COMMERCIAL AT
U+0041 A 41 LATIN CAPITAL LETTER A
U+0042 B 42 LATIN CAPITAL LETTER B
U+0043 C 43 LATIN CAPITAL LETTER C
U+0044 D 44 LATIN CAPITAL LETTER D
U+0045 E 45 LATIN CAPITAL LETTER E
U+0046 F 46 LATIN CAPITAL LETTER F
U+0047 G 47 LATIN CAPITAL LETTER G
U+0048 H 48 LATIN CAPITAL LETTER H
U+0049 I 49 LATIN CAPITAL LETTER I
U+004A J 4a LATIN CAPITAL LETTER J
U+004B K 4b LATIN CAPITAL LETTER K
U+004C L 4c LATIN CAPITAL LETTER L
U+004D M 4d LATIN CAPITAL LETTER M
U+004E N 4e LATIN CAPITAL LETTER N
U+004F O 4f LATIN CAPITAL LETTER O
U+0050 P 50 LATIN CAPITAL LETTER P
U+0051 Q 51 LATIN CAPITAL LETTER Q
U+0052 R 52 LATIN CAPITAL LETTER R
U+0053 S 53 LATIN CAPITAL LETTER S
U+0054 T 54 LATIN CAPITAL LETTER T
U+0055 U 55 LATIN CAPITAL LETTER U
U+0056 V 56 LATIN CAPITAL LETTER V
U+0057 W 57 LATIN CAPITAL LETTER W
U+0058 X 58 LATIN CAPITAL LETTER X
U+0059 Y 59 LATIN CAPITAL LETTER Y
U+005A Z 5a LATIN CAPITAL LETTER Z
U+005B [ 5b LEFT SQUARE BRACKET
U+005C \ 5c REVERSE SOLIDUS
U+005D ] 5d RIGHT SQUARE BRACKET
U+005E ^ 5e CIRCUMFLEX ACCENT
U+005F _ 5f LOW LINE
U+0060 ` 60 GRAVE ACCENT
U+0061 a 61 LATIN SMALL LETTER A
U+0062 b 62 LATIN SMALL LETTER B
U+0063 c 63 LATIN SMALL LETTER C
U+0064 d 64 LATIN SMALL LETTER D
U+0065 e 65 LATIN SMALL LETTER E
U+0066 f 66 LATIN SMALL LETTER F
U+0067 g 67 LATIN SMALL LETTER G
U+0068 h 68 LATIN SMALL LETTER H
U+0069 i 69 LATIN SMALL LETTER I
U+006A j 6a LATIN SMALL LETTER J
U+006B k 6b LATIN SMALL LETTER K
U+006C l 6c LATIN SMALL LETTER L
U+006D m 6d LATIN SMALL LETTER M
U+006E n 6e LATIN SMALL LETTER N
U+006F o 6f LATIN SMALL LETTER O
U+0070 p 70 LATIN SMALL LETTER P
U+0071 q 71 LATIN SMALL LETTER Q
U+0072 r 72 LATIN SMALL LETTER R
U+0073 s 73 LATIN SMALL LETTER S
U+0074 t 74 LATIN SMALL LETTER T
U+0075 u 75 LATIN SMALL LETTER U
U+0076 v 76 LATIN SMALL LETTER V
U+0077 w 77 LATIN SMALL LETTER W
U+0078 x 78 LATIN SMALL LETTER X
U+0079 y 79 LATIN SMALL LETTER Y
U+007A z 7a LATIN SMALL LETTER Z
U+007B { 7b LEFT CURLY BRACKET
U+007C ` ` 7c
U+007D } 7d RIGHT CURLY BRACKET
U+007E ~ 7e TILDE
U+00A1 ¡ c2 a1 INVERTED EXCLAMATION MARK
U+00A2 ¢ c2 a2 CENT SIGN
U+00A3 £ c2 a3 POUND SIGN
U+00A4 ¤ c2 a4 CURRENCY SIGN
U+00A5 ¥ c2 a5 YEN SIGN
U+00A6 ¦ c2 a6 BROKEN BAR
U+00A7 § c2 a7 SECTION SIGN
U+00A8 ¨ c2 a8 DIAERESIS
U+00A9 © c2 a9 COPYRIGHT SIGN
U+00AA ª c2 aa FEMININE ORDINAL INDICATOR
U+00AB « c2 ab LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
U+00AC ¬ c2 ac NOT SIGN
U+00AE ® c2 ae REGISTERED SIGN
U+00AF ¯ c2 af MACRON
U+00B0 ° c2 b0 DEGREE SIGN
U+00B1 ± c2 b1 PLUS-MINUS SIGN
U+00B2 ² c2 b2 SUPERSCRIPT TWO
U+00B3 ³ c2 b3 SUPERSCRIPT THREE
U+00B4 ´ c2 b4 ACUTE ACCENT
U+00B5 µ c2 b5 MICRO SIGN
U+00B6 c2 b6 PILCROW SIGN
U+00B7 · c2 b7 MIDDLE DOT
U+00B8 ¸ c2 b8 CEDILLA
U+00B9 ¹ c2 b9 SUPERSCRIPT ONE
U+00BA º c2 ba MASCULINE ORDINAL INDICATOR
U+00BB » c2 bb RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
U+00BC ¼ c2 bc VULGAR FRACTION ONE QUARTER
U+00BD ½ c2 bd VULGAR FRACTION ONE HALF
U+00BE ¾ c2 be VULGAR FRACTION THREE QUARTERS
U+00BF ¿ c2 bf INVERTED QUESTION MARK
U+00C0 À c3 80 LATIN CAPITAL LETTER A WITH GRAVE
U+00C1 Á c3 81 LATIN CAPITAL LETTER A WITH ACUTE
U+00C2 Â c3 82 LATIN CAPITAL LETTER A WITH CIRCUMFLEX
U+00C3 Ã c3 83 LATIN CAPITAL LETTER A WITH TILDE
U+00C4 Ä c3 84 LATIN CAPITAL LETTER A WITH DIAERESIS
U+00C5 Å c3 85 LATIN CAPITAL LETTER A WITH RING ABOVE
U+00C6 Æ c3 86 LATIN CAPITAL LETTER AE
U+00C7 Ç c3 87 LATIN CAPITAL LETTER C WITH CEDILLA
U+00C8 È c3 88 LATIN CAPITAL LETTER E WITH GRAVE
U+00C9 É c3 89 LATIN CAPITAL LETTER E WITH ACUTE
U+00CA Ê c3 8a LATIN CAPITAL LETTER E WITH CIRCUMFLEX
U+00CB Ë c3 8b LATIN CAPITAL LETTER E WITH DIAERESIS
U+00CC Ì c3 8c LATIN CAPITAL LETTER I WITH GRAVE
U+00CD Í c3 8d LATIN CAPITAL LETTER I WITH ACUTE
U+00CE Î c3 8e LATIN CAPITAL LETTER I WITH CIRCUMFLEX
U+00CF Ï c3 8f LATIN CAPITAL LETTER I WITH DIAERESIS
U+00D0 Ð c3 90 LATIN CAPITAL LETTER ETH
U+00D1 Ñ c3 91 LATIN CAPITAL LETTER N WITH TILDE
U+00D2 Ò c3 92 LATIN CAPITAL LETTER O WITH GRAVE
U+00D3 Ó c3 93 LATIN CAPITAL LETTER O WITH ACUTE
U+00D4 Ô c3 94 LATIN CAPITAL LETTER O WITH CIRCUMFLEX
U+00D5 Õ c3 95 LATIN CAPITAL LETTER O WITH TILDE
U+00D6 Ö c3 96 LATIN CAPITAL LETTER O WITH DIAERESIS
U+00D7 × c3 97 MULTIPLICATION SIGN
U+00D8 Ø c3 98 LATIN CAPITAL LETTER O WITH STROKE
U+00D9 Ù c3 99 LATIN CAPITAL LETTER U WITH GRAVE
U+00DA Ú c3 9a LATIN CAPITAL LETTER U WITH ACUTE
U+00DB Û c3 9b LATIN CAPITAL LETTER U WITH CIRCUMFLEX
U+00DC Ü c3 9c LATIN CAPITAL LETTER U WITH DIAERESIS
U+00DD Ý c3 9d LATIN CAPITAL LETTER Y WITH ACUTE
U+00DE Þ c3 9e LATIN CAPITAL LETTER THORN
U+00DF ß c3 9f LATIN SMALL LETTER SHARP S
U+00E0 à c3 a0 LATIN SMALL LETTER A WITH GRAVE
U+00E1 á c3 a1 LATIN SMALL LETTER A WITH ACUTE
U+00E2 â c3 a2 LATIN SMALL LETTER A WITH CIRCUMFLEX
U+00E3 ã c3 a3 LATIN SMALL LETTER A WITH TILDE
U+00E4 ä c3 a4 LATIN SMALL LETTER A WITH DIAERESIS
U+00E5 å c3 a5 LATIN SMALL LETTER A WITH RING ABOVE
U+00E6 æ c3 a6 LATIN SMALL LETTER AE
U+00E7 ç c3 a7 LATIN SMALL LETTER C WITH CEDILLA
U+00E8 è c3 a8 LATIN SMALL LETTER E WITH GRAVE
U+00E9 é c3 a9 LATIN SMALL LETTER E WITH ACUTE
U+00EA ê c3 aa LATIN SMALL LETTER E WITH CIRCUMFLEX
U+00EB ë c3 ab LATIN SMALL LETTER E WITH DIAERESIS
U+00EC ì c3 ac LATIN SMALL LETTER I WITH GRAVE
U+00ED í c3 ad LATIN SMALL LETTER I WITH ACUTE
U+00EE î c3 ae LATIN SMALL LETTER I WITH CIRCUMFLEX
U+00EF ï c3 af LATIN SMALL LETTER I WITH DIAERESIS
U+00F0 ð c3 b0 LATIN SMALL LETTER ETH
U+00F1 ñ c3 b1 LATIN SMALL LETTER N WITH TILDE
U+00F2 ò c3 b2 LATIN SMALL LETTER O WITH GRAVE
U+00F3 ó c3 b3 LATIN SMALL LETTER O WITH ACUTE
U+00F4 ô c3 b4 LATIN SMALL LETTER O WITH CIRCUMFLEX
U+00F5 õ c3 b5 LATIN SMALL LETTER O WITH TILDE
U+00F6 ö c3 b6 LATIN SMALL LETTER O WITH DIAERESIS
U+00F7 ÷ c3 b7 DIVISION SIGN
U+00F8 ø c3 b8 LATIN SMALL LETTER O WITH STROKE
U+00F9 ù c3 b9 LATIN SMALL LETTER U WITH GRAVE
U+00FA ú c3 ba LATIN SMALL LETTER U WITH ACUTE
U+00FB û c3 bb LATIN SMALL LETTER U WITH CIRCUMFLEX
U+00FC ü c3 bc LATIN SMALL LETTER U WITH DIAERESIS
U+00FD ý c3 bd LATIN SMALL LETTER Y WITH ACUTE
U+00FE þ c3 be LATIN SMALL LETTER THORN
U+00FF ÿ c3 bf LATIN SMALL LETTER Y WITH DIAERESIS

Create an openssl certificate

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365
openssl pkcs12 -export -out certificate.pfx -inkey key.pem -in cert.pem

An example certificate is in the folder. Password for all is test.

Change history

See the Changelog.

About

NetCoreMQTTExampleCluster is a project to run a multi MQTT broker system based on MQTTnet, Dapper (for database connection) and Orleans.

Resources

License

Stars

Watchers

Forks

Packages

No packages published