Loggregator Component Properties

Q: What do the Doppler properties mean?

Here is some more detailed info on some of the Doppler properties and what they mean and do within the code. Of course, there may be a tendency for this list to diverge from actual properties.

Code Config/Manifest Property Name	Description
BlackListIps/ doppler.blacklisted_syslog_ranges	Blacklist for IPs that should not be used as syslog drains, e.g. internal ip addresses.
ContainerMetricTTLSeconds/ doppler.container_metric_ttl_seconds	TTL (in seconds) for container usage metrics. It is used to obtain the latest container metrics for the app.
LegacyIncomingMessagesPort/ doppler.incoming_port	Defaults to 3456. This is legacy. It is no longer used in code and will be removed from the template spec.
IncomingUDPPort/ doppler.dropsonde_incoming_port	Defaults to 3457. This is the port Doppler listens on for messages from Metron via UDP. Metrics like `dropsondeListener.{receivedByteCount
IncomingTCPPort/ doppler.incoming_tcp_port	Defaults to 3458. Metrics like `tcpListener.{receivedByteCount
EnableTLSTransport/ doppler.tls.enable	Defaults to false. This makes Doppler listen over TLS. If this property is true then the TLSListenerConfig property must be configured as well.
TLSListenerConfig	Requires the TLS port, Doppler cert and key, and Loggregator CA Cert.
EtcdRequireTLS/ loggregator.etcd.require_ssl	Require TLS for communicating with etcd.
EtcdTLSClientConfig	Cert, key and CA files for communicating with etcd over TLS.
MaxRetainedLogMessages/ doppler.maxRetainedLogMessages	Defaults to 100. This is the buffer size for the Dump Sinks. Dump Sinks are created for every app because it contains the most n recentLogs of the app. If you want more logs for cf logs app_name --recent then configure this property.
MessageDrainBufferSize/ doppler.message_drain_buffer_size	Defaults to 10000. Size of the internal buffer used by doppler to store messages. If the buffer gets full doppler will drop the messages. This is used by the websocket and syslog sinks.
MetricBatchIntervalMilliseconds	Defaults to 5000ms or 5s. This doesn't have a corresponding spec property. This is the interval for how often Doppler will emit batched metrics.
MetronAddress	Defaults to 127.0.0.1:3457. Doppler also needs to send metrics/logs about itself to Metron.
MonitorIntervalSeconds	Defaults to 60 seconds. This property is not in the template spec. This is used for the uptime monitor to send Doppler's uptime via dropsonde as a metric.
OutgoingPort/ doppler.outgoing_port	Defaults to 8081. This is the port Doppler listens on for requests from Traffic Controller.
SharedSecret/ doppler_endpoint.shared_secret	This secret is shared with metron. It is used to verify cryptographically signed dropsonde messages.
SinkDialTimeoutSeconds /doppler.sink_dial_timeout_seconds	Defaults to 1s. Dial timeout for syslog sinks.
SinkIOTimeoutSeconds /doppler.sink_io_timeout_seconds	Defaults to 0s. Write timeout for syslog sinks. This will only be set if greater than zero. See here
SinkInactivityTimeoutSeconds /doppler.sink_inactivity_timeout_seconds	Defaults to 3600s. Interval before removing a sink due to inactivity. This is used by Dump Sinks and ContainerMetrics Sinks.
SinkSkipCertVerify /doppler.syslog_skip_cert_verify	Defaults to true. When connecting over TLS, don't verify certificates for syslog sink.
Syslog	Defaults to vcap.doppler. This is used to setup a gosteno logger syslog sink when starting up Doppler. Has nothing to do with Doppler's Syslog Sinks for apps.
UnmarshallerCount /doppler.unmarshaller_count	Defaults to 5. Number of parallel dropsonde unmarshallers to run within Doppler
WebsocketWriteTimeoutSeconds /doppler.websocket_write_timeout_seconds	Defaults to 60s. This sets the write deadline for websocket sinks when streaming logs or firehose. See here
PPROFPort /doppler.pprof_port	Defaults to 0 (Random port will be picked). This is the port for pprof runtime profiling data.

Back to FAQ

Q: What do the Metron properties mean?

Here is some more detailed info on some of the Metron properties and what they mean and do within the code. Of course, there may be a tendency for this list to diverge from actual properties.

Code Config/Manifest Property Name	Description
Syslog	Defaults to vcap.metron_agent. This is used to setup a gosteno logger syslog sink when starting up Metron.
IncomingUDPPort/ metron_agent.dropsonde_incoming_port	Defaults to 3457. This is the port on which metron listens to locally for incoming dropsonde messages via UDP.
LoggregatorDropsondePort/ loggregator.dropsonde_incoming_port	Defaults to 3457. This is a duplicate of IncomingUDPPort. This seems to be a legacy property. Its usage should be switched to IncomingUDPPort.
EtcdQueryIntervalMilliseconds	This is not used. This needs to be removed soon.
EtcdRequireTLS/ loggregator.etcd.require_ssl	Require TLS for communicating with etcd.
EtcdTLSClientConfig	Cert, key and CA files for communicating with etcd over TLS.
SharedSecret/metron_endpoint.shared_secret	This secret is shared with Doppler. It is used to prefix/sign each UDP message bytes. So when Doppler gets a message on its UDP listener it uses this shared secret to verify that the message bytes haven't been tampered with.
MetricBatchIntervalMilliseconds	Defaults to 5000ms or 5s. This doesn't have a corresponding spec property. This is the interval on which aggregated metrics are sent.
RuntimeStatsIntervalMilliseconds	Defaults to 15000ms or 15s. This doesn't have a corresponding spec property. This is the interval on which the memoryStats metrics are emitted.
TCPBatchSizeBytes/ metron_agent.tcp.batching_buffer_bytes	Defaults to 10240 bytes. The number of bytes which can be buffered prior to TCP writes (applies to TLS over TCP). Metron will error out if this is less than 1024 bytes.
TCPBatchIntervalMilliseconds/ ...batching_buffer_flush_interval_milliseconds	Defaults to 100. The maximum time that a message can stay in the batching buffer before being sent to Doppler.
PreferredProtocol/ metron_agent.preferred_protocol	As of the latest implementation, this is more of a required protocol. That is if this set to tls and there aren't any Dopplers with TLS listeners, Metron will panic with a message of No available Dopplers.
PPROFPort /metron_agent.pprof_port	Defaults to 0 (Random port will be picked). This is the port for pprof runtime profiling data.

Back to FAQ

Q: What do the Traffic Controller properties mean?

Here is some more detailed info on some of the traffic controller properties and what they mean and do within the code. Of course, there may be a tendency for this list to diverge from actual properties.

Code Config/Manifest Property Name	Description
EtcdUrls/ loggregator.etcd.machines	The ETCD urls that Doppler advertises itself to.
EtcdMaxConcurrentRequests/ loggregator.etcd.maxconcurrentrequests	Sets the max workers for the work pool when setting up the etcdStoreAdapter
EtcdRequireTLS/ loggregator.etcd.require_ssl	Require TLS for communicating with etcd.
EtcdTLSClientConfig	Cert, key and CA files for communicating with etcd over TLS.
Syslog	Set to vcap.trafficcontroller here. This is used to setup a gosteno logger syslog sink when starting up TC. This has nothing to do with Doppler's syslog sink.
ApiHost/cc.srv_api_uri	TC uses this to verify if an app has log access permissions
DopplerPort/loggregator.doppler_port	Defaults to 8081. TC uses this as part of the dopplerservice.LegacyFinder to build the doppler url by appending to the doppler IP retrieved from the legacy etcd key healthstatus/doppler.... These urls are used to create a websocket connection to dopplers. This is a legacy property.
OutgoingPort/ traffic_controller.outgoing_port	Defaults to 8080. This is legacy. TC listens on this port for requests in legacy format. That is /tail/, /dump, /recent which is converted to /apps/<appId>/stream and /apps/<appId>/recentlogs
OutgoingDropsondePort/ loggregator.outgoing_dropsonde_port	Defaults to 8081. This is related to OutgoingPort. TC listens on this port for requests in the new format of /apps/<appId>/{stream/recentlogs/containermetrics}
MetronPort/ metron_endpoint.dropsonde_port	Defaults to 3457. TC initializes dropsonde with this port. It sends its logs and metrics to Metron on this port
SystemDomain/system_domain	TC uses this to set a cookie domain with the structure of doppler.<SystemDomain> when a request is made to TC with path /set-cookie. MORE DETAILS HERE. WHY IS THIS ENDPOINT USED?
SkipCertVerify/ssl.skip_cert_verify	Skips cert verification to the CC and the UAA.
MonitorIntervalSeconds	Defaults to 60 seconds. This property is not in the template spec. This is used for the uptime monitor to send TC's uptime via dropsonde as a metric.
Zone/traffic_contoller.zone	DEPRECATED. We removed this from our spec because it isn't being used in the traffic controller job.
PPROFPort /traffic_controller.pprof_port	Defaults to 0 (Random port will be picked). This is the port for pprof runtime profiling data.

Back to FAQ