enhancement(socket sink): Integrate encoding::Encoder with socket sink
#10684
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for vector-project canceled.
|
github-actions
bot
added
domain: codecs
binarylogic
changed the title
encoding::Encoder with socket sourceencoding::Encoder with socket sink
pablosichert
changed the title
encoding::Encoder with socket sinkencoding::Encoder with socket sink
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
c2cf5f8
to
59a3a96
Compare
pablosichert
changed the base branch from
master
to
pablosichert/codec-json-serializer
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
59a3a96
to
4a04ae3
Compare
github-actions
bot
removed
the
domain: codecs
pablosichert
force-pushed
the
pablosichert/codec-json-serializer
branch
from
eadf6c8
to
b4354b2
Compare
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
4a04ae3
to
7113ed0
Compare
lukesteensen
approved these changes
Jan 7, 2022
Comment on lines
-13
to
-16
| // TODO: add back when serde-rs/serde#1358 is addressed | ||
| // #[serde(deny_unknown_fields)] |
There was a problem hiding this comment.
Just curious, why remove all of these (even the working ones)?
There was a problem hiding this comment.
They don't work any more since we have two adjacent #[serde(flatten)] attributes. E.g. deserializing TcpSinkConfig for mode will unfortunately reject flattened attributes from encoding when #[serde(flatten)] is present.
pablosichert
force-pushed
the
pablosichert/codec-json-serializer
branch
2 times, most recently
from
8d01d34
to
650c17f
Compare
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
7113ed0
to
1e7b577
Compare
|
Rebased it on top of the changes to all encoding support structures for zero-copy operations and the new soak tests. Very curious to see the results! ...for which I'd need to implement a soak test first. Sounds like a good idea though, will do that. |
pablosichert
force-pushed
the
pablosichert/codec-json-serializer
branch
from
650c17f
to
129cdef
Compare
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
1e7b577
to
bdc8657
Compare
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
bdc8657
to
83dca4d
Compare
Soak Test ResultsBaseline: 573cda3 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Test units below are bytes/second/CPU, except for "skewness". The further "skewness" is from 0.0 the more indication that vector lacks consistency in behavior, making predictions of fitness in the field challenging. The abbreviated table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. The abbreviated table will be omitted if no statistically interesting changes are observed.
Fine details of change detection per experiment.
Fine details of each soak run.
|
github-actions
bot
added
the
domain: codecs
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
724f4d7
to
61e4af2
Compare
Soak Test ResultsBaseline: ad6ae66 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Test units below are bytes/second/CPU, except for "skewness". The further "skewness" is from 0.0 the more indication that vector lacks consistency in behavior, making predictions of fitness in the field challenging. The abbreviated table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. The abbreviated table will be omitted if no statistically interesting changes are observed.
Fine details of change detection per experiment.
Fine details of each soak run.
|
Soak Test ResultsBaseline: 9afb079 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±7.0% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±7.0%:
Fine details of change detection per experiment.
Fine details of each soak run.
|
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
f3d5dea
to
bd4fc7f
Compare
pablosichert
changed the base branch from
master
to
pablosichert/standard-encodings-migrator
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
bd4fc7f
to
d5665a2
Compare
lukesteensen
approved these changes
Apr 11, 2022
Soak Test ResultsBaseline: e2530d8 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±7.0% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±7.0%:
Fine details of change detection per experiment.
Fine details of each soak run.
|
Soak Test ResultsBaseline: 04f78f2 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±7.0% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±7.0%:
Fine details of change detection per experiment.
Fine details of each soak run.
|
pablosichert
force-pushed
the
pablosichert/standard-encodings-migrator
branch
from
04f78f2
to
90ead98
Compare
pablosichert
force-pushed
the
pablosichert/standard-encodings-migrator
branch
from
90ead98
to
c3d3e92
Compare
Base automatically changed from
pablosichert/standard-encodings-migrator
to
master
April 13, 2022 16:23
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
48e8a91
to
453bcb6
Compare
…h `flatten` Signed-off-by: Pablo Sichert <mail@pablosichert.com>
Signed-off-by: Pablo Sichert <mail@pablosichert.com>
…orrowing Signed-off-by: Pablo Sichert <mail@pablosichert.com>
…on top of it Signed-off-by: Pablo Sichert <mail@pablosichert.com>
Signed-off-by: Pablo Sichert <mail@pablosichert.com>
… `vector::codecs::Encoder::encode` Signed-off-by: Pablo Sichert <mail@pablosichert.com>
Signed-off-by: Pablo Sichert <mail@pablosichert.com>
Signed-off-by: Pablo Sichert <mail@pablosichert.com>
pablosichert
force-pushed
the
pablosichert/sinks-socket-codec-integration
branch
from
453bcb6
to
cbcc080
Compare
Soak Test ResultsBaseline: 06aca83 ExplanationA soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core. The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±7.0% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed. Changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±7.0%:
Fine details of change detection per experiment.
Fine details of each soak run.
|
Filed #12227 for that @lukesteensen |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #9459.
Closes: #3454