Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Add future deprecation note for component_(sent|received)_bytes_total metrics #20412

Conversation

jszwedko
Copy link
Member

@jszwedko jszwedko commented May 1, 2024

Vector originally published two component bytes metrics:

  • component_sent_bytes_total the number of serialized bytes successfully sent by sinks (before
    compression)
  • component_received_bytes_total the number of serialized bytes successfully received by sources
    (after decompression and filtering)

These were intended to be used by users to understand the volume of data flowing in/out of sources
and sinks. However, they were dependent on the exact serialization format such that the same event
going into the socket source vs. vector source might be measured differently. To have
a consistent metric we then added:

  • component_sent_event_bytes_total the number of "estimated JSON event" bytes successfully sent by a components
  • component_received_event_bytes_total the number of "estimated JSON event" bytes successfully received by a components

This is a metric that is able to be compared between any two components. This removed a use-case of
component_(sent|received)_bytes_total. The remaining use-case users seemed to use this metric for
was as a proxy for network bytes, but what they really are interested in is the actual network
bytes. So we proposed two new metrics:

  • component_sent_network_bytes_total the total number of bytes sent out on the network by a
    component
  • component_received_network_bytes_total the total number of bytes received on the network by a
    component

Once those metrics exist, it isn't clear what use-case remains for
component_(sent|received)_bytes_total and so it would make sense to deprecate and remove them.

Signed-off-by: Jesse Szwedko [email protected]

…es_total` metrics

Vector originally published two component bytes metrics:

- `component_sent_bytes_total` the number of serialized bytes successfully sent by sinks (before
  compression)
- `component_received_bytes_total` the number of serialized bytes successfully received by sources
  (after decompression and filtering)

These were intended to be used by users to understand the volume of data flowing in/out of sources
and sinks. However, they were dependent on the exact serialization format such that the same event
going into the `socket` source vs. `vector` source might be measured differently. To have
a consistent metric we then added:

- `component_sent_event_bytes_total` the number of "estimated JSON event" bytes successfully sent by a components
- `component_received_event_bytes_total` the number of "estimated JSON event" bytes successfully received by a components

This is a metric that is able to be compared between any two components. This removed a use-case of
`component_(sent|received)_bytes_total`. The remaining use-case users seemed to use this metric for
was as a proxy for network bytes, but what they really are interested in is the _actual_ network
bytes. So we proposed two new metrics:

- `component_sent_network_bytes_total` the total number of bytes sent out on the network by a
  component
- `component_received_network_bytes_total` the total number of bytes received on the network by a
  component

Once those metrics exist, it isn't clear what use-case remains for
`component_(sent|received)_bytes_total` and so it would make sense to deprecate and remove them.

Signed-off-by: Jesse Szwedko <[email protected]>
@jszwedko jszwedko requested a review from a team as a code owner May 1, 2024 19:03
@jszwedko jszwedko added the no-changelog Changes in this PR do not need user-facing explanations in the release changelog label May 1, 2024
@jszwedko jszwedko enabled auto-merge May 1, 2024 19:16
@jszwedko jszwedko added this pull request to the merge queue May 1, 2024
@datadog-vectordotdev
Copy link

Datadog Report

Branch report: jszwedko/add-future-deprecation-note-for-component-bytes-metrics
Commit report: ffe2198
Test service: vector

✅ 0 Failed, 7 Passed, 0 Skipped, 25.45s Wall Time

Merged via the queue into master with commit 61b0b36 May 1, 2024
53 of 55 checks passed
@jszwedko jszwedko deleted the jszwedko/add-future-deprecation-note-for-component-bytes-metrics branch May 1, 2024 20:45
AndrooTheChen pushed a commit to discord/vector that referenced this pull request Sep 23, 2024
…es_total` metrics (vectordotdev#20412)

Vector originally published two component bytes metrics:

- `component_sent_bytes_total` the number of serialized bytes successfully sent by sinks (before
  compression)
- `component_received_bytes_total` the number of serialized bytes successfully received by sources
  (after decompression and filtering)

These were intended to be used by users to understand the volume of data flowing in/out of sources
and sinks. However, they were dependent on the exact serialization format such that the same event
going into the `socket` source vs. `vector` source might be measured differently. To have
a consistent metric we then added:

- `component_sent_event_bytes_total` the number of "estimated JSON event" bytes successfully sent by a components
- `component_received_event_bytes_total` the number of "estimated JSON event" bytes successfully received by a components

This is a metric that is able to be compared between any two components. This removed a use-case of
`component_(sent|received)_bytes_total`. The remaining use-case users seemed to use this metric for
was as a proxy for network bytes, but what they really are interested in is the _actual_ network
bytes. So we proposed two new metrics:

- `component_sent_network_bytes_total` the total number of bytes sent out on the network by a
  component
- `component_received_network_bytes_total` the total number of bytes received on the network by a
  component

Once those metrics exist, it isn't clear what use-case remains for
`component_(sent|received)_bytes_total` and so it would make sense to deprecate and remove them.

Signed-off-by: Jesse Szwedko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
no-changelog Changes in this PR do not need user-facing explanations in the release changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants