Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(axiom sink): Rebase sink on http sink and remove elasticsearch compatibility #21362

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

darach
Copy link

@darach darach commented Sep 26, 2024

Fixes #21292

The axiom sink has been rebased from elasticsearch to http per the issue.
Deprecated headers removed in favor of aligning with the http request configuration
scope, and the compression configuration now defaults to gzip.

The sink now conforms to Axiom's native ingest API:

https://axiom.co/docs/send-data/ingest#ingest-api

The _time field vector event data is no longer rejected. This was a source of
confusion for vector axiom users in the past as elasticsearch semantics were
imposed as the sink was based on the elasticsearch sink and hence field
semantics and constraints.

For existing vector users, the _time field is documented and aligns well with
vectors own timestamp mechanics:

https://axiom.co/docs/reference/field-restrictions#requirements-of-the-timestamp-field

If a @timestamp field is sent to the axiom sink it is now treated as a normal field. This
is a non breaking change.

If a _time field is sent to Axiom it no longer raises a 4XX error, so this is effectively a fix.

As the axiom sink is now HTTP based the full wealth of vectors http configuration apparatus
are now available to users of the axiom sink.

@darach darach requested review from a team as code owners September 26, 2024 15:58
@bits-bot
Copy link

bits-bot commented Sep 26, 2024

CLA assistant check
All committers have signed the CLA.

@github-actions github-actions bot added domain: sinks Anything related to the Vector's sinks domain: external docs Anything related to Vector's external, public documentation labels Sep 26, 2024
Signed-off-by: Darach Ennis <[email protected]>
…ection to changelog entry

Signed-off-by: Darach Ennis <[email protected]>
Copy link
Member

@jszwedko jszwedko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @darach ! I'll aim to review this next week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rework the Vector Axiom Sink to use Axiom's ingest endpoint instead of the ElasticSearch _bulk endpoint
4 participants