[logstash bridge]: bootstrap #107973
[logstash bridge]: bootstrap #107973
Conversation
Adds a new `logstash-bridge` project in `/libs` that: - publishes bridge methods for accessing various constructors, static functions, and static fields that are required by Logstash's Elastic Integration Filter - adds "duck type" interface validation tests to ensure that the object methods that Logstash's Elastic Integration Filter rely on maintain a stable API.
elasticsearchmachine
added
external-contributor
elasticsearchmachine
added
Team:Core/Infra
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
yaauie
added
>enhancement
needs:triage
elasticsearchmachine
added
Team:Core/Infra
|
Hi @yaauie, I've created a changelog YAML for you. |
Is this a new or existing plugin? Is that plugin going to be part of this repository or do we need to publish this artifact somewhere for external consumption? |
The elastic/logstash-filter-elastic_integration exists, and is a plugin for Logstash, whose artifacts are published as self-contained rubygems (no maven). The plugin already builds and bundles a minimal subset of Elasticsearch as part of its build process, and is capable of also building this artifact for self-consumption without requiring it to be published elsewhere. |
| apply plugin: 'elasticsearch.build' | ||
|
|
||
| dependencies { | ||
| compileOnly project(':server') |
There was a problem hiding this comment.
@rjernst I vaguely remember us not wanting libs to depend on server but this isn't really a "lib" of traditional sorts.
There was a problem hiding this comment.
I'm glad to place it elsewhere, even in x-pack project as this does not need to be published under an OSS license. Discussions with @rjernst offline indicated that /libs was an appropriate starting-point.
[The] way i would do this is have a lib, it will depend on server and painless. That lib will only live inside ES source, you can build/pull in the jars within logstash
-- @rjernst via Slack
There was a problem hiding this comment.
This is a tricky and special case. I think we are ok to depend on server here.
| compileOnly project(':modules:lang-painless:spi') | ||
|
|
||
| testImplementation(project(":test:framework")) { | ||
| exclude group: 'org.elasticsearch', module: 'logstashbridge' |
There was a problem hiding this comment.
Why is this required? Nothing should actually depend on this project so there should be no reason to exclude it.
There was a problem hiding this comment.
Looks lime a remnant from copying a different lib to bootstrap. I'll snip.
|
|
||
| public class ThreadPoolBridgeTests extends ESTestCase { | ||
|
|
||
| public void testThreadPoolAPIStability() throws Exception { |
There was a problem hiding this comment.
I don't understand the purpose of these tests. The point of this lib is that it is a bridge that will maintain all compatibility promises. If you need access to ThreadPool, have your own ThreadPool wrapper, and delegate in this bridge lib to the ES ThreadPool. By doing that, any refactorings (eg renamings/adjustments/whatever) of ThreadPool methods will require ES devs to update this code before merging, no notification to logstash devs necessary.
There was a problem hiding this comment.
Got it.
For many of these objects, we need to pass actual instances of these objects to ES-owned code, or to work with instances that are provided by ES-owned code.\
I had hoped to avoid many short-lived wrappers for each object by applying some pressure to the stability of the API, but I understand now that we are not to expect that to work.
I'll convert this PR back to 'draft' and work up a wrapper pattern for each of the objects we need to work with.
Adds a new
logstash-bridgeproject in/libsthat:TODO:
:libs:elasticsearch-logstash-bridge:testinto the build/CI, so that when someone breaks an API Logstash relies on we have direct feedback.