TIKA-4252: add request metadata #1753
Conversation
| FetchKey fetchKey = t.getFetchKey(); | ||
| protected MetadataListAndEmbeddedBytes parseFromTuple(FetchEmitTuple fetchEmitTuple, Fetcher fetcher) { | ||
| FetchKey fetchKey = fetchEmitTuple.getFetchKey(); | ||
| Metadata fetchResponseMetadata = new Metadata(); |
There was a problem hiding this comment.
The metadata that goes in the fetchemittuple was envisioned to be user-injected metadata that was injected after the parse and then emitted (e.g. provenance metadata).
I think we need to put both metadatas on the fetchemittuple.
This is what I'm thinking...let me know what you think.
So, there will be three metadatas in play. The fetchemit tuple will have a fetchRequestMetadata (???) and a userMetadata (???). At parse time, we'll create a fresh metadata object, which we'll call "responseMetadata" in the following call: fetcher.fetch(requestMetadata, responseMetadata).
The parse will then use the responseMetadata and, after the parse, inject the userMetadata from the fetchEmitTuple.
The fetcher may use the fetchRequestMetadata to carry out its request, but info from that one should not make it into the "responseMetadata" nor make it into the emit data.
There was a problem hiding this comment.
@nddipiazza any chance you can revert this in main so that we have a working build? Thank you!
There was a problem hiding this comment.
shoot i didn't realize i was deplying broken builds! reverted. i'll make this change and make a new pr
This reverts commit b068e42.
add request metadata