Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Htsget POST request support #1529

Merged
merged 7 commits into from
Mar 18, 2021

Conversation

andersleung
Copy link
Contributor

Provide classes implementing the new Htsget POST request API, as well as integrate this into the HtsgetBamFileReader class so that it makes use of the API when available.

@lbergelson

Things to think about before submitting:

  • Make sure your changes compile and new tests pass locally.
  • Add new tests or update existing ones:
    • A bug fix should include a test that previously would have failed and passes now.
    • New features should come with new tests that exercise and validate the new functionality.
  • Extended the README / documentation, if necessary
  • Check your code style.
  • Write a clear commit title and message
    • The commit message should describe what changed and is targeted at htsjdk developers
    • Breaking changes should be mentioned in the commit message.

Sorry, something went wrong.

andersleung and others added 5 commits August 19, 2020 13:34

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.

Verified

This commit was signed with the committer’s verified signature.
stipsan Cody Olsen
@andersleung andersleung marked this pull request as draft January 11, 2021 17:26
@codecov-io
Copy link

codecov-io commented Jan 11, 2021

Codecov Report

Merging #1529 (e7e6974) into master (0e3e2b2) will increase coverage by 0.060%.
The diff coverage is 76.923%.

@@               Coverage Diff               @@
##              master     #1529       +/-   ##
===============================================
+ Coverage     69.417%   69.477%   +0.060%     
- Complexity      8924      8981       +57     
===============================================
  Files            602       606        +4     
  Lines          35520     35770      +250     
  Branches        5904      5944       +40     
===============================================
+ Hits           24657     24852      +195     
- Misses          8530      8565       +35     
- Partials        2333      2353       +20     
Impacted Files Coverage Δ Complexity Δ
...htsjdk/samtools/util/htsget/HtsgetPOSTRequest.java 75.294% <75.294%> (ø) 20.000 <20.000> (?)
...ava/htsjdk/samtools/util/htsget/HtsgetRequest.java 73.846% <80.000%> (+11.941%) 46.000 <2.000> (+12.000)
...main/java/htsjdk/samtools/HtsgetBAMFileReader.java 76.735% <81.481%> (+0.936%) 28.000 <6.000> (+4.000)
.../java/htsjdk/samtools/util/BufferedLineReader.java 48.387% <0.000%> (-20.034%) 7.000% <0.000%> (+1.000%) ⬇️
...htsjdk/samtools/util/nio/DeleteOnExitPathHook.java 80.952% <0.000%> (-9.524%) 3.000% <0.000%> (-1.000%)
...sjdk/samtools/util/BlockCompressedInputStream.java 85.560% <0.000%> (-0.722%) 90.000% <0.000%> (ø%)
src/main/java/htsjdk/io/IOPath.java 78.788% <0.000%> (-0.522%) 15.000% <0.000%> (+1.000%) ⬇️
src/main/java/htsjdk/samtools/CRAMFileReader.java 74.762% <0.000%> (-0.238%) 47.000% <0.000%> (+2.000%) ⬇️
.../htsjdk/variant/variantcontext/VariantContext.java 78.245% <0.000%> (-0.128%) 240.000% <0.000%> (+2.000%) ⬇️
...jdk/samtools/util/BlockCompressedOutputStream.java 79.710% <0.000%> (ø) 34.000% <0.000%> (+2.000%)
... and 10 more

@brainstorm brainstorm mentioned this pull request Jan 13, 2021
Copy link
Member

@lbergelson lbergelson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andersleung This looks good. I don't have any comments except for 1 question about a todo that I think might be unecessary now.

// Only use POST request if supported and there are multiple intervals
if (this.usePOSTRequest && intervals.size() > 1) {
// POST request does not guarantee that all returned records are overlapped/contained
// by requested intervals, so we still need to filter, but don't need to filter duplicates
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks fo this comment.

}

@Override
public HtsgetPOSTRequest withFormat(final HtsgetFormat format) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's so frustrating that java doesn't have a way of expressing this without overriding all the methods. It really needs a self type for just this situation.

@@ -199,6 +211,11 @@ public static void testQueryOverlapped(final HtsgetBAMFileReader htsgetReader) t

@Test(dataProvider = "readerProvider")
public static void testRemovesDuplicates(final HtsgetBAMFileReader htsgetReader) throws IOException {
// TODO: temporary workaround as reference server does not properly merge regions and remove duplicates yet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@andersleung Is this fixed now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We started the work on this but with the onramp projects (DRS and Passports) and now the file formats work there's probably not going to enough bandwidth to push it through any time soon, so it's not fixed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an issue # for it we can link to somewhere to monitor the progress? It's one of those things that's confusing in the future if we don't have somewhere to look.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created an issue and linked to it in the comment ga4gh/htsget-refserver#27

@lbergelson
Copy link
Member

@andersleung Thank you. This is still marked as in progress. Is there any reason we can't merge it? Did I miss some important missing thing when I looked ai it. It seemed pretty sane to me.

@andersleung
Copy link
Contributor Author

Ah sorry just force of habit. It should be ready to merge

@andersleung andersleung marked this pull request as ready for review March 17, 2021 17:38
@lbergelson
Copy link
Member

Excellent. I'll merge when tests pass which they definitely should because you just changed a comment..

@lbergelson lbergelson merged commit 7719274 into samtools:master Mar 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants