Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dask ip resolution. #6475

Merged
merged 3 commits into from Dec 8, 2020
Merged

Fix dask ip resolution. #6475

merged 3 commits into from Dec 8, 2020

Conversation

trivialfis
Copy link
Member

@trivialfis trivialfis commented Dec 7, 2020

This adopts the solution used in dask/dask-xgboost#40 which employs the get_host_ip from dmlc-core tracker. It was referred in #5765 .

Close #6469 .

TODOs

  • Test on k8s.

@hcho3
Copy link
Collaborator

hcho3 commented Dec 7, 2020

Maybe this should be blocking?

@trivialfis trivialfis mentioned this pull request Dec 7, 2020
12 tasks
@codecov-io
Copy link

codecov-io commented Dec 7, 2020

Codecov Report

Merging #6475 (a547d3b) into master (703c2d0) will increase coverage by 0.21%.
The diff coverage is 94.73%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6475      +/-   ##
==========================================
+ Coverage   80.14%   80.35%   +0.21%     
==========================================
  Files          13       13              
  Lines        3515     3533      +18     
==========================================
+ Hits         2817     2839      +22     
+ Misses        698      694       -4     
Impacted Files Coverage Δ
python-package/xgboost/tracker.py 95.11% <94.11%> (-0.07%) ⬇️
python-package/xgboost/dask.py 82.10% <100.00%> (+0.88%) ⬆️
python-package/xgboost/core.py 80.62% <0.00%> (+0.04%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55bdf08...a547d3b. Read the comment docs.

@hcho3 hcho3 merged commit 0ffaf0f into dmlc:master Dec 8, 2020
hcho3 pushed a commit that referenced this pull request Dec 8, 2020
This adopts the solution used in dask/dask-xgboost#40 which employs the get_host_ip from dmlc-core tracker.
@trivialfis trivialfis deleted the fix-dask-ip branch December 8, 2020 00:47
Copy link

@drobison00 drobison00 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified that it works in GKE.

hostIP = socket.gethostbyname(socket.gethostname())
if hostIP.startswith("127."):
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# doesn't have to be reachable

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May just leave a more informative comment here, for posterity. 1) Why local host isn't appropriate in all contexts, 2) Why '10.255.255.255' on port 1. Did you choose this because its a broadcast addr and won't be forwarded, or just because it's unlikely to be hosting anything?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @drobison00 , let me add some comments there later. But if you have suggestions on how to do this better feel free to share! I'm happy to revise it. I think you are much more familiar with this than me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dask xgboost fit: munmap_chunk(): invalid pointer: 0x00007fa5380304b0
4 participants