Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add distance-to-edge and distance-to-centre metrics to pipeline #696

Open
ddobie opened this issue Aug 31, 2023 · 4 comments
Open

Add distance-to-edge and distance-to-centre metrics to pipeline #696

ddobie opened this issue Aug 31, 2023 · 4 comments
Assignees
Labels
enhancement New feature or request

Comments

@ddobie
Copy link
Contributor

ddobie commented Aug 31, 2023

During the 2023 workshop we discovered that there are some edge-of-field effects that can induce false variability. It would therefore be helpful to be able to tag, filter or remove sources that are close to the edge of the field.

@ddobie ddobie added the enhancement New feature or request label Aug 31, 2023
@ddobie ddobie self-assigned this Aug 31, 2023
@github-actions github-actions bot added this to To do in Pipeline Backlog Aug 31, 2023
@ajstewart
Copy link
Contributor

Just a note that I remember I tried to implement this once and scrapped it because it was quite expensive.

Unfortunately I can't find the code, I thought I committed it at some point.

I think this function in tools was the result of it in the end:
https://github.com/askap-vast/vast-tools/blob/3abf03e47a111821094bd4eac54fedd3b89911bd/vasttools/utils.py#L632-L646

I remember the taxicab having something to do with it.

@ddobie
Copy link
Contributor Author

ddobie commented Sep 1, 2023

I did think it would be quite expensive. In that case it's something that we should be doing in the last stage of filtering with vast-tools rather than making it part of the pipeline.

@ajstewart
Copy link
Contributor

I guess distance from centre is easy but perhaps not that useful? Distance from edge is the hard but most useful.

You could let the pipeline calculate the metric on sources only with a certain threshold of variability.

But I reckon you could put this into the image import, you only have to run it once. If it only adds a little bit of processing I'd guess it's worth it.

@ddobie
Copy link
Contributor Author

ddobie commented Sep 6, 2023

Yeah distance from centre isn't useful at all, it really has to be distance from the edge.

I think if we're going to calculate it for a subset then it makes sense to do that post-pipeline (e.g. vast-tools) because otherwise we're almost certainly going to end up with a situation where someone finds a source that they want it for, but it doesn't exist. The code already partially/mostly exists in vast-tools anyway.

This is also a bit less relevant if we're cropping the fields - there's no rubbish on the image edges because that's all been removed anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

2 participants