can not find sample image #6
Comments
|
Hello, @cindyyyl ! |
|
Copy that, I saw it , but i find maybe the key problem is i did not run them sucessfully. I stuck in the evaluation part, i try to adjust the number of samples from 10001 to 10000( i think it was a bug ) ,and i am sure that the both dir has file end with xxx.npz. could you give me any instructions? |
|
Hello @cindyyyl, Sorry for the late reply. To better assist you, I need that you describe better your issue could you please fill out the following details, please:
Best regards, Giulio |
|
Hi Giulio,
Sorry for the late reply. the steps of what i did are :
[image: image.png]
[image: image.png]
[image: image.png]
[image: image.png]
[image: image.png]
the bug of fid.compute() i have already solved , it because this function
need 3 chanels but the .npy file sampled in evaluation only 1 chanel.
however, after i solved this problem. my fid is nan and fid clip was three
times of yours exp. I think email communication maybe not a efficient way.
do you have time to have a zoom meeting? Besides that, your work is really
wonderful ! Thank you !
best,
xinxin
Giulio Corallo ***@***.***> 于2024年3月30日周六 11:42写道：
… Hello, @cindyyyl <https://github.com/cindyyyl> !
Does the sampled image are correctly displayed on wandb?
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQ44PHFKUTJX3DMR2ADY23FM3AVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRYGE2DCMRTGI>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hello @cindyyyl, It appears there's some confusion regarding the FID calculations for MNIST. The FID Clean and FID Clip scores cannot be computed with the current setup as the feature extractor networks require images with 3 channels. However, for MNIST, we've utilized a pretrained LeNet5, using its penultimate layer to extract features for the standard FID computation. You can verify in the configuration file that LeNet is specified as the feature extractor for MNIST at this link: .Additionally, we have provided the pretrained LeNet5 model weights in the Therefore, it is expected that you won’t be able to obtain the FID CLIP and FID Clean scores—it’s not a bug. Could you confirm if you were able to calculate the standard FID score? If so, I'll initiate a pull request to clarify that FID CLIP and FID Clean scores should not be computed when dealing with single-channel images, to avoid further confusion. Best regards, |
|
Hi Giulio，
Thank you so much for you details instructions. It really helps me a lot !
however, i just want to the fid score and do not have much interest for FID
clean and clip. For fid score caculation , i did not do any other
operations(I do not change the channel of .npz for fid score caculation),
just follow the instruction of github. however , the fid score is nan.
Thank you again !
best,
xinxin
[image: image.png]
Giulio Corallo ***@***.***> 于2024年5月2日周四 05:57写道：
… Hello @cindyyyl <https://github.com/cindyyyl>,
It appears there's some confusion regarding the FID calculations for
MNIST. The FID Clean and FID Clip scores cannot be computed with the
current setup as the feature extractor networks require images with 3
channels. However, for MNIST, we've utilized a pretrained LeNet5, using its
penultimate layer to extract features for the standard FID computation.
You can verify in the configuration file that LeNet is specified as the
feature extractor for MNIST at this link:
https://github.com/giulio98/functional-diffusion-processes/blob/cb13fb08b8f1a5449d627c6c9dd2b964e52852dd/conf/metrics/metrics_mnist.yaml#L8
.
Additionally, we have provided the pretrained LeNet5 model weights in the
models/lenet5 for reproducibility.
Therefore, it is expected that you won’t be able to obtain the FID CLIP
and FID Clean scores—it’s not a bug. Could you confirm if you were able to
calculate the standard FID score? If so, I'll initiate a pull request to
clarify that FID CLIP and FID Clean scores should not be computed when
dealing with single-channel images, to avoid further confusion.
Best regards,
Giulio
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQ5X6QLCOJIHYCDRU3DZAIEYVAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJQGA3DMMJTHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hi, Can you send here a screenshot of a batch for the sampled image for mnist? |
|
HI Giulio,
how about this, i have already zip projects (include the images i sampled
in evaluation process). i pull it to your repo?
best,
xinxin
Giulio Corallo ***@***.***> 于2024年4月26日周五 19:49写道：
… Hello @cindyyyl <https://github.com/cindyyyl>,
Sorry for the late reply. To better assist you, I need that you describe
better your issue could you please fill out the following details, please:
1.
*Steps to Reproduce:*
- Step 1:
- Step 2:
- Step 3:
2.
*Expected Behavior:*
- What you expected to happen after completing the steps above.
3.
*Actual Behavior:*
- What actually happened. Please include any error messages or
screenshots if possible.
4.
*Additional Information:*
- Any other details or context you think might be helpful.
Best regards,
Giulio
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQ2ON2RXDPX2PWQ7YF3Y7LRZBAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBQGI2DCNJQGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
[image: image.png][image: image.png]
[image: image.png]
Giulio Corallo ***@***.***> 于2024年5月3日周五 02:38写道：
… Hi,
Can you send here a screenshot of a batch for the sampled image for mnist?
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQ5LMYZBJNZ6AZLXJGTZAMWFBAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGM4TGOBRHA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hey sorry I can't see your screenshot. I just see [image.png] can you try again? |
|
Hi Giulio,
Could we talk about it recently or what other information you want to know
? since the DDL of my master thesis is coming soon, r my exp did not have
any result as so far \cry\cry.
best,
xinxin
…On Sat, May 4, 2024 at 10:45 PM cv zx ***@***.***> wrote:
On Sat, May 4, 2024 at 10:04 PM Giulio Corallo ***@***.***>
wrote:
> Hey sorry I can't see your screenshot. I just see [image.png] can you try
> again?
>
> —
> Reply to this email directly, view it on GitHub
> <#6 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AY3WWQ5RTEI6DE7N7ZR533TZAWHTNAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUGU2TEMBRGI>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
|
|
Hello, I run the script and i was able to get the FID score for mnist Please do the following: Carefully check your please notice the following, leave the others environment variables as is except you have to fill Let me know if after following this steps you are able to get the FID score. |
|
Hi Giulio,
I hope this email finds you well. I wanted to express my gratitude for your
assistance – I've successfully run the FID score! Life can be quite
challenging at times, but moments like these make it worthwhile. 😊
However, during the evaluation process in W&B, I noticed that the sampled
images contain noise. Could you please advise on the configurations
necessary to replicate the experimental results outlined in your paper?
[image: image.png]
Thank you once again for your help.
Best regards,
Xinxin
Giulio Corallo ***@***.***> 于2024年4月26日周五 19:49写道：
… Hello @cindyyyl <https://github.com/cindyyyl>,
Sorry for the late reply. To better assist you, I need that you describe
better your issue could you please fill out the following details, please:
1.
*Steps to Reproduce:*
- Step 1:
- Step 2:
- Step 3:
2.
*Expected Behavior:*
- What you expected to happen after completing the steps above.
3.
*Actual Behavior:*
- What actually happened. Please include any error messages or
screenshots if possible.
4.
*Additional Information:*
- Any other details or context you think might be helpful.
Best regards,
Giulio
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQ2ON2RXDPX2PWQ7YF3Y7LRZBAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOBQGI2DCNJQGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Hello, I can't see the images you share with me. |
|
sorry about that. how about this time ?
Giulio Corallo ***@***.***> 于2024年5月6日周一 04:48写道：
… Hello,
I can't see the images you share with me.
This is what I see from your message:
image.png (view on web)
<https://github.com/giulio98/functional-diffusion-processes/assets/79860892/6979042f-ee4c-4221-9420-07945c2ed0c9>
Anyway this is a batch i get from the provided checkpoint
image.png (view on web)
<https://github.com/giulio98/functional-diffusion-processes/assets/79860892/864a9630-02ee-4230-883e-62b88c7949a9>
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQZKQ2ZPTXCMP66DJCLZA47WXAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJVGQ4DAMRUGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
No I can't see it Can you try directly on GitHub? |
|
|
hahahahaha |
|
Hey, This is not supposed to happen, please run command |
|
hi ,
you mean git push my changes?
best,
xinxin
Giulio Corallo ***@***.***> 于2024年5月6日周一 14:27写道：
… Hey,
This is not supposed to happen, please run command
git pull
And let me know if you will get the correct image. You should get
something similar to my previous comment
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQ2FP27ZJI53BUJJFC3ZA7DSDAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJWGY2TGNZRGM>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
No i mean to pull my changes |
|
i have already git pull your changes last time
Giulio Corallo ***@***.***> 于2024年5月7日周二 04:46写道：
… No i mean to pull my changes
git pull
—
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AY3WWQ4XPJDGSXNMSIOYGULZBCIIFAVCNFSM6AAAAABE6MZ37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJXG43TENZXGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
I'm unable to reproduce your experiment, if i run with this .env file I get this logs: and this are the sampled image on wandb Did you follow the exact same steps? |
|
export WANDB_API_KEY= |
|
zxcvzxcv980234Projectsthird_timeRunsinr_mnistLogs cindyyyl Overview 100 |
|
Thank you so much for the help !! |
|
Hi I hope we can have a meeting for efficiency , since the result of exp by i run still incorrect,i.e. with larger N, should be have small fid, however as so far, my results still larger N larger fid. and i can not write these results in my work. ./cry /cry best, |
|
Hi from your logs it appear that it skips sampling because it has found checkpoints from previous run, please clean your logs directory, rm your and run again. |
|
i think it should be here ? there should be logs_test not logs ? |
|
I get fid score 1.28 using our config |
|
Please rm your mnist_stats since could be broken before the changes i made |
|
Emmm, I am a little confused about this: Please note that you will get a higher FID score because the checkpoint we provided does not include y-corrupted data at the input of the INR. Also, since I've set up a new environment and performed a new Git clone, I did all operations from the beginning. Could this issue be because I did not run the training script 'sh scripts/maml/train_mnist.sh'?" |
|
i see, but this time i totally create all by the beging , what ever environment and git repo " Please rm your mnist_stats since could be broken before the changes i made" |
|
i still think is the problem of when i caculate the FID . not the sample probllem |
|
Please recalculate the mnist_stats.npz You should get FID 1.28 |
|
hi , it is still \cry [2024-05-08 12:21:35,639][tensorflow][WARNING] - From /cis/home/shuan124/anaconda3/envs/lxxfdp/lib/python3.10/site-packages/tensorflow_gan/python/estimator/tpu_gan_estimator.py:42: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead. [2024-05-08 12:21:35,664][main][INFO] - Instantiating <functional_diffusion_processes.metrics.fid_metric.FIDMetric> |
|
In your logs you don't have the steps where it calculates the mnist_stats.npz meaning that is reusing one already precalculated, did you remove it? |
|
Please run this |
|
yes, removed it all .npz file when i run a new fid. i think maybe the problem in this part code since i remeber last time you fix about the jax .... , will it affect there ? |
|
|
i will run this Please run this |
|
I think this time maybe on the way !! than you ~~ |
|
it stills, iclear both *.npz in inr_minist and meta_0_30 and rm -rf./data: [2024-05-08 12:59:02,223][tensorflow][WARNING] - From /cis/home/shuan124/anaconda3/envs/lxxfdp/lib/python3.10/site-packages/tensorflow_gan/python/estimator/tpu_gan_estimator.py:42: The name tf.estimator.tpu.TPUEstimator is deprecated. Please use tf.compat.v1.estimator.tpu.TPUEstimator instead. [2024-05-08 12:59:02,246][main][INFO] - Instantiating <functional_diffusion_processes.metrics.fid_metric.FIDMetric> |
|
I'm sorry but I don't understand what could be the reason. if a run the code using our config file I get the following: please share your |
|
Hi this is my eval_mnist.yaml @Package globaldefaults:
trainers: evaluation_config: sdes: correctors: samplers: models: datasets: |
|
You changed N to be 3 is normal you get an high fid scores, please use our configurations to get the same result |
|
I totolly understand the problem , if i run your configurations with N=50 , i am almost 100 times larger than you |
you can see this. i think is still the problem of function of fid.compute. since i have already sampled image successfully |
|
Sorry, i cloned the repository from scratch, loaded the model from scratch and run the script and i still get fid:1.28 |
|
I think when you changed from 3 to 50 you still skipping the sampling |
|
Please run this PYTHONPATH=. python3 src/functional_diffusion_processes/run.py --multirun +experiments_maml=eval_mnist trainers.evaluation_config.eval_dir= ${oc.env:LOGS_ROOT}/inr_mnist_new This way you will specify a new folder for eval and it will generate images from scratch |
|
Please rember that when you do changes on the config you have to change also the eval dir otherwise the code detect that there are already samples and use them for the fid computation |
|
yes, i see . since today's code is a totally new git repo and environment i downloaded in the morning, and the first time i run this code did not change the N , it is 100 times larger than you,. imean waht i did is totally follow the instructions of github without any chage. after that , i change N = 3 and find it fid score is less than N = 50. I think althogh git clone, with a unknown reason. we still have something different( like in yoour project is logs and for me is logs.test ...) so , now i will try to use other way not your code to caculate the fid . since as so far i can see the image from wandb and it is seems right image with N =3 and N =50 |
yes, each time i run before i will rm .data you told me and move *.npz include meta file to another folder to keep the clean of eval.dir |
|
You don't have to remove the data, you just have to change the eval dir name, have you tried that? |
|
Hello i'm able to reproduce your experiment and i get indeed fid score: 75.73 and these sampled image when i run instead: I get fid score: 1.28 If i run instead: Here i am specifying on porpouse the eval_dir corresponding to N=3 Please whenever you do changes on I hope the message is clear and you will be able to get the FID score. |
Thanks so much for your kind help. i copy that, i am not good at argpasers opearations .. but , this is the score i use probality flow ode to eval.minist when N=10, and the fid_score is : 1.1349964141845703:
|
|
Look at the sampled image and see the results. Remember always to change the eval dir when you compute the fid calculation. |
|
Thank you so much for the whole help ! i really appreciate alll your help! |
Hi, i met a problem when i run sh scripts/maml/sample_mnist.sh. i can not find the sample image. i open the dir ( inr_minst/samples) but it was empty. however , when i run exp_ninist. it will generate image. could you give me some instructions?
best,
xinxin
The text was updated successfully, but these errors were encountered: