[bug] conan config install skips files if ".git" is in the path

[MrPointer]

I had set my CONAN_USER_HOME env-var to "$HOME/.gitlab-conan" (doesn't matter why).
Later, I ran the conan config install command, providing a git repo that stores shared-company config, but that command has appeared to silently fail, or do nothing at all, as the config hasn't been applied, at all.
The configuration repo is properly configured as I've tested it on other environments where conan's home is set to the default path, ~/.conan, and it works as expected.

When I run the command, I always get the output Repo cloned! and that's it, it quits right after. So I've decided to dig in and find out why, and I found that inside the conans/client/conf/config_installer.py file, on this line, there's a check for the .git word being in the root string returned from walk(). I get the idea here, but it might be required to re-implement this to look for .git only in the last component of the root, as this is what actually matters, allowing other components of the path to anything the user desires.
It might not be that trivial to solve this nicely, so an alternative could be to warn users in the docs about setting conan home to paths that include .git in it.

Environment Details (include every applicable attribute)

• Operating System+version: Ubuntu 16.04 (Docker container)
• Conan version: 1.34.0
• Python version: 3.8.8

Steps to reproduce (Include if Applicable)

1. Set the conan home to a path that has .git in it, no matter where exactly, e.g. .gitlab
2. Try to install some config using conan config install

[danimtb]

Hi @MrPointer,

I tried to reproduce your issue but it working fine for me. I set CONAN_USER_HOME to a local cloned repo (with .git folder inside it). Then I did a conan config install from a github repo. Conan copied the contents of the github repo to $CONAN_USER_HOME/.conan folder as expected.

As you pointed out, the .git folder from the github repo is skipped, as you only need the contents of the repo and not to create a repo itself in the config folder. There are other files skipped on purpose such as license.txt or readme.md files.

If you could provide me with a github repo to conan config install I would be willing to try to reproduce it further. Thanks a lot.

[MrPointer]

You probably misunderstood me, or the exact scenario I'm describing. I'll try to be more specific 😅

Let's assume I have the following path set as CONAN_USER_HOME: ~/.gitlab-conan
It means that conan would create a .conan subdir under this path, resulting in ~/.gitlab-conan/.conan, under which all the configuration resides.
If you'll try to conan config install in this scenario, you won't be able to, because the code that processes directories will skip all of the directories found in the config-repo, as the .git string would be part of the root string variable all the time. Why? Because .gitlab-conan contains the substr .git, and AFAIK, Python will match against it when checking if this substr is in a string.

I'm talking about this piece of code:

for root, dirs, files in walk(folder):
        dirs[:] = [d for d in dirs if d != ".git"]
        if ".git" in root:
            continue

I hope it makes it a bit clearer 🙂

[danimtb]

Thanks for the clarification. Do you mean you are installing from that ~/.gitlab-conan folder as well? I mean, doing something like conan config install ~/.gitlab-conan/.conan?

[MrPointer]

No, this is only where I'm installing to. I can't provide the exact git repo that stores my config because it's proprietary, but the full command looks something like this: conan config install <url-to-git-repo> --type git

[danimtb]

@MrPointer I have added a test here #8605 trying to reproduce your case. It uses a local git repository for simplification but it should work in the same way. Let me know if that is similar enough to issue you described. Thank you.

[MrPointer]

@danimtb Yeah it seems so, thanks! Just make sure CONAN_USER_HOME is indeed set to .gitlab-conan (I suspect the cache_folder parameter covers this, but just to be sure)

[danimtb]

yep, the cache_folder is the parameter to initialize the Conan class by the TestClient() we use for convenience. In the Conan client that logic can be found here

conan/conans/client/conan_api.py

Line 231 in a0d81b3

self.cache_folder = cache_folder or os.path.join(get_conan_user_home(), ".conan")

[danimtb]

please, let me know if you managed to solve the issue as I am not able to reproduce it from my side. Thanks!

[MrPointer]

@danimtb I've simply removed the dot from the dir's name, becoming ~/gitlab-conan instead of ~/.gitlab-conan, but it doesn't really solve anything, does it?
I'll see if I can solve it myself, but I can't promise that I'll have time for it in the upcoming days 😕

[danimtb]

Seems that the issue is legit. We have updated the test at #8605 and it fails as you reported and now I understand better the issue. Thank you!