Dependencies: refactor requirements into server/client/dev, pin dependencies for server and dev

[rdimaio]

Main commits

• ba3d46a: splitting requirements.txt into three files, for client, server and development/testing.
• 7c2293c: unpin dependencies and remove upper bounds for client dependencies - this addresses Remove upper bound version constraints for rucio-clients #6663
• f7d12b1: this adds pip-tools to the development dependencies. pip-tools includes pip-compile, which lets us also pin the secondary dependencies installed by our main dependencies. With this, we can avoid issues such as this one: xmlsec-related failures in CI/CD #6694. We can also add a pre-commit action to run pip-compile automatically.

File structure

To avoid cluttering up the top-level directory, I've moved the requirements file into the requirements folder.

For dev and server, the requirements file with the .in suffix represent the input files to pip-compile; the .txt files are the requirements files generated by pip-compile. client does not have an .in file, because we are not pinning client dependencies.

Integration test failure

The integration test failure is due to this part of the dev Dockerfile in rucio/containers:

https://github.com/rucio/containers/blob/e75e7194cc9f404b46c977751c334b2b8f71b677/dev/Dockerfile#L75-L83

requirements.txt does not exist anymore, so this fails. I can make a PR to address this in rucio/containers, but the change would not be reflected until the next release, so I think the best way to handle this (once both PRs are approved) would be to:

1. merge PR in rucio/containers
2. manually update the rucio/dev image on Docker Hub
3. merge this PR

[rdimaio]

Latest force-pushes:

• ce0cbda: rebased, raised requests to 2.32.2 in both server and client (see Address Dependabot alert #15 #6771
• https://github.com/rucio/rucio/compare/ce0cbdac1d4fca56d3655d27c336bab18f490676..99c9dcb9a2dae7ec99ab265f6e794f5aac32bb2b: re-compiled reqs from top-level folder instead of requirements subfolder (not needed, it just changes the path shown in the comments in the generated requirements file, but I wanted to remain consistent with the previous version)

[rdimaio]

Rebased in latest forcepush