Real API Gateway endpoints unreachable when moto submodules

[getglad]

Moto Version: 2.2.15
Boto3 Version: 1.20.7

I have a test that mocks ssm, but attempts to reach a real excute-api API Gateway endpoint. It appears that the responses_mock may be catching it? Despite not loading an API Gateway mock, I am getting a "The method is not implemented" message.

https://github.com/spulec/moto/blob/a912fc4cac1d4e66401e1772fe3c7a4182c2040e/moto/core/models.py#L450

https://github.com/spulec/moto/blob/master/moto/core/models.py#L485-L500

An example test

from moto import mock_ssm

@pytest.fixture
def set_ssm(monkeypatch):
    monkeypatch.setenv("AWS_DEFAULT_REGION", "us-east-2")
    with mock_ssm() as _:
      ... create client and do things ...
      ssm_c.put_parameter(
            Name=PLATFORM_CONFIG_NAME,
            Value=json.dumps(data),
            Type="String",
            Overwrite=True,
            DataType="text",
        )

        yield ssm_c

def test_token(self, set_ssm):
   real_function()

If I put a breakpoint() in moto/core/models.py(322)_find_first_match(), the request below catches in that function for reasons that do not make sense to me. I would not expect moto to even be involved at this point, but I have to imagine the fact that it is a execute-api.<region>.amazonaws.com url is contributing to the issue.

https://github.com/spulec/moto/blob/master/moto/core/models.py#L315-L319

import requests

def real_function():
   ... use ssm (this works with moto as expected) ...
   real_endpoint = "http://<api-id>.execute-api.<region>.amazonaws.com/<path>"
   payload = {}
   response = requests.request("PUT", url, headers=headers, data=payload) # return 400
   print(response.text) # returns 'The method is not implemented'

I'm happy to try to contribute a fix, but not sure where to start.

[bblommers]

Hi @getglad, that is on purpose, actually - we intercept all requests to AWS to prevent people from accidentally changing their real infrastructure.

One option would be to at least make this behaviour configurable via an environment variable. The majority would want to intercept everything, as a fail-safe, but for expert users the option should be there to connect to AWS itself.

If you'd like to contribute, a PR is always welcome!
Our documentation has info on how to get started as a contributor: http://docs.getmoto.org/en/latest/docs/contributing/index.html

For this specific feature, the logic that intercepts all AWS calls currently resides here: https://github.com/spulec/moto/blob/master/moto/core/models.py#L485:L500
All environment variables are located in moto/settings.py, so we should add another called something like 'MOTO_ALLOW_AWS_ACCESS' = 'true'/'false'

I haven't tested it out yet - but it might be as simple as placing an if-statement around that block, if not settings.allow_aws_access(): # add the not_implemented callbacks (and write a test for it of course).

Let me know if that makes sense - happy to help out if you get stuck!

[bblommers]

Hi @getglad, it's been a while, but this feature is now part of the latest alpha-release: moto==5.0.0alpha2

The new mock_aws-decorator has an option to configure exactly which URL's you want to passthrough:

@mock_aws(
    config={"core": {
        "mock_credentials": False, 
        "passthrough": {
            "urls": ["execute-api.*.amazonaws.com"],
            "services": []}}}
)

See the release announcement here: #7198

Are you able to upgrade and verify that this works for you?

[bblommers]

Moto V5 has now been released, so I'll close this.

Release: https://pypi.org/project/moto/5.0.0/

Documentation about this feature: http://docs.getmoto.org/en/5.0.0/docs/configuration/index.html