$ airflow connections import fails with AttributeError

[MatrixManAtYrService]

Apache Airflow version

6f8c204

Environment

OS (e.g. from /etc/os-release): Mac OS 11.3
Kernel: Darwin Kernel Version 20.4.0
Install tools: pip install -e .

What happened:

Tried to import valid connection json via the cli. Got an error:

❯ echo '{"sqlite_conn_id": {
    "conn_type": "sqlite",
    "description": "",
    "host": "localhost",
    "login": "",
    "password": null,
    "schema": "/usr/local/airflow/test.db",
    "port": null,
    "extra": ""
  }}' | jq . > connections.json

❯ airflow connections import connections.json
Traceback (most recent call last):
  File "/Users/matt/src/airflow/venv/bin/airflow", line 33, in <module>
    sys.exit(load_entry_point('apache-airflow', 'console_scripts', 'airflow')())
  File "/Users/matt/src/airflow/airflow/__main__.py", line 40, in main
    args.func(args)
  File "/Users/matt/src/airflow/airflow/cli/cli_parser.py", line 48, in command
    return func(*args, **kwargs)
  File "/Users/matt/src/airflow/airflow/utils/cli.py", line 91, in wrapper
    return f(*args, **kwargs)
  File "/Users/matt/src/airflow/airflow/cli/commands/connection_command.py", line 244, in connections_import
    _import_helper(args.file)
  File "/Users/matt/src/airflow/airflow/cli/commands/connection_command.py", line 272, in _import_helper
    key: value for key, value in conn_values.items() if key in allowed_fields
AttributeError: 'Connection' object has no attribute 'items'

Several of the tests assume that load_connections_dict returns a dict of dicts

airflow/tests/cli/commands/test_connection_command.py

Line 790 in 6f8c204

# We're not testing the behavior of load_connections_dict, assume successfully reads JSON, YAML or env

Actually, load_connections_dict returns a dict of Connections

airflow/airflow/secrets/local_filesystem.py

Line 278 in 6f8c204

connection_by_conn_id[key] = _create_connection(key, secret_values)

When we call .items() on the Connection, it fails.

What you expected to happen:

The import is sucessful.

How to reproduce it:

See commands above.

Anything else we need to know:

This commit fixes it (I think): MatrixManAtYrService@5159059

I started making a PR, but while trying to make the tests pass I started to worry that I was on the wrong path, so I made an issue instead.

[ephraimbuddy]

I was not able to reproduce this. Can you check the content inside the connections.json? I'm suspecting that it has errors.

This worked for me:

echo '{"sqlite_conn_id": {
    "conn_type": "sqlite",
    "description": "",
    "host": "localhost",
    "login": "",
    "password": null,
    "schema": "/usr/local/airflow/test.db",
    "port": null,
    "extra": ""
  }}'  > connections.json

airflow connections import connections.json

[MatrixManAtYrService]

Hmm, maybe there's something different about our environments. I can indeed replicate it with that json.

To protect against environmental differences, here's a session that does it in docker:

$ docker run -it --entrypoint bash "apache/airflow@sha256:305f237eb0ce38ed4f707ba1d08353111d0f36c36fab24661483cd365af1278d"
/opt/airflow$ airflow version
2.1.0
/opt/airflow$ airflow db init
...
Initialization done
/opt/airflow$ echo '{"sqlite_conn_id": {
>     "conn_type": "sqlite",
>     "description": "",
>     "host": "localhost",
>     "login": "",
>     "password": null,
>     "schema": "/usr/local/airflow/test.db",
>     "port": null,
>     "extra": ""
>   }}'  > connections.json
/opt/airflow$ cat connections.json
{"sqlite_conn_id": {
    "conn_type": "sqlite",
    "description": "",
    "host": "localhost",
    "login": "",
    "password": null,
    "schema": "/usr/local/airflow/test.db",
    "port": null,
    "extra": ""
  }}
/opt/airflow$ airflow connections import connections.json
Traceback (most recent call last):
  File "/home/airflow/.local/bin/airflow", line 8, in <module>
    sys.exit(main())
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/__main__.py", line 40, in main
    args.func(args)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/cli_parser.py", line 48, in command
    return func(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/utils/cli.py", line 91, in wrapper
    return f(*args, **kwargs)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/connection_command.py", line 244, in connections_import
    _import_helper(args.file)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/connection_command.py", line 272, in _import_helper
    key: value for key, value in conn_values.items() if key in allowed_fields
AttributeError: 'Connection' object has no attribute 'items'

And an asciicast: https://asciinema.org/a/86Y40kiOwPXcF4jKw3n59FA3Z?theme=solarized-dark&speed=1.5

(I used 305f... because it's the current digest of apache/airflow:latest-python3.8).

@ephraimbuddy Do you get something different when you run it in docker like that?

[MatrixManAtYrService]

This was a duplicate of #15425

It's fixed now.