.env file cannot be used: unexpected character "»" in variable name near "\ufeff"

[Jeb135]

Description

I am trying to get an application running with docker compose up that relies on environment variables that are set in a .env file. Anytime I attempt to use a .env file however, I get the error 'unexpected character "»" in variable name near "\ufeffENVIRONMENTVAR1=myvalue"'. I have gone so far as to make the .env file completely empty and this same error still occurs.

Steps to reproduce the issue:

1. Create an empty .env file
2. Create a docker-compose.yml file in the same directory, The content of this file doesn't appear to matter, but I will supply one similar to what I was using. The microsoft sample app for aspnet core on docker would work with this sample config file: https://github.com/dotnet/dotnet-docker/tree/56f6d1d7671e9a1fd99c50ec5f229e5fbc0369e7/samples/aspnetapp

version: "3.9"

services:
  webapp:
    build: .
    ports:
      - "5001:80"

4. run docker compose up from an administrator powershell window

Describe the results you received:
Docker compose appears to be failing to parse the .env file for a character that does not exist. I have triple checked that there is no hidden unicode characters in the file, and the failure occurs even on totally empty .env files.

Describe the results you expected:
The environment variables would be parsed from the file and used in the same way that docker run --env-file .env <image> uses them.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker compose version:

Docker Compose version v2.9.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc., v0.8.2)
  compose: Docker Compose (Docker Inc., v2.9.0)
  extension: Manages Docker extensions (Docker Inc., v0.2.8)
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc., 0.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 31
  Running: 0
  Paused: 0
  Stopped: 31
 Images: 205
 Server Version: master-dockerproject-2022-03-26
 Storage Driver: windowsfilter
  Windows:
 Logging Driver: json-file
 Plugins:
  Volume: local
  Network: ics internal l2bridge l2tunnel nat null overlay private transparent
  Log: awslogs etwlogs fluentd gcplogs gelf json-file local logentries splunk syslog
 Swarm: active
  NodeID: qtua9l1p2l9halezs4l0ae6uv
  Is Manager: true
  ClusterID: ua8jix4r9tihua2fo75pblyb6
  Managers: 1
  Nodes: 1
  Default Address Pool: 10.0.0.0/8
  SubnetSize: 24
  Data Path Port: 4789
  Orchestration:
   Task History Retention Limit: 5
  Raft:
   Snapshot Interval: 10000
   Number of Old Snapshots to Retain: 0
   Heartbeat Tick: 1
   Election Tick: 10
  Dispatcher:
   Heartbeat Period: 5 seconds
  CA Configuration:
   Expiry Duration: 3 months
   Force Rotate: 0
  Autolock Managers: false
  Root Rotation In Progress: false
  Node Address: 192.168.58.4
  Manager Addresses:
   192.168.58.4:2377
 Default Isolation: process
 Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
 Operating System: Microsoft Windows Server Version 1809 (OS Build 17763.737)
 OSType: windows
 Architecture: x86_64
 CPUs: 2
 Total Memory: 16GiB
 Name: DEV01
 ID: BX6Y:OQBU:JRIL:W4A6:X6O2:77H5:I3BJ:KUMB:PR33:SICG:OHB5:ZXZF
 Docker Root Dir: C:\ProgramData\docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Additional environment details:
The Windows Server 2019 instance I am running docker in is a VM on a Windows 10 host machine. Other operations with running containers directly, or using compose files that do not rely on environment files works fine.

[Lenormju]

I could not reproduce just by creating an empty .env file.
But I did reproduce it this way :

$ python3 -c 'with open(".env", "wb") as file: file.write(bytes("\uFEFF", "utf-8"))'
$ docker compose --env-file .env up
unexpected character "»" in variable name near "\ufeff"

Notably, (U+FEFF) is the famous "Zero Width No-Break Space" which is known for breaking things.
But like you said :

I have triple checked that there is no hidden unicode characters in the file, and the failure occurs even on totally empty .env files.
So I don't know ...

[Lenormju]

Indeed, U+FEFF is used as UTF-16 Byte Order Mark. Did it got automatically added by your IDE on Windows, as suggested in the PR comment ?

[milas]

I made a fix for this in compose-spec/compose-go#301, which is the upstream library that handles the actual parsing.

We'll get a new version of that tagged and included in the next release of Compose 🙂

[milas]

This should be fixed in the latest release of Compose (v2.11.0+).