Potential data corruption issue for multipart stream upload for OCI Python SDK

[bhagwatvyas]

If you are using oci.object_storage.UploadManager.upload_stream, this issue may affect you. This issue is currently under investigation.

Update (1/12): The root cause of the issue causing data corruption has been fixed with the release of OCI Python SDK v2.54.0. Please use oci v2.54.0 or above to avoid data corruption. The behavior of older versions of the OCI Python SDK regarding this issue have been explained below.

Update (12/22): After more investigation and with the release of OCI Python SDK v2.53.1, here's more details about the issue and possible resolutions/workarounds:
Customers using oci.object_storage.UploadManager.upload_stream in FIPS mode may be vulnerable to silent data corruption. The SDK reports success for this operation, but a 0-length object is uploaded. The resolution differs across these scenarios:

1. Using UploadManager.upload_stream() in an environment which uses a FIPS compliant openssl, but the SDK is not operating in FIPS mode:

   • Customers using FIPS compliant openssl but not operating the SDK in FIPS mode using one of the ways mentioned in the OCI Python SDK's docs fall under this scenario.
   • If oci.fips.is_fips_mode() returns True, then the SDK is in FIPS mode.
   • You can verify if you are using a FIPS compliant openssl by running the command openssl version. If "fips" is part of the version name, and you are not operating the SDK in FIPS mode, then you would fall under this scenario.
     Workaround: Upgrade the OCI Python SDK to v2.53.1 or above, and operate the SDK in FIPS mode as mentioned in the OCI Python SDK's docs. Not operating the SDK in FIPS mode while using a FIPS compliant openssl would still result in data corruption with UploadManager.upload_stream().

2. Using UploadManager.upload_stream() when the SDK is operating in FIPS mode, using OCI Python SDK v2.53.0 or lower:

   • Customers operating the SDK in FIPS mode, using one of the ways mentioned in the OCI Python SDK's docs fall under this scenario.
   • If oci.fips.is_fips_mode() returns True, then the SDK is in FIPS mode.
     Resolution: Upgrade the OCI Python SDK to v2.53.1 or above.

Update (12/20): The issue seems to affect these specific scenarios:

• When using oci.object_storage.UploadManager.upload_stream in FIPS mode, regardless of platform.
• In Cloud shell for the same operation regardless of FIPS mode being enabled.

This was reported for OCI CLI which uses the OCI Python SDK's APIs for object upload. More details on the OCI CLI issue here - oracle/oci-cli#490 (comment)

[bhagwatvyas]

Updated the original issue with the specific scenarios being affected. Posting as a comment again so the customers who are watching this issue would be notified by github:

The issue seems to affect these specific scenarios:

• When using oci.object_storage.UploadManager.upload_stream in FIPS mode, regardless of platform.
• In Cloud shell for the same operation regardless of FIPS mode being enabled.

We are testing the fixes for this issue.

[bhagwatvyas]

After more investigation and with the release of OCI Python SDK v2.53.1, here's more details about the issue and possible resolutions/workarounds:
Customers using oci.object_storage.UploadManager.upload_stream in FIPS mode may be vulnerable to silent data corruption. The SDK reports success for this operation, but a 0-length object is uploaded. The resolution differs across these scenarios:

1. Using UploadManager.upload_stream() in an environment which uses a FIPS compliant openssl, but the SDK is not operating in FIPS mode:

   • Customers using FIPS compliant openssl but not operating the SDK in FIPS mode using one of the ways mentioned in the OCI Python SDK's docs fall under this scenario.
   • If oci.fips.is_fips_mode() returns True, then the SDK is in FIPS mode.
   • You can verify if you are using a FIPS compliant openssl by running the command openssl version. If "fips" is part of the version name, and you are not operating the SDK in FIPS mode, then you would fall under this scenario.

   Workaround: Upgrade the OCI Python SDK to v2.53.1 or above, and operate the SDK in FIPS mode as mentioned in the OCI Python SDK's docs. Not operating the SDK in FIPS mode while using a FIPS compliant openssl would still result in data corruption with UploadManager.upload_stream().

2. Using UploadManager.upload_stream() when the SDK is operating in FIPS mode, using OCI Python SDK v2.53.0 or lower:

   • Customers operating the SDK in FIPS mode, using one of the ways mentioned in the OCI Python SDK's docs fall under this scenario.
   • If oci.fips.is_fips_mode() returns True, then the SDK is in FIPS mode.

   Resolution: Upgrade the OCI Python SDK to v2.53.1 or above.

[bhagwatvyas]

The root cause of the issue causing data corruption has been fixed with the release of v2.54.0. Please use oci v2.54.0 or above to avoid data corruption. The behavior of older versions of the OCI Python SDK regarding this issue have been explained in the older comments.