s3.selectObjectContent continues receiving data after abort

[letto4135]

Describe the bug

When using selectObjectContent if the client aborts the request the request continues to receive payloads.

"express": "^4.17.2",
"aws-sdk": "^2.1388.0",

Expected Behavior

selectObjectContent ends properly

Current Behavior

selectObjectContent continues to receive payloads

Reproduction Steps

  app.get("/query", async (req: any, res, next) => {
      const s3options = {
        Bucket: options.s3Bucket as string,
        Key: getFeedObjectPath(),
      };
      const selectRequest = createSelectRequest(req, s3options);
      s3.selectObjectContent(selectRequest, async (err, data) => {
        if (err) {
          return console.error(err);
        }
        let eventStream: SelectObjectContentEventStream = data.Payload as ReadStream;
        eventStream.on("data", (event) => {
          if (req.aborted) {
            logger.debug("Request closed");
          }
        });
        eventStream.on("error", (err) => {
          logger.error("Error in /query", err);
        });
        eventStream.on("end", () => {
          logger.debug("End of /query");
        });
      });
  });

make a request to /query, cancel it before it finishes. "Request closed" is continuously logged. "Error in /query" is never logged.

Possible Solution

No response

Additional Information/Context

A sample of the expression being used is this

    {
      ExpressionType: "SQL",
      InputSerialization: {
        CompressionType: "GZIP",
        JSON: {
          Type: "LINES",
        },
      },
      OutputSerialization: {
        JSON: {
          RecordDelimiter: "\n",
        },
      },
    },
    {
      'select * from S3Object s where 1 = 1 limit 10000',
    },

every 5000 records or so its 5-7mb

SDK version used

2.1388.0

Environment details (OS name and version, etc.)

mac ventura 13.3.1, node:latest, and node 16.2.0

[letto4135]

Any ideas on this? I still can't figure out how to stop the request...

[yenfryherrerafeliz]

Hi @letto4135, thanks for opening this issue. Basically this is not an issue with the SDK, rather is just that you are not implementing any logic to stop the request from the SDK in case of any failures of the request from the web server. What I mean with this is that by aborting a request from the client not necessarily mean that it will stop any process started by that same request.

Something I can suggest you is to add an on error event to the request from the web server, and abort the request done with the SDK in case of any failures.
Here is a quick example using the sample code that you have provided:

app.get("/query", async (req: any, res, next) => {
    const s3options = {
        Bucket: options.s3Bucket as string,
        Key: getFeedObjectPath(),
    };
    const selectRequest = createSelectRequest(req, s3options);
    const s3Request = s3.selectObjectContent(selectRequest, async (err, data) => {
        if (err) {
            return console.error(err);
        }
        let eventStream: SelectObjectContentEventStream = data.Payload as ReadStream;
        eventStream.on("data", (event) => {
            if (req.aborted) {
                logger.debug("Request closed");
            }
        });
        eventStream.on("error", (err) => {
            logger.error("Error in /query", err);
        });
        eventStream.on("end", () => {
            logger.debug("End of /query");
        });
    });

    // Handling failures
    req.on('error', (err) => {
        // To abort just when the request is aborted
        // If you want to abort in case of any failure please remove the if condition
        if (err.message.includes('aborted')) {
            s3Request.abort();
        }
        
    });
});

I hope this helps!

Thanks!

[letto4135]

Thats great, I have a lot of error handling stuff that I left out, but ultimately I was always trying to cancel the request inside s3.selectObjectContent(selectRequest, async (err, data) => {
Thanks @yenfryherrerafeliz

[letto4135]

Actually @yenfryherrerafeliz I do have one more question..

const selectRequest = createSelectRequest(req, s3options);
const { Payload: eventStream } = await s3.selectObjectContent(selectRequest).promise();
await createStreamPipeline(eventStream, res, model);

Do you know how I would be able to cancel the request in this instance?