Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report is null when trying to get a report that is already in JSON by default #214

Open
Philomath88 opened this issue Jun 3, 2023 · 0 comments

Comments

@Philomath88
Copy link

First of all, I think the error is with Amazon response for the GET_SALES_AND_TRAFFIC_REPORT (since it says it is plain but it is already in JSON):

headers: {
'x-amz-id-2': 'rlgFAYDwHlr2LGbKaD5I2RJNCe6VWfKVu+75xr4AC/RwWxYrMXQLEMrbkWEiemDF2TuWYhrNkoU=',
'x-amz-request-id': 'CM5WHTZ3B9X2S4EG',
date: 'Sat, 03 Jun 2023 10:29:09 GMT',
'last-modified': 'Sat, 03 Jun 2023 10:28:52 GMT',
'x-amz-expiration': 'expiry-date="Sat, 02 Sep 2023 00:00:00 GMT", rule-id="Rule for: NinetyDays"',
etag: '"50d09cf57b235b1a66159a41f1c0f43c"',
'x-amz-server-side-encryption': 'aws:kms',
'x-amz-server-side-encryption-aws-kms-key-id': 'arn:aws:kms:eu-west-1:538593642818:key/eefdcca3-2e83-416d-b342-a5f2ccdb7ea5',
'x-amz-server-side-encryption-bucket-key-enabled': 'true',
'x-amz-version-id': '3mnV9LmfZEZsg46QKh_KD3cABN7OWa6T',
'accept-ranges': 'bytes',
'content-type': 'text/plain; charset=utf-8',
server: 'AmazonS3',
'content-length': '5825',
connection: 'close'
}
}

This resulted in the report being null (since the CSV library can't parse a JSON). Since I am a newbie programmer (and I have changed other things in the library, am not comfortable with opening up a PR). I have added functionality to try to parse it as JSON and if it is, skip the CSV step. It works now:

async download(details, options = {}){
options = Object.assign({
unzip:true
}, options);
this._validateEncryptionDetails(details);
// Result will be a tab-delimited flat file or an xml document

function isJSONString(str) {
    try {
        JSON.parse(str);
    } catch (e) {
        return false;
    }
    return true;
}


let res = await request({
  url:details.url,
  timeouts:options.timeouts
});
this._validateUpOrDownloadSuccess(res, 'DOWNLOAD');

console.log('res: ', res);

let decrypted_buffer = this._decryptBuffer(details, res.chunks);



// Decompress if content is compressed and unzip option is true
if (details.compressionAlgorithm && options.unzip){
  decrypted_buffer = await this._unzip(decrypted_buffer);
}

console.log("Decrypted Buffer:", decrypted_buffer.toString());
let decrypted = decrypted_buffer;
if (!details.compressionAlgorithm || options.unzip){
  decrypted = this._decodeBuffer(decrypted, res.headers, options.charset);
  if (options.json) {
        try {
            if (isJSONString(decrypted)) {
                // If content is JSON, parse it directly
                decrypted = JSON.parse(decrypted);
            } else if (res.headers['content-type'].includes('xml')) {
                decrypted = this._xml_parser.parse(decrypted);
            } else if (res.headers['content-type'].includes('plain')) {
                decrypted = await csv({
                    delimiter: '\t',
                    quote: 'off',
                }).fromString(decrypted);
                if (options.debug) {
                    console.log("CSV Parsed JSON:", decrypted);
                }
            }
        } catch (e) {
            if (options.debug) {
                console.error("Error while parsing result to JSON:", e);
            }
            throw new CustomError({
                code: 'PARSE_ERROR',
                message: 'Could not parse result to JSON.',
                details: decrypted,
            });
        }
    }
}
if (options.file){
  await this._saveFile(decrypted, options);
}
return decrypted;

}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant