Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add options to customize parsing/stringifying JSON #1298

Merged
merged 23 commits into from
Jun 5, 2020
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
17 changes: 17 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,6 +317,23 @@ const body = await got(url).json();
const body = await got(url, {responseType: 'json', resolveBodyOnly: true});
```

###### parseJson

Type: `(text: string) => unknown`\
Default: `(text) => JSON.parse(text)`
Giotino marked this conversation as resolved.
Show resolved Hide resolved

Function used to parse JSON responses.

Example:
Giotino marked this conversation as resolved.
Show resolved Hide resolved

```js
Giotino marked this conversation as resolved.
Show resolved Hide resolved
const Bourne = require('@hapi/bourne');

const parsed = await got('https://example.com', {
parseJson: text => Bourne.parse(text)
}).json();
```

###### resolveBodyOnly

Type: `boolean`\
Expand Down
6 changes: 3 additions & 3 deletions source/as-promise/core.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import {
ParseError,
Response
} from './types';
import Request, {knownHookEvents, RequestError, Method} from '../core';
import Request, {knownHookEvents, RequestError, Method, ParseJsonFn} from '../core';

if (!knownHookEvents.includes('beforeRetry' as any)) {
knownHookEvents.push('beforeRetry' as any, 'afterResponse' as any);
Expand All @@ -17,7 +17,7 @@ if (!knownHookEvents.includes('beforeRetry' as any)) {
export const knownBodyTypes = ['json', 'buffer', 'text'];

// @ts-ignore The error is: Not all code paths return a value.
export const parseBody = (response: Response, responseType: ResponseType, encoding?: string): unknown => {
export const parseBody = (response: Response, responseType: ResponseType, parseJson: ParseJsonFn, encoding?: string): unknown => {
const {rawBody} = response;

try {
Expand All @@ -26,7 +26,7 @@ export const parseBody = (response: Response, responseType: ResponseType, encodi
}

if (responseType === 'json') {
return rawBody.length === 0 ? '' : JSON.parse(rawBody.toString()) as unknown;
return rawBody.length === 0 ? '' : parseJson(rawBody.toString());
}

if (responseType === 'buffer') {
Expand Down
4 changes: 2 additions & 2 deletions source/as-promise/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ export default function asPromise<T>(options: NormalizedOptions): CancelableRequ

// Parse body
try {
response.body = parseBody(response, options.responseType, options.encoding);
response.body = parseBody(response, options.responseType, options.parseJson, options.encoding);
} catch (error) {
// Fallback to `utf8`
response.body = rawBody.toString();
Expand Down Expand Up @@ -244,7 +244,7 @@ export default function asPromise<T>(options: NormalizedOptions): CancelableRequ
// Wait until downloading has ended
await promise;

return parseBody(globalResponse, responseType, options.encoding);
return parseBody(globalResponse, responseType, options.parseJson, options.encoding);
})();

Object.defineProperties(newPromise, Object.getOwnPropertyDescriptors(promise));
Expand Down
4 changes: 4 additions & 0 deletions source/core/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ type CacheableRequestFn = (
) => CacheableRequest.Emitter;

type CheckServerIdentityFn = (hostname: string, certificate: DetailedPeerCertificate) => Error | void;
export type ParseJsonFn = (text: string) => unknown;
Giotino marked this conversation as resolved.
Show resolved Hide resolved

interface RealRequestOptions extends https.RequestOptions {
checkServerIdentity: CheckServerIdentityFn;
Expand Down Expand Up @@ -150,6 +151,7 @@ export interface Options extends URLOptions {
lookup?: CacheableLookup['lookup'];
headers?: Headers;
methodRewriting?: boolean;
parseJson?: ParseJsonFn;

// From `http.RequestOptions`
localAddress?: string;
Expand Down Expand Up @@ -201,6 +203,7 @@ export interface NormalizedOptions extends Options {
methodRewriting: boolean;
username: string;
password: string;
parseJson: ParseJsonFn;
[kRequest]: HttpRequestFunction;
[kIsNormalizedAlready]?: boolean;
}
Expand All @@ -224,6 +227,7 @@ export interface Defaults {
allowGetBody: boolean;
https?: HTTPSOptions;
methodRewriting: boolean;
parseJson?: ParseJsonFn;
szmarczak marked this conversation as resolved.
Show resolved Hide resolved
szmarczak marked this conversation as resolved.
Show resolved Hide resolved

// Optional
agent?: Agents | false;
Expand Down
3 changes: 2 additions & 1 deletion source/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,8 @@ const defaults: InstanceDefaults = {
countLimit: Infinity,
requestLimit: 10000,
stackAllItems: true
}
},
parseJson: (text: string) => JSON.parse(text)
},
handlers: [defaultHandler],
mutableDefaults: false
Expand Down
9 changes: 9 additions & 0 deletions test/response-parse.ts
Original file line number Diff line number Diff line change
Expand Up @@ -242,3 +242,12 @@ test('responseType is optional when using template', withServer, async (t, serve

t.deepEqual(body, data);
});

test('JSON response custom parser', withServer, async (t, server, got) => {
server.get('/', defaultHandler);

t.deepEqual((await got({
responseType: 'json',
parseJson: text => ({...JSON.parse(text), custom: 'parser'})
})).body, {...dog, custom: 'parser'});
});