Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General improvements to HTTPS API #1255

Merged
merged 39 commits into from
May 27, 2020
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
7dc662e
Added checkServerIdentity to options
Giotino May 12, 2020
687a23a
Fixed HTTPS tests
Giotino May 12, 2020
263bebe
Added checkServerIdentity tests
Giotino May 12, 2020
750d8cf
First draft of the HTTPS documentation
Giotino May 13, 2020
0ac003a
Merge branch 'master' into checkServerIdentity
Giotino May 13, 2020
49fcb1a
Update readme.md
Giotino May 13, 2020
6aa19ed
Update readme.md
Giotino May 13, 2020
a283a5f
Update readme.md
Giotino May 13, 2020
401cbde
Update readme.md
Giotino May 13, 2020
565376c
Some HTTPS properties renamed
Giotino May 13, 2020
99b28dc
Damn whitespace
Giotino May 13, 2020
dfb682e
Added `key` type
Giotino May 14, 2020
314308a
Update readme.md
Giotino May 14, 2020
6463c8e
Updated documentation with requested links
Giotino May 14, 2020
b205d9b
Added validation for the new options
Giotino May 14, 2020
b5b3cbe
Added "WARNING: " before the two warnings
Giotino May 15, 2020
b3ef4cc
Updated documentation
Giotino May 15, 2020
d0ecc71
Added `https` option
Giotino May 16, 2020
cb17d88
Small documentation fixes
Giotino May 16, 2020
98d312d
Removed deprecation warning name parameter
Giotino May 16, 2020
4a3f8bd
Merge branch 'master' into checkServerIdentity
Giotino May 16, 2020
45f86ad
Fixed deprecation warning tests
Giotino May 16, 2020
4656276
Update source/utils/deprecation-warning.ts
Giotino May 16, 2020
25e7bd1
Little test cleanup
Giotino May 16, 2020
b9eeb99
Update source/utils/deprecation-warning.ts
Giotino May 16, 2020
73655ce
Update readme.md
Giotino May 21, 2020
6fe45a0
Update readme.md
Giotino May 21, 2020
b4eeb02
Update test/https.ts
Giotino May 21, 2020
257d0a7
Update test/https.ts
Giotino May 21, 2020
59bace2
Update readme.md
Giotino May 21, 2020
3dafcd3
Update readme.md
Giotino May 21, 2020
90b60b2
Update readme.md
Giotino May 21, 2020
a8d02ff
Update test/https.ts
Giotino May 21, 2020
785c028
Updated tests with `p-event`
Giotino May 27, 2020
bd87e0a
Update readme.md
Giotino May 27, 2020
0f9e1b3
Update source/core/index.ts
Giotino May 27, 2020
a25c288
Update test/https.ts
Giotino May 27, 2020
8054797
Applied requested changes
Giotino May 27, 2020
66ee8d3
Merge remote-tracking branch 'origin/checkServerIdentity' into checkS…
Giotino May 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
112 changes: 112 additions & 0 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ For browser usage, we recommend [Ky](https://github.com/sindresorhus/ky) by the
- [Errors with metadata](#errors)
- [JSON mode](#json-mode)
- [WHATWG URL support](#url)
- [HTTPS](#https)
Giotino marked this conversation as resolved.
Show resolved Hide resolved
- [Hooks](#hooks)
- [Instances with custom defaults](#instances)
- [Types](#types)
Expand Down Expand Up @@ -819,6 +820,92 @@ Type: `string`

The IP address used to send the request from.

### HTTPS
Giotino marked this conversation as resolved.
Show resolved Hide resolved

##### ca
Giotino marked this conversation as resolved.
Show resolved Hide resolved

Type: `string` | `string[]` | `Buffer` | `Buffer[]`
Giotino marked this conversation as resolved.
Show resolved Hide resolved

Override the default CAs ([from Mozilla](https://ccadb-public.secure.force.com/mozilla/IncludedCACertificateReport))
Giotino marked this conversation as resolved.
Show resolved Hide resolved

```js
// Single CA
got('https://example.com', {ca: fs.readFileSync('./my_ca.pem')});
Giotino marked this conversation as resolved.
Show resolved Hide resolved

// Multiple CAs
got('https://example.com', {
ca: [
fs.readFileSync('./my_ca1.pem'),
fs.readFileSync('./my_ca2.pem')
]
});
Giotino marked this conversation as resolved.
Show resolved Hide resolved
```

##### key
Giotino marked this conversation as resolved.
Show resolved Hide resolved

Type: `string` | `string[]` | `Buffer` | `Buffer[]` | `Object[]`

Private keys in PEM format.\
Giotino marked this conversation as resolved.
Show resolved Hide resolved
PEM allows the option of private keys being encrypted. Encrypted keys will be decrypted with `options.passphrase`.\
Multiple keys with different passphrases can be provided as an array of `{pem: <string|Buffer>, passphrase: <string>}`
Giotino marked this conversation as resolved.
Show resolved Hide resolved

##### cert
Giotino marked this conversation as resolved.
Show resolved Hide resolved

Type: `string` | `string[]` | `Buffer` | `Buffer[]`

Cert chains in PEM format.\
Giotino marked this conversation as resolved.
Show resolved Hide resolved
One cert chain should be provided per private key (`options.key`).\
When providing multiple cert chains, they do not have to be in the same order as their private keys in `options.key`.\
If the intermediate certificates are not provided, the peer will not be able to validate the certificate, and the handshake will fail.

##### passphrase

Type: `string`

The passphrase to decrypt the `key` (if different keys have different passphrases refer to `key` documentation).

Giotino marked this conversation as resolved.
Show resolved Hide resolved

##### Examples for key, cert and passphrase
Giotino marked this conversation as resolved.
Show resolved Hide resolved

```js
// Single key with cert
Giotino marked this conversation as resolved.
Show resolved Hide resolved
got('https://example.com', {
key: fs.readFileSync('./client_key.pem'),
cert: fs.readFileSync('./client_cert.pem'),
Giotino marked this conversation as resolved.
Show resolved Hide resolved
});

// Multiple keys with certs (out of order)
Giotino marked this conversation as resolved.
Show resolved Hide resolved
got('https://example.com', {
key: [
fs.readFileSync('./client_key1.pem'),
fs.readFileSync('./client_key2.pem')
],
cert: [
fs.readFileSync('./client_cert2.pem'),
fs.readFileSync('./client_cert1.pem')
]
});

// Single key with passphrase
got('https://example.com', {
key: fs.readFileSync('./client_key.pem'),
cert: fs.readFileSync('./client_cert.pem'),
passphrase: 'client_key_passphrase'
});

// Multiple keys with different passphrases
got('https://example.com', {
key: [
{pem: fs.readFileSync('./client_key1.pem'), passphrase: 'passphrase1'},
{pem: fs.readFileSync('./client_key2.pem'), passphrase: 'passphrase2'},
],
cert: [
fs.readFileSync('./client_cert1.pem'),
fs.readFileSync('./client_cert2.pem')
]
});
```

##### rejectUnauthorized

Type: `boolean`\
Expand All @@ -844,6 +931,31 @@ const got = require('got');
})();
```

##### checkServerIdentity

Type: `Function`\
Signature: `(hostname: string, certificate: DetailedPeerCertificate) => Error | undefined`\
Default: `tls.checkServerIdentity` (from the `tls` module)

This function enable a custom check of the certificate.\
N.B. In order to have the function called the certificate must not be `expired`, `self-signed` or with an `untrusted-root`.\
Giotino marked this conversation as resolved.
Show resolved Hide resolved
The function parameters are
Giotino marked this conversation as resolved.
Show resolved Hide resolved
- `hostname`: the server hostname (used when connecting)
Giotino marked this conversation as resolved.
Show resolved Hide resolved
- `certificate`: the server certificate
Giotino marked this conversation as resolved.
Show resolved Hide resolved

The function must return `undefined` if the check succeeded or and `Error` if it failed.
Giotino marked this conversation as resolved.
Show resolved Hide resolved

```js
await got('https://example.com', {
checkServerIdentity: (hostname, certificate) => {
if (hostname === 'example.com')
Giotino marked this conversation as resolved.
Show resolved Hide resolved
return; // Certificate OK

return new Error('Invalid Hostname');
}
});
```

#### Response

The response object will typically be a [Node.js HTTP response stream](https://nodejs.org/api/http.html#http_class_http_incomingmessage), however, if returned from the cache it will be a [response-like object](https://github.com/lukechilds/responselike) which behaves in the same way.
Expand Down
3 changes: 2 additions & 1 deletion source/core/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import {Duplex, Writable, Readable} from 'stream';
import {ReadStream} from 'fs';
import {URL, URLSearchParams} from 'url';
import {Socket} from 'net';
import {SecureContextOptions} from 'tls';
import {SecureContextOptions, DetailedPeerCertificate} from 'tls';
szmarczak marked this conversation as resolved.
Show resolved Hide resolved
import http = require('http');
import {ClientRequest, RequestOptions, IncomingMessage, ServerResponse, request as httpRequest} from 'http';
import https = require('https');
Expand Down Expand Up @@ -142,6 +142,7 @@ export interface Options extends URLOptions, SecureContextOptions {
allowGetBody?: boolean;
lookup?: CacheableLookup['lookup'];
rejectUnauthorized?: boolean;
checkServerIdentity?: (hostname: string, certificate: DetailedPeerCertificate) => Error | void;
headers?: Headers;
methodRewriting?: boolean;

Expand Down
3 changes: 2 additions & 1 deletion test/helpers/with-server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ const generateHook = ({install, options: testServerOptions}: {install?: boolean;
const server = await createTestServer(is.plainObject(testServerOptions) ? testServerOptions : {
bodyParser: {
type: () => false
}
},
certificate: 'example.com'
}) as ExtendedTestServer;

const options: InstanceDefaults = {
Expand Down
69 changes: 66 additions & 3 deletions test/https.ts
Original file line number Diff line number Diff line change
@@ -1,27 +1,90 @@
import test from 'ava';
import got from '../source';
import withServer from './helpers/with-server';
import {PeerCertificate} from 'tls';

test('https request without ca', withServer, async (t, server, got) => {
server.get('/', (_request, response) => {
response.end('ok');
});

t.truthy((await got({rejectUnauthorized: false})).body);
t.truthy((await got.secure({rejectUnauthorized: false})).body);
});

test('https request with ca', withServer, async (t, server, got) => {
server.get('/', (_request, response) => {
response.end('ok');
});

const {body} = await got({
const {body} = await got.secure({
ca: server.caCert,
headers: {host: 'sindresorhus.com'}
headers: {host: 'example.com'}
});

t.is(body, 'ok');
});

test('https request with checkServerIdentity OK', withServer, async (t, server, got) => {
Giotino marked this conversation as resolved.
Show resolved Hide resolved
server.get('/', (_request, response) => {
response.end('ok');
});

const {body} = await got.secure({
ca: server.caCert,
checkServerIdentity: (hostname: string, certificate: PeerCertificate) => {
t.is(hostname, 'example.com');
t.is(certificate.subject.CN, 'example.com');
t.is(certificate.issuer.CN, 'localhost');
},
headers: {host: 'example.com'}
});

t.is(body, 'ok');
});

test('https request with checkServerIdentity NOT OK', withServer, async (t, server, got) => {
Giotino marked this conversation as resolved.
Show resolved Hide resolved
server.get('/', (_request, response) => {
response.end('ok');
});

const promise = got.secure({
ca: server.caCert,
checkServerIdentity: (hostname: string, certificate: PeerCertificate) => {
t.is(hostname, 'example.com');
t.is(certificate.subject.CN, 'example.com');
t.is(certificate.issuer.CN, 'localhost');

return new Error('CUSTOM_ERROR');
},
headers: {host: 'example.com'}
});

await t.throwsAsync(
promise,
{
message: 'CUSTOM_ERROR'
}
);
});

test('https request with expired certificate', async t => {
await t.throwsAsync(
got('https://expired.badssl.com/'),
{
code: 'CERT_HAS_EXPIRED'
}
);
});

test('https request with wrong host', async t => {
await t.throwsAsync(
got('https://wrong.host.badssl.com/'),
{
code: 'ERR_TLS_CERT_ALTNAME_INVALID'
}
);
});

test('http2', async t => {
const promise = got('https://httpbin.org/anything', {
http2: true
Expand Down