Update Amazon Root 2019 RDS certificate #2280

codebykenny · 2019-10-10T16:24:29Z

Amazon RDS has released their 2019-2024 CA’s and has given a formal warning to all RDS users to upgrade their CA’s to the new one before February 5th 2020.

All new RDS Instances will have the new 2019-2024 CA’s starting in November 2019.

You can read about it here:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL-certificate-rotation.html

And their CA’s are posted here:
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html

This PR adds their new 2019-2024 Global CA that works for all AWS Regions

dougwilson · 2019-10-15T17:20:06Z

In addition this just the profile; you can already supply this new cert directly in the "ca" config of this module, so the merging of this PR does not in any way prevent you from using this cert with any existing version of this module.

AashishAsh · 2019-10-15T17:23:32Z

@dougwilson We updated the cert on the rds cluster from ca-2015 to ca-2019 which went through. but the nodejs application which has a dependency on this ssl_profile is failed due to this cert is not merged yet.

Error:
unable to get local issuer certificate
Error: unable to get local issuer certificate

dougwilson · 2019-10-15T17:25:26Z

Thanks both of you :) so it sounds like there are some options

Set your instance to use the older cert for now
Just supply the new root cert in the ca option https://github.com/mysqljs/mysql/blob/master/Readme.md#ssl-options
Wait for the new release

felixge · 2020-01-14T15:26:54Z

@dougwilson hey, I just saw a few people complain about this on node-mysql@googlegroups.com. I'm happy and grateful to let you run this project as you see fit these days, but I'm wondering what's going on here?

I understand that users can deal with this issue in their applications, but is there anything wrong with just merging this PR? And what's up with all the deleted comments and restricting the issue to repo contributors?

Anyway, you don't owe anybody any timely action on this, but I just wanted to check if everything is okay : ). If hitting merge & cutting a release is all that's needed to make people happy here, I'd also be willing to help with it.

dougwilson · 2020-01-14T15:32:07Z

I'm happy and grateful to let you run this project as you see fit these days, but I'm wondering what's going on here?

I am working to roll out a security update and roll this change into a second module to shared with mysql2.

And what's up with all the deleted comments and restricting the issue to repo contributors?

I was simply deleting all the "me too" and "+1" comments and out of date information I had posted to keep the thread clean and so it wasn't confusing to folks coming by. In the future I will use the "hide comment" feature instead, though it does make the thread still a but longer, but it is a newer feature that I should probably use for everyone's sanity.

Anyway, you don't owe anybody any timely action on this, but I just wanted to check if everything is okay : ). If hitting merge & cutting a release is all that's needed to make people happy here, I'd also be willing to help with it.

Splitting out the profile into a separate module will be better in the long run which is why I'm doing that. It will mean mysql and mysql2 will be able to not have duplicate PRs and we can update the profile more streamline.

As a post script, I am heading to work now, but if this is a hot topic I will merge and release this change at work later today and delay the security fixes until a later release.

P.P.S (sorry, I'm trying not to clutter up this issue, but figure folks are looking here currently) there are is a backlog of things that generally need a 3.0 release, and prior to this I was working to get it put together. I will be opening a meta issue to track and though 2.0 was a good long run, it is time to close out a lot of outstanding issues/features in a 3.0.

felixge · 2020-01-14T15:37:42Z

@dougwilson sounds good, I'll let you handle it : ).

That being said, I'm kinda disappointed. I was hoping for some nice conspiracy. This could have been the beginning of the uprising against Jeff Bezos! TBH I'd probably support that ; ).

Anyway, thanks for all the work on this project! The node community is lucky to have you.

SimonHooker · 2020-01-17T08:30:30Z

As a stop-gap, I copied the entirety of the lib/protocol/constants/ssl_profiles.js file on this PR into a new file. I then updated the connection parameters as follows :

params_for_connection.ssl = 'Amazon RDS';

... becomes ...

const MysqlSSLProfiles = require( './path/to/imported/ssl_profiles.js' );
...
params_for_connection.ssl = MysqlSSLProfiles[ 'Amazon RDS' ];

Hopefully this is of help to anyone that needs to get a band-aid up ahead of this being merged.

dougwilson · 2020-01-17T08:32:38Z

I am heading to sleep now, but I wanted to provide an update on this issue (reminded by the post above): I will have a new minor version with this change published on either Jan 18 or 19 (depending on the time zone you are in).

Edit: I just resumed work to get this done, only to realize I had the dates off by one day, as they should have matched Sat/Sun, so updated them.

Edit: apologies 🙏 this weekend where I live has been having power failures, so I got a lot less done than I anticipated. Power is back and running, though it is about time for me to go to sleep, but no worries as Jan 20 is a holiday for me, and so I will be spending the morning getting a new mysql version published.

brettneese · 2020-01-20T20:08:08Z

@dougwilson thanks for your transparency and keep on keeping on 👍 😄

do let us know if you need any help!!!

CovertLeopard · 2020-01-20T20:09:19Z

Doug, I appreciate the work you are doing and understand you are busy. Many of us are trying to plan migrations to be done before February 5th when AWS starts swapping out the root CA of our RDS instances during the scheduled maintenance, which I believe is why this keeps coming up. I'll continue plugging away at the issue we are having in hopes to find something else, but was hoping I could at least try the new RDS profile implemented by the developers of this module, just in case my implementation of manually specifying the RDS root ca cert is flawed. Thanks.

…

On Mon, Jan 20, 2020 at 3:04 PM Douglas Wilson ***@***.***> wrote: TBH, I'm trying to be open and transparent here, but the more I am just "hounded" the less I feel motivated to get going. Please, can we please stop posting the same stuff over and over so I can just keep my head down and get everything wrapped op? I would very much appreciate it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2280?email_source=notifications&email_token=ACTRJJAMGCXY4JOMKAJH65LQ6X7UFA5CNFSM4I7PHMLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJNWAJI#issuecomment-576413733>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACTRJJHTUFKXY74CKOWTAO3Q6X7UFANCNFSM4I7PHMLA> .

dougwilson · 2020-01-20T20:12:56Z

This profile is just a preset config given to the ssl option of this module... it does nothing you cannot already do. All it is doing is translating the string "Amazon RDS" into the object that is in the ssl_profiles.js ... there is no requirement to wait for this unless you have to just see ssl: "Amazon RDS" in your config.

dougwilson · 2020-01-20T20:17:12Z

hoping I could at least try the new RDS profile implemented by the developers of this module, just in case my implementation of manually specifying the RDS root ca cert is flawed

The time to try it out to make sure your issue is resolved is now, prior to merging and releasing... the changes are in the "Changes" tab above, as everything is open source. If you're not sure these changes will actually work for your issue, ideally if you can find out before I release that will have the best turn around time. That said, the date you gave is over a week away and I promised Jan 20, so what specific time on Jan 20 I'd be surprised that would have a bearing on such a long lead time?

CovertLeopard · 2020-01-20T20:19:55Z

I will take the changes into my local copy and test it right now. Thanks!

…

On Mon, Jan 20, 2020 at 3:13 PM Douglas Wilson ***@***.***> wrote: This profile is just a preset config given to the ca option of this module... it does nothing you cannot already do. All it is doing is translating the string "Amazon RDS" into the object that is in the ssl_profiles.js ... there is no *requirement* to wait for this unless you have to just see ca: "Amazon RDS" in your config. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2280?email_source=notifications&email_token=ACTRJJAHUAH5L3SYGK7LKNLQ6YAU5A5CNFSM4I7PHMLKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJNWTGA#issuecomment-576416152>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACTRJJGANR7N63IEZLADUSDQ6YAU5ANCNFSM4I7PHMLA> .

berzniz · 2020-01-20T20:50:55Z

There is an npm module if you need it to bridge this gap: https://github.com/toriihq/mysql-rds-ca — it's been tested on production in our various servers, with RDS and Aurora.

I think going forward it would make sense to keep that part outside of this package.

dougwilson · 2020-01-20T22:42:34Z

I think going forward it would make sense to keep that part outside of this package.

It is definitely on my mind for when the next major comes up. There are a lot of benefits, from just general decoupling like users who never use Amazon RDS endpoints do not need to have all the certs loaded up, to management where it could be in the hands of folks who use Amazon RDS every day, which would help especially around things like #2276 .

After all these years, the only profile that has never been has been Amazon RDS, so it's not like there are many to choose from... and it seems like the "newer" Amazon MySQL system presents certs that are trusted by Node.js itself by default, and I would guess Amazon RDS would eventually move to that making the need for a profile obsolete at some point.

Plus you can use that module with mysql2 as well, making just one place to keep up-to-date for everyone, nice!

closes mysqljs#2280

closes mysqljs#2280 closes mysqljs#2281

dougwilson · 2020-01-21T09:00:52Z

Hi everyone, as you may have gotten the notice, this PR has been merged. Sorry for the delay, a lot of time was spent waiting for CI to complete on the various changes. The new version is not published just yet, as I have learned my lesson in the past when publishing a release and immediately going to sleep :) I will be publishing in approx 8 hours from now, though you are welcome to test out master before hand. I will also be around after that in case there is a regression or other issue in the new version to address promptly.

dougwilson · 2020-01-22T08:23:27Z

Did anyone in this thread test this PR before I merged and released it? Can someone chime in on #2292 ?

SimonHooker · 2020-01-22T08:30:34Z

@dougwilson as noted in my comment, I manually replicated this PR within my source code and confirmed it operational against a ca-2019 RDS instance using the steps detailed, which I believe should be fundamentally identical to the PR.

However there are additional CA certificates for different access points. This PR only actually included the global one. The additional APs probably also need merging, and then later on a cleanup after March 2020 can remove the 2015-2020 CA certificates as they will be useless att hat point.

jthele · 2020-01-22T08:32:09Z

@SimonHooker: What version of MySQL was your RDS instance running? Perhaps there may be different behavior between older and newer server versions.

SimonHooker · 2020-01-22T08:35:05Z

@jthele MySQL 5.7.26

The certificate added in #2280 was the global CA, however there are intermediates though not sure why it would work for me without the intermediate, but others would require it. It sounds like this may be related to using AWS GovCloud or similar setup, detailed on https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html

For consistency #2292 probably should be merged in also

jthele · 2020-01-22T08:36:55Z

I am not using GovCloud. I am on MySQL 5.6.37.

dougwilson · 2020-01-22T08:45:10Z

Maybe it is a difference in Node.js version being used?

CovertLeopard · 2020-01-22T13:16:58Z

Did anyone in this thread test this PR before I merged and released it? Can someone chime in on #2292 ?

I'm using it with AWS Lambda Functions. v2.18.0 works with nodejs8.10, nodejs10.x and nodejs12.x lambda runtimes in the US-East-1 region when connecting to an Aurora MySQL RDS instance on MySQL 5.7.12 engine. It's a standard account, not GovCloud or any other specialty. I'll add this comment to #2292 as well.

See mysqljs/mysql#2280

ewarren · 2020-01-29T15:32:06Z

Hello,
We have moved from this implementation:

dialectOptions: {
        ssl: 'Amazon RDS'
}

To this instead:

var Sequelize = require('sequelize');
const fs = require('fs');
const rdsCa = fs.readFileSync('rds-combined-ca-bundle.pem');

// configuration
var sequelize = new Sequelize(process.env.DB_DATABASE, process.env.DB_USER, process.env.DB_PASSWORD, {
    dialect: 'mysql',
    dialectOptions: {
        ssl: {
            rejectUnauthorized: true,
            ca: [rdsCa]
        }
    },

However, now we have to build the .pem file into our zip file that we deploy on our EC2 instances.
Is that what is recommended? We never did this before, as just specifiying 'Amazon RDS' as the ssl string under dialectOptions seemed to used to pull the contents of the .pem file automatically.

Is there any risk to us bundling the .pem file into our deployable zip file?

dougwilson mentioned this pull request Oct 15, 2019

Amazon certs expiring in March 2020 #2281

Closed

dashmug mentioned this pull request Oct 15, 2019

Is this library affected by Amazon's new SSL cert? brianmario/mysql2#1073

Closed

mysqljs deleted a comment from AashishAsh Jan 6, 2020

mysqljs deleted a comment from arkintz Jan 6, 2020

mysqljs deleted a comment from AashishAsh Jan 6, 2020

mysqljs deleted a comment from winzig Jan 6, 2020

dougwilson self-assigned this Jan 14, 2020

dougwilson added the feature label Jan 14, 2020

dougwilson force-pushed the master branch 3 times, most recently from d7aaf04 to afb27af Compare January 15, 2020 07:23

This comment has been minimized.

Sign in to view

dougwilson force-pushed the master branch 2 times, most recently from 551786e to ccc106e Compare January 20, 2020 06:19

This comment has been minimized.

Sign in to view

dougwilson pushed a commit to codebykenny/mysql that referenced this pull request Jan 20, 2020

Add new Amazon RDS Root 2019 CA to Amazon RDS SSL profile

7e8ed68

closes mysqljs#2280

dougwilson force-pushed the master branch from ccc106e to 7e8ed68 Compare January 20, 2020 23:00

Add new Amazon RDS Root 2019 CA to Amazon RDS SSL profile

1428049

closes mysqljs#2280 closes mysqljs#2281

dougwilson force-pushed the master branch from 7e8ed68 to 1428049 Compare January 21, 2020 04:32

dougwilson closed this in 1428049 Jan 21, 2020

dougwilson merged commit 1428049 into mysqljs:master Jan 21, 2020

jthele mentioned this pull request Jan 22, 2020

Amazon RDS 2019 Intermediate Certificates #2292

Merged

bubbltrubl added a commit to bubbltrubl/machinepack-mysql that referenced this pull request Jan 22, 2020

Update mysql to 2.18.0 to support new Amazon RDS certificates

427090c

See mysqljs/mysql#2280

bubbltrubl mentioned this pull request Jan 22, 2020

Update mysql to 2.18.0 to support new Amazon RDS certificates sailshq/machinepack-mysql#17

Merged

goldsziggy mentioned this pull request Jan 27, 2020

Update MySQL Package for the latest Amazon RDS support CodeFoodPixels/node-promise-mysql#134

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Amazon Root 2019 RDS certificate #2280

Update Amazon Root 2019 RDS certificate #2280

codebykenny commented Oct 10, 2019

dougwilson commented Oct 15, 2019

AashishAsh commented Oct 15, 2019

dougwilson commented Oct 15, 2019 •

edited

felixge commented Jan 14, 2020 •

edited

dougwilson commented Jan 14, 2020 •

edited

felixge commented Jan 14, 2020

This comment has been minimized.

This comment has been minimized.

SimonHooker commented Jan 17, 2020 •

edited

dougwilson commented Jan 17, 2020 •

edited

This comment has been minimized.

This comment has been minimized.

brettneese commented Jan 20, 2020

CovertLeopard commented Jan 20, 2020 via email

dougwilson commented Jan 20, 2020 •

edited

dougwilson commented Jan 20, 2020

CovertLeopard commented Jan 20, 2020 via email

berzniz commented Jan 20, 2020 •

edited

dougwilson commented Jan 20, 2020

dougwilson commented Jan 21, 2020

dougwilson commented Jan 22, 2020

SimonHooker commented Jan 22, 2020 •

edited

jthele commented Jan 22, 2020

SimonHooker commented Jan 22, 2020 •

edited

jthele commented Jan 22, 2020

dougwilson commented Jan 22, 2020

CovertLeopard commented Jan 22, 2020

ewarren commented Jan 29, 2020 •

edited by dougwilson

Update Amazon Root 2019 RDS certificate #2280

Update Amazon Root 2019 RDS certificate #2280

Conversation

codebykenny commented Oct 10, 2019

dougwilson commented Oct 15, 2019

AashishAsh commented Oct 15, 2019

dougwilson commented Oct 15, 2019 • edited

felixge commented Jan 14, 2020 • edited

dougwilson commented Jan 14, 2020 • edited

felixge commented Jan 14, 2020

This comment has been minimized.

This comment has been minimized.

SimonHooker commented Jan 17, 2020 • edited

dougwilson commented Jan 17, 2020 • edited

This comment has been minimized.

This comment has been minimized.

brettneese commented Jan 20, 2020

CovertLeopard commented Jan 20, 2020 via email

dougwilson commented Jan 20, 2020 • edited

dougwilson commented Jan 20, 2020

CovertLeopard commented Jan 20, 2020 via email

berzniz commented Jan 20, 2020 • edited

dougwilson commented Jan 20, 2020

dougwilson commented Jan 21, 2020

dougwilson commented Jan 22, 2020

SimonHooker commented Jan 22, 2020 • edited

jthele commented Jan 22, 2020

SimonHooker commented Jan 22, 2020 • edited

jthele commented Jan 22, 2020

dougwilson commented Jan 22, 2020

CovertLeopard commented Jan 22, 2020

ewarren commented Jan 29, 2020 • edited by dougwilson

dougwilson commented Oct 15, 2019 •

edited

felixge commented Jan 14, 2020 •

edited

dougwilson commented Jan 14, 2020 •

edited

SimonHooker commented Jan 17, 2020 •

edited

dougwilson commented Jan 17, 2020 •

edited

dougwilson commented Jan 20, 2020 •

edited

berzniz commented Jan 20, 2020 •

edited

SimonHooker commented Jan 22, 2020 •

edited

SimonHooker commented Jan 22, 2020 •

edited

ewarren commented Jan 29, 2020 •

edited by dougwilson