Add async server.start() function (#4981)

Previously, server startup worked like this:

- `new ApolloServer`
  - If no gateway, calculate schema and schema derived data immediately
  - If gateway, kick off gateway.load from the end of the constructor, and if it
    async-throws, log an error once and make the server kinda broken forever
- At various spots in the framework integration code, call (but don't await)
  the protected `willStart` function, which is an async function that first
  waits for the gateway to load the schema if necessary and then runs
  serverWillStart plugin functions; save the Promise returned by calling this.
- At request time in the framework integration code, await that Promise.
  And also, if there's no schema, fail with an error.

Now server startup works like this:
- ApolloServer represents its state explicitly with a new ServerState
- `new ApolloServer`
  - If no gateway, initialize all the schema-derived state directly like
    before (though the state now lives inside ServerState)
  - If gateway, the constructor DOES NOT KICK OFF `gateway.load()`
- You can now call `await server.start()` yourself, which will first await
  `gateway.load` if necessary, and then await all serverWillStart calls.
- If you're using `apollo-server` rather than an integration, `server.listen()`
  will just transparently do this for you; explicit `start()` is just for
  integrations!
- Serverless frameworks also call it automatically for you in the background
  (kicked off by the constructor) because their startup has to be
  synchronous; if it fails then future requests will all fail (and log) as before.
- The integration places that used to call willStart now call
  `server.ensureStarting()` instead which will kick off server.start in the
  background if you didn't (and log any errors thrown).
- The places that used to await promiseWillStart no longer do so; generally
  right after that code we end up calling `graphqlServerOptions`
- `graphqlServerOptions` now awaits `server.ensureStarted` which will start the
  server if necessary and throw if it threw.

The overall change to user experience:
- If you're using `apollo-server`, startup errors will cause `listen` to reject;
  no code changes are necessary.
- If you're using a serverless integration, the behavior will be relatively similar,
  except that the startup error will be logged on all requests instead of just
  the first one.
- If you're using an integration you are encouraged to call `await
  server.start()` yourself immediately after the constructor, which will let
  you detect startup errors.
- But if you don't do that, the server will call `start` itself eventually. When
  you try to execute your first GraphQL request, `start` will happen if it
  hasn't already. Also an integration call like `server.applyMiddleware` will
  initiate a background `start`. If startup fails, the startup error will be
  logged on *every* failed graphql request, not just the first time like
  happened before.
- If you have your own ApolloServer subclass that calls the protected
  `willStart` method, it will still work (the method isn't deleted) but you
  should rewrite it to either `await this.start()` or `this.ensureStarting()` instead.

This is close enough to backwards-compatible to be appropriate for a v2 minor
release. We are likely to make `start()` required in Apollo Server 3 for
non-serverless integrations.

Also:
- Previously we used the deprecated `ApolloServer.schema` field to determine
  whether to install ApolloServerPluginInlineTrace, which we want to have active
  by default for federated schemas only. If you're using a gateway, this field
  isn't actually set at the time that ensurePluginInstantiation reads it.
  That's basically OK because we don't want to turn on the plugin automatically
  in the gateway, but in the interest of avoiding use of the deprecated field, I
  refactored it so that `ApolloServerPluginInlineTrace` is installed by default
  (ie, if you don't install your own version or install
  `ApolloServerPluginInlineTraceDisabled`) without checking the schema, and
  then (if it's installed automatically) it decides whether or not to be active
  by checking the schema at `serverWillStart` time.
- Similarly, schema reporting now throws in its `serverWillStart` if the schema
  is federated, instead of in `ensurePluginInstantiation`. (This does mean that
  if you're not using the new `start()` or `apollo-server`, that failure won't
  make your app fail as fast as if the `ApolloServer` constructor threw.)
- Fix some fastify tests that used a fixed listen port to not do that.
- I am doing my best to never accidentally run `prettier` on whole files and
  instead to very carefully select specific blocks of the file to format them
  several times per minute. Apparently I screwed up once and ran it once on
  `packages/apollo-server-core/src/ApolloServer.ts`. The ratio of "prettier
  changes" to "actual changes" in that file is low enough that I'd rather just
  leave the changes in this PR rather than spending time carefully reverting
  them. (It's one of the files I work on the most and being able to keep it
  prettier-clean will be helpful anyway.)
- Replace a hacky workaround for the lack of `start` in the op reg tests!
- Replace a use of a `Barrier` class I added recently in tests with the
  `@josephg/resolvable` npm package, which does basically the same thing.
  Use that package in new tests and in the core state machine itself.
- While running tests I found that some test files hung if run separately due to
  lack of cleanup. I ended up refactoring the cache tests to:
  - make who is responsible for calling cache.close more consistent
  - make the Redis client mocks self-contained mocks of the ioredis API instead
    of starting with an actual ioredis implementation and mocking out some
    internals
  - clean up Jest fake timers when a certain test is done
  I'm not super certain exactly which of these changes fixed the hangs but it
  does seem better this way. (Specifically I think the fake timer fix, which I
  did last, is what actually fixed it, but the other changes made it easier for
  me to reason about what was going on.) Can factor out into another PR if
  helpful.

Fixes #4921. Fixes apollographql/federation#335.

Co-authored-by: Stephen Barlow <stephen@apollographql.com>

CHANGELOG.md

@@ -11,6 +11,8 @@ The version headers in this history reflect the versions of Apollo Server itself
 
 > The changes noted within this `vNEXT` section have not been released yet.  New PRs and commits which introduce changes should include an entry in this `vNEXT` section as part of their development.  With few exceptions, the format of the entry should follow convention (i.e., prefix with package name, use markdown `backtick formatting` for package names and code, suffix with a link to the change-set à la `[PR #YYY](https://link/pull/YYY)`, etc.).  When a release is being prepared, a new header will be (manually) created below and the appropriate changes within that release will be moved into the new section.
 
+- Improve startup error handling by ensuring that your server has loaded its schema and executed its `serverWillStart` handlers successfully before starting an HTTP server. If you're using the `apollo-server` package, no code changes are necessary. If you're using an integration such as `apollo-server-express` that is not a "serverless framework", you can insert [`await server.start()`](https://www.apollographql.com/docs/apollo-server/api/apollo-server/#start) between `server = new ApolloServer()` and `server.applyMiddleware`. (If you don't call `server.start()` yourself, your server will still work, but the previous behavior of starting a web server that may fail to load its schema still applies.) The serverless framework integrations (Lambda, Azure Functions, and Cloud Functions) do not support this functionality. While the protected method `willStart` still exists for backwards compatibility, you should replace calls to it with `start` or the new protected method `ensureStarting`. [PR #4981](https://github.com/apollographql/apollo-server/pull/4981)
+
 ## v2.21.2
 
 - `apollo-server-core`: The `SIGINT` and `SIGTERM` signal handlers installed by default (when not disabled by `stopOnTerminationSignals: false`) now stay active (preventing process termination) while the server shuts down, instead of letting a second signal terminate the process. The handlers still re-signal the process after `this.stop()` concludes. Also, if `this.stop()` throws, the signal handlers will now log and exit 1 instead of throwing an uncaught exception. [Issue #4931](https://github.com/apollographql/apollo-server/issues/4931)

docs/source/api/apollo-server.md

@@ -617,7 +617,7 @@ A lifecycle hook that's called whenever a subscription connection is terminated
 
 #### `listen`
 
-> This method is provided only by the `apollo-server` package. If you're integrating with Node.js middleware via a different package (such as `apollo-server-express`), instead see [`applyMiddleware`](#applymiddleware).
+> This method is provided only by the `apollo-server` package. If you're integrating with Node.js middleware via a different package (such as `apollo-server-express`), instead see both [`start`](#start) and [`applyMiddleware`](#applymiddleware).
 
 Instructs Apollo Server to begin listening for incoming requests:
 
@@ -697,11 +697,35 @@ The full URL of the server's subscriptions endpoint.
 </tbody>
 </table>
 
+#### `start`
+
+The async `start` method instructs Apollo Server to prepare to handle incoming operations.
+
+> Call `start` **only** if you are using a [middleware integration](../integrations/middleware/) for a non-"serverless" environment (e.g., `apollo-server-express`).
+>
+> * If you're using the core `apollo-server` library, call [`listen`](#listen) instead.
+> * If you're using a "serverless" middleware integration (such as `apollo-server-lambda`), this method isn't necessary because the integration doesn't distinguish between starting the server and serving a request.
+
+Always call `await server.start()` *before* calling `server.applyMiddleware` and starting your HTTP server. This allows you to react to Apollo Server startup failures by crashing your process instead of starting to serve traffic.
+
+##### Triggered actions
+
+The `start` method triggers the following actions:
+
+1. If your server is a [federated gateway](https://www.apollographql.com/docs/federation/managed-federation/overview/), it attempts to fetch its schema. If the fetch fails, `start` throws an error.
+2. Your server calls all of the [`serverWillStart` handlers](../integrations/plugins/#serverwillstart) of your installed plugins. If any of these handlers throw an error, `start` throws an error.
+
+##### Backward compatibility
+
+To ensure backward compatibility, calling `await server.start()` is optional. If you don't call it yourself, your integration package invokes it when you call `server.applyMiddleware`. Incoming GraphQL operations wait to execute until Apollo Server has started, and those operations fail if startup fails (a redacted error message is sent to the GraphQL client).
+
+We recommend calling `await server.start()` yourself, so that your web server doesn't start accepting GraphQL requests until Apollo Server is ready to process them.
+
 #### `applyMiddleware`
 
 Connects Apollo Server to the HTTP framework of a Node.js middleware library, such as hapi or express.
 
-You call this method instead of [`listen`](#listen) if you're using an `apollo-server-{integration}` package.
+You call this method instead of [`listen`](#listen) if you're using a [middleware integration](../integrations/middleware/), such as `apollo-server-express`. You should call [`await server.start()`](#start) _before_ calling this method.
 
 Takes an `options` object as a parameter. Supported fields of this object are described below.
 
@@ -712,18 +736,24 @@ const express = require('express');
 const { ApolloServer } = require('apollo-server-express');
 const { typeDefs, resolvers } = require('./schema');
 
-const server = new ApolloServer({
-  typeDefs,
-  resolvers,
-});
+async function startApolloServer() {
+  const server = new ApolloServer({
+    typeDefs,
+    resolvers,
+  });
+  await server.start();
 
-const app = express();
+  const app = express();
 
-// Additional middleware can be mounted at this point to run before Apollo.
-app.use('*', jwtCheck, requireAuth, checkScope);
+  // Additional middleware can be mounted at this point to run before Apollo.
+  app.use('*', jwtCheck, requireAuth, checkScope);
 
-// Mount Apollo middleware here.
-server.applyMiddleware({ app, path: '/specialUrl' });
+  // Mount Apollo middleware here.
+  server.applyMiddleware({ app, path: '/specialUrl' });
+  await new Promise(resolve => app.listen({ port: 4000 }, resolve));
+  console.log(`🚀 Server ready at http://localhost:4000${server.graphqlPath}`);
+  return { server, app };
+}
 ```
 
 ##### Options

docs/source/data/subscriptions.mdx

@@ -411,28 +411,29 @@ In case of an authentication error, the Promise will be rejected, which prevents
 
 You can use subscriptions with any of Apollo Server's supported [middleware integrations](../integrations/middleware/). To do so, you call `installSubscriptionHandlers` on your `ApolloServer` instance.
 
-
 This example enables subscriptions for an Express server that uses `apollo-server-express`:
 
 ```js
 const http = require('http');
 const { ApolloServer } = require('apollo-server-express');
 const express = require('express');
 
-const PORT = 4000;
-const app = express();
-const server = new ApolloServer({ typeDefs, resolvers });
-
-server.applyMiddleware({app})
-
-const httpServer = http.createServer(app);
-server.installSubscriptionHandlers(httpServer); // highlight-line
-
-// Make sure to call listen on httpServer, NOT on app.
-httpServer.listen(PORT, () => {
-  console.log(`🚀 Server ready at http://localhost:${PORT}${server.graphqlPath}`)
-  console.log(`🚀 Subscriptions ready at ws://localhost:${PORT}${server.subscriptionsPath}`)
-})
+async function startApolloServer() {
+  const PORT = 4000;
+  const app = express();
+  const server = new ApolloServer({ typeDefs, resolvers });
+  await server.start();
+  server.applyMiddleware({app})
+
+  const httpServer = http.createServer(app);
+  server.installSubscriptionHandlers(httpServer); // highlight-line
+
+  // Make sure to call listen on httpServer, NOT on app.
+  await new Promise(resolve => httpServer.listen(PORT, resolve));
+  console.log(`🚀 Server ready at http://localhost:${PORT}${server.graphqlPath}`);
+  console.log(`🚀 Subscriptions ready at ws://localhost:${PORT}${server.subscriptionsPath}`);
+  return { server, app, httpServer };
+}
 ```
 
 ## Production `PubSub` libraries

docs/source/deployment/azure-functions.md

@@ -64,7 +64,7 @@ npm init -y
 npm install apollo-server-azure-functions graphql
 ```
 
-Copy the code below and paste at you **index.js** file.
+Copy the code below and paste in your **index.js** file.
 
 ```javascript
 const { ApolloServer, gql } = require('apollo-server-azure-functions');
@@ -84,7 +84,6 @@ const resolvers = {
 };
 
 const server = new ApolloServer({ typeDefs, resolvers });
-
 exports.graphqlHandler = server.createHandler();
 ```
 

docs/source/deployment/netlify.md

@@ -90,7 +90,7 @@ exports.handler = server.createHandler();
 
 Now, make sure you've run `NODE_ENV=development npm run start:lambda`, and navigate to `localhost:9000/graphql` in your browser. You should see GraphQL Playground, where you can run queries against your API!
 
-*Note - The GraphQL Playground will only run if your `NODE_ENV` is set to `development`. If you don't pass this, or your `NODE_ENV` is set to `production`, you will not see the GraphQL Playground.* 
+*Note - The GraphQL Playground will only run if your `NODE_ENV` is set to `development`. If you don't pass this, or your `NODE_ENV` is set to `production`, you will not see the GraphQL Playground.*
 
 ![Local GraphQL Server](../images/graphql.png)
 

docs/source/integrations/middleware.md

@@ -36,23 +36,26 @@ const express = require('express');
 const { ApolloServer } = require('apollo-server-express');
 const { typeDefs, resolvers } = require('./schema');
 
-const app = express();
-const server = new ApolloServer({
-  typeDefs,
-  resolvers,
-});
-
-server.applyMiddleware({ app });
-
-app.use((req, res) => {
-  res.status(200);
-  res.send('Hello!');
-  res.end();
-});
-
-app.listen({ port: 4000 }, () =>
-  console.log(`🚀 Server ready at http://localhost:4000${server.graphqlPath}`)
-)
+async function startApolloServer() {
+  const app = express();
+  const server = new ApolloServer({
+    typeDefs,
+    resolvers,
+  });
+  await server.start();
+
+  server.applyMiddleware({ app });
+
+  app.use((req, res) => {
+    res.status(200);
+    res.send('Hello!');
+    res.end();
+  });
+
+  await new Promise(resolve => app.listen({ port: 4000 }, resolve));
+  console.log(`🚀 Server ready at http://localhost:4000${server.graphqlPath}`);
+  return { server, app };
+}
 ```
 
 The parameter you provide to `applyMiddleware` is your middleware's top-level representation of your application. In Express applications, this variable is commonly named `app`.

docs/source/security/terminating-ssl.md

@@ -12,48 +12,51 @@ library](/integrations/middleware/).
 Here's an example that uses HTTPS in production and HTTP in development:
 
 ```javascript
-import express from 'express'
-import { ApolloServer } from 'apollo-server-express'
-import typeDefs from './graphql/schema'
-import resolvers from './graphql/resolvers'
-import fs from 'fs'
-import https from 'https'
-import http from 'http'
-
-const configurations = {
-  // Note: You may need sudo to run on port 443
-  production: { ssl: true, port: 443, hostname: 'example.com' },
-  development: { ssl: false, port: 4000, hostname: 'localhost' }
-}
-
-const environment = process.env.NODE_ENV || 'production'
-const config = configurations[environment]
-
-const apollo = new ApolloServer({ typeDefs, resolvers })
-
-const app = express()
-apollo.applyMiddleware({ app })
-
-// Create the HTTPS or HTTP server, per configuration
-var server
-if (config.ssl) {
-  // Assumes certificates are in a .ssl folder off of the package root. Make sure 
-  // these files are secured.
-  server = https.createServer(
-    {
-      key: fs.readFileSync(`./ssl/${environment}/server.key`),
-      cert: fs.readFileSync(`./ssl/${environment}/server.crt`)
-    },
-    app
-  )
-} else {
-  server = http.createServer(app)
-}
-
-server.listen({ port: config.port }, () =>
+import express from 'express';
+import { ApolloServer } from 'apollo-server-express';
+import typeDefs from './graphql/schema';
+import resolvers from './graphql/resolvers';
+import fs from 'fs';
+import https from 'https';
+import http from 'http';
+
+async function startApolloServer() {
+  const configurations = {
+    // Note: You may need sudo to run on port 443
+    production: { ssl: true, port: 443, hostname: 'example.com' },
+    development: { ssl: false, port: 4000, hostname: 'localhost' },
+  };
+
+  const environment = process.env.NODE_ENV || 'production';
+  const config = configurations[environment];
+
+  const server = new ApolloServer({ typeDefs, resolvers });
+  await server.start();
+
+  const app = express();
+  server.applyMiddleware({ app });
+
+  // Create the HTTPS or HTTP server, per configuration
+  let server;
+  if (config.ssl) {
+    // Assumes certificates are in a .ssl folder off of the package root. Make sure
+    // these files are secured.
+    server = https.createServer(
+      {
+        key: fs.readFileSync(`./ssl/${environment}/server.key`),
+        cert: fs.readFileSync(`./ssl/${environment}/server.crt`)
+      },
+      app,
+    );
+  } else {
+    server = http.createServer(app);
+  }
+
+  await new Promise(resolve => server.listen({ port: config.port }, resolve));
   console.log(
     '🚀 Server ready at',
-    `http${config.ssl ? 's' : ''}://${config.hostname}:${config.port}${apollo.graphqlPath}`
-  )
-)
+    `http${config.ssl ? 's' : ''}://${config.hostname}:${config.port}${server.graphqlPath}`
+  );
+  return { server, app };
+}
 ```

package.json

@@ -57,6 +57,7 @@
   },
   "devDependencies": {
     "@apollographql/graphql-upload-8-fork": "8.1.3",
+    "@josephg/resolvable": "^1.0.0",
     "@types/async-retry": "1.4.2",
     "@types/aws-lambda": "8.10.66",
     "@types/body-parser": "1.19.0",