Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove no-return-await as it sets traps during code refactoring and causes inferior stack traces #1442

Closed
CherryDT opened this issue Oct 19, 2019 · 43 comments

Comments

@CherryDT
Copy link

CherryDT commented Oct 19, 2019

What version of this package are you using?
14.3.1

What problem do you want to solve?
The no-return-await rule introduced in #695 is the only one with which I continue to disagree, and I think since the arrival of async stack traces in Node, there is now another reason to rethink it.

Therefore I want to open a discussion here. My point is that the no-return-await rule (which disallows "redundant" use of return await promise) has two negative effects:

1) It reduces the usefulness of error stack traces

Node now has zero-cost async stack traces with the --async-stack-traces flag. There, it makes a difference whether return promise or return await promise is used - the former omits a stack frame.

Example:

const delay = require('util').promisify(setTimeout)
const throws = async () => { await delay(100); throw new Error('oh noez') }

const a = async () => { return throws() }
const b = async () => { return await throws() }

async function main () {
    try { await a() } catch (e) { console.error(e) }
    try { await b() } catch (e) { console.error(e) }
}

main()

Result:

david@CHE-X1:~/z/Temp $ node --async-stack-traces example.js
Error: oh noez
    at throws (/mnt/c/Users/david/Temp/example.js:2:54)
    at async main (/mnt/c/Users/david/Temp/example.js:8:8)
Error: oh noez
    at throws (/mnt/c/Users/david/Temp/example.js:2:54)
    at async b (/mnt/c/Users/david/Temp/example.js:5:32)
    at async main (/mnt/c/Users/david/Temp/example.js:9:8)

Note how there is no stack frame for a.

2) It lays out a trap for code refactoring by making it easy to overlook that an async function is being called

Example with subtle unintended behavior:

return somethingAsync()
// refactor to
try {
  return somethingAsync()
} catch (e) {
  // ...
}
// whoops, now suddenly it makes a difference - errors are caught only
// if they happen in the synchronous part of `somethingAsync` - hard to spot!

Example with more obvious unintended behavior:

return somethingAsync()
// refactor to
return somethingAsync() || null
// whoops, that did nothing, because the promise won't be null

Another example:

return somethingAsync()
// refactor to
const value = somethingAsync()
await moreStuff()
return value
// whoops now somethingAsync runs in parallel to moreStuff, which
// may not be immediately obvious and cause a race condition

Or even:

return somethingAsync()
// refactor to
return { value: somethingAsync() }
// whoops now the outer code gets a promise in an unexpected place
// and if it previously was a boolean, it may not even be noticed, but
// conditions will now get always a truthy value...!

There are many more examples like this which don't immediately crash (it is easy to spot something like somethingAsync().toUpperCase() is not a function, but my examples above are more subtle). If I modify existing code, I may not be familiar with every function that is being called (and I may even use a refactor tool where I just select the returned value and extract to a variable for example). Usually, I'd easily recognize that await somethingAsync() should probably stay this way, but with return somethingAsync() I may not realize that in any place other than the return statement, I'd need an await in front (and even if it stays in a return, other things like the aforementioned try block may suddenly require a change here - and imagine that the return is somewhere inside a larger block of code which I'm now wrapping with try)!

I had all of these issues from my examples happen in real life.

To work around this issue, I manually mark return somethingAsync() with a comment, e.g. return somethingAsync() // async but that seems ugly and much less useful than having the obvious await there (plus, it doesn't solve the try case).

What do you think?

What do you think is the correct solution to this problem?
Remove the no-return-await rule

Are you willing to submit a pull request to implement this change?
I guess so...? (but I guess it would be easier for you)

@mightyiam
Copy link
Member

I value your input, @CherryDT.

So, your claim is that the no-return-await rule is bad practice and that the opposite is good practice.

Your examples make sense to me. Personally, when I am using TypeScript, I am protected from that kind of refactoring errors. But that's just a side note.

Is there discussion in ESLint about the usefulness of no-return-await? Yup. Here it is.

I feel that what @ilyavolodin suggested is reasonable and would allow us in Standard to choose the opposite. Rename the rule to allow the opposite or make a new rule that does the opposite. @CherryDT would you be interested in making that contribution? That would allow you to enjoy your preferred style regardless of what Standard ends up deciding.

And I would appreciate input from more users on this ❤️ .

@LinusU
Copy link
Member

LinusU commented Oct 22, 2019

Sounds very reasonable, it's not fun to lose the frame in the stack trace 👍

@CherryDT
Copy link
Author

CherryDT commented Oct 30, 2019

@mightyiam Unfortunately I'm not familiar with how ESLint rules work internally, I'm not sure right now how to make said contribution. But nonetheless, I appreciate your open ear on this.

Did I understand it correctly in case you would actually reconsider this rule in the Standard, that you would then only consider switching to the opposite rule (hence the need for it to be exist), but not consider merely removing the existing rule from the ruleset?

(Oh and just out of interest - does TypeScript also prevent the try/catch trap?)

@mightyiam
Copy link
Member

Since we have no implementation of an alternative and since the rule seems to me to be more detriment than use, I would have it removed.

@mightyiam
Copy link
Member

@feross

ricealexander added a commit to ricealexander/eslint-config-webtrinkets that referenced this issue Jan 23, 2020
Contrary to ESLint Documentation, returning await has valid use cases. By explicitly marking that the return value needs to be awaited, code is more clear and easier to refactor. See standard/standard#1442
@CherryDT
Copy link
Author

May I ask what the status on this is? I'm just confused whether it is now decided that it'll be removed or not, since @mightyiam wrote it should be removed after all but that was already almost half a year ago. Thank you!

@LinusU
Copy link
Member

LinusU commented Feb 26, 2020

There is some discussion about this in a few places, e.g. mightyiam/eslint-config-love#206, eslint/eslint#12246, typescript-eslint/typescript-eslint#1378

The more I think of it the more I'm leaning towards:

  • Removing the rule completely in the JavaScript part
  • Enforcing the use of return await in the TypeScript part
    • (that is, enforcing return await when you are trying to return a PromiseLike<_> from an async function, the always setting of the @typescript-eslint/return-await rule)

One of the last arguments was that that part of the stack frame can be a "red herring", but I really don't see how that is? When would it not be useful to see the full call-chain that led up the failing call?

In fact, I see the opposite of this all the time in Node.js. You get an error that just says "ENOENT", but you don't see that it was the function loadConfigFile that called fs.readFile. This for me is a really really bad experience, and something that we can prevent here!


@mightyiam how do you feel about this?

@feross it would also be great if you could chime in with your thoughts!

❤️

@mightyiam
Copy link
Member

@LinusU thank you for the input. I am not a big async function user. So I'll wait for more input from others.

@ljharb
Copy link

ljharb commented Feb 27, 2020

If it requires a node flag to get a benefit from return await (one which overrides the cost of writing redundant code), then it should require a node flag to disable the rule.

@CherryDT
Copy link
Author

@ljharb this is secondary. It doesn't require a node flag to mess up your refactoring.

@LinusU
Copy link
Member

LinusU commented Feb 27, 2020

If it requires a node flag to get a benefit from [...]

I'm not aware that you need a specific flag in the newer versions of Node.js?

In fact, as far as I know, it's supported in every currently active release (>= 10.13.0)

Or am I missing something? 🤔

@ljharb
Copy link

ljharb commented Feb 27, 2020

@CherryDT what i mean is, return await, outside of a try, is redundant and incorrect and makes your code artificially slower. If someone wants to incur this cost in exchange for better stack traces in one JS engine, that's fine! but they should have to disable the rule for that.

@CherryDT
Copy link
Author

CherryDT commented Feb 27, 2020

The better stack traces are a nice side effect but the major issue is that it covers up what the code is actually doing, namely waiting for another asynchronous result. Hence all my examples about how it hinders readability and adds traps for later modifications. My point is that it is bad practice to encourage that. I and my colleagues have wasted several hours of our lives due to traps created by this rule. That can't be the goal...

There are many other rules following the philosophy of making the code "normalized" and easier to read and less likely to break on changes (to code or environment), so why should this one actually push into the opposite direction? For example, there is no-path-concat, and I'm quite sure that path.join is slower than + by at least an order of magnitude. Or no-throw-literal, I bet throw new Error('xyz') and if (e && e.message === 'xyz') are slower than throw 'xyz' and if (e === 'xyz'). Yet, these rules are worth the (small) speed difference, since they avoid trouble.

(In the path case, it would break on another platform, and in the error case it would break when someone makes the assumption that errors have a property stack or message for example - just like the assumption that turning return getCustomer() into return getCustomer() || defaultCustomer will work as expected...)

Correct me if I'm wrong, but I thought the goal of StandardJS was to eventually save time developing, increase code clarity and avoid common pitfalls (such as these refactoring issues) - not micro-optimize code. For example, the extra parentheses in if ((x = y)) are also redundant for sure, yet they add an important hint to the reader and avoid human errors - just like return await. (I think the opposite rule should exist - always use return await in this case.)

I quote:

This module saves you (and others!) time in three ways:

[...]

  • Catch style issues & programmer errors early. Save precious code review time by eliminating back-and-forth between reviewer & contributor.

Adopting standard style means ranking the importance of code clarity and community conventions higher than personal style. This might not make sense for 100% of projects and development cultures, however open source can be a hostile place for newbies. Setting up clear, automated contributor expectations makes a project healthier.

(Empasis mine.)

Maybe I should edit this issue to reorder my points, since actually point 2 is the major issue, not point 1. I don't understand why everyone is arguing about "stack traces for one engine" (here and at eslint) when the main problem is another one.

@ljharb
Copy link

ljharb commented Feb 27, 2020

It's an async function, that's what makes it wait for an async result. The extra await is what covers up what the async function is actually doing.

When I say "slower", i don't mean "takes a few extra ms", i mean "takes another tick" which might be seconds, depending on your app.

@CherryDT
Copy link
Author

CherryDT commented Feb 27, 2020

It's an async function, that's what makes it wait for an async result. The extra await is what covers up what the async function is actually doing.

That may be technically right but logically wrong. This is not how humans think. Again, the standard should aid humans, not computers. When I'm modifying one line of code I'm not always scrolling up and down and reading the whole rest of the file in detail. Plus, here the point is not that it covers up the async function's job, but the fact that the thing I'm returning is also an async function call so it requires special care upon almost any modification, even if it's done at a totally different point in my function (such as wrapping a piece of code in try - I'm repeating myself now, but in my example above the issue is that you'd have to read all the lines inside your try and see if any of them return something that itself is a promise, and that was just one of the traps I mentioned!)... Having the await there would instantly remove all those traps and avoid subtle errors that are sometimes detected only much later on.

When I say "slower", i don't mean "takes a few extra ms", i mean "takes another tick" which might be seconds, depending on your app.

No it doesn't take another tick.

node:

image

Chrome (assuming that setTimeout(..., 0) is equivalent to setImmediate in node):

image

It comes actually at the end of the current tick (after the other synchronous code, but before going idle - not even the next tick, if you look closely!) and not the next-next-next-next-next tick despite my ridiculous use of multiple awaits and Promise.resolves.

@ljharb
Copy link

ljharb commented Feb 27, 2020

It's how this human thinks, and how the JavaScript language works. Something that encourages an incorrect understanding of what the language does is worse for humans, even if it appears to be better for some humans in the short term.

You're right, "tick" is the wrong term. It's a new microtask, though.

@CherryDT
Copy link
Author

Sorry, I didn't mean to call you not human. That didn't come out right.

Anyway, it seems you are not addressing any of the code clarity and refactoring-traps issues that I mentioned, from what I understand your point is that it is more important to work "low-level" than to create maintainable code. In this case let's just agree to disagree because I'm not sure what to respond to that.

@ljharb
Copy link

ljharb commented Feb 27, 2020

From my perspective, code is clearer when it avoids the unnecessary await - so yes, I think we just won't agree on that subjective opinion, even though we agree that clarity should be prioritized, and that code is for humans.

@CherryDT
Copy link
Author

CherryDT commented Feb 27, 2020

For the record (and for anyone interested): I wanted to get to the bottom of how much it affects the performance.

Using a native debugger I found that it creates the same amount of microtasks either way, but I'm not sure about how reliable my findings there are because I simply put breakpoints on all symbols in node that contained anything related to "enqueue microtask" and counted how often they were hit.

I then also checked async hooks and how often they fire. The result is that they fire the same amount of times, just the order in which things execute is a tad different:

image

Then I tried to actually time the execution. The method is a bit crude (I tried to make sure that I really catch the relevant part only, that's why it times until the last then ran - and the setImmediate is there just to verify when the next tick starts), but I got results:

image

So, yes, the extra await is indeed slower, but in my example the effect was minimal: around 12 microseconds per iteration - and in real code you'd probably have much more code than return 123 inside your inner async function, so this is where the majority of your processor time will be spent on, not the await...

I also verified that I'm not measuring some sort of sleeping time by accident, by running twice the amount of iterations, and yes the numbers are also around twice as big:

image

@mightyiam
Copy link
Member

Thank you @LinusU, @ljharb and @CherryDT. Is there agreement now that the performance hit seems negligible?

@ljharb
Copy link

ljharb commented Feb 28, 2020

@mightyiam in node and v8, sure, but standard applies to code that's used in other engines. I'd want to see results in a lot more of them before conceding a performant argument.

However, I don't think perf actually matters here; i think return await is incorrect code, and tools allowing it (or worse, requiring it) will confuse people about how async functions, and await, actually work.

@CherryDT
Copy link
Author

I still don't see how it is "incorrect code" though.

return undefined is also valid, even though you could say it's redundant because return does the same thing from a technical standpoint. Yet it conveys a different message: I'd use return if I basically don't care what's returned because the function is not supposed to return a value at all (technically, it'd always be undefined then), and I'd use return undefined when the function is supposed to return a value but I want that value to be undefined.

@ljharb
Copy link

ljharb commented Feb 28, 2020

Sure, i agree - and you should use return await x when you need to await the result of x within the function, ie, inside a try block. Otherwise, you’re awaiting for no apparent purpose.

@LinusU
Copy link
Member

LinusU commented Feb 28, 2020

[...] is redundant and incorrect and makes your code artificially slower.

Hmm, I don't really see how it "incorrect"? and it isn't really "redundant" since the behaviour is different.

If someone wants to incur this cost in exchange for better stack traces in one JS engine, that's fine! but they should have to disable the rule for that.

This has been brought up in another thread about this, but this isn't because of one JS engine. I don't see how one engine could choose to add a function that isn't running anymore to the stack trace?

It's an async function, that's what makes it wait for an async result.

Not really though, if you return a Promise from an async function it doesn't wait for that result. It immediately returns that promise and the function is no no longer executing. This is why it would also be wrong for a specific engine to keep the function on the stack trace, and why return await is necessary for that.

Something that encourages an incorrect understanding of what the language does is worse for humans, even if it appears to be better for some humans in the short term.

I really don't understand why this would give an incorrect understanding of the language. To me it seems like the root of the problem here is that Promises flat maps automatically when resolved with another promise.

This is convenient, but this is really what hides what is happening here. Without this feature return a promise without doing await on it from an async function would actually return a Promise<Promise<...>>.

In e.g. Rust this is very nice because you can see this very clearly, and thus it requires you to do their equivalent of return await unless you want to return a future of another future.

However, I don't think perf actually matters here

I really agree with this, an extra await is not going to be your performance bottleneck, and we are optimising for code readability and correctness firstly.

i think return await is incorrect code, and tools allowing it (or worse, requiring it) will confuse people about how async functions, and await, actually work.

I really don't think that it will confuse people though 🤔 As I said earlier, I feel that the thing confusing people is really the automagic-flat-mapping...


I still strongly feel that we should remove no-return-await in Standard, and add @typescript-eslint/return-await = always in Standard TS.

@CherryDT
Copy link
Author

CherryDT commented Feb 28, 2020

(@ljharb - First: I respect your opinion. I just think that "incorrect" is a very absolute statement that goes beyond opinions, and I think that the way you view the language and how to work with it - which is totally valid as everyone can build the mental model that works best for them - will for most other people lead to worse code quality, hence I think it should not be advocated by default. Of course I cannot really be sure about what "most other people" think, but that's why we are discussing it here. I thought your perspective was that you ignore the refactoring and readability issues because the "language purity" has higher priority for you - that's why I said to agree to disagree originally - but now I understand that you don't view those issues as issues at all because they are not issues for you. I personally believe that they would be issues for most other people though, that's why I continue to elaborate.)

Otherwise, you’re awaiting for no apparent purpose.

My point was that the purpose does exist: The purpose is to do exactly that, logically keep the awaiting inside the function - both for computers (stack issue - which could even be relevant to code in some edge cases, e.g. profiling, since the stack can accessed programmatically) and humans (refactoring issue).

Definition of "return*" for further use: From a callee's perspective, it is what is passed to return. From a caller's perspective, it is what is received from syncFunction() or await asyncFunction(). The case of not awaiting an asyncFunction() is what I actually interpret a different, advanced operation. I'm defining this here because this is where the implementation details start.

Logically, these are different function types for me:

  • sync function returning* a string
  • async function returning* a string
  • sync function returning* a promise to a string
  • async function returning* a promise to a string < this cannot be done properly in JavaScript

...even though technically the middle two happen to be the same. I view this as a consequence of "the concept of await was implemented using promises in JavaScript", causing it to leak promise semantics into async function semantics, and that is an implementation detail. (Sure you have to know its gotchas - here: it is impossible to actually return* a non-awaited promise from an async function -, but this is the case with many things. It is much easier logically to consider a feature as its ideal implementation plus an exception to keep in mind, instead of looking at it the other way round.)

In other languages, the concept of await was implemented differently. Look at Rust like @LinusU mentioned, or modern C++ or others.

Therefore, I think, leaving out the await there should be discouraged, much like no-use-before-defined (x = 1; var x;) and no-redeclare (if (a) { var x = 1; } else { var x = 2; } console.log(x);) even though those are also technically correct since hoisting and function-scoped variables are part of "how the language works". It's a matter of implementation detail vs logical mental framework that developers can use to write readable, maintainable and robust code.

It's just like how the BOOL type in Windows' classic API is just a typedef to the type int (because C didn't have a native boolean type), and TRUE and FALSE are constants for 1 and 0 respectively, so you could get away with having a function that returns int and then writing return FALSE instead of return 0 inside of it. Yet, the meaning is different: If I see the return FALSE I subconciously assume that the function returns BOOL and will be surprised if later on I see 2 returned elsewhere. Or worse, the other way round - you could declare the return type as BOOL and always return arbitrary numbers (and save the result in an int variable at the caller's site as well) and it would still work. It's an implementation detail that int semantics leaked into BOOL semantics.

I see it as an implementation detail that it actually works without the await even though logically you'd expect to get an unresolved promise at the end. Sure, await and async functions technically do a certain very specific thing. But the purpose for the developer is a different, logical, one.

In fact I think the current implementation in JS is actually a bit counterintuitive, because it is impossible to actually return* a pending promise from an async function. (This is the gotcha I meant to keep in mind above.) But, this is much easier to keep in mind and you much more rarely bump into it as a limitation (and then you can use return { promise } or function wrapper (...args) { return asyncFunction(...args) } instead) than the refactoring issues mentioned above.


EDIT: I just thought about another example to maybe illustrate my point more clearly. This example does not use async function itself, it's about the abstract idea I'm trying to advocate here.

Consider this code (based on real-life examples):

// Let's assume `req.query` is a parsed URL query string
// Let's also assume that `data` is an array with items.

const PAGE_SIZE = 10
const page = req.query.page

const items = data.slice((page - 1) * PAGE_SIZE, PAGE_SIZE)
const totalPages = Math.floor(data.length / PAGE_SIZE)

return {
  items,
  totalPages,
  description: `Page ${page} of ${totalPages}`
}

In my eyes, this code is bad and has a hidden trap. As mentioned in my comment, req.query is a parsed URL query string, so page=123 would become { page: '123' }. Note how req.query.page is a string.

I would therefore change it to:

const page = Number(req.query.page)

Now, you could argue that it doesn't matter, since page - 1 would automatically convert it to a number and that's how JavaScript works, and ${page} would convert it back to a string anyway, and therefore the Number(...) is redundant and "incorrect" and that the conversion back and forth artificially slows down the code.

However, I disagree. It has essentially the same benefits as in my return await example:

a) Benefit for tooling - here it's not a stack trace, but it would make IntelliSense-type tools work better because they would know for sure that page is a number (and maybe at runtime cause different optimizations later on in the JS engine as well). And yes, this also applies only to some environments obviously.

b) Benefit of preventing refactoring traps - see this example where later on, we add more pagination output (again, from real life):

return {
  items,
  totalPages,
  prevPage: page > 1 ? page - 1 : null,
  nextPage: page < totalPages ? page + 1 : null,
  description: `Page ${page} of ${totalPages}`
}

Do you spot the problem? If the URL has page=3 and totalPages is 5, then we'll get this:

result = {
  items: [...],
  totalPages: 5,
  prevPage: 2,
  nextPage: '31', // oops
  description: 'Page 3 of 5'
}

So, why not prevent this possible future coding mistake proactively in the first place?

And in this example it was easy to spot during a first test. But now think about what will happen if it's not req.query.page but req.body.page from a POST body, and the body object can come from parsing different formats. If it was JSON, req.body.page may actually be a number and the code will work, but if it was formdata, it will be a string again (even if we also passed it as number to an AJAX function on the client side). If 90% of the time the data would be JSON, then in basic tests you wouldn't notice the bug immediately. (This sort of bug has happened to us in real life several times.)

Yes I know that TypeScript would have solved this as well, but that's not the point.

Bottom line: Code should convey intent.

@CherryDT
Copy link
Author

red herring

I also don't see where this argument comes from. I posted a comment at typescript-eslint/typescript-eslint#1378 (comment) with another real life example where the missing frame would have pointed to the culprit.

@ljharb
Copy link

ljharb commented Feb 28, 2020

@LinusU the promise returned from an Async function is never the same promise you eventually return from it; the spec requires it have a different identity. Promise.resolve(x) === x is the only place the behavior you describe is observable.

Separately, you can never have a promise of a promise, it’s not a monad.

I’m seeing arguments here in favor of return await that directly contradict how the language is specified, so i hope my point that allowing it increases confusion comes across.

@ljharb
Copy link

ljharb commented Feb 28, 2020

@CherryDT note that the concept of await wasn’t implemented using promises, it is promises. Promises aren’t an implementation detail that you can ever avoid thinking about - async/await simply does not replace the need to directly use, and understand, promises. Why do you think we keep adding new Promise combinators to the spec, and have no await syntax for any of them?

@CherryDT
Copy link
Author

CherryDT commented Feb 28, 2020

I don't think so. Promises in the way we were using this word in this thread, more specifically ECMAScript Promise Objects (or earlier, with differences that are actually not overlapping the point of this discussion, Promises/A+) are a JS-specific interface of the much broader language-independent promise/future/deferred concept.

async/await is a syntax construct to create synchronous-looking code that is actually asynchronous, and this is also a language-independent programming concept, much like an array or a class or a switch statement.

The "auto-flattening" is convenient for the promise/feature/deferred concept, but counter-productive for the async/await concept. But for neither it is part of the concept itself, just an implementation decision.


For me, the concept of async/await is definable like this:

Assume that the execution environment can be viewed as a number of parallelly-executing threads with cooperative multitasking, each of them running synchronously and blocking all other activities until they are yielding control.

This execution environment makes it impossible to physically have parallel activities. However, we can create this concept virtually. This requires establishing a distinction between physical and logical control flow, where logical control flow is what is written in the code. There can now be long-running processes which execute "in chunks" in between other activities from a physical perspective, yet they can logically be viewed as one long process running in parallel with other activities.

A conventional function will always run in a blocking manner and return a value of type T, both physically and logically. The caller physically waits until the function physically returns and then receives its return value which is of type T. If an exception happens, it physically bubbles up to the next exception handler in the call stack. No other "parallel" activity can take place during any of this. The logical behavior of a conventional function is equal to its physical behavior.

The behavior of an async function has to be described from two perspectives now:

1 - Logically:

An async function can run in a non-blocking manner. Other activities can take place while the function is running. An async function is able to call another async function in such a way that it waits for the called function and, if that function returns a value of type T, the caller will receive the same value of type T. If the called async function throws an exception, the caller can catch it with the same logical semantics as it can with a conventional function.

If an async function logically calls another function (regardless of whether that function is conventional or async), the logical behavior is the same as the physical behavior would be when a conventional function calls another conventional function.

2 - Physically:

An async function physically doesn't differ from a conventional function. However, the return type is defined to be always a Future<...>. If the async function logically returns a value of type T, it physically returns a value of type Future<T>. Physically, the async function is also blocking (since "non-blocking" is physically an impossible concept), and it physically returns before it logically returns. The T value of the returned Future<T> will only become known once the function also logically returned, which can happen only after the caller yielded control, so the Future<T> return value is meaningless for the caller and can only be passed around but not interpreted. An async function usually shouldn't throw a physical exception.

It is the job of some sort of outside force (scheduler, event loop, OS, ...) - which becomes active after the user code yielded control, i.e. physically returned - to physically execute the next "chunks" of the code of async functions which are currently logically "running".

Finally, await is a syntax construct to indicate whether a function call should be interpreted physically or logically. With await it is logical, without await it is physical. Since interpreting a function call logically requires us to already be in a logical frame of mind in the first place, await can be used only inside an async function. Furthermore, await can be used to "convert" an existing Future<T> to a T - again a logical operation. (The first usage is actually just a combination of the second plus a physical call, but it is most common to use them together, so the mental model can be made more practical by looking at x() and await x() as two different call semantics.)

Marking a function async causes the compiler/interpreter to interpret the containing code in its logical meaning instead of its physical meaning, allowing to developer to use purely logical constructs like await inside of it which cannot be interpreted physically.

Note that this concept doesn't describe how to interact with Future<...> values from user code. It does however assume that Future<...> is just another type. This implies that logically returning a Future<T> from an async function will result in a Future<T> for the logical caller (or a Future<Future<T>> for the physical caller). Any other behavior would be surprising in the same way as it would be surprising if [1, 2, [3, 4], 5] would behave like [1, 2, 3, 4, 5].

Caller function type Callee function type await used? Value type logically sent by callee Value type physically returned Value type logically received by caller
conventional conventional no T T T
conventional conventional yes T invalid invalid
conventional async no T Future<T> Future<T>
conventional async yes T invalid invalid
async conventional no T T T
async conventional yes T T T
async async no T Future<T> Future<T>
async async yes T Future<T> T

In a nutshell:

  • Conventional callee = physical return type equals type of value logically sent by callee
  • Async callee = physical return type equals type of value logically sent by callee wrapped in Future<...>
  • Caller doesn't use await = logical type received is physical type sent by callee
  • Caller does use await = logical type received is logical type sent by callee
    • This is invalid if caller is conventional

This concept maps almost perfectly to JS's implementation, except for the behavior that we are discussing now.


Now that it is clear why async/await (and the generic concept of a Future) are its own thing, a concept separate of ECMAScript Promise Objects, let's look at how they overlap in JS.

async/await in JavaScript has inherited some of the semantics of ECMAScript Promise Objects because the existing promises were probably the obvious choice of implementation, like in most languages, especially since promises were already very popular and async/await now allowed for a new, neat way of writing code that interacts with the existing ECMAScript Promise Objects-based ecosystem.

Not all futures/promises are self-flattening in other languages. They just happen to be in ECMAScript Promise Objects which is what JS happened to implement into the language, and I'm quite sure that async/await was not on the table when ECMAScript Promise Objects came about.

In other words: If the concept of promises/futures is telephony, and the concept of async/await is mobile telephony, then ECMAScript Promise Objects would be the US numbering plan where numbers are given on a strictly geographic basis (since it didn't matter to the idea of telephony and "having a unique number" itself but was useful at that time). Nowadays, the fact that mobile phone numbers have to belong to the owner's home's geographical region is certainly not part of the idea of mobile telephony and has become actually inconvenient (you'd have to change the number when moving even though there is no need anymore due to physical wiring) and useless (since it can no longer tell you to which physical location you are currently calling). (At least that's what I understand about how things are in the US - I hope my example is correct.)


This can be compared to how ES6 classes inherited some prototype semantics and existing new operator can be used there as well, and then you can "legally" write code like this:

class X {
  constructor () {
    return { a: 1 }
  }

  xyz () {}
}

const x = new X()
console.log(x instanceof X) // false !
console.log(x.a) // 1
console.log(x.xyz) // undefined !

...which is a result of how the concept of a "class" was implemented in JS, namely by putting syntactic sugar on the existing prototype system and constructor semantics (which was not designed with this future usage in mind). It doesn't mean this is how classes should work ideally, so I'm happy if my linter tells me I created "surprising" code, just like with the return await.


Now that I've spent some time to self-analyze how my mental model of async/await as a language-independent concept exactly looks, I'd be very curious to know if others have the same mental model. I know you, @ljharb, don't, but I wonder how others see that! It is hard to find definite descriptions of concepts like this, since they are not standardized but just evolved.

@ljharb
Copy link

ljharb commented Feb 28, 2020

imo the JS implementation is the only concept that matters to JS devs; the long tail of JS devs likely have never written another language. Promises/A+ was useful until Promises were added to the spec, at which point it's no longer relevant except as historical background.

@CherryDT
Copy link
Author

You are right. My mistake. I changed my post accordingly. It doesn't change anything though, since the flattening behavior is the same in Promises/A+ and ECMAScript Promise Objects.

And you are also right regarding JS devs, but I think that ultimately programmers get to understand concepts, ideally. (Obviously, you have to understand the implementation in your current language/framework/etc. as well, but you should be aware what is part of the concept and what is not.)

@LinusU
Copy link
Member

LinusU commented Mar 2, 2020

I’m seeing arguments here in favor of return await that directly contradict how the language is specified, so i hope my point that allowing it increases confusion comes across.

Would you mind elaborating on this? 🤔

@ljharb
Copy link

ljharb commented Mar 2, 2020

@LinusU see the rest of that comment for examples.

@LinusU
Copy link
Member

LinusU commented Mar 2, 2020

Do you mean this part?

the promise returned from an Async function is never the same promise you eventually return from it; the spec requires it have a different identity.

I'm not sure how that affects the arguments 🤔

@ljharb
Copy link

ljharb commented Mar 2, 2020

@LinusU

if you return a Promise from an async function it doesn't wait for that result. It immediately returns that promise and the function is no no longer executing

This is false.

I seem to have slightly misread part of your comment; you weren't claiming you can have a Promise of a Promise, but contrasting JS to a language where that is possible.

To me it seems like the root of the problem here is that Promises flat maps automatically when resolved with another promise.

None the less, that is the semantic the language has, and return await thus doesn't offer any value (except as the OP points out, in one implementation's non-spec-mandated non-standard stack trace modification)

@CherryDT
Copy link
Author

CherryDT commented Mar 2, 2020

... and other than this secondary usefulness, the primary extra value of not provoking a number of foreseeable developer errors in the future... which is independent of any implementation or standard and the main point of my issue

@LinusU
Copy link
Member

LinusU commented Mar 3, 2020

This is false.

@ljharb why is this? 🤔 do you have any links where I can read up on this?

@ljharb
Copy link

ljharb commented Mar 3, 2020

Because that’s the way the spec works? An async function produces a new promise and returns it at the first await, or at the return, before resolving the awaited/returned value. If your statement was true, const p = Promise.resolve(); (async function () { return p; }()) === p would be true; it is not.

@LinusU
Copy link
Member

LinusU commented Mar 4, 2020

Ah, okay, I understand how you mean. What I was trying to convey was that the promise of the async function will be settled with the new promise that is being returned, and the async function will no longer be executing even though the final promise hasn't settled yet.

async function sleep (ms) {
  await new Promise((resolve) => setTimeout(resolve, ms))
}

async function example () {
  return sleep(1000)
}

const a = example()

// the "example" function won't be on the stack here
await a

[...] or at the return, before resolving the awaited/returned value

I'm not sure this is correct, but I might be misinterpreting what you are trying to say here. If it resolves the value before returning, than the following code should catch the rejection, right?

async function throwLater () {
  await new Promise((resolve) => setTimeout(resolve, 100))
  throw new Error('Test')
}

async function example () {
  try {
    return throwLater()
  } catch () {
    return 1337
  }
}

// This will actually reject
assert(await example() === 1337)

I think that just the example above is justification enough for always doing return await. Why should it be treated different if we are in a try or not, it's just one more rule to have to remember. It seems more consistent to me to always await async values before returning them from an async function.

@ljharb
Copy link

ljharb commented Mar 4, 2020

Certainly if you're returning from an async function inside a try, return await might make sense. However, that's the only case where it might make sense. There's already an eslint rule that forbids return await outside of a try, so you don't have to remember anything :-)

@CherryDT
Copy link
Author

CherryDT commented Mar 4, 2020

It's not in the standard though (for JS), and I doubt there can be rules that catch the other cases I described above unless the code is TypeScript. So again it should be at the developer's discretion in this case...

@LinusU
Copy link
Member

LinusU commented Mar 4, 2020

There's already an eslint rule that forbids return await outside of a try, so you don't have to remember anything

Well, I have to remember that I need to add it inside the try, which is very easy to forget if the linter normally forces you to not add it.

I've actually seen this caught in code review more than once, people forgetting to add await in the try, and I think that this could be somewhat mitigated by having the mindset that async values should always be awaited.


I still strongly feel that we should:

  • Removing the rule completely in the JavaScript part
  • Enforcing the use of return await in the TypeScript part
    • (that is, enforcing return await when you are trying to return a PromiseLike<_> from an async function, the always setting of the @typescript-eslint/return-await rule)

@LinusU
Copy link
Member

LinusU commented May 11, 2020

Fixed in 14.3.4 :shipit:

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

5 participants
@ljharb @LinusU @mightyiam @CherryDT and others