New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
windows: fsevents v1 is getting built #301
Comments
You should update to chokidar 3 / fsevents 2. fsevents 1 is unsupported. This happens because of NPM bug. See the comment below from @guidobouman: If you really need {
...
"optionalDependencies": {
"fsevents": "1.2.9"
},
"_comment": "fsevent@1.2.9 is locked in to prevent broken builds on windows for v1.2.11"
} This works until the issue is resolved in a more sustainable manner at Do note: This breaks when you run Node v13, as the binaries are not available for that version. |
If it's unsupported, why 1.2.11 was released 12 days ago breaking existing projects where 1.2.9 did not? |
1.2.9 was downloading binaries from AWS to which we no longer have access to. There were no binaries for node 13 which made 1.2.9 unusable for people on node 13. Which is why we released 1.2.11, so that people are able to build it now. If the software doesn’t build on your system, it’s system’s problem. You need python now etc. |
2.x doesn’t require any of those, which is why we recommend it. |
yep, sorry, I did not look into original report, which relates to missing build dependencies. However, my Windows system has the build dependencies installed, it reports a different error, which actually does not fail the install.
|
Interesting. fsevents should not build at windows at all. Not sure why this happens -- package.json field "os" should block the compilation. cc @pipobscure |
I guess this was my point. If windows had python or not, the package shouldn’t even be trying to compile on Windows because it’s not needed. My fault for not explicitly saying I was on a Windows machine in the description. |
Yeah it’s interesting, I would get a successful install but the exit code returned from npm was 1. This set the lastexitcode environment variable to be set. This caused my automated CI build to fail. |
We have the same issue. |
Running We are using Windows 10 x64. Here is our dependency tree
Here is the log of the fsevents build attempt
Updating to the latest
Node / npm version:
For now I'm going to switch our CI/CD process to use |
So this is what I have been talking about @pipobscure webpack/webpack-dev-server#2351 Webpack, and tons of other dependencies cannot just bump fsevents to v2, because they still support node v6 etc. We should not just "ditch" fsevents v1 for at least 3-6 months YET. Can we get new AWS binaries? Or can you describe what do I need to do to set up an AWS account with the binaries? I understand how to host files, but I don't know which exactly files should lie there. |
|
Bonus: Because 1.2.11 was released as a patch version, I now have to manually rollback each package-lock.json that updates a dependency that has fsevent 1.x as a deep dependency, even if the package that leans on fsevents has not changed (webpack in my case). This gets really annoying really quickly. Especially when you have a set of shared packages that you actively develop on and update multiple times a day throughout multiple different projects. I think you can imagine this matrix of changes where I constantly have to rollback a package.json. |
I know this is annoying, but GYP is a ticking time bomb. All the dependencies really need to upgrade to fsevents v2. |
I get that. To help I've also raised an issue with |
I talked with @pipobscure. Initially i've thought he has added binary hosting on AWS. Apparently, the S3 storage was added in 96e1ace via PR #59. We (maintainers of fsevents) don't have access to the S3 storage. Which makes it unsafe to use. Adversaries could've replaced binaries with rogue software that steals credentials. This could be fixed by someone diving into .travis.yml for fsevents v1.2 and deciphering the way fsevents binaries are published. As soon as we understand how they're built, we can set up a new S3 account, and publish binaries for v1.2.11. |
In 1.2.10 binaries were still being published to: See 909af26#diff-b9cfc7f2cdf78a7f4b91a753d10865a2L27 for more info. This URL can be changed to a new S3 bucket. Combined with a new environment var in Travis, |
I don't understand. If i'm publishing the binary from macos with node v12, how will a node v13 binary appear? |
The old travis.yml ran with all versions of node each then pushing the compile result to S3 somehow. |
Travis is configured with a matrix: Lines 7 to 14 in 909af26
Adding node 13 (and 14 in may) to this matrix wil cause these additional binaries to be published. The AWS SDK is used to publish: Line 44 in 909af26
Which uses the API credentials stored here: Lines 5 to 6 in 909af26
Whether or not binaries should be published is decided upon here: Line 63 in 909af26
Actual publishing is done here: Line 69 in 909af26
See the node-pre-gyp docs for more info on setting up the environment: If you can provide an S3 bucket, I can build you a PR. You / we can then update the secrets in .travis.yml needed to publish to that S3 bucket. Note: Yes, I know this is only until all dependent packages publish their newer versions with |
SidenoteIf you really need {
...
"optionalDependencies": {
"fsevents": "1.2.9"
},
"_comment": "fsevent@1.2.9 is locked in to prevent broken builds on windows for v1.2.11"
} This works until the issue is resolved in a more sustainable manner at Do note: This breaks when you run Node v13, as the binaries are not available for that version. |
works a treat thanks @guidobouman |
My issue disappeared after the weekend. |
If you use {
"resolutions": {
"webpack-dev-server/chokidar": "^3.3.1",
"webpack/watchpack/chokidar": "^3.3.1"
}
} |
is this problem just apper into windows CI enviroments? now i am try to build in a CI/CD Linux eviroments and give some similar errors i am using node 10.5.1, npm 6.0.1 and fsevent 1.2.11 (i was using fsevents 1.2.9 but the same error is used)
i check dependencies with npm list fsevent and another package use lowest version of fsevent like chokidar, any advice? |
Just encountered the same issue, npm ci 2>nul This allows the build to continue without errors, |
@idanwork That's a very reckless approach. More info: Better yet, use this temporary setup: |
@guidobouman is there a reason S3 needs to be used? Instead the artifacts could be uploaded to the "Releases" page, for example. That would avoid problems with the maintainers losing access to AWS or another service |
Fundamentally v1.x is deprecated. There is no valid reason anyone should use it. Please bug upstream maintainers to upgrade to v2.x which has been available for a year already. Anyone not doing so is a serious threat to security. |
Sure, but implementors like Sorry to be so blunt, but in the end this is all caused by you guys breaking builds & semver support in a patch release. |
@rajivshah3 S3 is used in the old setup, so this is the easiest route to get an old version to work again. For v2 all this is not needed. |
@guidobouman sorry to be so blunt, but this is caused by people neglecting their dependencies. The only difference between chokidar@2 and chokidar@3 is that the latter has an async close method to stop monitoring. This has been an issue for a year now and we’ve been trying to minimise the impact while asking upstream users to upgrade quickly due to security issues. So if webpack (for any value of webpack) can’t be asked to upgrade for a security issue, when there are only miniscule api differences, then they are part of the problem. |
This is not simply a case of neglecting dependencies.
I've pushed the upgrade with the webpack maintainers before: Packages that have supported Yes, And security in front-end packages... This is only ever an issue if external sources get to trigger execution trough those packages with custom content or properties. 99% of usage scenarios are not within this domain. The only case I can think of is online code editors like CodePen. That's why security fixes should always be pushed in patch releases. Compare this to the ~25% (rough guess) of webpack / watchpack builds that are on Windows, that now all fail with |
Don't get me wrong, I want to fix the situation. But every step of the way I am met with resistance. Some resistance is grounded, some is less so. I would love to write the whole pull request and add support back for the old platforms. |
I've tried setting up builds using your comments (thanks for that), but got nowhere. It's complicated. We're open if someone is able to resolve it. |
Until S3 is sorted out, I think this can help in the meantime: #322 |
Non-Mac will no longer attempt building since v1.2.13 |
When doing an install of dependecies on my windows based system, fsevents 1.2.11 attempts to run node-gyp, asking for Python. When Python is not found node-gyp sets the exitcode to npm as non-zero. This causes our CI to fail the build.
It seems that fsevents could do a check ahead of time for a darwin based system before calling node-gyp since fsevents is strictly for darwin systems.
Below is the captured output:
The text was updated successfully, but these errors were encountered: