The previous logic was checking for the env var `ENABLE_EXPERIMENTAL_COREPACK` to determine if corepack was being used by a project. However, this value only means that the build system should consider corepack, not that it's actively being used.
We need to check that flag AND the existence of a `packageManager` property in the project's `package.json`.
There were scenarios where the warning "Node.js Version defined in your Project Settings ("18.x") will not apply" would not be triggered. For example:
1. Project Setting is 18.x
2. package.json has "engines.node": ">=18.x"
3. semver.intersects('18.x', '>=18.x') // => true (In this [code path](8ea93839cc/packages/build-utils/src/fs/run-user-scripts.ts (L258))) -> No warning message that Node.js Version was changed
4. After the error message we evaluate the latest node Version to use (20.x) in getSupportedNodeVersion()(this [code path](8ea93839cc/packages/build-utils/src/fs/run-user-scripts.ts (L274)))
5. User doesn't get notified that we changed the node version to something different than their project setting
This fixes that scenario by getting the supported node version first.
Both `pnpm@6` and `pnpm@7` can parse lockfile versions `5.3` and `5.4`, but if there's a mismatch, `pnpm` will output a warning saying so. In order to prevent confusing warnings from being displayed to the user, this PR aligns the pnpm version with the exact lockfile verison.
It also adds a debug log to show which package manager version was determined based on the lockfile version.
The code we use to detect the version of `pnpm` based on lockfiles is hard to follow. It doesn't pick and executable, it instead of *sometimes* overrides the `PATH` for these utilities with a prepended alternate path.
This means that there is no one single place where we definitively specify which version of `pnpm` that is used, so much as *hope* that the `PATH` has the correct version on it. I don't know that we *can* specify an executable outside of the build container, and it's unclear, since this is a package, whether this utility is used outside of a build container setting.
In this pull request I cleaned up the code and updated the logic to include the `pnpm 7` and `pnpm 8` changes requested, with no changes yet to implement `pnpm 9`.
It's possible that we want to rewrite this detection logic from scratch to avoid this roundabout way of "finding" the executable, but I haven't done that here.
Adds two new helper functions that custom runtimes can utilize in order
to determine whether they should use `provided.al2` or `provided.al2023`
Lambda "runtime" value, based on the build environment.
The new `getProvidedRuntime()` helper function is being used in
`@vercel/go`, so that Go endpoints may be used when the `20.x` Node
Version project setting is used.
* Forces the AL2 build container image for fixtures that depend on it,
via `engines.node` in package.json for most cases.
* The `testDeployment()` function was updated to send
`projectSettings.nodeVersion` in the POST body, to mimic the behavior in
CLI.
* For Go, Ruby, and Python tests, the `projectSettings.nodeVersion`
property is set "globally" in the Jest setup file, so that individual
fixtures didn't need to be adjusted.
This reverts commit de63e35622 / #11104.
We had no great way to test this without release but we have other
things we'd like to deploy today. Will revert the revert when we're
closer to fixing just this issue.
This PR changes the default package manager from `yarn` to `npm` in the case that no lockfile is present.
Many years ago when `yarn` was first released, it was much faster than `npm` so we used it by default. That is no longer the case today and `yarn@1` is no longer receiving new features.
For example, `sharp` and `esbuild` no longer works with `yarn@1`:
### Related
- https://github.com/lovell/sharp/issues/3750
- https://github.com/evanw/esbuild/issues/2949
Note that this change will not impact most projects because most used a lockfile.
We're currently using the directory name at `./vendor/bundle/ruby/`[0]
to determine ruby version. This directory is created as part of running
`bundle install` during the build process. However, when multiple ruby
versions are available, you can end up with multiple directories and the
0th entry at `./vendor/bundle/ruby/` isn't guaranteed to be the Ruby
version specified in the project.
Other companies in this space seem to do some order of
- check for value in `.ruby-version` file
- check for value passed into `ruby` directive in the Gemfile
- check the value in the `*.gemspec` file if the Gemfile has a `gemspec`
directive.
We'll do that work in a future PR but for _now_ our only supported Ruby
version is Ruby 3.2.x [as of
2023-11-22](https://vercel.com/changelog/upgrading-ruby-v2-7-to-v3-2).
So this can be hard-coded for the moment.
Skipping ruby tests now since these target _current_ prod where Ruby
3.2.x is not in `PATH` until we redeploy after this gets deployed.
To use this outside of CLI we want a way to suppress the `console.log`s in `getEnvForPackageManager`.
For achieving this, we introduce a new helper `getPathForPackageManager()` which contains the core logic.
Best to review commit by commit + hide whitespace.
**X-Ref**
- [Internal Context](https://vercel.slack.com/archives/C03F2CMNGKG/p1701970097725689)
Makes `getLatestNodeVersion()` and `getSupportedNodeVersion()` (and thus by extension - `getNodeVersion()`) be aware of the available Node.js versions when running inside the build-container. This is to address the situation with the new AL2023 build-container image which has different Node versions installed compared to the existing AL2 build image.
### Project Settings `20.x` with package.json `"engines": "18.x"`
If the Project Settings Node Version is set to `20.x`, but the package.json has `"engines": "18.x"`, then the build will fail like so, because Node 18 is not present on the AL2023 build image:
<img width="1044" alt="Screenshot 2023-11-09 at 1 25 41 PM" src="https://github.com/vercel/vercel/assets/71256/572c544b-6574-4eb1-98f7-787075a60000">
### Project Settings `18.x` with package.json `"engines": "20.x"`
If the Project Settings Node Version is set to `18.x`, but the package.json has `"engines": "20.x"`, then the build will fail like so, because Node 20 is not present on the AL2 build image:
<img width="1042" alt="Screenshot 2023-11-09 at 1 34 43 PM" src="https://github.com/vercel/vercel/assets/71256/c6a2d955-9453-4ef5-a99d-b49a6d9af766">
### Project Settings `18.x` with no package.json `"engines"`
If Project Settings Node Version is set to `18.x`, but the package.json has no "engines" (and thus wants "latest"), then the latest available Node version in the build-container, which would be Node 18.
This adds a new `pnpm type-check` that leverages `turbo` to validate the TypeScript code. This can be run at the top-level or for an individual package.
The `test-lint` workflow will run it after linting and doing the prettier check.
As apart of this effort, each package's `tsconfig.json` has been simplified. There's a new top-level `tsconfig.base.json` file that extends the Vercel Style Guide for TypeScript. Each package's `tsconfig.json` has been audited and previously suppressed rules that no longer apply have been removed. The result is each package's `tsconfig.json` is greatly simplified and we can control common settings in the base config while keeping the flexibility of package-level overrides.
Lastly, in `package/cli`, `pnpm build` calls `scripts/build.mjs` which calls `scripts/compile-templates.mjs`. The `compile-templates.mjs` file was generating invalid TypeScript code. I've fixed it and now it's happier than ever.
Note: In order to run `pnpm type-check`, you must first `pnpm build` because we need the `.d.ts` definition files.
Updates package manager detection to account for two lock files. All other managers will only have the one lock file. Bun, however, may have both a `bun.lockb` _and_ a `yarn.lock` file. To ensure bun is properly detected, the presence of `bun.lockb` with `yarn.lock` must occur before `yarn.lock` so we don't mistake the presence of a `yarn.lock` to mean "Yarn".
This PR also adds a test for this situation in `fs-detectors`. The behavior is currently correct there, but was not tested initially. It is now to avoid future regressions.
This adds an experimental flag to `Prerender` outputs as a way to programmatically bypass the cache and hit the lambda directly, using a similar interface to `has`.
(Note: I copied over `HasField` from `@vercel/router-utils` since it wasn't available for import in `build-utils`, but can add it as a dep if that's preferred)
The specific use-case being targeted here relates to https://github.com/vercel/next.js/pull/51534 -- a Next.js page marked static should still be able to initiate server actions.
The `getNodeBinPath()` function is problematic because it assumes that commands are installed in the `node_modules` directory alongside the detected lockfile. This works fine the majority of the time, but ends up not being the case when using a monorepo that uses a package manager in "linked" mode (i.e. pnpm by default).
Consider the following:
```
.
├── pnpm-lock.yaml
├── node_modules
├── blog
│ ├── node_modules
│ │ ├── hexo -> .pnpm/hexo@3.9.0/node_modules/hexo
```
In this setup, adding the root-level `node_modules/.bin` would not make the `hexo` command be visible in the `$PATH`.
To solve this issue, the new `getNodeBinPaths()` function returns an array of all directories up to the specified `root`, which can then be placed into the `$PATH`. It's also more efficient (synchronous) since it does not need to scan for a lockfile anymore (the `root` needs to be specified explicitly).
The new function is being used in `@vercel/next` and `@vercel/static-build`.
The `traverseUpDirectories()` function from CLI was moved to `build-utils` to implement this function. Consequently, that makes the implementations of `walkParentDirs()` and `walkParentDirsMulti()` simpler, since it's using this generator now.
When a project has a `.npmrc` containing `use-node-version`, package managers (notably `pnpm`) will download the specified Node.js version. This is not the correct way as it can lead to `pnpm` downloading Node.js 18 or newer which depends on a version of GLIBC that is not present in the current AWS image. The proper way is to set the `"engines"` in the `package.json`.
<img width="468" alt="image" src="https://github.com/vercel/vercel/assets/97262/0974cf05-6a11-4d95-88e8-13affc4aad2a">
Discussion: https://github.com/orgs/vercel/discussions/2436
In an effort to speed up CI, we should update the lowest common
denominator to Node.js 16
Note: In April, Node.js 14 will reach EOL so we can update tsconfig
targets and ship a major semver at that time.
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Around 6 months ago, @styfle brought to my attention how `rename()` in build-utils used `reduce()` and could be written better. So, I rewrote it.
Before, the code would create a new `Files` object and copy the contents of the previous `Files` object. This caused heavy garbage collection and memory thrashing.
Instead, I created a single `Files` object, then add the files to it.
Results:
| # Files | Before | After |
|---|---|---|
| 1,000 | 75 ms | 1 ms |
| 10,000 | 10.6 s | 7 ms |
| 20,000 | 44.6 s | 16 ms |
| 30,000 | 105.5 s | 22 ms |
| 100,000 | Too long | 73 ms |
This function can sometimes give a TypeScript error if @types/node says 'code' should be a `number | null` and yet the code is returns as a `number`:
```
@vercel/build-utils:build: src/fs/run-user-scripts.ts(140,13): error TS2322: Type 'number \| null' is not assignable to type 'number'.
@vercel/build-utils:build: Type 'null' is not assignable to type 'number'.
```
I'm not sure if there are any other projects that depend on `@vercel/build-utils`, but it doesn't appear that the Vercel CLI nor the build container use this `execAsync()` function. If I'm mistaken, feel free to close this PR.
As this removal is a breaking change, setting the semver to major and not auto-merging unless others are in agreement.
Instead of always including empty directories in `glob()`, make it an
opt-in behavior because technically it's a breaking change to include
them by default.
