There are some edge-case situations where a dependency or build process is expecting an empty directory to exist, and this expectation would fail currently because `glob()` does not return directory entries.
So update `glob()` to return entries for empty directories, which are also re-created properly when passed to the `download()` function.
### Related Issues
x-ref: https://github.com/vercel/vercel/pull/8743
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with
tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a
reviewer
- [ ] Issue from task tracker has a link to this PR
There was a case where the npm version wasn't decided base on Node.js version but instead based on the lockfile.
This PR fixes the case when a newer npm version is detected base on the lockfile.
- Follow up to #8598
- Follow up to #8550
In this PR, we try to detect when npm install fails due to invalid peer deps and retry with `--legacy-peer-deps`.
This is a follow up to PR #8550 that explains more.
Steps to reproduce the npm bug:
```sh
corepack enable npm
echo '{"packageManager":"npm@8.5.5","dependencies":{"swr":"1.3.0","react":"16.8.0"}}' > package.json
npm install --legacy-peer-deps
npm install # works fine now that a lockfile exists
echo '{"packageManager":"npm@8.6.0","dependencies":{"swr":"1.3.0","react":"16.8.0"}}' > package.json
npm install # fails with code ERESOLVE (Conflicting peer dependency)
```
Windows doesn't like it when the env has both a `PATH` and a `Path`, so `cloneEnv()` must remove `Path` after copying it to `PATH`.
Renamed the next builder `01-normalize-routes` test to `01-normalize-paths` as it's test normalization of output paths, then re-enabled the test for Windows.
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
Prior to `npm@8.6.0`, running `npm install --legacy-peer-deps` to generate a lock file and then later running `npm install` would work fine.
After `npm@8.6.0`, running `npm install` with that same lock file will fail with `code ERESOLVE` and `Conflicting peer dependency`.
Steps to reproduce:
```sh
corepack enable npm
echo '{"packageManager":"npm@8.5.5","dependencies":{"next":"12.3.0","react":"16.8.0"}}' > package.json
npm install --legacy-peer-deps
npm install # works
echo '{"packageManager":"npm@8.6.0","dependencies":{"next":"12.3.0","react":"16.8.0"}}' > package.json
npm install # fails with code ERESOLVE
```
This PR introduces a flag so we can safely upgrade npm to a newer version without breaking thousands of deployments.
The [npm docs](https://docs.npmjs.com/cli/v8/using-npm/config#legacy-peer-deps) say the flag is not recommended so we also make sure that once we upgrade to Node.js 18, we stop flagging `--legacy-peer-deps` because its safe to introduce breaking changes.
On Windows 10 and 11 machines, environment variables are not case sensitive. The system PATH is actually defined as `process.env.Path`, however Node.js kindly handles the case sensitivity and will automatically return the system path when specifying `process.env.PATH`.
When we clone the environment variables via `{ ...process.env }`, we lose the automatic resolving of `Path` to `PATH`. To fix this, we need to explicitly copy the `PATH`.
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
### Related Issues
https://nx.dev/more-concepts/folder-structure#integrated-repo-folder-structure
Nx monorepo has an option to use Nx workspaces.
Nx workspace is defined within the root `workspace.json` file
Within this `workspace.json` file the workspace packages are under
projects
```{
"$schema": "./node_modules/nx/schemas/workspace-schema.json",
"version": 2,
"projects": {
"myblog": "apps/myblog",
"svelte-app": "apps/svelte-app",
}
}
```
Within `getNxWorkspacePackagePaths` get the projects object values for
the paths
Nx is listed as the last workspace manager within
`workspace-managers.ts` because other workspace managers could exist to
check for before Nx workspaces because the `workspace.json` could exist
but not be the correct workspace manager
Nx workspace file can exist when yarn/npm workspaces exist. When this
happens, the workspace.json file is empty with no projects so it will
not add any package paths to the list to look through for projects.
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with
tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a
reviewer
- [ ] Issue from task tracker has a link to this PR
This consolidates the logic to get the framework-specific prefixed System Environment Variables into a single shared function so each builder can reuse the same function.
- Related to #7009
- Related to #8306
In the future, this feature could be added to any other missing builders as well as `vc dev` but we'll save that for a new PR.
Some builders, such as `@vercel/next`, return both the symlinked directory and the resolved file.
When `vc build` iterates over the files to recreate them in `.vercel/output`, it fails with `EEXIST: file already exists` when creating the symlink because it first creates the file `node_modules/<symlink>/package.json` and then attempts to create the symlink `node_modules/<symlink>`.
This happened to work before `vc build` because yazl would accept the symlinked directory instead of the package.json file, so this PR is created to match that behavior.
This PR update the tests suite to wait for Vercel CLI tarball and then use that tarball to run E2E tests.
This is valuable because it will package all the packages in this monorepo to make the tests follow more closely what will happen in production once merged.
Since the current "Find Changes" step takes about 2 minutes, we run that first (it happens concurrently with the tarball preparation). Then once we complete "Find Changes" we wait for the tarball but it will likely be ready by that point since it also takes about 2 minutes. After both of those steps, the E2E tests continue as usual but with the `VERCEL_CLI_VERSION` set to the tarball.
- Related to #7967
- Closes#8245
- Closes#8227
This PR fixes a couple issues where `vercel build` was not correctly detecting the package.json files
```
Error: @vercel/node:test: ERROR: frontend/index.ts(1,12): error TS2580: Cannot find name 'require'. Do you need to install type definitions for node? Try `npm i --save-dev @types/node`.
```
It also fixes an issue where all deployments were incorrectly detecting the lock file because the lock file doesn't always live in the same directory as the package.json file. So we need to do 2 passes: one to find the nearest package.json and one to find the nearest lock file.
This test sometimes times out on Windows. When it succeeds, it takes ~6 seconds. When it fails, we don't know how long it would have taken, but the current timeout is 7 seconds. Let's try 10 seconds and see if that's better.
This PR fixes a bug where corepack (#7871) was not correctly setup because the lockfile autodetection and node version autodetection was overriding the PATH.
It also fixes a bug where the log output was printed twice because we incorrectly prepended the PATH twice.
The `@vercel/build-utils` package was meant be shared functions necessary for writing a Vercel Builder (aka Runtime).
This package has since bloated into the catch-all package for anything that wasn't a Builder.
This PR removes the bloat in favor of a new package, `@vercel/fs-detectors`. It also removes the need for `@vercel/build-utils` to have a dependency on `@vercel/frameworks`.
- Related to #7951
In some rare cases, `npm bin` exits with code 7, but still outputs the right bin path.
To reproduce, try:
```
npm init -y
echo "//registry.npmjs.org/:_authToken=${NPM_TOKEN}" > .npmrc
vc
# enter "echo build" for the build command, leave the other configuration as default
```
The build will fail with `Error: Command exited with 7` because `npm bin` fails with code 7, for some reason.
In this PR, we do 2 things:
(1) Ignore exit codes from `npm bin`. It still outputs the right path when it exits with code 7 so we just read the output and check if it's a valid path.
(2) Throw a more specific error message when `npm bin` fails to give us the bin path. The current error was hard to debug because it looked like it was coming from the install commmand. We can do better by emitting a custom error.
Alternative considered for (2): Do not throw errors. If `npm bin` fails, emit a warning and let the build continue.
Related Issues:
- https://github.com/vercel/customer-issues/issues/585 (internal)
Gets rid of the annoying ts-jest update warning:
```
ts-jest[ts-jest-transformer] (WARN) Use 'ts-jest' entry point in v28 will resolve in refactored transformer. If you wish to use legacy transformer, please adjust your Jest 'transform' config. For example:
module.exports = {
transform: {
'^.+\\.tsx?$': 'ts-jest/legacy'
}
}
```
Adds support for `config.middleware` property in `@vercel/node` to output an Edge Middleware (by including a catch-all route in the output).
Also updates the zero-config detection logic to schedule a build of root-level `middleware.js`/`middleware.ts` files with `@vercel/node` with this middleware option enabled.
<img width="1174" alt="Screen Shot 2022-05-19 at 3 53 55 PM" src="https://user-images.githubusercontent.com/35947020/169402845-4eb70a24-9954-44ac-8726-a6b56161d58c.png">
If there is no workspace detected, we still want to check if there are potential projects in the monorepo. An example of this case would be:
<img width="202" alt="Screen Shot 2022-05-19 at 10 29 12 AM" src="https://user-images.githubusercontent.com/35947020/169336400-e69845e3-616e-4857-80fe-c2614a65352a.png">
The directories `backend/app-three`, `frontend/app-one`, `frontend/app-two` all have definitions within their `package.json` files that define a framework assosiated with the project. This is what distinguishes them from the `package.json` files within `backend` and `frontend`
`getProjectPaths` loops through a depth of 3 (this can be changed but in most cases of this it won't be more than 3). For each directory it recursively calls itself until it finds a `package.json` file. It then checks if the directory has a framework defined and if so it adds the path to the final return value of all projects.
If any `skipPaths` are sent into the function, no projects will be looked for within those path(s). This would include the case for a hybrid monorepo where a workspace has already been detected
### Related Issues
> Fixes#1
> Related to #2
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
Gets the package paths from the workspace definition for a given git repository
```ts
/**
* Gets the list of workspace package paths based on the configuration
*
* @example
*
* my-repo
* |-- packages
* | |-- api-1
* | |-- api-2
* |-- package.json
* |-- pnpm-workspaces.yaml // packages: ["packages/*"]
*
* const fs = new ... // based on 'my-repo'
* const workspace = { ..., type: "pnpm" }
* getWorkspacePackagePaths({ fs, workspace }) => [
* "my-repo/packages/api-1",
* "my-repo/packages/api-2"
* ]
*/
function getWorkspacePackagePaths(fs: DetectorFilesystem, workspace: Workspace): string[]
```
### Related Issues
Closes https://github.com/vercel/vercel/issues/7749
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [x] Issue from task tracker has a link to this PR
This PR updates the way we handle warning for engines.node in `package.json`.
- should not warn when the engines version satisfies the project settings (previously it was an exact match)
- should warn when engines version is exact instead of range since it cannot be satisfied exactly
- should warn when engines version is greater than since it might introduce breaking changes for a future node.js version
These tests are asynchronous and would sometimes fail if the second call started before the first call finished. This PR fixes the usage since the function returns a promise.
```ts
/*
Example 1:
my-repo
|-- packages
| |-- api-1
| |-- api-2
|-- package.json
|-- pnpm-workspaces.yaml // packages: ["packages/*"]
getWorkspaces("my-repo") => [
{
implementation: "pnpm",
rootPath: "https://github.com/.../my-repo"
}
]
Example 2:
my-repo
|-- backend
| |-- api-1
| |-- api-2
| |-- package.json // workspaces: ["api-1", "api-2"]
|-- frontend
| |-- nextjs-app
| |-- gatsby-app
| |-- package.json
| |-- pnpm-workspaces.yaml // packages: ["nextjs-app", "gatsby-app"]
getWorkspaces("my-repo") => [
{
implementation: "yarn",
rootPath: "https://github.com/.../my-repo/backend"
},
{
implementation: "pnpm",
rootPath: "https://github.com/.../my-repo/frontend"
}
]
*/
type Workspace = {
implementation: "yarn" | "npm" | "pnpm"
rootPath: string // the path to the root of the workspace
}
function getWorkspaces(fs: DetectorFilesystem): Workspace[]
```
### Related Issues
> Closes https://github.com/vercel/vercel/issues/7730
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [x] Issue from task tracker has a link to this PR
This file will mimic the structure of `vercel/frameworks/frameworks.ts` in the sense that it defines the list of supported monorepo managers. These are the ones were targeting for the initial launch,
- Turborepo
```ts
export const monorepoManagers = [
{
name: "Turborepo",
slug: "turbo",
detectors: {
every: [
{
path: 'turbo.json'
},
],
},
},
]
```
### Related Issues
> Closes https://github.com/vercel/vercel/issues/7757
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [x] Issue from task tracker has a link to this PR
Node.js 12 reached EOL April 2022 so its time to notify customers.
This PR will warn starting today and then error once the discontinue date is reached in a couple months.
### Related Issues
Fixes pnpm 7 support. Now uses a yarn installed version and drops an unnecessary check for node version.
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
In order to support various `fs` operations for monorepo detection, the team needs to add two new abstract functions to `DetectorFilesystem` in order to traverse down into children directories
* readdir
* chdir
```ts
interface Stat {
name: string
path: string
type: "file" | "dir"
}
export abstract class DetectorFilesystem {
...
/**
* Returns a list of Stat objects from the current working directory.
*
* @example
*
* const url = "https://github.com/vercel/front"
* const fs = new GitDetectorFilesystem(...) // based on url
*
* // calls "https://api.github.com/repos/vercel/front/contents" behind the scenes
* await fs.readdir() => [
* { name: "docs", path: "docs", type: "dir" },
* { name: "front", path: "front", type: "dir" },
* ...,
* { name: "package.json", path: "package.json", type: "file" },
* ]
*/
protected abstract _readdir(name: string): Promise<Stat[]>;
/**
* Changes the current directory to the specified path and returns a new instance of DetectorFilesystem.
*
* @example
*
* my-repo
* |-- backend
* | |-- api-1
* | |-- api-2
* | |-- package.json // workspaces: ["api-1", "api-2"]
* | |-- yarn.lock
* |-- frontend
* | |-- nextjs-app
* | |-- gatsby-app
* | |-- package.json
* | |-- pnpm-workspaces.yaml // packages: ["nextjs-app", "gatsby-app"]
*
* const fs = new (...) // based on "my-repo" as the root
* const backendFs = fs.chdir("backend")
* const frontendFs = fs.chdir("frontend")
*
* const backendWorkspaceManager = detectFramework({ fs: backendFs, frameworkList: workspaceManagers }) // "yarn"
* const frontendWorkspaceManager = detectFramework({ fs: frontendFs, frameworkList: workspaceManagers }) // "pnpm"
*/
protected abstract _chdir(name: string): DetectorFilesystem
...
}
```
### Related Issues
> Related to https://github.com/vercel/vercel/issues/7750
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [x] Issue from task tracker has a link to this PR
### Related Issues
Adds support for `pnpm@7` using a similar lock file detection and path setting method that `npm@7` uses
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
In order to support monorepo detection, we need to build out the list of supported workspace managers so that our helper functions and API endpoints can utilize the same `detectFramework` function / logic to detect a workspace manager for a given git repository.
### Related Issues
- Closes https://github.com/vercel/vercel/issues/7731
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [x] Issue from task tracker has a link to this PR
### Related Issues
improves package manager auto detection so that when multiple lock files are found, the highest priority one is selected rather than defaulting to yarn.
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
Adds a best-effort optimization to only run `npm install` once per
`pacakge.json` file. This will save a lot of time in the single-sandbox
build world (i.e. `vc build`).
