With the release of Turborepo 2 we're renaming `pipeline` to `tasks`.
This PR updates the default settings logic to look in `tasks` for a
`build` task definition in addition to looking at `pipeline`. It also
updates the message to no longer mention Turbo configuration in
`package.json` as this is fully ignored in Turborepo 2.
Added a quick unit test to verify `build` task definitions are found in
the `tasks` section of `turbo.json`.
Please let me know if there are other tests/places I should update.
Adds support for Remix apps which use the new Remix Vite plugin.
* The vanilla Remix + Vite template deploys correctly out-of-the-box,
however only a single Node.js function will be used, and a warning will
be printed saying to configure the `vercelPreset()` Preset.
* When used in conjunction with the `vercelPreset()` Preset
(https://github.com/vercel/remix/pull/81), allows for the application to
utilize Vercel-specific features, like per-route `export const config`
configuration, including multi-runtime (Node.js / Edge runtimes) within
the same app.
## To test this today
1. Generate a Remix + Vite project from the template:
```
npx create-remix@latest --template remix-run/remix/templates/vite
```
1. Install `@vercel/remix`:
```
npm i --save-dev @vercel/remix
```
1. **(Before Remix v2.8.0 is released)** - Update the `@remix-run/dev`
dependency to use the "pre" tag which contains [a bug
fix](https://github.com/remix-run/remix/pull/8864):
```
npm i --save--dev @remix-run/dev@pre @remix-run/serve@pre
```
1. Configure the `vercelPreset()` in the `vite.config.ts` file:
```diff
--- a/vite.config.ts
+++ b/vite.config.ts
@@ -1,10 +1,11 @@
import { vitePlugin as remix } from "@remix-run/dev";
import { installGlobals } from "@remix-run/node";
import { defineConfig } from "vite";
+import { vercelPreset } from "@vercel/remix/vite";
import tsconfigPaths from "vite-tsconfig-paths";
installGlobals();
export default defineConfig({
- plugins: [remix(), tsconfigPaths()],
+ plugins: [remix({ presets: [vercelPreset()] }), tsconfigPaths()],
});
```
1. Create a new Vercel Project in the dashboard, and ensure the Vercel
preset is set to "Remix" in the Project Settings. The autodetection will
work correctly once this PR is merged, but for now it gets incorrectly
detected as "Vite" preset.
* **Hint**: You can create a new empty Project by running the `vercel
link` command.
<img width="545" alt="Screenshot 2024-02-27 at 10 37 11"
src="https://github.com/vercel/vercel/assets/71256/f46baf57-5d97-4bde-9529-c9165632cb30">
1. Deploy to Vercel, setting the `VERCEL_CLI_VERSION` environment
variable to use the changes in this PR:
```
vercel deploy -b
VERCEL_CLI_VERSION=https://vercel-git-tootallnate-zero-1217-research-remix-v-next-vite.vercel.sh/tarballs/vercel.tgz
```
Updates package manager detection to account for two lock files. All other managers will only have the one lock file. Bun, however, may have both a `bun.lockb` _and_ a `yarn.lock` file. To ensure bun is properly detected, the presence of `bun.lockb` with `yarn.lock` must occur before `yarn.lock` so we don't mistake the presence of a `yarn.lock` to mean "Yarn".
This PR also adds a test for this situation in `fs-detectors`. The behavior is currently correct there, but was not tested initially. It is now to avoid future regressions.
The `supersedes` property on the `Framework` interface may be specified by a framework in which the superseded framework should _not_ be included in the resulting framework matches during detection.
For example, a Hydrogen v2 project matches both the "remix" and "hydrogen" framework detectors, but we really only want "remix" to be selected.
Follow-up to https://github.com/vercel/vercel/pull/10319, which didn't exhibit this failing test since the fs-detectors package was not changed, and thus those tests didn't run.
Similar to `detectFramework()`, but this version returns all detected frameworks at the given path. It also returns the full `Framework` objects instead of just the slug.
This will be useful for detecting when multiple Projects exist at the same root directory (i.e. Storybook).
**Note:** I rearranged some tests, so probably review with [whitespace changes hidden](https://github.com/vercel/vercel/pull/10195/files?diff=split&w=1).
`LocalFileSystemDetector#readdir()` was throwing an error when a symlink was encountered, due to `fs.readdir()` `withFileTypes: true` option performing an lstat instead of a stat operation.
So re-implement the `readdir()` logic to use `fs.stat()` so that the symlink is resolved (only "dir" and "file" types are expected in the result).
The code for these two are almost identical, so consolidate into one codebase.
Also adjusts the `pnpm test` script to allow for specifying a file name to be executed, instead of running all tests.
In a Next.js app, if you have Node.js Functions in both `api` and `pages/api`, you'll get a warning about moving those `api` functions into `pages/api`. This does specify Node.js Functions already, but a user was confused and tried to move their non-Node.js functions into `pages/api` as well.
This PR tries to make it clearer that you can do both. Note that this still requires a non-ideal local dev setup where you:
- update next config to fall back to the `vc dev` port for `api` handlers
- run `next dev` for the Next.js app on one port
- run `vc dev` for the top-level `api` Functions on another port
---
Example repo: https://github.com/rexfordessilfie/vercel-api-languages
`ignoreRuntimes: ['@vercel/node']` will also disable root-level middleware, which is not intended. So ensure that middleware still work, even for frameworks that have this `ignoreRuntimes` configuration enabled (Remix, RedwoodJS).
Missed this occurrence so it still prevents the upload of serverless
functions that have a mem value that is not dividable by 64.
Should be the last place before we can ship the documentation update.
#### Related PRs
- #9440
Co-authored-by: Steven <steven@ceriously.com>
If there is a monorepo that has one project at the root level we want to
return commands without the relative to root prefixes
---------
Co-authored-by: Chris Barber <chris.barber@vercel.com>
1. `commandForIgnoringBuildStep` should be run at the project directory level not the monorepo root level
2. Simplifying the `installCommand` because doesn't need the relative root unless it is `npm`
In [a different PR](https://github.com/vercel/vercel/pull/9009), detecting frameworks by package name will also provide framework version metadata to the build. Should we update these framework detectors to look up their respective packages or were they not doing that already for a reason?
I left the old detectors in place as fallbacks, which looks like:
```
some: [
{
path: 'package.json',
matchContent:
'"(dev)?(d|D)ependencies":\\s*{[^}]*"remix":\\s*".+?"[^}]*}',
},
{
path: 'remix.config.js',
},
],
```
Please review carefully.
This PR:
- updates `packages/frameworks` to have most supported frameworks specify which dependency version should reflect the overall framework version
- updates `packages/fs-detectors` to allow framework detection that returns the full `Framework` record instead of just the slug
- updates `packages/next` to return the detected Next.js version in the build result
- updates `packages/cli` to leverage these changes so that `vc build` can add `framework: { version: string; }` to `config.json` output
The result is that Build Output API and supported frameworks will return their framework version in the build result of `vc build` when possible, which is used by the build container when creating the deployment. The dashboard later retrieves this value to display in richer deployment outputs.
Supports:
- https://github.com/vercel/api/pull/15601
- https://github.com/vercel/front/pull/18319
---
With the related build container updates, we get to see Next.js version in the build output. You'll see this with BOA+Prebuilt or a normal deploy:
<img width="1228" alt="Screen Shot 2022-12-09 at 2 48 12 PM" src="https://user-images.githubusercontent.com/41545/206793639-f9cd3bdf-b822-45dd-9564-95b94994271d.png">
---
### The Path to this PR
I went through all the supported frameworks and figured out how to best determine their versions. For most of them, we can check a known dependency's installed version number.
We can get most of the way only checking npm. For a handful, we'd have to support Go/Ruby/Rust/Whatever dependencies.
I started with a more complex method signature to allow for later expansion without changing the signature. It looked like this, in practice:
```
async getVersion(dependencies: DependencyMap) => depedencies['next']
```
However, after checking all currently supported frameworks, I don't think this will end up being necessary. It also has the constraint that all dependencies have to be gathered and presented to the function even though it only needs to check for one or two. That's not a huge deal if we have them already where we need them, but we don't. We could use a variant here where this function does its own lookups, but this seemed unnecessary and would beg for duplication and small variances that could cause bugs.
Further, if we only look at `package.json`, we're going to either see a specific version of a version range. To be precise, we have to look at the installed version of the package. That means checking one of the various types of lockfiles that can exist or poking into node_modules.
If we poke into node_modules to detect the installed version, we introduce another point where Yarn 3 (default mode) will not be supported. If we read lockfiles, we have to potentially parse `npm`, `pnpm`, and `yarn` lockfiles.
If we use `npm ls <package-name>`, that also fails in Yarn 3 (default mode). We could accept that and go forward anyway, which would look like:
```
const args = `ls ${packageName} --depth=0 --json`.split(' ');
const { stdout } = await execa('npm', args, { cwd });
const regex = new RegExp(String.raw`${packageName}@([\.\d]+)`);
const matches = stdout.match(regex);
if (matches) {
return matches[1];
}
```
But it turns out there's a `--json` option! That's what I ended up using, for now.
We could explore the lockfile route more, but after some initial digging, it' non-trivial. There are 3 main lockfiles we'd want to check for (npm, pnpm, and yarn) and there are different lockfile versions that put necessary data in different places. I looked for existing tools that parse this, but I didn't find any. We could certainly go down this path, but the effort doesn't seem worth it when `npm ls` gets us really close.
---
### Follow-up Versioning
Now that we know how to determine version per framework, we can vary configuration by version. In a future PR, we could allow a given value to vary by version number:
```
name: (version) => {
if (semver.gt(version, '9.8.7')) {
return 'some-framework-2''
}
return 'some-framework';
}
```
However, it may still be easier to differentiate significant versions by adding multiple entries in the list.
### Related Issues
Improves how `vc build` handles monorepos. In short, this introduces
monorepo manager detection logic and then some helpful defaults so users
don't have to manually specify a `buildCommand` or `installCommand` when
linking a project within a monorepo.
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with
tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a
reviewer
- [ ] Issue from task tracker has a link to this PR
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
### Related Issues
Parse `rush.json` files with `json5` because it is very common for these
to have comments in them
[Template for people to clone for
Rush](https://rushjs.io/pages/configs/rush_json/) which has comments in
it as a default which most people will clone
Docs of Rush showing to not use `JSON.parse`
https://rushjs.io/pages/help/faq/#why-do-rushs-json-config-files-contain--comments-that-github-shows-in-red
Added in tests with block comments and single line comments
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with
tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a
reviewer
- [ ] Issue from task tracker has a link to this PR
Co-authored-by: Sean Massa <EndangeredMassa@gmail.com>
This is to fix the issue with processing new entries of `potentialFiles` when there was a `readdir` cache hit
### Related Issues
Related to https://linear.app/vercel/issue/HIT-57/monorepo-detection-api-prevent-rate-limits
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
### Related Issues
Add in getting the package paths for Rush workspaces.
First get the contents of the `rush.json` file. If the `projects`
property is an array, map through that array and return the
`projectFolder` as the package path.
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with
tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a
reviewer
- [ ] Issue from task tracker has a link to this PR
### Related Issues
Add in Rush workspace
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with
tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a
reviewer
- [ ] Issue from task tracker has a link to this PR
### Related Issues
https://nx.dev/more-concepts/folder-structure#integrated-repo-folder-structure
Nx monorepo has an option to use Nx workspaces.
Nx workspace is defined within the root `workspace.json` file
Within this `workspace.json` file the workspace packages are under
projects
```{
"$schema": "./node_modules/nx/schemas/workspace-schema.json",
"version": 2,
"projects": {
"myblog": "apps/myblog",
"svelte-app": "apps/svelte-app",
}
}
```
Within `getNxWorkspacePackagePaths` get the projects object values for
the paths
Nx is listed as the last workspace manager within
`workspace-managers.ts` because other workspace managers could exist to
check for before Nx workspaces because the `workspace.json` could exist
but not be the correct workspace manager
Nx workspace file can exist when yarn/npm workspaces exist. When this
happens, the workspace.json file is empty with no projects so it will
not add any package paths to the list to look through for projects.
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [ ] The code changed/added as part of this PR has been covered with
tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a
reviewer
- [ ] Issue from task tracker has a link to this PR
### Related Issues
Adding in Nx and Rush as monorepo managers.
This will allow to help with starting zero config for both the above managers.
I have added in unit tests for both Nx and Rush.
### 📋 Checklist
#### Tests
- [ ] The code changed/added as part of this PR has been covered with tests
- [ ] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
Adds a `writeFile` function to `DetectorFilesystem` that will be used to update the various file cache maps.
**Why is this needed?**
When detecting npm7+ monorepos, we identified a performance improvement where the service can inspect the `package-lock.json` file for workspaces, and reuse the package information for each workspace in framework-detection.
The pseudo code in `vercel/api` will look something like this
For a given lockfile
```json
{
...,
"packages": {
"": {
"name": "npm-workspaces",
"version": "1.0.0",
"license": "ISC",
"workspaces": {
"packages": [
"apps/*"
]
}
},
"apps/admin": {
"version": "0.1.0",
"dependencies": {
"next": "12.2.5",
"react": "18.2.0",
"react-dom": "18.2.0"
},
"devDependencies": {
"eslint": "8.23.0",
"eslint-config-next": "12.2.5"
}
},
...,
}
```
```ts
// for each projectPath we detect in package-lock.json
// switch the cwd of the fs to the project directory
const projectFs = fs.chdir(projectPath);
// gets the package info from the lockfile
const projectPackageInfo = lockFile.packages[projectPath];
// insert this content into fs cache
await projectFs.writeFile('package.json', projectPackageInfo)
// call detectFramework, which should now have a cached "package.json" file
const projectFramework = await detectFramework(projectFs);
```
### Related Issues
Related to https://linear.app/vercel/issue/HIT-57/monorepo-detection-api-prevent-rate-limits
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [x] Issue from task tracker has a link to this PR
Adding the ability to explicitly set the `hasPath` and `isFile` caches via `readdir`
**Why is this needed?**
For framework detection, this allows us to fetch the contents of a directory (the root directory) of a git repo, and make assumptions about the existence of files,
```ts
import frameworks from "@vercel/frameworks";
// we can make the set of all file paths needed for framework detection like this
const setOfPaths = new Set(
frameworks.frameworks
.flatMap((f) => [
...(f.detectors?.every ?? []),
...(f.detectors?.some ?? []),
])
.map((d) => d.path)
);
// then we can read the contents of the root directory of a git repo via the `DetectorFilesystem`
fs.readdir('./', { potentialFiles: [...setOfPaths] });
// The filesystem has been fully primed - any network calls from here will only happen if needed.
// This works because the logic of `detectFramework` calls `hasPath` and then `isFile` before
// fetching file contents. Since the filesystem knows this information, it doesn't have to make
// any additional network calls 🙌
const framework = detectFramework(fs);
```
### Related Issues
Related to [HIT-57](https://linear.app/vercel/issue/HIT-57/monorepo-detection-api-prevent-rate-limits)
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [x] This PR has a concise title and thorough description useful to a reviewer
- [x] Issue from task tracker has a link to this PR
Adds a `writeFile` function to `DetectorFilesystem` that will be used to update the various file cache maps.
When detecting npm7+ monorepos, we identified a performance improvement where the service can inspect the `package-lock.json` file for workspaces, and reuse the package information for each workspace in framework-detection.
The pseudo code in `vercel/api` will look something like this
For a given lockfile
```json
{
...,
"packages": {
"": {
"name": "npm-workspaces",
"version": "1.0.0",
"license": "ISC",
"workspaces": {
"packages": [
"apps/*"
]
}
},
"apps/admin": {
"version": "0.1.0",
"dependencies": {
"next": "12.2.5",
"react": "18.2.0",
"react-dom": "18.2.0"
},
"devDependencies": {
"eslint": "8.23.0",
"eslint-config-next": "12.2.5"
}
},
...,
}
```
```ts
// for each projectPath we detect in package-lock.json
// switch the cwd of the fs to the project directory
const projectFs = fs.chdir(projectPath);
// gets the package info from the lockfile
const projectPackageInfo = lockFile.packages[projectPath];
// insert this content into fs cache
projectFs.writeFile('package.json', projectPackageInfo)
// call detectFramework, which should now have a cached "package.json" file
const projectFramework = await detectFramework(projectFs);
```
### Related Issues
Related to [HIT-57](https://linear.app/vercel/issue/HIT-57/monorepo-detection-api-prevent-rate-limits)
### 📋 Checklist
<!--
Please keep your PR as a Draft until the checklist is complete
-->
#### Tests
- [x] The code changed/added as part of this PR has been covered with tests
- [x] All tests pass locally with `yarn test-unit`
#### Code Review
- [ ] This PR has a concise title and thorough description useful to a reviewer
- [ ] Issue from task tracker has a link to this PR
