Compare commits

..

39 Commits

Author SHA1 Message Date
Joe Haddad
c1049985af Publish
- @now/bash@0.1.3-canary.0
 - @now/build-utils@0.4.37-canary.0
 - @now/next@0.0.85-canary.8
2019-02-26 14:33:47 -05:00
Nathan Rajlich
214388ccf3 [now-bash] Use base64 --decode flag
Instead of `-d`. Seems the short flag BSD's `base64` is `-D`,
however the long option name is consistent between both versions.
2019-02-26 11:28:45 -08:00
Joe Haddad
b1d6b7bfc0 feat(@now/next): optional assets directory (#240)
* Make assets part of the build output

* Add console output for user
2019-02-26 14:18:03 -05:00
Nathan Rajlich
ece3564dfd [now-build-utils] Use os.tmpdir() for getWritableDirectory() (#238)
`os.tmpdir()` abstracts away platform differences related to retrieving
the writable temp directory, for example MacOS returns a user-specific
temp directory instead of `/tmp`.

On AWS Lambda, [it returns `/tmp`](https://nexec-v2.n8.io/api/node?arg=-p&arg=require(%27os%27).tmpdir()).

More importantly, it removes the `__dirname/tmp` special-case when not
running in prod. This was problematic for `now dev` because the module
directory is not always writable (i.e. for a `pkg` binary).
2019-02-25 16:56:41 -08:00
Igor Klopov
a88af1f077 Publish
- @now/build-utils@0.4.36
2019-02-24 23:26:14 +03:00
Igor Klopov
d92f7b26c0 lint fixes for build-utils 2019-02-24 23:08:09 +03:00
Igor Klopov
52198af750 Publish
- @now/build-utils@0.4.35-canary.3
2019-02-24 11:56:01 +03:00
Igor Klopov
d58bff2453 [build-utils] chmod +x before running user script 2019-02-24 11:50:07 +03:00
Joe Haddad
8c0a144ae4 Publish
- @now/next@0.0.85-canary.7
2019-02-22 17:13:46 -05:00
Joe Haddad
106e4d5f36 Cache Next.js node_modules for faster rebuilds (#235)
* Add `node_modules` caching for Next.js projects

* Use existing work directory instead of redownloading dependencies

* Reorder fns

* Add prettier configuration

* Move entry files to cache folder

* Correct glob base path

* Be sure to include .yarn-integrity

* Save `package.json` and `yarn.lock`, too

* Increase compatibility so Yarn can dedupe
2019-02-22 16:30:29 -05:00
Nathan Rajlich
66c28bd695 Publish
- @now/next@0.0.85-canary.6
 - @now/node-server@0.5.0-canary.3
 - @now/node@0.5.0-canary.5
2019-02-21 13:20:49 -08:00
Nathan Rajlich
55e75296ff [now-next] Update @now/node-bridge to v1.0.0-canary.2 (#232)
Update `@now/next` to use the latest `@now/node-bridge`, which removes the hard-coded port 3000 that was previously in-place, which was problematic for `now dev`. Now the `http.Server` instance listens on an ephemeral port which is detected by the Now node runtime.

(Similar to 6101ba9d95 and 8dc0c92c58)
2019-02-21 12:55:33 +01:00
Nathan Rajlich
36cbb36737 Add --tags flag when determining if commit it tagged
This fixes matching for "unannotated tags".

See: https://stackoverflow.com/questions/1474115/how-to-find-the-tag-associated-with-a-given-git-commit#comment9135284_1474161
2019-02-20 18:11:11 -08:00
Nathan Rajlich
978ca328ef Publish
- @now/node@0.5.0-canary.4
 - @now/node-server@0.5.0-canary.2
2019-02-20 16:49:58 -08:00
Nathan Rajlich
7b383e0f7c [now-node] Fix build.sh script 2019-02-20 16:40:52 -08:00
Nathan Rajlich
faa5ab36aa Publish
- @now/node@0.5.0-canary.3
2019-02-20 16:32:31 -08:00
Nathan Rajlich
c0a21969dd Regenerate yarn.lock file
Not really sure why this is happening, since `@now/node` and
`@now/node-server` both use this version, but without this commit, then
publish step on CircleCI fails due to a dirty working tree.

Go figure 🤷
2019-02-20 16:31:04 -08:00
Nathan Rajlich
73d0a1723f [now-node] Ensure that @now/node-bridge is installed for build 2019-02-20 14:59:24 -08:00
Nathan Rajlich
7c515544ae [now-node] Don't commit the bridge.d.ts file 2019-02-20 14:52:28 -08:00
Nathan Rajlich
b53c9a6299 Publish
- @now/node@0.5.0-canary.2
2019-02-20 14:51:02 -08:00
Nathan Rajlich
35ff11e6e4 Remove prepublish step for now
It's causing these kinds of failures:

https://circleci.com/gh/zeit/now-builders/839
2019-02-20 14:50:18 -08:00
Nathan Rajlich
64ee4905cd Publish
- @now/node-server@0.5.0-canary.1
 - @now/node@0.5.0-canary.1
2019-02-20 13:53:37 -08:00
Nathan Rajlich
e50dd7e50a Bump now/node and now/node-server to v0.5.0 canary 2019-02-20 13:53:00 -08:00
Nathan Rajlich
6101ba9d95 [now-node-server] Update @now/node-bridge to v1.0.0-canary.2 (#231)
Update `@now/node-server` to use the latest `@now/node-bridge`, which
removes the hard-coded port 3000 that was previously in-place,
which was problematic for `now dev`. Now the `http.Server`
instance listens on an ephemeral port which is detected by the
Now node runtime.
2019-02-20 13:47:17 -08:00
Nathan Rajlich
8dc0c92c58 [now-node] Migrate to TypeScript and use @now/node-bridge v1 (#212)
This moves `@now/node` to being implemented in TypeScript.

It also updates it to use the latest `@now/node-bridge`, which
removes the hard-coded port 3000 that was previously in-place,
which was problematic for `now dev`. Now the `http.Server`
instance listens on an ephemeral port which is detected by the
Now node runtime.
2019-02-20 12:33:21 -08:00
Nathan Rajlich
44c9f3765a Publish
- @now/node-bridge@1.0.0-canary.2
 - @now/node-server@0.4.27-canary.7
 - @now/node@0.4.29-canary.7
2019-02-19 20:49:58 -08:00
Nathan Rajlich
92c05ca338 Pin @now/node-bridge to v0.1.11-canary.0
To address: https://github.com/zeit/now-builders/pull/224#issuecomment-465410905
2019-02-19 20:26:38 -08:00
Nathan Rajlich
069b557906 Publish
- @now/node-bridge@1.0.0-canary.1
2019-02-19 17:26:04 -08:00
Nathan Rajlich
692a0df909 Publish
- @now/node-bridge@0.1.11-canary.1
 - @now/node-server@0.4.27-canary.6
 - @now/node@0.4.29-canary.6
2019-02-19 16:33:18 -08:00
Nathan Rajlich
aeafeb5441 [now-node-bridge] Refactor API to be http.Server focused (#224)
* [now-node-bridge] Refactor API to be `http.Server` focused

This commit refactors the `@now/node-bridge` helper module to work with
`http.Server` instances directly, instead of expecting the `port` to be
set.

The `Bridge` instance calls the `listen()` function on the server instance
which binds to an ephemeral port. This is especially important for
`now dev`, where using a hard-coded port will cause port conflicts for
multiple lambdas using the same builder.

Also converts to TypeScript and adds some basic unit tests.

Example usage:

```js
const server = new Server(() => {});
const bridge = new Bridge(server);
bridge.listen();

const info = await bridge.listening;
assert.equal(info.address, '127.0.0.1');
assert.equal(typeof info.port, 'number');
```

* Update `yarn.lock`

* Add `pretest` script

* Enable TypeScript `strict` mode

* Throw if a string is returned from `server.address()`

Defensive programming ftw

Co-Authored-By: TooTallNate <n@n8.io>

* Prettier

* Prettier

* Fixes

* Attempt to fix CI

* Add `files` array to package.json

* Check for the `Action` property to avoid type casting

Co-Authored-By: TooTallNate <n@n8.io>

* Split the normalizing functions into separate ones

Also adds additional unit tests.

* export the `NowProxyRequest` and `NowProxyResponse`

* Remove last `as` casting

* Move some up

* Debug CircleCI tests :/

* Fix "Invoke" check

* Attempt to fix CircleCI again

* Fix bad

* Convert tests to use `jest`
2019-02-19 16:17:58 -08:00
Nathan Rajlich
a09d5fb355 Add publish.sh script for CircleCI (#228)
* Add `publish.sh` script for CircleCI

Centralize the publishing logic for CircleCI to use.

There was a bug in the previous "Potentially publish stable release"
branch, such that it would be executed for non-tagged commits. The new
publish script checks if there are indeed any tags for the commit, and
bails if there are none.

As an additional bonus, there's now only one publish step in the
CircleCI config, because the publish script determines stable vs. canary
based on the tag name.

* Remove `echo`
2019-02-19 14:45:04 -08:00
Nathan Rajlich
d8017aa9aa Publish
- @now/python@0.0.41-canary.2
2019-02-19 12:14:49 -08:00
Honza Javorek
702f56b9b5 [now-python] Add missing base64 import and fix coding style (#226) 2019-02-19 12:11:12 -08:00
Nathan Rajlich
183b117152 Publish
- @now/python@0.0.41-canary.1
2019-02-19 11:29:26 -08:00
Nathaniel Hill
75b3fb4981 [now-python] Pass the request body to the HTTP request (#99)
* pass body to lambda

* reformat

* look for requirements.txt in lambda root

* single issue PR

* support base64 encoded request body

* support local requirements.txt

* formatting cleanup

* Update packages/now-python/now_handler.py

Co-Authored-By: NathanielHill <nata@goguna.com>

* Avoid error on empty POST body

Just realized I forgot to address the empty POST body error. This should do it

* quick formatting fix

remove double quotes for consistency
2019-02-19 11:28:21 -08:00
Steven
49e63de5fe Publish
- @now/next@0.0.85-canary.5
 - @now/node-server@0.4.27-canary.5
 - @now/node@0.4.29-canary.5
2019-02-17 19:09:27 -05:00
Steven
4742cd32f2 [now-node] add sourceMap flag to ncc (#222)
This adds a {sourceMap:true} flag to ncc builds
2019-02-17 19:08:19 -05:00
Steven
377b73105d [now-node] bump ncc to 0.15.2 (#221)
This has a couple fixes.

See release notes here https://github.com/zeit/ncc/releases/tag/0.15.2
2019-02-17 23:55:55 +01:00
Connor Davis
a5577efb3d Prepublish @now/next 2019-02-15 17:07:14 -06:00
36 changed files with 1421 additions and 273 deletions

12
.circleci/build.sh Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
set -euo pipefail
circleci_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
needs_build="$(grep -rn '"build"' packages/*/package.json | cut -d: -f1)"
for pkg in $needs_build; do
dir="$(dirname "$pkg")"
cd "$circleci_dir/../$dir"
echo "Building \`$dir\`"
yarn build
done

View File

@@ -23,6 +23,9 @@ jobs:
- run:
name: Linting
command: yarn lint
- run:
name: Building
command: ./.circleci/build.sh
- run:
name: Tests
command: yarn test
@@ -30,11 +33,8 @@ jobs:
name: Potentially save npm token
command: "([[ ! -z $NPM_TOKEN ]] && echo \"//registry.npmjs.org/:_authToken=$NPM_TOKEN\" >> ~/.npmrc) || echo \"Did not write npm token\""
- run:
name: Potentially publish canary release
command: "if ls ~/.npmrc >/dev/null 2>&1 && [[ $(git describe --exact-match 2> /dev/null || :) =~ -canary ]]; then yarn run lerna publish from-git --npm-tag canary --yes; else echo \"Did not publish\"; fi"
- run:
name: Potentially publish stable release
command: "if ls ~/.npmrc >/dev/null 2>&1 && [[ ! $(git describe --exact-match 2> /dev/null || :) =~ -canary ]]; then yarn run lerna publish from-git --yes; else echo \"Did not publish\"; fi"
name: Potentially publish releases to npm
command: ./.circleci/publish.sh
workflows:
version: 2
build-and-deploy:

24
.circleci/publish.sh Executable file
View File

@@ -0,0 +1,24 @@
#!/bin/bash
set -euo pipefail
if [ ! -e ~/.npmrc ]; then
echo "~/.npmrc file does not exist, skipping publish"
exit 0
fi
npm_tag=""
tag="$(git describe --tags --exact-match 2> /dev/null || :)"
if [ -z "$tag" ]; then
echo "Not a tagged commit, skipping publish"
exit 0
fi
if [[ "$tag" =~ -canary ]]; then
echo "Publishing canary release"
npm_tag="--npm-tag canary"
else
echo "Publishing stable release"
fi
yarn run lerna publish from-git $npm_tag --yes

3
.prettierrc.json Normal file
View File

@@ -0,0 +1,3 @@
{
"singleQuote": true
}

View File

@@ -21,7 +21,7 @@
"pre-commit": "lint-staged",
"lint-staged": {
"*.js": [
"prettier --write --single-quote",
"prettier --write",
"eslint --fix",
"git add"
]

View File

@@ -1,6 +1,6 @@
{
"name": "@now/bash",
"version": "0.1.2",
"version": "0.1.3-canary.0",
"description": "Now 2.0 builder for HTTP endpoints written in Bash",
"main": "index.js",
"author": "Nathan Rajlich <nate@zeit.co>",

View File

@@ -76,7 +76,7 @@ _lambda_runtime_next() {
_lambda_runtime_body() {
if [ "$(jq --raw-output '.body | type' < "$1")" = "string" ]; then
if [ "$(jq --raw-output '.encoding' < "$1")" = "base64" ]; then
jq --raw-output '.body' < "$1" | base64 -d
jq --raw-output '.body' < "$1" | base64 --decode
else
# assume plain-text body
jq --raw-output '.body' < "$1"

View File

@@ -1,12 +1,10 @@
const path = require('path');
const fs = require('fs-extra');
const prod = process.env.AWS_EXECUTION_ENV || process.env.X_GOOGLE_CODE_LOCATION;
const TMP_PATH = prod ? '/tmp' : path.join(__dirname, 'tmp');
const { join } = require('path');
const { tmpdir } = require('os');
const { mkdirp } = require('fs-extra');
module.exports = async function getWritableDirectory() {
const name = Math.floor(Math.random() * 0x7fffffff).toString(16);
const directory = path.join(TMP_PATH, name);
await fs.mkdirp(directory);
const directory = join(tmpdir(), name);
await mkdirp(directory);
return directory;
};

View File

@@ -13,9 +13,18 @@ function spawnAsync(command, args, cwd) {
});
}
async function chmodPlusX(fsPath) {
const s = await fs.stat(fsPath);
const newMode = s.mode | 64 | 8 | 1; // eslint-disable-line no-bitwise
if (s.mode === newMode) return;
const base8 = newMode.toString(8).slice(-3);
await fs.chmod(fsPath, base8);
}
async function runShellScript(fsPath) {
assert(path.isAbsolute(fsPath));
const destPath = path.dirname(fsPath);
await chmodPlusX(fsPath);
await spawnAsync(`./${path.basename(fsPath)}`, [], destPath);
return true;
}

View File

@@ -1,6 +1,6 @@
{
"name": "@now/build-utils",
"version": "0.4.35-canary.2",
"version": "0.4.37-canary.0",
"license": "MIT",
"repository": {
"type": "git",

View File

@@ -9,6 +9,7 @@ const {
runPackageJsonScript,
} = require('@now/build-utils/fs/run-user-scripts.js'); // eslint-disable-line import/no-extraneous-dependencies
const glob = require('@now/build-utils/fs/glob.js'); // eslint-disable-line import/no-extraneous-dependencies
const fs = require('fs-extra');
const semver = require('semver');
const nextLegacyVersions = require('./legacy-versions');
const {
@@ -39,7 +40,7 @@ async function readPackageJson(entryPath) {
try {
return JSON.parse(await readFile(packagePath, 'utf8'));
} catch (err) {
console.log('no package.json found in entry');
console.log('package.json not found in entry');
return {};
}
}
@@ -68,6 +69,36 @@ async function writeNpmRc(workPath, token) {
);
}
function getNextVersion(packageJson) {
let nextVersion;
if (packageJson.dependencies && packageJson.dependencies.next) {
nextVersion = packageJson.dependencies.next;
} else if (packageJson.devDependencies && packageJson.devDependencies.next) {
nextVersion = packageJson.devDependencies.next;
}
return nextVersion;
}
function isLegacyNext(nextVersion) {
// If version is using the dist-tag instead of a version range
if (nextVersion === 'canary' || nextVersion === 'latest') {
return false;
}
// If the version is an exact match with the legacy versions
if (nextLegacyVersions.indexOf(nextVersion) !== -1) {
return true;
}
const maxSatisfying = semver.maxSatisfying(nextLegacyVersions, nextVersion);
// When the version can't be matched with legacy versions, so it must be a newer version
if (maxSatisfying === null) {
return false;
}
return true;
}
exports.config = {
maxLambdaSize: '5mb',
};
@@ -86,38 +117,14 @@ exports.build = async ({ files, workPath, entrypoint }) => {
const pkg = await readPackageJson(entryPath);
let nextVersion;
if (pkg.dependencies && pkg.dependencies.next) {
nextVersion = pkg.dependencies.next;
} else if (pkg.devDependencies && pkg.devDependencies.next) {
nextVersion = pkg.devDependencies.next;
}
const nextVersion = getNextVersion(pkg);
if (!nextVersion) {
throw new Error(
'No Next.js version could be detected in "package.json". Make sure `"next"` is installed in "dependencies" or "devDependencies"',
);
}
const isLegacy = (() => {
// If version is using the dist-tag instead of a version range
if (nextVersion === 'canary' || nextVersion === 'latest') {
return false;
}
// If the version is an exact match with the legacy versions
if (nextLegacyVersions.indexOf(nextVersion) !== -1) {
return true;
}
const maxSatisfying = semver.maxSatisfying(nextLegacyVersions, nextVersion);
// When the version can't be matched with legacy versions, so it must be a newer version
if (maxSatisfying === null) {
return false;
}
return true;
})();
const isLegacy = isLegacyNext(nextVersion);
console.log(`MODE: ${isLegacy ? 'legacy' : 'serverless'}`);
@@ -278,6 +285,18 @@ exports.build = async ({ files, workPath, entrypoint }) => {
);
}
// An optional assets folder that is placed alongside every page entrypoint
const assets = await glob(
'assets/**',
path.join(entryPath, '.next', 'serverless'),
);
const assetKeys = Object.keys(assets);
if (assetKeys.length > 0) {
console.log('detected assets to be bundled with lambda:');
assetKeys.forEach(assetFile => console.log(`\t${assetFile}`));
}
await Promise.all(
pageKeys.map(async (page) => {
// These default pages don't have to be handled as they'd always 404
@@ -291,6 +310,7 @@ exports.build = async ({ files, workPath, entrypoint }) => {
lambdas[path.join(entryDirectory, pathname)] = await createLambda({
files: {
...launcherFiles,
...assets,
'page.js': pages[page],
},
handler: 'now__launcher.launcher',
@@ -308,7 +328,9 @@ exports.build = async ({ files, workPath, entrypoint }) => {
const staticFiles = Object.keys(nextStaticFiles).reduce(
(mappedFiles, file) => ({
...mappedFiles,
[path.join(entryDirectory, `_next/static/${file}`)]: nextStaticFiles[file],
[path.join(entryDirectory, `_next/static/${file}`)]: nextStaticFiles[
file
],
}),
{},
);
@@ -326,3 +348,39 @@ exports.build = async ({ files, workPath, entrypoint }) => {
return { ...lambdas, ...staticFiles, ...staticDirectoryFiles };
};
exports.prepareCache = async ({ cachePath, workPath, entrypoint }) => {
console.log('preparing cache ...');
const entryDirectory = path.dirname(entrypoint);
const entryPath = path.join(workPath, entryDirectory);
const cacheEntryPath = path.join(cachePath, entryDirectory);
const pkg = await readPackageJson(entryPath);
const nextVersion = getNextVersion(pkg);
const isLegacy = isLegacyNext(nextVersion);
if (isLegacy) {
// skip caching legacy mode (swapping deps between all and production can get bug-prone)
return {};
}
console.log('clearing old cache ...');
fs.removeSync(cacheEntryPath);
fs.mkdirpSync(cacheEntryPath);
console.log('copying build files for cache ...');
fs.renameSync(entryPath, cacheEntryPath);
console.log('producing cache file manifest ...');
const cacheEntrypoint = path.relative(cachePath, cacheEntryPath);
return {
...(await glob(
path.join(cacheEntrypoint, 'node_modules/{**,!.*,.yarn*}'),
cachePath,
)),
...(await glob(path.join(cacheEntrypoint, 'package-lock.json'), cachePath)),
...(await glob(path.join(cacheEntrypoint, 'yarn.lock'), cachePath)),
};
};

View File

@@ -4,10 +4,8 @@ const { Server } = require('http');
const { Bridge } = require('./now__bridge.js');
const page = require('./page.js');
const bridge = new Bridge();
bridge.port = 3000;
const server = new Server(page.render);
server.listen(bridge.port);
const bridge = new Bridge(server);
bridge.listen();
exports.launcher = bridge.launcher;

View File

@@ -3,9 +3,6 @@ const next = require('next-server');
const url = require('url');
const { Bridge } = require('./now__bridge.js');
const bridge = new Bridge();
bridge.port = 3000;
process.env.NODE_ENV = 'production';
const app = next({});
@@ -14,6 +11,8 @@ const server = new Server((req, res) => {
const parsedUrl = url.parse(req.url, true);
app.render(req, res, 'PATHNAME_PLACEHOLDER', parsedUrl.query, parsedUrl);
});
server.listen(bridge.port);
const bridge = new Bridge(server);
bridge.listen();
exports.launcher = bridge.launcher;

View File

@@ -1,6 +1,6 @@
{
"name": "@now/next",
"version": "0.0.85-canary.4",
"version": "0.0.85-canary.8",
"license": "MIT",
"repository": {
"type": "git",
@@ -8,8 +8,9 @@
"directory": "packages/now-next"
},
"dependencies": {
"@now/node-bridge": "0.1.4",
"@now/node-bridge": "1.0.0-canary.2",
"execa": "^1.0.0",
"fs-extra": "^7.0.0",
"fs.promised": "^3.0.0",
"semver": "^5.6.0"
}

View File

@@ -0,0 +1,24 @@
{
"extends": ["prettier", "airbnb-base"],
"rules": {
"no-console": 0,
"import/no-unresolved": 0,
"import/no-dynamic-require": 0,
"global-require": 0
},
"overrides": [
{
"files": ["test/**"],
"rules": {
"import/no-extraneous-dependencies": 0
},
"globals": {
"describe": true,
"it": true,
"test": true,
"expect": true
}
}
]
}

1
packages/now-node-bridge/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
/bridge.*

View File

@@ -1,110 +0,0 @@
const http = require('http');
function normalizeEvent(event) {
let isApiGateway = true;
if (event.Action === 'Invoke') {
isApiGateway = false;
const invokeEvent = JSON.parse(event.body);
const {
method, path, headers, encoding,
} = invokeEvent;
let { body } = invokeEvent;
if (body) {
if (encoding === 'base64') {
body = Buffer.from(body, encoding);
} else if (encoding === undefined) {
body = Buffer.from(body);
} else {
throw new Error(`Unsupported encoding: ${encoding}`);
}
}
return {
isApiGateway, method, path, headers, body,
};
}
const {
httpMethod: method, path, headers, body,
} = event;
return {
isApiGateway, method, path, headers, body,
};
}
class Bridge {
constructor() {
this.launcher = this.launcher.bind(this);
}
launcher(event) {
// eslint-disable-next-line consistent-return
return new Promise((resolve, reject) => {
if (this.userError) {
console.error('Error while initializing entrypoint:', this.userError);
return resolve({ statusCode: 500, body: '' });
}
if (!this.port) {
return resolve({ statusCode: 504, body: '' });
}
const {
isApiGateway, method, path, headers, body,
} = normalizeEvent(event);
const opts = {
hostname: '127.0.0.1',
port: this.port,
path,
method,
headers,
};
const req = http.request(opts, (res) => {
const response = res;
const respBodyChunks = [];
response.on('data', chunk => respBodyChunks.push(Buffer.from(chunk)));
response.on('error', reject);
response.on('end', () => {
const bodyBuffer = Buffer.concat(respBodyChunks);
delete response.headers.connection;
if (isApiGateway) {
delete response.headers['content-length'];
} else
if (response.headers['content-length']) {
response.headers['content-length'] = bodyBuffer.length;
}
resolve({
statusCode: response.statusCode,
headers: response.headers,
body: bodyBuffer.toString('base64'),
encoding: 'base64',
});
});
});
req.on('error', (error) => {
setTimeout(() => {
// this lets express print the true error of why the connection was closed.
// it is probably 'Cannot set headers after they are sent to the client'
reject(error);
}, 2);
});
if (body) req.write(body);
req.end();
});
}
}
module.exports = {
Bridge,
};

View File

@@ -1,10 +1,26 @@
{
"name": "@now/node-bridge",
"version": "0.1.11-canary.0",
"version": "1.0.0-canary.2",
"license": "MIT",
"main": "./index.js",
"repository": {
"type": "git",
"url": "https://github.com/zeit/now-builders.git",
"directory": "packages/now-node-bridge"
},
"files": [
"bridge.*",
"index.js"
],
"scripts": {
"build": "tsc",
"test": "npm run build && jest",
"prepublish": "npm run build"
},
"devDependencies": {
"@types/aws-lambda": "8.10.19",
"@types/node": "11.9.4",
"jest": "24.1.0",
"typescript": "3.3.3"
}
}

View File

@@ -0,0 +1,183 @@
import { AddressInfo } from 'net';
import { APIGatewayProxyEvent } from 'aws-lambda';
import {
Server,
IncomingHttpHeaders,
OutgoingHttpHeaders,
request
} from 'http';
interface NowProxyEvent {
Action: string;
body: string;
}
export interface NowProxyRequest {
isApiGateway?: boolean;
method: string;
path: string;
headers: IncomingHttpHeaders;
body: Buffer;
}
export interface NowProxyResponse {
statusCode: number;
headers: OutgoingHttpHeaders;
body: string;
encoding: string;
}
function normalizeNowProxyEvent(event: NowProxyEvent): NowProxyRequest {
let bodyBuffer: Buffer | null;
const { method, path, headers, encoding, body } = JSON.parse(event.body);
if (body) {
if (encoding === 'base64') {
bodyBuffer = Buffer.from(body, encoding);
} else if (encoding === undefined) {
bodyBuffer = Buffer.from(body);
} else {
throw new Error(`Unsupported encoding: ${encoding}`);
}
} else {
bodyBuffer = Buffer.alloc(0);
}
return { isApiGateway: false, method, path, headers, body: bodyBuffer };
}
function normalizeAPIGatewayProxyEvent(
event: APIGatewayProxyEvent
): NowProxyRequest {
let bodyBuffer: Buffer | null;
const { httpMethod: method, path, headers, body } = event;
if (body) {
if (event.isBase64Encoded) {
bodyBuffer = Buffer.from(body, 'base64');
} else {
bodyBuffer = Buffer.from(body);
}
} else {
bodyBuffer = Buffer.alloc(0);
}
return { isApiGateway: true, method, path, headers, body: bodyBuffer };
}
function normalizeEvent(
event: NowProxyEvent | APIGatewayProxyEvent
): NowProxyRequest {
if ('Action' in event) {
if (event.Action === 'Invoke') {
return normalizeNowProxyEvent(event);
} else {
throw new Error(`Unexpected event.Action: ${event.Action}`);
}
} else {
return normalizeAPIGatewayProxyEvent(event);
}
}
export class Bridge {
private server: Server | null;
private listening: Promise<AddressInfo>;
private resolveListening: (info: AddressInfo) => void;
constructor(server?: Server) {
this.server = null;
if (server) {
this.setServer(server);
}
this.launcher = this.launcher.bind(this);
// This is just to appease TypeScript strict mode, since it doesn't
// understand that the Promise constructor is synchronous
this.resolveListening = (info: AddressInfo) => {};
this.listening = new Promise(resolve => {
this.resolveListening = resolve;
});
}
setServer(server: Server) {
this.server = server;
server.once('listening', () => {
const addr = server.address();
if (typeof addr === 'string') {
throw new Error(`Unexpected string for \`server.address()\`: ${addr}`);
} else if (!addr) {
throw new Error('`server.address()` returned `null`');
} else {
this.resolveListening(addr);
}
});
}
listen() {
if (!this.server) {
throw new Error('Server has not been set!');
}
return this.server.listen({
host: '127.0.0.1',
port: 0
});
}
async launcher(
event: NowProxyEvent | APIGatewayProxyEvent
): Promise<NowProxyResponse> {
const { port } = await this.listening;
const { isApiGateway, method, path, headers, body } = normalizeEvent(
event
);
const opts = {
hostname: '127.0.0.1',
port,
path,
method,
headers
};
// eslint-disable-next-line consistent-return
return new Promise((resolve, reject) => {
const req = request(opts, res => {
const response = res;
const respBodyChunks: Buffer[] = [];
response.on('data', chunk => respBodyChunks.push(Buffer.from(chunk)));
response.on('error', reject);
response.on('end', () => {
const bodyBuffer = Buffer.concat(respBodyChunks);
delete response.headers.connection;
if (isApiGateway) {
delete response.headers['content-length'];
} else if (response.headers['content-length']) {
response.headers['content-length'] = String(bodyBuffer.length);
}
resolve({
statusCode: response.statusCode || 200,
headers: response.headers,
body: bodyBuffer.toString('base64'),
encoding: 'base64'
});
});
});
req.on('error', error => {
setTimeout(() => {
// this lets express print the true error of why the connection was closed.
// it is probably 'Cannot set headers after they are sent to the client'
reject(error);
}, 2);
});
if (body) req.write(body);
req.end();
});
}
}

View File

@@ -0,0 +1,71 @@
const assert = require('assert');
const { Server } = require('http');
const { Bridge } = require('../bridge');
test('port binding', async () => {
const server = new Server();
const bridge = new Bridge(server);
bridge.listen();
// Test port binding
const info = await bridge.listening;
assert.equal(info.address, '127.0.0.1');
assert.equal(typeof info.port, 'number');
server.close();
});
test('`APIGatewayProxyEvent` normalizing', async () => {
const server = new Server((req, res) => res.end(
JSON.stringify({
method: req.method,
path: req.url,
headers: req.headers,
}),
));
const bridge = new Bridge(server);
bridge.listen();
const result = await bridge.launcher({
httpMethod: 'GET',
headers: { foo: 'bar' },
path: '/apigateway',
body: null,
});
assert.equal(result.encoding, 'base64');
assert.equal(result.statusCode, 200);
const body = JSON.parse(Buffer.from(result.body, 'base64').toString());
assert.equal(body.method, 'GET');
assert.equal(body.path, '/apigateway');
assert.equal(body.headers.foo, 'bar');
server.close();
});
test('`NowProxyEvent` normalizing', async () => {
const server = new Server((req, res) => res.end(
JSON.stringify({
method: req.method,
path: req.url,
headers: req.headers,
}),
));
const bridge = new Bridge(server);
bridge.listen();
const result = await bridge.launcher({
Action: 'Invoke',
body: JSON.stringify({
method: 'POST',
headers: { foo: 'baz' },
path: '/nowproxy',
body: 'body=1',
}),
});
assert.equal(result.encoding, 'base64');
assert.equal(result.statusCode, 200);
const body = JSON.parse(Buffer.from(result.body, 'base64').toString());
assert.equal(body.method, 'POST');
assert.equal(body.path, '/nowproxy');
assert.equal(body.headers.foo, 'baz');
server.close();
});

View File

@@ -0,0 +1,16 @@
{
"compilerOptions": {
"target": "es6",
"module": "commonjs",
"outDir": ".",
"strict": true,
"sourceMap": true,
"declaration": true
},
"include": [
"src/**/*"
],
"exclude": [
"node_modules"
]
}

View File

@@ -48,7 +48,7 @@ async function downloadInstallAndBundle(
data: JSON.stringify({
license: 'UNLICENSED',
dependencies: {
'@zeit/ncc': '0.15.1',
'@zeit/ncc': '0.15.2',
},
}),
}),
@@ -64,7 +64,7 @@ async function downloadInstallAndBundle(
async function compile(workNccPath, downloadedFiles, entrypoint) {
const input = downloadedFiles[entrypoint].fsPath;
const ncc = require(path.join(workNccPath, 'node_modules/@zeit/ncc'));
const { code, assets } = await ncc(input);
const { code, assets } = await ncc(input, { sourceMap: true });
const preparedFiles = {};
const blob = new FileBlob({ data: code });

View File

@@ -4,22 +4,16 @@ const { Bridge } = require('./bridge.js');
const bridge = new Bridge();
const saveListen = Server.prototype.listen;
Server.prototype.listen = function listen(...args) {
this.on('listening', function listening() {
bridge.port = this.address().port;
});
saveListen.apply(this, args);
Server.prototype.listen = function listen() {
bridge.setServer(this);
Server.prototype.listen = saveListen;
return bridge.listen();
};
try {
if (!process.env.NODE_ENV) {
process.env.NODE_ENV = 'production';
}
// PLACEHOLDER
} catch (error) {
console.error(error);
bridge.userError = error;
if (!process.env.NODE_ENV) {
process.env.NODE_ENV = 'production';
}
// PLACEHOLDER
exports.launcher = bridge.launcher;

View File

@@ -1,6 +1,6 @@
{
"name": "@now/node-server",
"version": "0.4.27-canary.4",
"version": "0.5.0-canary.3",
"license": "MIT",
"repository": {
"type": "git",
@@ -8,7 +8,7 @@
"directory": "packages/now-node-server"
},
"dependencies": {
"@now/node-bridge": "^0.1.11-canary.0",
"@now/node-bridge": "1.0.0-canary.2",
"fs-extra": "7.0.1"
},
"scripts": {

2
packages/now-node/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
/dist
/src/bridge.d.ts

View File

@@ -1 +0,0 @@
/test

12
packages/now-node/build.sh Executable file
View File

@@ -0,0 +1,12 @@
#!/bin/bash
set -euo pipefail
bridge_entrypoint="$(node -p 'require.resolve("@now/node-bridge")')"
bridge_defs="$(dirname "$bridge_entrypoint")/bridge.d.ts"
if [ ! -e "$bridge_defs" ]; then
yarn install
fi
cp -v "$bridge_defs" src
tsc

View File

@@ -1,22 +0,0 @@
const { Server } = require('http');
const { Bridge } = require('./bridge.js');
const bridge = new Bridge();
bridge.port = 3000;
let listener;
try {
if (!process.env.NODE_ENV) {
process.env.NODE_ENV = 'production';
}
// PLACEHOLDER
} catch (error) {
console.error(error);
bridge.userError = error;
}
const server = new Server(listener);
server.listen(bridge.port);
exports.launcher = bridge.launcher;

View File

@@ -1,17 +1,27 @@
{
"name": "@now/node",
"version": "0.4.29-canary.4",
"version": "0.5.0-canary.5",
"license": "MIT",
"main": "./dist/index",
"repository": {
"type": "git",
"url": "https://github.com/zeit/now-builders.git",
"directory": "packages/now-node"
},
"dependencies": {
"@now/node-bridge": "^0.1.11-canary.0",
"@now/node-bridge": "1.0.0-canary.2",
"fs-extra": "7.0.1"
},
"scripts": {
"build": "./build.sh",
"test": "jest"
},
"files": [
"dist"
],
"devDependencies": {
"@types/node": "11.9.4",
"jest": "24.1.0",
"typescript": "3.3.3"
}
}

View File

@@ -1,14 +1,14 @@
const { createLambda } = require('@now/build-utils/lambda.js'); // eslint-disable-line import/no-extraneous-dependencies
const download = require('@now/build-utils/fs/download.js'); // eslint-disable-line import/no-extraneous-dependencies
const FileBlob = require('@now/build-utils/file-blob.js'); // eslint-disable-line import/no-extraneous-dependencies
const FileFsRef = require('@now/build-utils/file-fs-ref.js'); // eslint-disable-line import/no-extraneous-dependencies
const fs = require('fs-extra');
const glob = require('@now/build-utils/fs/glob.js'); // eslint-disable-line import/no-extraneous-dependencies
const path = require('path');
const {
import { join, dirname } from 'path';
import { remove, readFile } from 'fs-extra';
import * as glob from '@now/build-utils/fs/glob.js';
import * as download from '@now/build-utils/fs/download.js';
import * as FileBlob from '@now/build-utils/file-blob.js';
import * as FileFsRef from '@now/build-utils/file-fs-ref.js';
import { createLambda } from '@now/build-utils/lambda.js';
import {
runNpmInstall,
runPackageJsonScript,
} = require('@now/build-utils/fs/run-user-scripts.js'); // eslint-disable-line import/no-extraneous-dependencies
runPackageJsonScript
} from '@now/build-utils/fs/run-user-scripts.js';
/** @typedef { import('@now/build-utils/file-ref') } FileRef */
/** @typedef {{[filePath: string]: FileRef}} Files */
@@ -27,16 +27,16 @@ const {
*/
async function downloadInstallAndBundle(
{ files, entrypoint, workPath },
{ npmArguments = [] } = {},
{ npmArguments = [] } = {}
) {
const userPath = path.join(workPath, 'user');
const nccPath = path.join(workPath, 'ncc');
const userPath = join(workPath, 'user');
const nccPath = join(workPath, 'ncc');
console.log('downloading user files...');
const downloadedFiles = await download(files, userPath);
console.log("installing dependencies for user's code...");
const entrypointFsDirname = path.join(userPath, path.dirname(entrypoint));
const entrypointFsDirname = join(userPath, dirname(entrypoint));
await runNpmInstall(entrypointFsDirname, npmArguments);
console.log('writing ncc package.json...');
@@ -46,12 +46,12 @@ async function downloadInstallAndBundle(
data: JSON.stringify({
license: 'UNLICENSED',
dependencies: {
'@zeit/ncc': '0.15.1',
},
}),
}),
'@zeit/ncc': '0.15.2',
}
})
})
},
nccPath,
nccPath
);
console.log('installing dependencies for ncc...');
@@ -59,43 +59,41 @@ async function downloadInstallAndBundle(
return [downloadedFiles, nccPath, entrypointFsDirname];
}
async function compile(workNccPath, downloadedFiles, entrypoint) {
async function compile(workNccPath: string, downloadedFiles, entrypoint: string) {
const input = downloadedFiles[entrypoint].fsPath;
const ncc = require(path.join(workNccPath, 'node_modules/@zeit/ncc'));
const ncc = require(join(workNccPath, 'node_modules/@zeit/ncc'));
const { code, assets } = await ncc(input);
const preparedFiles = {};
const blob = new FileBlob({ data: code });
// move all user code to 'user' subdirectory
preparedFiles[path.join('user', entrypoint)] = blob;
preparedFiles[join('user', entrypoint)] = blob;
// eslint-disable-next-line no-restricted-syntax
for (const assetName of Object.keys(assets)) {
const { source: data, permissions: mode } = assets[assetName];
const blob2 = new FileBlob({ data, mode });
preparedFiles[
path.join('user', path.dirname(entrypoint), assetName)
] = blob2;
preparedFiles[join('user', dirname(entrypoint), assetName)] = blob2;
}
return preparedFiles;
}
exports.config = {
maxLambdaSize: '5mb',
export const config = {
maxLambdaSize: '5mb'
};
/**
* @param {BuildParamsType} buildParams
* @returns {Promise<Files>}
*/
exports.build = async ({ files, entrypoint, workPath }) => {
export async function build({ files, entrypoint, workPath }) {
const [
downloadedFiles,
workNccPath,
entrypointFsDirname,
entrypointFsDirname
] = await downloadInstallAndBundle(
{ files, entrypoint, workPath },
{ npmArguments: ['--prefer-offline'] },
{ npmArguments: ['--prefer-offline'] }
);
console.log('running user script...');
@@ -103,36 +101,34 @@ exports.build = async ({ files, entrypoint, workPath }) => {
console.log('compiling entrypoint with ncc...');
const preparedFiles = await compile(workNccPath, downloadedFiles, entrypoint);
const launcherPath = path.join(__dirname, 'launcher.js');
let launcherData = await fs.readFile(launcherPath, 'utf8');
const launcherPath = join(__dirname, 'launcher.js');
let launcherData = await readFile(launcherPath, 'utf8');
launcherData = launcherData.replace(
'// PLACEHOLDER',
[
'process.chdir("./user");',
`listener = require("./${path.join('user', entrypoint)}");`,
'if (listener.default) listener = listener.default;',
].join(' '),
`listener = require("./${join('user', entrypoint)}");`,
'if (listener.default) listener = listener.default;'
].join(' ')
);
const launcherFiles = {
'launcher.js': new FileBlob({ data: launcherData }),
'bridge.js': new FileFsRef({ fsPath: require('@now/node-bridge') }),
'bridge.js': new FileFsRef({ fsPath: require('@now/node-bridge') })
};
const lambda = await createLambda({
files: { ...preparedFiles, ...launcherFiles },
handler: 'launcher.launcher',
runtime: 'nodejs8.10',
runtime: 'nodejs8.10'
});
return { [entrypoint]: lambda };
};
}
exports.prepareCache = async ({
files, entrypoint, workPath, cachePath,
}) => {
await fs.remove(workPath);
export async function prepareCache({ files, entrypoint, workPath, cachePath }) {
await remove(workPath);
await downloadInstallAndBundle({ files, entrypoint, workPath: cachePath });
return {
@@ -141,6 +137,6 @@ exports.prepareCache = async ({
...(await glob('user/yarn.lock', cachePath)),
...(await glob('ncc/node_modules/**', cachePath)),
...(await glob('ncc/package-lock.json', cachePath)),
...(await glob('ncc/yarn.lock', cachePath)),
...(await glob('ncc/yarn.lock', cachePath))
};
};
}

View File

@@ -0,0 +1,16 @@
import { Server } from 'http';
import { Bridge } from './bridge';
let listener;
if (!process.env.NODE_ENV) {
process.env.NODE_ENV = 'production';
}
// PLACEHOLDER
const server = new Server(listener);
const bridge = new Bridge(server);
bridge.listen();
exports.launcher = bridge.launcher;

View File

@@ -0,0 +1,15 @@
{
"compilerOptions": {
"target": "es6",
"module": "commonjs",
"outDir": "dist",
"sourceMap": false,
"declaration": false
},
"include": [
"src/**/*"
],
"exclude": [
"node_modules"
]
}

View File

@@ -40,8 +40,16 @@ exports.build = async ({ files, entrypoint }) => {
await pipInstall(pipPath, srcDir, 'requests');
if (files['requirements.txt']) {
console.log('found "requirements.txt"');
const entryDirectory = path.dirname(entrypoint);
const requirementsTxt = path.join(entryDirectory, 'requirements.txt');
if (files[requirementsTxt]) {
console.log('found local "requirements.txt"');
const requirementsTxtPath = files[requirementsTxt].fsPath;
await pipInstall(pipPath, srcDir, '-r', requirementsTxtPath);
} else if (files['requirements.txt']) {
console.log('found global "requirements.txt"');
const requirementsTxtPath = files['requirements.txt'].fsPath;
await pipInstall(pipPath, srcDir, '-r', requirementsTxtPath);

View File

@@ -1,23 +1,35 @@
import base64
from http.server import HTTPServer
import json
import requests
from __NOW_HANDLER_FILENAME import handler
import _thread
server = HTTPServer(('', 3000), handler)
def now_handler(event, context):
_thread.start_new_thread(server.handle_request, ())
payload = json.loads(event['body'])
path = payload['path']
headers = payload['headers']
method = payload['method']
res = requests.request(method, 'http://0.0.0.0:3000' + path, headers=headers)
encoding = payload.get('encoding')
body = payload.get('body')
if (
(body is not None and len(body) > 0) and
(encoding is not None and encoding == 'base64')
):
body = base64.b64decode(body)
res = requests.request(method, 'http://0.0.0.0:3000' + path,
headers=headers, data=body, allow_redirects=False)
return {
'statusCode': res.status_code,
'headers': dict(res.headers),
'body': res.text
'body': res.text,
}

View File

@@ -1,6 +1,6 @@
{
"name": "@now/python",
"version": "0.0.41-canary.0",
"version": "0.0.41-canary.2",
"main": "index.js",
"license": "MIT",
"repository": {

831
yarn.lock

File diff suppressed because it is too large Load Diff