Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrangler: some additions #4013

Merged
merged 33 commits into from
Sep 26, 2023
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
15f17b4
vectorize: add wrangler vectorize
elithrar Sep 22, 2023
2e3a53d
vectorize: add changeset
elithrar Sep 22, 2023
f0c4185
vectorize: remove bad mTLS test, add snapshot to index
elithrar Sep 22, 2023
f0d05af
vectorize: update @cloudflare/workers-types to tip
elithrar Sep 22, 2023
a5db98b
vectorize: update lockfile
elithrar Sep 22, 2023
c930463
Apply suggestions from code review
elithrar Sep 22, 2023
df983e0
vectorize: fix test output
elithrar Sep 22, 2023
fe967c0
vectorize: skip-confirmation -> force
elithrar Sep 22, 2023
c22b95b
vectorize: fix ai command rebase issue?
elithrar Sep 22, 2023
00288b6
vectorize: fix release notes
elithrar Sep 22, 2023
0b61c6d
vectorize: indexes -> indices
elithrar Sep 22, 2023
35cf1d0
vectorize: improve wrangler HTTP 204+205 handling
elithrar Sep 22, 2023
3adf7ae
vectorize: test output spacing...
elithrar Sep 23, 2023
921201c
vectorize: fix unused import
elithrar Sep 23, 2023
0056b04
vectorize: add binding support for [[vectorize]]
elithrar Sep 23, 2023
84687e2
vectorize: update tests
elithrar Sep 23, 2023
2289787
vectorize: add lockfile
elithrar Sep 23, 2023
2805704
Upgrade XDG package (#4005)
JacobMGEvans Sep 23, 2023
fa8cae5
vectorize: fix missing binding config
elithrar Sep 23, 2023
1f0cc8d
vectorize: npm run bundle
elithrar Sep 23, 2023
d00888d
Merge branch 'main' into silverlock/wrangler-vectorize
elithrar Sep 23, 2023
d54207b
vectorize: npm run bundle
elithrar Sep 23, 2023
865619a
vectorize: fix binding output
elithrar Sep 23, 2023
14605a1
vectorize: use indexes > indices
elithrar Sep 23, 2023
73a28b2
vectorize: fix formatting
elithrar Sep 24, 2023
1c0b4db
vectorize: address comments
elithrar Sep 25, 2023
027a137
vectorize: add mocks
elithrar Sep 25, 2023
1e586a7
running formatter
JacobMGEvans Sep 25, 2023
810385d
vectorize: comment HTTP 204/205
elithrar Sep 25, 2023
f7cbcae
vectorize: add validation tests
elithrar Sep 25, 2023
f965c3a
Fix tests & lockfile
penalosa Sep 25, 2023
cc82471
Reduce lockfile diff
penalosa Sep 25, 2023
4692d92
Merge remote-tracking branch 'origin/main' into silverlock/wrangler-v…
penalosa Sep 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/good-ads-fold.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
"wrangler": minor
---

Adds wrangler support for Vectorize, Cloudflare's new vector database, with
`wrangler vectorize`. Visit the developer documentation
(https://developers.cloudflare.com/vectorize/) to learn more and create your
first vector database with `wrangler vectorize create my-first-index`.
10 changes: 5 additions & 5 deletions packages/wrangler/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -116,11 +116,11 @@
"xxhash-wasm": "^1.0.1"
},
"devDependencies": {
"@cloudflare/pages-shared": "workspace:^",
"@cloudflare/eslint-config-worker": "*",
"@cloudflare/pages-shared": "workspace:^",
"@cloudflare/types": "^6.18.4",
"@cloudflare/workers-tsconfig": "workspace:*",
"@cloudflare/workers-types": "^4.20230724.0",
"@cloudflare/workers-types": "^4.20230914.0",
"@iarna/toml": "^3.0.0",
"@microsoft/api-extractor": "^7.28.3",
"@types/body-parser": "^1.19.2",
Expand All @@ -130,6 +130,7 @@
"@types/glob-to-regexp": "0.4.1",
"@types/is-ci": "^3.0.0",
"@types/javascript-time-ago": "^2.0.3",
"@types/jest": "^29.5.5",
"@types/mime": "^2.0.3",
"@types/minimatch": "^5.1.2",
"@types/prompts": "^2.0.14",
Expand All @@ -141,9 +142,6 @@
"@types/ws": "^8.5.3",
"@types/yargs": "^17.0.10",
"@webcontainer/env": "^1.1.0",
"@types/jest": "^29.5.5",
"esbuild-jest": "0.5.0",
"jest": "^29.7.0",
"body-parser": "^1.20.0",
"chalk": "^2.4.2",
"cli-table3": "^0.6.3",
Expand All @@ -153,6 +151,7 @@
"concurrently": "^7.2.2",
"devtools-protocol": "^0.0.955664",
"dotenv": "^16.0.0",
"esbuild-jest": "0.5.0",
"execa": "^6.1.0",
"express": "^4.18.1",
"finalhandler": "^1.2.0",
Expand All @@ -169,6 +168,7 @@
"ink-text-input": "^4.0.3",
"is-ci": "^3.0.1",
"javascript-time-ago": "^2.5.4",
"jest": "^29.7.0",
"jest-fetch-mock": "^3.0.3",
"jest-websocket-mock": "^2.5.0",
"mime": "^3.0.0",
Expand Down
1 change: 1 addition & 0 deletions packages/wrangler/src/__tests__/configuration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ describe("normalizeAndValidateConfig()", () => {
compatibility_flags: [],
configPath: undefined,
d1_databases: [],
vectorize: [],
constellation: [],
dev: {
ip: "0.0.0.0",
Expand Down
2 changes: 2 additions & 0 deletions packages/wrangler/src/__tests__/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ describe("wrangler", () => {
wrangler d1 🗄 Interact with a D1 database
wrangler ai 🤖 Interact with AI models
wrangler constellation 🤖 Interact with Constellation models
wrangler vectorize 🧮 Interact with Vectorize indexes
wrangler pubsub 📮 Interact and manage Pub/Sub Brokers
wrangler mtls-certificate 🪪 Manage certificates used for mTLS connections
wrangler login 🔓 Login to Cloudflare
Expand Down Expand Up @@ -104,6 +105,7 @@ describe("wrangler", () => {
wrangler d1 🗄 Interact with a D1 database
wrangler ai 🤖 Interact with AI models
wrangler constellation 🤖 Interact with Constellation models
wrangler vectorize 🧮 Interact with Vectorize indexes
wrangler pubsub 📮 Interact and manage Pub/Sub Brokers
wrangler mtls-certificate 🪪 Manage certificates used for mTLS connections
wrangler login 🔓 Login to Cloudflare
Expand Down
36 changes: 7 additions & 29 deletions packages/wrangler/src/__tests__/mtls-certificates.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -356,39 +356,17 @@ describe("wrangler", () => {
describe("commands", () => {
describe("help", () => {
elithrar marked this conversation as resolved.
Show resolved Hide resolved
it("should show the correct help text", async () => {
await runWrangler("mtls-certifiate --help");
await runWrangler("mtls-certificate --help");
expect(std.err).toMatchInlineSnapshot(`""`);
expect(std.out).toMatchInlineSnapshot(`
"wrangler
"wrangler mtls-certificate

🪪 Manage certificates used for mTLS connections

Commands:
wrangler docs [command..] 📚 Open wrangler's docs in your browser
wrangler init [name] 📥 Initialize a basic Worker project, including a wrangler.toml file
wrangler generate [name] [template] ✨ Generate a new Worker project from an existing Worker template. See https://github.com/cloudflare/templates
wrangler dev [script] 👂 Start a local server for developing your worker
wrangler deploy [script] 🆙 Deploy your Worker to Cloudflare. [aliases: publish]
wrangler delete [script] 🗑 Delete your Worker from Cloudflare.
wrangler tail [worker] 🦚 Starts a log tailing session for a published Worker.
wrangler secret 🤫 Generate a secret that can be referenced in a Worker
wrangler secret:bulk [json] 🗄️ Bulk upload secrets for a Worker
wrangler kv:namespace 🗂️ Interact with your Workers KV Namespaces
wrangler kv:key 🔑 Individually manage Workers KV key-value pairs
wrangler kv:bulk 💪 Interact with multiple Workers KV key-value pairs at once
wrangler pages ⚡️ Configure Cloudflare Pages
wrangler queues 🇶 Configure Workers Queues
wrangler r2 📦 Interact with an R2 store
wrangler dispatch-namespace 📦 Interact with a dispatch namespace
wrangler d1 🗄 Interact with a D1 database
wrangler ai 🤖 Interact with AI models
wrangler constellation 🤖 Interact with Constellation models
wrangler pubsub 📮 Interact and manage Pub/Sub Brokers
wrangler mtls-certificate 🪪 Manage certificates used for mTLS connections
wrangler login 🔓 Login to Cloudflare
wrangler logout 🚪 Logout from Cloudflare
wrangler whoami 🕵️ Retrieve your user info and test your auth config
wrangler types 📝 Generate types from bindings & module rules in config
wrangler deployments 🚢 List and view details for deployments
wrangler rollback [deployment-id] 🔙 Rollback a deployment
wrangler mtls-certificate upload Upload an mTLS certificate
wrangler mtls-certificate list List uploaded mTLS certificates
wrangler mtls-certificate delete Delete an mTLS certificate

Flags:
-j, --experimental-json-config Experimental: Support wrangler.json [boolean]
Expand Down
116 changes: 116 additions & 0 deletions packages/wrangler/src/__tests__/vectorize/vectorize.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
import { endEventLoop } from "../helpers/end-event-loop";
import { mockConsoleMethods } from "../helpers/mock-console";
import { runInTempDir } from "../helpers/run-in-tmp";
import { runWrangler } from "../helpers/run-wrangler";

describe("vectorize", () => {
elithrar marked this conversation as resolved.
Show resolved Hide resolved
const std = mockConsoleMethods();
runInTempDir();

it("should show help when no argument is passed", async () => {
await runWrangler("vectorize");
await endEventLoop();

expect(std.out).toMatchInlineSnapshot(`
"wrangler vectorize

🧮 Interact with Vectorize indexes

Commands:
wrangler vectorize create <name> Create a Vectorize index
wrangler vectorize delete <name> Delete a Vectorize index
wrangler vectorize get <name> Get a Vectorize index by name
wrangler vectorize list List your Vectorize indexes

Flags:
-j, --experimental-json-config Experimental: Support wrangler.json [boolean]
-c, --config Path to .toml configuration file [string]
-e, --env Environment to use for operations and .env files [string]
-h, --help Show help [boolean]
-v, --version Show version number [boolean]

--------------------
📣 Vectorize is currently in open beta
📣 See the Vectorize docs for how to get started and known issues: https://developers.cloudflare.com/vectorize
📣 Please report any bugs to https://github.com/cloudflare/workers-sdk/issues/new/choose
📣 To give feedback, visit https://discord.cloudflare.com/
--------------------"
`);
});

it("should show help when an invalid argument is passed", async () => {
await expect(() => runWrangler("vectorize foobarfofum")).rejects.toThrow(
"Unknown argument: foobarfofum"
);

expect(std.err).toMatchInlineSnapshot(`
"X [ERROR] Unknown argument: foobarfofum

"
`);
expect(std.out).toMatchInlineSnapshot(`
"
wrangler vectorize

🧮 Interact with Vectorize indexes

Commands:
wrangler vectorize create <name> Create a Vectorize index
wrangler vectorize delete <name> Delete a Vectorize index
wrangler vectorize get <name> Get a Vectorize index by name
wrangler vectorize list List your Vectorize indexes

Flags:
-j, --experimental-json-config Experimental: Support wrangler.json [boolean]
-c, --config Path to .toml configuration file [string]
-e, --env Environment to use for operations and .env files [string]
-h, --help Show help [boolean]
-v, --version Show version number [boolean]

--------------------
📣 Vectorize is currently in open beta
📣 See the Vectorize docs for how to get started and known issues: https://developers.cloudflare.com/vectorize
📣 Please report any bugs to https://github.com/cloudflare/workers-sdk/issues/new/choose
📣 To give feedback, visit https://discord.cloudflare.com/
--------------------"
`);
});

it("should show help when the get command is passed without an index", async () => {
await expect(() => runWrangler("vectorize get")).rejects.toThrow(
"Not enough non-option arguments: got 0, need at least 1"
);

expect(std.err).toMatchInlineSnapshot(`
"X [ERROR] Not enough non-option arguments: got 0, need at least 1

"
`);
expect(std.out).toMatchInlineSnapshot(`
"
wrangler vectorize get <name>

Get a Vectorize index by name

Positionals:
name The name of the Vectorize index. [string] [required]

Flags:
-j, --experimental-json-config Experimental: Support wrangler.json [boolean]
-c, --config Path to .toml configuration file [string]
-e, --env Environment to use for operations and .env files [string]
-h, --help Show help [boolean]
-v, --version Show version number [boolean]

Options:
--json return output as clean JSON [boolean] [default: false]

--------------------
📣 Vectorize is currently in open beta
📣 See the Vectorize docs for how to get started and known issues: https://developers.cloudflare.com/vectorize
📣 Please report any bugs to https://github.com/cloudflare/workers-sdk/issues/new/choose
📣 To give feedback, visit https://discord.cloudflare.com/
--------------------"
`);
});
});
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ function createWorkerBundleFormData(workerBundle: BundleResult): FormData {
queues: undefined,
r2_buckets: undefined,
d1_databases: undefined,
vectorize: undefined,
constellation: undefined,
services: undefined,
analytics_engine_datasets: undefined,
Expand Down
5 changes: 5 additions & 0 deletions packages/wrangler/src/cfetch/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,11 @@
logger.debug("RESPONSE:", jsonText);
logger.debug("-- END CF API RESPONSE");

if (!jsonText && (response.status === 204 || response.status === 205)) {
const emptyBody = `{"result": {}, "success": true, "errors": [], "messages": []}`;
return parseJSON<ResponseType>(emptyBody);

Check warning on line 91 in packages/wrangler/src/cfetch/internal.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/cfetch/internal.ts#L90-L91

Added lines #L90 - L91 were not covered by tests
}

elithrar marked this conversation as resolved.
Show resolved Hide resolved
try {
return parseJSON<ResponseType>(jsonText);
} catch (err) {
Expand Down
16 changes: 16 additions & 0 deletions packages/wrangler/src/config/environment.ts
Original file line number Diff line number Diff line change
Expand Up @@ -458,6 +458,22 @@ interface EnvironmentNonInheritable {
database_internal_env?: string;
}[];

/**
* Specifies Vectorize indexes that are bound to this Worker environment.
*
* NOTE: This field is not automatically inherited from the top level environment,
* and so must be specified in every named environment.
*
* @default `[]`
* @nonInheritable
*/
vectorize: {
/** The binding name used to refer to the Vectorize index in the worker. */
binding: string;
/** The name of the index. */
index_name: string;
}[];

/**
* Specifies Constellation projects that are bound to this Worker environment.
*
Expand Down
13 changes: 13 additions & 0 deletions packages/wrangler/src/config/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@
send_email,
queues,
d1_databases,
vectorize,
constellation,
r2_buckets,
logfwdr,
Expand Down Expand Up @@ -209,6 +210,18 @@
});
}

if (vectorize !== undefined && vectorize.length > 0) {
output.push({

Check warning on line 214 in packages/wrangler/src/config/index.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/index.ts#L214

Added line #L214 was not covered by tests
type: "Vectorize Indexes",
entries: vectorize.map(({ binding, index_name }) => {
return {

Check warning on line 217 in packages/wrangler/src/config/index.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/index.ts#L216-L217

Added lines #L216 - L217 were not covered by tests
key: binding,
value: index_name,
};
}),
});
}

if (constellation !== undefined && constellation.length > 0) {
output.push({
type: "Constellation Projects",
Expand Down
38 changes: 38 additions & 0 deletions packages/wrangler/src/config/validation.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1234,6 +1234,16 @@
validateBindingArray(envName, validateD1Binding),
[]
),
vectorize: notInheritable(
diagnostics,
topLevelEnv,
rawConfig,
rawEnv,
envName,
"vectorize",
validateBindingArray(envName, validateVectorizeBinding),
[]
),
constellation: notInheritable(
diagnostics,
topLevelEnv,
Expand Down Expand Up @@ -2253,6 +2263,34 @@
return isValid;
};

const validateVectorizeBinding: ValidatorFn = (diagnostics, field, value) => {
if (typeof value !== "object" || value === null) {
diagnostics.errors.push(

Check warning on line 2268 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2268

Added line #L2268 was not covered by tests
`"vectorize" bindings should be objects, but got ${JSON.stringify(value)}`
);
return false;

Check warning on line 2271 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2271

Added line #L2271 was not covered by tests
}
let isValid = true;

Check warning on line 2273 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2273

Added line #L2273 was not covered by tests
// Vectorize bindings must have a binding and a project.
if (!isRequiredProperty(value, "binding", "string")) {
diagnostics.errors.push(

Check warning on line 2276 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2276

Added line #L2276 was not covered by tests
`"${field}" bindings should have a string "binding" field but got ${JSON.stringify(
value
)}.`
);
isValid = false;

Check warning on line 2281 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2281

Added line #L2281 was not covered by tests
}
if (!isRequiredProperty(value, "index_name", "string")) {
diagnostics.errors.push(

Check warning on line 2284 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2284

Added line #L2284 was not covered by tests
`"${field}" bindings must have an "index_name" field but got ${JSON.stringify(
value
)}.`
);
isValid = false;

Check warning on line 2289 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2289

Added line #L2289 was not covered by tests
}
return isValid;

Check warning on line 2291 in packages/wrangler/src/config/validation.ts

View check run for this annotation

Codecov / codecov/patch

packages/wrangler/src/config/validation.ts#L2291

Added line #L2291 was not covered by tests
};

elithrar marked this conversation as resolved.
Show resolved Hide resolved
const validateConstellationBinding: ValidatorFn = (
diagnostics,
field,
Expand Down
1 change: 1 addition & 0 deletions packages/wrangler/src/deploy/deploy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -544,6 +544,7 @@ See https://developers.cloudflare.com/workers/platform/compatibility-dates for m
}),
r2_buckets: config.r2_buckets,
d1_databases: config.d1_databases,
vectorize: config.vectorize,
constellation: config.constellation,
services: config.services,
analytics_engine_datasets: config.analytics_engine_datasets,
Expand Down
Loading