Configuration

Where configuration lives

All configuration is stored as Worker Secrets on your searchkeep Worker. None of it is in the repository, which is what lets your fork stay identical to upstream and upgrade with “Sync fork”.

• Secrets are encrypted, never shown again after you set them, and never printed in logs by Searchkeep. setup_status reports only whether each one is present.
• The repository’s wrangler.jsonc sets keep_vars: true, so if you prefer to keep a non-sensitive value such as BRAND_REGEX as a plain variable in the dashboard, deployments will not overwrite it.
• Setting or changing a secret deploys a new version of the Worker within seconds. No build is needed.
• Problems in the values (invalid JSON in SITES, a bad regex) never crash the Worker; they are listed by setup_status and the dashboard’s status page.

Secrets reference

SITES

SITES is a JSON array with one object per site. It maps the site to its identifier in each data source. Only domain is required; every source-specific field is optional, and a source is simply skipped for a site that does not set it.

[
  {
    "id": "acme",
    "domain": "acme.com",
    "gscProperty": "sc-domain:acme.com",
    "bingSiteUrl": "https://acme.com/",
    "cloudflareZoneId": "023e105f4ecef8ad9ca31a8372d0c353",
    "webAnalyticsSiteTag": "abc123",
    "brandRegex": "acme"
  }
]

Several sites, with different sources each, look like this:

[
  { "id": "acme", "domain": "acme.com", "gscProperty": "sc-domain:acme.com",
    "bingSiteUrl": "https://acme.com/", "cloudflareZoneId": "023e105f4ecef8ad9ca31a8372d0c353",
    "webAnalyticsSiteTag": "abc123", "brandRegex": "acme" },
  { "id": "acme-docs", "domain": "docs.acme.com", "gscProperty": "https://docs.acme.com/",
    "webAnalyticsSiteTag": "def456" }
]

Where to find each identifier

• gscProperty: the property selector in Search Console. Domain properties are written sc-domain:example.com; URL-prefix properties are the full URL with a trailing slash.
• bingSiteUrl: the site URL exactly as listed in Bing Webmaster Tools.
• cloudflareZoneId: the zone’s Overview page in the Cloudflare dashboard, under API.
• webAnalyticsSiteTag: Analytics & Logs → Web Analytics, then select the site; the site tag appears in the address bar as siteTag~in=<tag>. It is not the token value in the JavaScript snippet. Web Analytics also needs CLOUDFLARE_ACCOUNT_ID.

Brand and non-brand

Every query-level tool and dashboard view accepts brand: "all" | "brand" | "non_brand". A query counts as brand when it matches the site’s brandRegex, or BRAND_REGEX if the site has none. Both are JavaScript regular expression sources, matched case-insensitively anywhere in the query text (anchor with ^ and $ if you need to). Changing the regex re-classifies stored queries on the next sync or query.

acme|acme\s?corp|acmeapp

• Do not wrap the pattern in slashes and do not add flags.
• Include common misspellings and the product names people search for.
• If neither is set, every query counts as non-brand, and results say so in their notes.

Enabling writes

Searchkeep is read-only by default. The only tool that changes anything outside your account, submit_sitemap, is not even registered with MCP clients until you opt in:

1. Set ENABLE_WRITES to true.
2. In Search Console, change the service account’s permission on the property from Restricted to Full. Submitting sitemaps requires it.

The tool is annotated as a write, so well-behaved MCP clients ask before running it even when you have auto-approved the read-only tools. Remove the secret to turn writes off again.

Sync budgets

GSC_ROWS_PER_DAY, GSC_BACKFILL_MONTHS, SYNC_DAILY_WRITE_BUDGET and SYNC_MAX_CALLS_PER_RUN control how much each sync pass does. The defaults keep a typical site within the free plans: D1 Free allows 100,000 rows written per day, and Workers Free allows 10 ms of CPU per invocation. If the sync keeps skipping units after interrupted attempts, lower GSC_ROWS_PER_DAY or move to Workers Paid. Values outside a setting’s range are clamped to it. See How syncing works.

Setting and rotating secrets

Use npx wrangler from any directory (it does not need a clone of the repository), or the dashboard: Workers & Pages → searchkeep → Settings → Variables and Secrets → Add, with type Secret.

# Overwrite a secret with a new value; a new version deploys within seconds.
npx wrangler secret put BING_API_KEY --name searchkeep

# List which secrets exist (names only, never values).
npx wrangler secret list --name searchkeep

# Remove one.
npx wrangler secret delete ENABLE_WRITES --name searchkeep

To rotate a credential, create the new one at the source, overwrite the secret, confirm setup_status is still green, then revoke the old credential at the source.

Local development

For working on Searchkeep itself, pnpm dev at the repository root builds the dashboard and starts the Worker locally with wrangler dev in dev mode. Local values go in apps/cloudflare/.dev.vars, which is git-ignored; start from the example file. The local D1 is migrated automatically on the first request.

cp apps/cloudflare/.dev.vars.example apps/cloudflare/.dev.vars
pnpm dev                                         # builds the dashboard, then wrangler dev on :8787

# In another terminal: fill the local D1 with demo data for the site in SITES.
curl -X POST localhost:8787/api/dev/seed

# apps/cloudflare/.dev.vars — git-ignored, never commit it
SITES=[{"id":"demo","domain":"demo.example.com","gscProperty":"sc-domain:demo.example.com","cloudflareZoneId":"demo-zone","webAnalyticsSiteTag":"demo-tag"}]
BRAND_REGEX=acme

Dev mode skips Cloudflare Access and treats the deployment as ready, but only for requests to localhost, 127.0.0.1 or [::1]; the seed endpoint works only in dev mode. Without real credentials the sync has nothing to pull, which is what the demo data is for. The repository’s CONTRIBUTING.md and AGENTS.md describe the full workflow, including tests.