118 lines
15 KiB
Markdown
118 lines
15 KiB
Markdown
# Tasks — Search completion
|
|
|
|
> Every checkbox MUST reflect work actually done in this change. The eight capabilities closed here were silently dropped during the original genesis migration (audit dated 2026-05-02). Do not retroactively tick boxes for surface that already shipped.
|
|
|
|
## 0. Pre-flight (non-negotiable)
|
|
|
|
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
|
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
|
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
|
|
|
## 1. Spec authoring
|
|
|
|
- [ ] 1.1 Read `../suaveplan/packages/universal/search/src/{search-client,query-builder,pipeline,hooks,adapters,types}/*.ts` end-to-end
|
|
- [ ] 1.2 Read `openspec/specs/search/spec.md` (if it exists; create the cap dir if not)
|
|
- [ ] 1.3 Read `openspec/AGENTS.md` Sections 0a (parity), 13 (spec-conformance), 14 (source mapping), 14a (dependents)
|
|
- [ ] 1.4 Draft `proposal.md` covering all eight gaps (`SearchClient`, `QueryBuilder`, `SearchPipeline` + 4 analyzers, `searchHooks`, facet/boost/scoring types, `ElasticsearchAdapter`, `OpenSearchAdapter` with AWS sigv4, `MeiliSearchAdapter`)
|
|
- [ ] 1.5 Fill `## Source mapping` in `proposal.md` with upstream line ranges for every requirement; every "no source line" requirement justified
|
|
- [ ] 1.6 Draft spec delta at `specs/search/spec.md`
|
|
- [ ] 1.7 Every requirement in the spec delta carries a `**Verified by:**` line naming concrete `it(...)` titles
|
|
- [ ] 1.8 `openspec change validate search-completion` exits with no errors
|
|
|
|
## 2. Implementation
|
|
|
|
- [ ] 2.1 `SearchClient<TDoc>` orchestrator — port from upstream `search-client/search-client.ts` into `packages/platform/search/src/client/search-client.ts`; thread `AbortSignal` through every async method
|
|
- [ ] 2.2 `QueryBuilder` fluent DSL — port from upstream `query-builder/query-builder.ts` into `packages/platform/search/src/query-builder/query-builder.ts`; add `must` / `should` / `mustNot` / `range` / `prefix` / `fuzzy` / `geo` / `boost` / `scoringFunction` clauses on top of upstream surface
|
|
- [ ] 2.3 `SearchPipeline` composable analyzer chain — port from upstream `pipeline/search-pipeline.ts` into `packages/platform/search/src/pipeline/search-pipeline.ts`; thread `AbortSignal` through `process(query, signal?)`
|
|
- [ ] 2.4 `Analyzer` interface + co-located built-ins:
|
|
- `pipeline/analyzers/lowercase/lowercase.ts` (verbatim from upstream)
|
|
- `pipeline/analyzers/stemming/stemming.ts` (verbatim 18-rule Porter-like table)
|
|
- `pipeline/analyzers/stopword/stopword.ts` (verbatim 51-word default + configurable set)
|
|
- `pipeline/analyzers/synonym/synonym.ts` (verbatim one-way / bidirectional)
|
|
- [ ] 2.5 `searchHooks` typed event bus — port from upstream `hooks.ts` into `packages/platform/search/src/hooks/search-hooks.ts`; rewire to `@suaveplan/events` `Hooks` class; add `'search:indexing'` and `'search:result'` symmetric hooks alongside upstream `'search:querying'`
|
|
- [ ] 2.6 Co-locate `FacetDistribution`, `FacetValue`, `FacetRequest`, `FieldBoost`, `ScoringFunction<T>`, `SearchSuggestion`, `SuggestOptions` shape types next to their consumers (no shared `types/` dump)
|
|
- [ ] 2.7 `BaseElasticAdapter<TDoc>` shared base class — port from upstream `adapters/base-elastic-adapter.ts` into `packages/platform/search/src/adapters/base-elastic/base-elastic.ts`; map upstream `HttpError` / `RateLimitError` / `TimeoutError` / `ConfigError` to genesis `SearchError` codes (`PROVIDER_*`)
|
|
- [ ] 2.8 `ElasticsearchAdapter<TDoc>` — port from upstream `adapters/elasticsearch-adapter.ts` into `packages/platform/search/src/adapters/elasticsearch/elasticsearch.ts`; ES 8 default + optional ES 7 compatibility mode
|
|
- [ ] 2.9 `OpenSearchAdapter<TDoc>` — port from upstream `adapters/opensearch-adapter.ts` into `packages/platform/search/src/adapters/opensearch/opensearch.ts`; pluggable `AwsSigningConfig` (no AWS-SDK runtime dep)
|
|
- [ ] 2.10 `MeiliSearchAdapter<TDoc>` — port from upstream `adapters/meilisearch-adapter.ts` into `packages/platform/search/src/adapters/meilisearch/meilisearch.ts`
|
|
- [ ] 2.11 Extend `SearchEngine<TDoc>` interface — add optional `bulkIndex(documents, options?)` and `ensureSchema()` methods; default-throw with `SEARCH.NOT_IMPLEMENTED` so `MemorySearchEngine` compiles unchanged
|
|
- [ ] 2.12 Add `SEARCH_*` codes (`PROVIDER_UNREACHABLE`, `PROVIDER_TIMEOUT`, `PROVIDER_RATE_LIMITED`, `PROVIDER_CONFIG_INVALID`, `HOOK_HANDLER_FAILED`, `ANALYZER_INVALID`, `NOT_IMPLEMENTED`, `AWS_SIGNING_FAILED`, `BULK_INDEX_PARTIAL_FAILURE`) to `errors/search-error.ts`
|
|
- [ ] 2.13 Wire OTel counters / spans on every public entrypoint: `search.client.queries`, `search.client.queries_duration_ms`, `search.client.indexed`, `search.pipeline.process_duration_ms`, `search.hooks.fired`, `search.adapter.requests`, `search.adapter.errors`, `search.adapter.bulk_indexed`
|
|
- [ ] 2.14 Wire Zod schemas for `SearchClientConfig`, `ElasticBaseConfig`, `ElasticAuth`, `ElasticsearchConfig`, `OpenSearchConfig`, `AwsSigningConfig`, `MeiliSearchConfig`, `FacetRequest`, `FieldBoost`, every `Analyzer` config to `schemas/search.schemas.ts`
|
|
- [ ] 2.15 Update `index.ts` barrel — re-export every new symbol; no `export *` (one explicit subpath per module)
|
|
- [ ] 2.16 Update `package.json` `exports` map — add subpaths `./client/search-client`, `./query-builder/query-builder`, `./pipeline/search-pipeline`, `./pipeline/analyzers/lowercase/lowercase`, `./pipeline/analyzers/stemming/stemming`, `./pipeline/analyzers/stopword/stopword`, `./pipeline/analyzers/synonym/synonym`, `./hooks/search-hooks`, `./adapters/base-elastic/base-elastic`, `./adapters/elasticsearch/elasticsearch`, `./adapters/opensearch/opensearch`, `./adapters/meilisearch/meilisearch`
|
|
- [ ] 2.17 Update `package.json` `dependencies` — add `@suaveplan/events` (`workspace:^`)
|
|
|
|
## 3. Tests
|
|
|
|
- [ ] 3.1 `packages/platform/search/src/client/search-client.test.ts` — `index` / `search` / `find` / `suggest` round-trip against an in-memory engine; `defaultLimit` applied; `fieldBoosts` and `scoringFunction` re-rank results; `removeMany` and bulk-index path
|
|
- [ ] 3.2 `packages/platform/search/src/client/search-client.spec-conformance.test.ts` — `searchHooks.hook('search:querying')` rewrites the query before the engine sees it; `'search:indexing'` rewrites documents; `'search:result'` post-processes; `AbortSignal` aborts mid-flight; injected `MemorySearchEngine` produces deterministic order
|
|
- [ ] 3.3 `packages/platform/search/src/query-builder/query-builder.test.ts` — `term` / `must` / `should` / `mustNot` / `range` / `prefix` / `fuzzy` / `geo` / `filter` / `facet` / `sort` / `paginate` build correctly; `boost` and `scoringFunction` round-trip; `reset()` clears state
|
|
- [ ] 3.4 `packages/platform/search/src/pipeline/search-pipeline.test.ts` — `LowercaseAnalyzer` / `StemmingAnalyzer` / `StopWordAnalyzer` / `SynonymAnalyzer` each tested in isolation; `SearchPipeline` walks analyzers in order; custom analyzer registers; analyzer name returned via `analyzerNames`
|
|
- [ ] 3.5 `packages/platform/search/src/pipeline/search-pipeline.spec-conformance.test.ts` — `AbortSignal` mid-pipeline rejects with `ANALYZER_INVALID` when an analyzer throws; bidirectional synonyms expand correctly; stop-word fallback returns original tokens when filtering would empty the list; stem rule guards against under-3 char results
|
|
- [ ] 3.6 `packages/platform/search/src/hooks/search-hooks.test.ts` — `'search:querying'` / `'search:indexing'` / `'search:result'` lifecycle bus tested with multiple handlers; handler-priority order respected; `unsubscribe` removes record
|
|
- [ ] 3.7 `packages/platform/search/src/hooks/search-hooks.spec-conformance.test.ts` — handler exception raises `HOOK_HANDLER_FAILED` with original `cause`; per-hook payload type pinned via `expectTypeOf`
|
|
- [ ] 3.8 `packages/platform/search/src/adapters/base-elastic/base-elastic.test.ts` — `index` / `remove` / `search` / `clear` / `size` / `suggest` round-trip against a `fetch` mock; `bulkIndex` builds NDJSON; `ensureIndex` / `deleteIndex` / `refresh`; auth basic / apiKey / bearer; `searchFields` honoured; facets translate to aggregations
|
|
- [ ] 3.9 `packages/platform/search/src/adapters/base-elastic/base-elastic.spec-conformance.test.ts` — 429 rate-limit response raises `PROVIDER_RATE_LIMITED` with `retryAfterMs`; 408 / 504 raises `PROVIDER_TIMEOUT`; missing host raises `PROVIDER_CONFIG_INVALID`; non-OK responses throw `SearchError` wrapping `HttpError`
|
|
- [ ] 3.10 `packages/platform/search/src/adapters/elasticsearch/elasticsearch.test.ts` — adapter sends ES 8 headers by default; `compatibilityMode: true` switches to ES 7 headers
|
|
- [ ] 3.11 `packages/platform/search/src/adapters/opensearch/opensearch.test.ts` — adapter without `awsSigning` behaves like `BaseElasticAdapter`; with `awsSigning` invokes the supplied signer once per request and uses the returned `RequestInit`; signer rejection raises `AWS_SIGNING_FAILED`
|
|
- [ ] 3.12 `packages/platform/search/src/adapters/meilisearch/meilisearch.test.ts` — `index` / `remove` / `search` / `clear` / `size` / `suggest` round-trip; filters translate to MeiliSearch filter strings; `ensureIndex` patches `searchableAttributes` / `filterableAttributes` / `sortableAttributes`; faceted search returns `facetDistribution`
|
|
- [ ] 3.13 `packages/platform/search/src/adapters/meilisearch/meilisearch.spec-conformance.test.ts` — 429 raises `PROVIDER_RATE_LIMITED`; 408 / 504 raises `PROVIDER_TIMEOUT`; 400 raises `QUERY_INVALID`; missing API key path
|
|
- [ ] 3.14 Cross-package `tests/dep-graph.test.ts` — `@suaveplan/search` declares `@suaveplan/events` as a runtime dep; the package still has zero browser-only deps; subpath count matches feature module count
|
|
- [ ] 3.15 Every `it(...)` carries `{ timeout: 5000 }` (or 30000 for genuinely-async adapter integration sweeps)
|
|
|
|
## 4. Documentation
|
|
|
|
- [ ] 4.1 Sibling `.md` for every new implementation `.ts` (≥200 words, basic + advanced examples, full API reference) — `client/search-client.md`, `query-builder/query-builder.md`, `pipeline/search-pipeline.md`, four analyzer `.md` files, `hooks/search-hooks.md`, `adapters/base-elastic/base-elastic.md`, `adapters/elasticsearch/elasticsearch.md`, `adapters/opensearch/opensearch.md`, `adapters/meilisearch/meilisearch.md`
|
|
- [ ] 4.2 README links every co-located `.md` under `## Modules`
|
|
- [ ] 4.3 Update `CLAUDE.md` `search` package listing — remove "MemorySearchEngine + BM25 + trigram + tokenizer only" wording; describe the full restored surface (orchestrator, builder, pipeline, hooks, three production adapters)
|
|
- [ ] 4.4 Refresh `STATUS.md` "Completed on genesis" entry for `search` to reflect parity
|
|
|
|
## 5. Pre-archive verification (the 5-gate triple)
|
|
|
|
- [ ] 5.1 `bun run check` — Biome 0 errors / 0 warnings / 0 info across the monorepo
|
|
- [ ] 5.2 `bunx turbo lint typecheck test --filter=@suaveplan/search` — green
|
|
- [ ] 5.3 `bunx turbo test:coverage --filter=@suaveplan/search` — 100/100/100/100
|
|
- [ ] 5.4 `bun scripts/openspec-conformance.ts @suaveplan/search` — exit 0, no `error` findings
|
|
- [ ] 5.5 `bunx turbo build --filter=@suaveplan/search` — green
|
|
- [ ] 5.6 Banned-pattern scan zero output: `grep -rn "@deprecated\|TODO\|FIXME\|ts-ignore\|ts-expect-error\|\.skip\|\.only" packages/platform/search/src/ --include="*.ts"`
|
|
- [ ] 5.7 Missing `.md` scan zero output (per AGENTS.md §13 step 5)
|
|
- [ ] 5.8 README check `test -f packages/platform/search/README.md`
|
|
- [ ] 5.9 Bare-version check zero output (per AGENTS.md §13 step 7)
|
|
- [ ] 5.10 Subpath count matches feature module count (per AGENTS.md §13 step 8)
|
|
|
|
## 6. Pre-archive 7-row checklist (AGENTS.md Section 13)
|
|
|
|
- [ ] 6.1 Generics threaded; `expectTypeOf` assertions on every literal-type claim (especially `SearchClient<TDoc>`, `BaseElasticAdapter<TDoc>`, `ElasticsearchAdapter<TDoc>`, `OpenSearchAdapter<TDoc>`, `MeiliSearchAdapter<TDoc>`)
|
|
- [ ] 6.2 Zod schemas wired at every declared boundary (`SearchClientConfig`, `ElasticBaseConfig`, `ElasticAuth`, `ElasticsearchConfig`, `OpenSearchConfig`, `AwsSigningConfig`, `MeiliSearchConfig`, `FacetRequest`, `FieldBoost`, every `Analyzer` config)
|
|
- [ ] 6.3 Every error code added in §2.12 thrown by at least one production code path; no orphans
|
|
- [ ] 6.4 Strict-mode behaviors tested both on and off (e.g. `compatibilityMode: true` vs `false`; `awsSigning` present vs absent; bulk-index partial failure vs success)
|
|
- [ ] 6.5 OTel counters / histograms emitted from production paths and asserted with a real in-memory `MeterProvider`
|
|
- [ ] 6.6 Lifecycle dedup tested where claimed (`SearchPipeline` analyzer registry idempotent on identical analyzer; `searchHooks.hook` returns a one-shot unsubscriber)
|
|
- [ ] 6.7 No retroactive tick — every box above reflects work done in this change
|
|
|
|
## 7. Archive
|
|
|
|
- [ ] 7.1 Fold spec delta into `openspec/specs/search/spec.md` (creating the cap file if missing)
|
|
- [ ] 7.2 Move `openspec/changes/search-completion/` → `openspec/archive/<yyyy-mm-dd>-search-completion/`
|
|
- [ ] 7.3 Refresh `MANIFEST.md` (dep set changed: `@suaveplan/search` now depends on `@suaveplan/events`)
|
|
- [ ] 7.4 Update `STATUS.md` to mark the eight dropped capabilities as restored
|
|
|
|
## Audit Note (2026-05-02)
|
|
|
|
Audit verdict: **DEFERRED — 0/65 tasks evidenced in code**.
|
|
|
|
Evidence reviewed:
|
|
|
|
- `packages/platform/search/src/` contains only `engine/memory-engine/`, `text/{tokenizer,scorer,trigram}/`, `errors/`, `schemas/`, `types/`, `constants/`. No `client/`, no `query-builder/`, no `pipeline/`, no `hooks/`, no `adapters/`.
|
|
- `package.json` does not depend on `@suaveplan/events`.
|
|
- The eight upstream capabilities (SearchClient, QueryBuilder, SearchPipeline + 4 analyzers, searchHooks, facet/boost/scoring types, ElasticsearchAdapter, OpenSearchAdapter with sigv4, MeiliSearchAdapter, SearchEngine.bulkIndex/ensureSchema) are absent.
|
|
|
|
Missing pieces:
|
|
|
|
1. `client/search-client.ts` orchestrator + `query-builder/query-builder.ts` fluent DSL with the upstream clauses.
|
|
2. `pipeline/search-pipeline.ts` analyzer chain with `lowercase`, `stemming`, `stopword`, `synonym` analyzers ported verbatim from upstream.
|
|
3. Three production adapters (`adapters/elasticsearch`, `adapters/opensearch` with pluggable `AwsSigningConfig`, `adapters/meilisearch`) on top of a shared `BaseElasticAdapter`, plus the 9 new `SEARCH_*` error codes.
|
|
|
|
Decision: leave change open. This is the largest of the deferred set (8 dropped capabilities, full dep graph rewire to `@suaveplan/events`, nine new error codes, parity audit).
|