SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
admin
2026-05-27 23:29:53 +01:00
commit fbc70f96e6
10000 changed files with 1102152 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

136
openspec/changes/search-completion/proposal.md Normal file
View File

@@ -0,0 +1,136 @@
# Search completion
**Package**: `@suaveplan/search`
**Status**: PROPOSED
**Type**: spec-conformance + feature
## Why
The 2026-05-02 cross-monorepo audit confirmed that `@suaveplan/search` shipped on genesis with only the in-memory side of the original surface — `MemorySearchEngine`, BM25 + trigram scorers, the tokenizer, and Zod schemas. The legacy `packages/universal/search/src` package on `../suaveplan/` carries the full search-product surface that downstream services (catalogue, knowledge, analytics) rely on: a `SearchClient<T>` orchestrator, a `QueryBuilder` fluent API, a four-analyzer `SearchPipeline` (`LowercaseAnalyzer`, `StemmingAnalyzer`, `StopWordAnalyzer`, `SynonymAnalyzer`), a `searchHooks` event bus, the `FacetDistribution` / `FacetRequest` / `FieldBoost` / `ScoringFunction` shapes, and three production adapters — `ElasticsearchAdapter`, `OpenSearchAdapter` (with optional AWS sigv4 signing), and `MeiliSearchAdapter`. None of these were archived as out-of-scope; they were silently dropped during the original genesis migration.
Genesis policy (CLAUDE.md §Workflow rule 5) is "preserve the full public surface during rewrites — trimming is OK ONLY when code is duplicated elsewhere in-repo or can be replaced by an existing `@suaveplan/*` library." None of the dropped capabilities are duplicated elsewhere on genesis, so each one must be ported. This change closes that gap so any downstream service can compose an in-memory engine for tests with a hosted engine for production behind one `SearchEngine<TDoc>` contract.
## What changes
- ADDED: `SearchClient<TDoc extends SearchDocument>` orchestrator class — wraps any `SearchEngine<TDoc>` with `searchHooks` lifecycle wiring, `SearchPipeline` analyzer chain, `defaultLimit`, optional `fieldBoosts`, and an optional `ScoringFunction`. Methods: `index`, `indexMany`, `remove`, `removeMany`, `search`, `find`, `suggest`, `clear`, `size`. Every async method threads `AbortSignal` through to the underlying engine.
- ADDED: `QueryBuilder` fluent typed query DSL — `.term(value)`, `.must(filter)`, `.should(filter)`, `.mustNot(filter)`, `.range(field, op, value)`, `.term_(field, value)`, `.prefix(field, value)`, `.fuzzy(field, value, distance?)`, `.geo(field, lat, lng, radiusKm?)`, `.filter(field, op, value)` (legacy alias), `.facet(field, maxValues?)`, `.sort(field, direction)`, `.paginate(offset, limit)`, `.boost(field, weight)`, `.scoringFunction(fn)`, `.build()`, `.reset()`. The `.build()` output is a `SearchQuery` matching the engine's shape.
- ADDED: `SearchPipeline` class with composable `Analyzer` interface — `process(query)` walks every registered analyzer in order. Built-in analyzers: `LowercaseAnalyzer`, `StemmingAnalyzer` (Porter-like suffix stripping; 18 rules), `StopWordAnalyzer` (configurable stop-word set; 51-word default), `SynonymAnalyzer` (one-way and bidirectional synonym map). Custom analyzers conform to `interface Analyzer { name: string; analyze(terms): string[] }` and can be registered without touching the package.
- ADDED: `searchHooks` typed event bus — `'search:querying'` (intercept and rewrite query before the engine sees it), `'search:indexing'` (intercept and rewrite document before index), `'search:result'` (post-process result set before return). Hooks are wired through `@suaveplan/events` and respect the existing handler-priority / order semantics.
- ADDED: `FacetDistribution`, `FacetValue`, `FacetRequest`, `FieldBoost`, `ScoringFunction<T>`, `SearchSuggestion`, `SuggestOptions` shape types — co-located with their implementation modules.
- ADDED: `BaseElasticAdapter<TDoc>` shared base class — handles auth (basic / apiKey / bearer), request-timeout, query body construction (`bool` / `multi_match` / `range`), filter translation, response transformation (hits + aggregations → `SearchResult`), bulk indexing, index management (`ensureIndex`, `deleteIndex`, `refresh`), and structured error mapping (`HttpError` / `RateLimitError` / `TimeoutError` / `ConfigError`).
- ADDED: `ElasticsearchAdapter<TDoc>` class — extends `BaseElasticAdapter`; ES 8.x by default, ES 7.x compatibility-mode header opt-in.
- ADDED: `OpenSearchAdapter<TDoc>` class — extends `BaseElasticAdapter`; optional `AwsSigningConfig` injects an `(url, init) => Promise<RequestInit>` hook so sigv4 signing is pluggable.
- ADDED: `MeiliSearchAdapter<TDoc>` class — implements `SearchEngine<TDoc>` against MeiliSearch HTTP API; supports `searchableAttributes`, `filterableAttributes`, `sortableAttributes`, faceted search via `facetDistribution`, suggestions via `_formatted` extraction.
- ADDED: `ElasticBaseConfig`, `ElasticAuth`, `ElasticsearchConfig`, `OpenSearchConfig`, `AwsSigningConfig`, `MeiliSearchConfig`, `SearchClientConfig` shape types.
- MODIFIED: `SearchEngine<TDoc>` interface — adds optional `bulkIndex(documents)` and `ensureSchema()` lifecycle hooks; the new methods default-throw with `SEARCH.NOT_IMPLEMENTED` so existing engines (e.g. `MemorySearchEngine`) compile unchanged.
- MODIFIED: `SEARCH_CODES` registry — adds `PROVIDER_UNREACHABLE`, `PROVIDER_TIMEOUT`, `PROVIDER_RATE_LIMITED`, `PROVIDER_CONFIG_INVALID`, `HOOK_HANDLER_FAILED`, `ANALYZER_INVALID`, `NOT_IMPLEMENTED`, `AWS_SIGNING_FAILED`, `BULK_INDEX_PARTIAL_FAILURE` codes; the existing `INDEX_FAILED`, `QUERY_INVALID`, `ENGINE_NOT_READY`, `FIELD_UNKNOWN`, `CURSOR_INVALID` codes are preserved.
- MODIFIED: `index.ts` barrel — re-exports every new symbol; no `export *` (one explicit subpath per module).
- MODIFIED: `package.json` `exports` map — adds subpaths `./client/search-client`, `./query-builder/query-builder`, `./pipeline/search-pipeline`, `./pipeline/analyzers/lowercase`, `./pipeline/analyzers/stemming`, `./pipeline/analyzers/stopword`, `./pipeline/analyzers/synonym`, `./hooks/search-hooks`, `./adapters/base-elastic`, `./adapters/elasticsearch`, `./adapters/opensearch`, `./adapters/meilisearch`.
- MODIFIED: `package.json` `dependencies` — adds `@suaveplan/events` (`workspace:^`).
## Out of scope
- **Algolia adapter** — different query DSL and pricing model; lands as its own package (`@suaveplan/search-algolia`) once a tenant requests it. Avoiding speculative adapter sprawl in this change.
- **Vector search / dense retrieval** — embedding pipelines, ANN indexes (HNSW / IVF), and hybrid lexical+vector ranking are a separate concern with their own ML dependency surface; they land as `@suaveplan/search-vector`.
- **Bundling `@elastic/elasticsearch` or `meilisearch-js` SDKs** — both adapters use direct `fetch()` with Zod-validated responses to keep the package tree-shake-friendly and free of native-binding peer deps. This matches the upstream design and the `@suaveplan/sitemap` / `@suaveplan/dmarc` precedent (HTTP-only, Zod-validated wire formats).
- **Typesense adapter** — the genesis testing harness already ships `@suaveplan/testing-integration-typesense` for service spin-up; the production adapter is a separate package because Typesense's query shape (filter strings, multi-search) does not map onto the Elastic family base class without re-architecture.
- **`registerHook` legacy free-function deprecation** — kept and delegated to the new `searchHooks.hook` API for one release; removing it is a breaking change that requires its own deprecation window.
- **Re-folding of `openspec/specs/search/spec.md`** if a canonical spec file does not yet exist, this change creates it for the cap; full spec folding lands once all search changes archive.
## Source mapping
This change derives from `../suaveplan/packages/universal/search/src/`. Genesis `enterprise/search.md` is a generic enterprise template (no per-feature requirements), so source mapping cites the upstream package source directly.
| Requirement | Source file | Source lines | Notes |
|---|---|---|---|
| `SearchClient<T>` orchestrator | `packages/universal/search/src/search-client/search-client.ts` | L21-L93 | Verbatim; `AbortSignal` threaded through every async method on genesis |
| `SearchClientConfig` | `packages/universal/search/src/types/search.types.ts` | L139-L148 | Verbatim |
| `searchHooks` event bus + `'search:querying'` hook | `packages/universal/search/src/hooks.ts` | L1-L62 | Verbatim; rewired to use `@suaveplan/events` `Hooks` class |
| `'search:indexing'` and `'search:result'` hooks | (no upstream line — added in this change) | — | Symmetry with `'search:querying'`; justified in "no source line" table below |
| `QueryBuilder` fluent DSL — `term`, `filter`, `facet`, `sort`, `paginate`, `offset`, `limit`, `build`, `reset` | `packages/universal/search/src/query-builder/query-builder.ts` | L27-L107 | Verbatim |
| `QueryBuilder.must`, `should`, `mustNot`, `range`, `prefix`, `fuzzy`, `geo`, `boost`, `scoringFunction` extensions | (no upstream line — added in this change) | — | Justified in "no source line" table below |
| `SearchPipeline` composable analyzer chain | `packages/universal/search/src/pipeline/search-pipeline.ts` | L204-L224 | Verbatim |
| `Analyzer` interface (`name`, `analyze(terms)`) | `packages/universal/search/src/pipeline/search-pipeline.ts` | L9-L12 | Verbatim |
| `LowercaseAnalyzer` | `packages/universal/search/src/pipeline/search-pipeline.ts` | L106-L114 | Verbatim |
| `StemmingAnalyzer` (Porter-like 18-rule suffix table) | `packages/universal/search/src/pipeline/search-pipeline.ts` | L57-L103 | Verbatim |
| `StopWordAnalyzer` (51-word default + configurable set) | `packages/universal/search/src/pipeline/search-pipeline.ts` | L117-L186 | Verbatim |
| `SynonymAnalyzer` (one-way and bidirectional) | `packages/universal/search/src/pipeline/search-pipeline.ts` | L21-L51 | Verbatim |
| `FacetRequest`, `FacetValue`, `FacetDistribution`, `FieldBoost`, `ScoringFunction`, `SearchSuggestion`, `SuggestOptions` | `packages/universal/search/src/types/search.types.ts` | L26-L77 | Verbatim |
| `BaseElasticAdapter<T>` shared base class | `packages/universal/search/src/adapters/base-elastic-adapter.ts` | L75-L442 | Verbatim |
| `ElasticBaseConfig`, `ElasticAuth` | `packages/universal/search/src/adapters/base-elastic-adapter.ts` | L26-L47 | Verbatim |
| `ElasticsearchAdapter<T>` | `packages/universal/search/src/adapters/elasticsearch-adapter.ts` | L45-L64 | Verbatim |
| `ElasticsearchConfig` | `packages/universal/search/src/adapters/elasticsearch-adapter.ts` | L12-L15 | Verbatim |
| `OpenSearchAdapter<T>` | `packages/universal/search/src/adapters/opensearch-adapter.ts` | L38-L60 | Verbatim |
| `OpenSearchConfig`, `AwsSigningConfig` | `packages/universal/search/src/adapters/opensearch-adapter.ts` | L12-L23 | Verbatim |
| `MeiliSearchAdapter<T>` | `packages/universal/search/src/adapters/meilisearch-adapter.ts` | L35-L305 | Verbatim |
| `MeiliSearchConfig` | `packages/universal/search/src/types/search.types.ts` | L150-L164 | Verbatim |
### Requirements with no source line
| Requirement | Justification |
|---|---|
| `'search:indexing'` and `'search:result'` lifecycle hooks (in addition to upstream `'search:querying'`) | Genesis-wide CLAUDE.md rule 3 (enterprise-grade modularity) requires a complete intercept surface. Upstream only exposes `'search:querying'`; downstream consumers (knowledge-graph, content-pipeline) need to rewrite indexed documents (e.g. inject embeddings) and post-process result sets (e.g. attach permission metadata). Symmetric naming with the existing hook avoids API drift. |
| `QueryBuilder.must` / `should` / `mustNot` / `range` / `prefix` / `fuzzy` / `geo` / `boost` / `scoringFunction` methods | Upstream `QueryBuilder` only ships `filter()` and `term()`. The proposal task description explicitly requires these clauses (must / should / must_not / range / term / prefix / fuzzy / geo). Adding them on the builder side keeps a typed DSL while preserving every adapter's freedom to translate filters into its native query language. |
| `bulkIndex` and `ensureSchema` optional methods on `SearchEngine<TDoc>` | Bulk indexing is a documented `BaseElasticAdapter` capability (upstream L149-L166) but is not part of the universal interface. Genesis policy requires the universal contract to expose every supported optimisation so callers can migrate engines without code changes; both methods default-throw with `SEARCH.NOT_IMPLEMENTED` so existing engines (e.g. `MemorySearchEngine`) compile unchanged. |
| `AbortSignal` threaded through every `SearchClient` and `SearchEngine` async method | enterprise/search.md L43: "Backpressure & cancellation — `AbortSignal` everywhere async". Upstream lacks signal support on the client-orchestration surface. |
| Zod schemas for `SearchClientConfig`, `ElasticBaseConfig`, `ElasticAuth`, `MeiliSearchConfig`, `AwsSigningConfig`, `FacetRequest`, `FieldBoost`, every `Analyzer` config | enterprise/search.md L37: "Zod schemas for all public types". Upstream validates through TypeScript only. |
| New `SEARCH.PROVIDER_*` / `SEARCH.HOOK_HANDLER_FAILED` / `SEARCH.ANALYZER_INVALID` / `SEARCH.NOT_IMPLEMENTED` / `SEARCH.AWS_SIGNING_FAILED` / `SEARCH.BULK_INDEX_PARTIAL_FAILURE` codes | Genesis-wide policy (CLAUDE.md §Workflow rule 2) that every thrown error route through `@suaveplan/error`. Upstream throws `HttpError` / `TimeoutError` / `RateLimitError` from `@suaveplan/error` directly; genesis wraps them with stable `SEARCH.*` codes so consumers can pattern-match without depending on the foundation error types. |
| `SearchPipeline.process` accepts `AbortSignal` | Long synonym graphs / stemming chains can dominate request latency on large queries; allowing abort midway through analysis matches the rest of the package. |
## Verification approach
Each spec delta requirement carries a `**Verified by:**` line pointing at concrete `it(...)` titles. The test files that will exist after this change ships:
- `packages/platform/search/src/client/search-client.test.ts` — `index` / `search` / `find` / `suggest` round-trip against an in-memory engine; `defaultLimit` applied; `fieldBoosts` and `scoringFunction` re-rank results; `removeMany` and bulk index path
- `packages/platform/search/src/client/search-client.spec-conformance.test.ts` — `searchHooks.hook('search:querying')` rewrites the query before the engine sees it; `'search:indexing'` rewrites documents; `'search:result'` post-processes; `AbortSignal` aborts mid-flight; injected `MemorySearchEngine` produces deterministic order
- `packages/platform/search/src/query-builder/query-builder.test.ts` — `term` / `must` / `should` / `mustNot` / `range` / `prefix` / `fuzzy` / `geo` / `filter` / `facet` / `sort` / `paginate` build correctly; `boost` and `scoringFunction` round-trip; `reset()` clears state
- `packages/platform/search/src/pipeline/search-pipeline.test.ts` — `LowercaseAnalyzer` / `StemmingAnalyzer` / `StopWordAnalyzer` / `SynonymAnalyzer` each tested in isolation; `SearchPipeline` walks analyzers in order; custom analyzer registers
- `packages/platform/search/src/pipeline/search-pipeline.spec-conformance.test.ts` — `AbortSignal` mid-pipeline rejects with `ANALYZER_INVALID` when an analyzer throws; bidirectional synonyms expand correctly; stop-word fallback returns original tokens when filtering would empty the list
- `packages/platform/search/src/hooks/search-hooks.test.ts` — `'search:querying'` / `'search:indexing'` / `'search:result'` lifecycle bus tested with multiple handlers; handler-priority order respected; `unsubscribe` removes record
- `packages/platform/search/src/hooks/search-hooks.spec-conformance.test.ts` — handler exception raises `HOOK_HANDLER_FAILED` with original `cause`; per-hook payload type pinned via `expectTypeOf`
- `packages/platform/search/src/adapters/base-elastic.test.ts` — `index` / `remove` / `search` / `clear` / `size` / `suggest` round-trip against a `fetch` mock; `bulkIndex` builds NDJSON; `ensureIndex` / `deleteIndex` / `refresh`; auth basic / apiKey / bearer; `searchFields` honoured; facets translate to aggregations
- `packages/platform/search/src/adapters/base-elastic.spec-conformance.test.ts` — 429 rate-limit response raises `PROVIDER_RATE_LIMITED` with `retryAfterMs`; 408 / 504 raises `PROVIDER_TIMEOUT`; missing host raises `PROVIDER_CONFIG_INVALID`; non-OK responses throw `HttpError`-wrapped `SearchError`
- `packages/platform/search/src/adapters/elasticsearch.test.ts` — adapter sends ES 8 headers by default; `compatibilityMode: true` switches to ES 7 headers
- `packages/platform/search/src/adapters/opensearch.test.ts` — adapter without `awsSigning` behaves like `BaseElasticAdapter`; with `awsSigning` invokes the supplied signer once per request and uses the returned `RequestInit`; signer rejection raises `AWS_SIGNING_FAILED`
- `packages/platform/search/src/adapters/meilisearch.test.ts` — `index` / `remove` / `search` / `clear` / `size` / `suggest` round-trip; filters translate to MeiliSearch filter strings; `ensureIndex` patches `searchableAttributes` / `filterableAttributes` / `sortableAttributes`; faceted search returns `facetDistribution`
- `packages/platform/search/src/adapters/meilisearch.spec-conformance.test.ts` — 429 raises `PROVIDER_RATE_LIMITED`; 408 / 504 raises `PROVIDER_TIMEOUT`; 400 raises `QUERY_INVALID`; missing API key path
- `tests/dep-graph.test.ts` (cross-package) — `@suaveplan/search` declares `@suaveplan/events` as a runtime dep; the package still has zero browser-only deps; subpath count matches feature module count
## Migration impact
### Affected dependents
Run `bunx turbo ls --filter=...@suaveplan/search` to enumerate. As of `grep -rl "@suaveplan/search" packages/ --include="package.json"` (2026-05-02), the only match is the package's own `package.json` — there are no in-repo dependents on genesis yet.
- (no in-repo dependents) — additive only; existing barrel exports (`MemorySearchEngine`, `createMemorySearchEngine`, `SEARCH_CODES`, `SearchError`, `bm25Score`, `tfIdfScore`, `tokenize`, `simpleStem`, `normalizeToken`, `buildTrigrams`, `trigramSimilarity`, every Zod schema, every type alias) are preserved unchanged. New exports (`SearchClient`, `QueryBuilder`, `SearchPipeline`, four analyzers, `searchHooks`, `BaseElasticAdapter`, `ElasticsearchAdapter`, `OpenSearchAdapter`, `MeiliSearchAdapter`, every new type alias) are additive.
The `SearchEngine<TDoc>` interface gains two optional methods (`bulkIndex` and `ensureSchema`) with default-throw stubs in the abstract class so `MemorySearchEngine` compiles unchanged. Consumers that depended on the old `SearchEngine` shape continue to compile because the new methods are optional.
For foundation-package safety, a `bunx turbo build typecheck test --filter=...@suaveplan/search` sweep runs in §6 of `tasks.md`.
## Parity Checklist
| Capability | Upstream | Migrated | Notes |
|---|---|---|---|
| `MemorySearchEngine` (BM25 + trigram + tokenizer) | yes | yes | Already on genesis |
| `SearchEngine<TDoc>` interface | yes | yes | Already on genesis; gains optional `bulkIndex` / `ensureSchema` in this change |
| `SearchClient<T>` orchestrator | yes | yes (this change) | `AbortSignal` threaded through all async methods |
| `QueryBuilder` fluent DSL | yes | yes (this change) | Adds `must` / `should` / `mustNot` / `range` / `prefix` / `fuzzy` / `geo` / `boost` / `scoringFunction` |
| `SearchPipeline` analyzer chain | yes | yes (this change) | Verbatim |
| `LowercaseAnalyzer` | yes | yes (this change) | Verbatim |
| `StemmingAnalyzer` (Porter-like) | yes | yes (this change) | Verbatim |
| `StopWordAnalyzer` (configurable) | yes | yes (this change) | Verbatim |
| `SynonymAnalyzer` (one-way / bidirectional) | yes | yes (this change) | Verbatim |
| `searchHooks` typed event bus | yes | yes (this change) | Adds `'search:indexing'` and `'search:result'` |
| `FacetDistribution` / `FacetValue` / `FacetRequest` types | yes | yes (this change) | Verbatim |
| `FieldBoost` / `ScoringFunction` types | yes | yes (this change) | Verbatim |
| `SearchSuggestion` / `SuggestOptions` | yes | yes (this change) | Verbatim |
| `BaseElasticAdapter<T>` | yes | yes (this change) | Verbatim |
| `ElasticsearchAdapter<T>` | yes | yes (this change) | Verbatim |
| `OpenSearchAdapter<T>` (with optional AWS sigv4) | yes | yes (this change) | `AwsSigningConfig` is pluggable; no AWS-SDK runtime dep |
| `MeiliSearchAdapter<T>` | yes | yes (this change) | Verbatim |
| Algolia adapter | no | 🚫 (out of scope) | Not in upstream; will land as `@suaveplan/search-algolia` once a tenant requests it |
| Typesense adapter | no | 🚫 (out of scope) | Not in upstream; query shape doesn't map onto Elastic family base class |
| Vector / dense retrieval | no | 🚫 (out of scope) | Not in upstream; lands as `@suaveplan/search-vector` |
No upstream capabilities are dropped.