SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
suaveplan/openspec/changes/search-completion/proposal.md
admin fbc70f96e6 Initial Commit
2026-05-27 23:29:53 +01:00

20 KiB
Raw Permalink Blame History

Search completion

Package: @suaveplan/search Status: PROPOSED Type: spec-conformance + feature

Why

The 2026-05-02 cross-monorepo audit confirmed that @suaveplan/search shipped on genesis with only the in-memory side of the original surface — MemorySearchEngine, BM25 + trigram scorers, the tokenizer, and Zod schemas. The legacy packages/universal/search/src package on ../suaveplan/ carries the full search-product surface that downstream services (catalogue, knowledge, analytics) rely on: a SearchClient<T> orchestrator, a QueryBuilder fluent API, a four-analyzer SearchPipeline (LowercaseAnalyzer, StemmingAnalyzer, StopWordAnalyzer, SynonymAnalyzer), a searchHooks event bus, the FacetDistribution / FacetRequest / FieldBoost / ScoringFunction shapes, and three production adapters — ElasticsearchAdapter, OpenSearchAdapter (with optional AWS sigv4 signing), and MeiliSearchAdapter. None of these were archived as out-of-scope; they were silently dropped during the original genesis migration.

Genesis policy (CLAUDE.md §Workflow rule 5) is "preserve the full public surface during rewrites — trimming is OK ONLY when code is duplicated elsewhere in-repo or can be replaced by an existing @suaveplan/* library." None of the dropped capabilities are duplicated elsewhere on genesis, so each one must be ported. This change closes that gap so any downstream service can compose an in-memory engine for tests with a hosted engine for production behind one SearchEngine<TDoc> contract.

What changes

  • ADDED: SearchClient<TDoc extends SearchDocument> orchestrator class — wraps any SearchEngine<TDoc> with searchHooks lifecycle wiring, SearchPipeline analyzer chain, defaultLimit, optional fieldBoosts, and an optional ScoringFunction. Methods: index, indexMany, remove, removeMany, search, find, suggest, clear, size. Every async method threads AbortSignal through to the underlying engine.
  • ADDED: QueryBuilder fluent typed query DSL — .term(value), .must(filter), .should(filter), .mustNot(filter), .range(field, op, value), .term_(field, value), .prefix(field, value), .fuzzy(field, value, distance?), .geo(field, lat, lng, radiusKm?), .filter(field, op, value) (legacy alias), .facet(field, maxValues?), .sort(field, direction), .paginate(offset, limit), .boost(field, weight), .scoringFunction(fn), .build(), .reset(). The .build() output is a SearchQuery matching the engine's shape.
  • ADDED: SearchPipeline class with composable Analyzer interface — process(query) walks every registered analyzer in order. Built-in analyzers: LowercaseAnalyzer, StemmingAnalyzer (Porter-like suffix stripping; 18 rules), StopWordAnalyzer (configurable stop-word set; 51-word default), SynonymAnalyzer (one-way and bidirectional synonym map). Custom analyzers conform to interface Analyzer { name: string; analyze(terms): string[] } and can be registered without touching the package.
  • ADDED: searchHooks typed event bus — 'search:querying' (intercept and rewrite query before the engine sees it), 'search:indexing' (intercept and rewrite document before index), 'search:result' (post-process result set before return). Hooks are wired through @suaveplan/events and respect the existing handler-priority / order semantics.
  • ADDED: FacetDistribution, FacetValue, FacetRequest, FieldBoost, ScoringFunction<T>, SearchSuggestion, SuggestOptions shape types — co-located with their implementation modules.
  • ADDED: BaseElasticAdapter<TDoc> shared base class — handles auth (basic / apiKey / bearer), request-timeout, query body construction (bool / multi_match / range), filter translation, response transformation (hits + aggregations → SearchResult), bulk indexing, index management (ensureIndex, deleteIndex, refresh), and structured error mapping (HttpError / RateLimitError / TimeoutError / ConfigError).
  • ADDED: ElasticsearchAdapter<TDoc> class — extends BaseElasticAdapter; ES 8.x by default, ES 7.x compatibility-mode header opt-in.
  • ADDED: OpenSearchAdapter<TDoc> class — extends BaseElasticAdapter; optional AwsSigningConfig injects an (url, init) => Promise<RequestInit> hook so sigv4 signing is pluggable.
  • ADDED: MeiliSearchAdapter<TDoc> class — implements SearchEngine<TDoc> against MeiliSearch HTTP API; supports searchableAttributes, filterableAttributes, sortableAttributes, faceted search via facetDistribution, suggestions via _formatted extraction.
  • ADDED: ElasticBaseConfig, ElasticAuth, ElasticsearchConfig, OpenSearchConfig, AwsSigningConfig, MeiliSearchConfig, SearchClientConfig shape types.
  • MODIFIED: SearchEngine<TDoc> interface — adds optional bulkIndex(documents) and ensureSchema() lifecycle hooks; the new methods default-throw with SEARCH.NOT_IMPLEMENTED so existing engines (e.g. MemorySearchEngine) compile unchanged.
  • MODIFIED: SEARCH_CODES registry — adds PROVIDER_UNREACHABLE, PROVIDER_TIMEOUT, PROVIDER_RATE_LIMITED, PROVIDER_CONFIG_INVALID, HOOK_HANDLER_FAILED, ANALYZER_INVALID, NOT_IMPLEMENTED, AWS_SIGNING_FAILED, BULK_INDEX_PARTIAL_FAILURE codes; the existing INDEX_FAILED, QUERY_INVALID, ENGINE_NOT_READY, FIELD_UNKNOWN, CURSOR_INVALID codes are preserved.
  • MODIFIED: index.ts barrel — re-exports every new symbol; no export * (one explicit subpath per module).
  • MODIFIED: package.json exports map — adds subpaths ./client/search-client, ./query-builder/query-builder, ./pipeline/search-pipeline, ./pipeline/analyzers/lowercase, ./pipeline/analyzers/stemming, ./pipeline/analyzers/stopword, ./pipeline/analyzers/synonym, ./hooks/search-hooks, ./adapters/base-elastic, ./adapters/elasticsearch, ./adapters/opensearch, ./adapters/meilisearch.
  • MODIFIED: package.json dependencies — adds @suaveplan/events (workspace:^).

Out of scope

  • Algolia adapter — different query DSL and pricing model; lands as its own package (@suaveplan/search-algolia) once a tenant requests it. Avoiding speculative adapter sprawl in this change.
  • Vector search / dense retrieval — embedding pipelines, ANN indexes (HNSW / IVF), and hybrid lexical+vector ranking are a separate concern with their own ML dependency surface; they land as @suaveplan/search-vector.
  • Bundling @elastic/elasticsearch or meilisearch-js SDKs — both adapters use direct fetch() with Zod-validated responses to keep the package tree-shake-friendly and free of native-binding peer deps. This matches the upstream design and the @suaveplan/sitemap / @suaveplan/dmarc precedent (HTTP-only, Zod-validated wire formats).
  • Typesense adapter — the genesis testing harness already ships @suaveplan/testing-integration-typesense for service spin-up; the production adapter is a separate package because Typesense's query shape (filter strings, multi-search) does not map onto the Elastic family base class without re-architecture.
  • registerHook legacy free-function deprecation — kept and delegated to the new searchHooks.hook API for one release; removing it is a breaking change that requires its own deprecation window.
  • Re-folding of openspec/specs/search/spec.md if a canonical spec file does not yet exist, this change creates it for the cap; full spec folding lands once all search changes archive.

Source mapping

This change derives from ../suaveplan/packages/universal/search/src/. Genesis enterprise/search.md is a generic enterprise template (no per-feature requirements), so source mapping cites the upstream package source directly.

Requirement Source file Source lines Notes
SearchClient<T> orchestrator packages/universal/search/src/search-client/search-client.ts L21-L93 Verbatim; AbortSignal threaded through every async method on genesis
SearchClientConfig packages/universal/search/src/types/search.types.ts L139-L148 Verbatim
searchHooks event bus + 'search:querying' hook packages/universal/search/src/hooks.ts L1-L62 Verbatim; rewired to use @suaveplan/events Hooks class
'search:indexing' and 'search:result' hooks (no upstream line — added in this change) — Symmetry with 'search:querying'; justified in "no source line" table below
QueryBuilder fluent DSL — term, filter, facet, sort, paginate, offset, limit, build, reset packages/universal/search/src/query-builder/query-builder.ts L27-L107 Verbatim
QueryBuilder.must, should, mustNot, range, prefix, fuzzy, geo, boost, scoringFunction extensions (no upstream line — added in this change) — Justified in "no source line" table below
SearchPipeline composable analyzer chain packages/universal/search/src/pipeline/search-pipeline.ts L204-L224 Verbatim
Analyzer interface (name, analyze(terms)) packages/universal/search/src/pipeline/search-pipeline.ts L9-L12 Verbatim
LowercaseAnalyzer packages/universal/search/src/pipeline/search-pipeline.ts L106-L114 Verbatim
StemmingAnalyzer (Porter-like 18-rule suffix table) packages/universal/search/src/pipeline/search-pipeline.ts L57-L103 Verbatim
StopWordAnalyzer (51-word default + configurable set) packages/universal/search/src/pipeline/search-pipeline.ts L117-L186 Verbatim
SynonymAnalyzer (one-way and bidirectional) packages/universal/search/src/pipeline/search-pipeline.ts L21-L51 Verbatim
FacetRequest, FacetValue, FacetDistribution, FieldBoost, ScoringFunction, SearchSuggestion, SuggestOptions packages/universal/search/src/types/search.types.ts L26-L77 Verbatim
BaseElasticAdapter<T> shared base class packages/universal/search/src/adapters/base-elastic-adapter.ts L75-L442 Verbatim
ElasticBaseConfig, ElasticAuth packages/universal/search/src/adapters/base-elastic-adapter.ts L26-L47 Verbatim
ElasticsearchAdapter<T> packages/universal/search/src/adapters/elasticsearch-adapter.ts L45-L64 Verbatim
ElasticsearchConfig packages/universal/search/src/adapters/elasticsearch-adapter.ts L12-L15 Verbatim
OpenSearchAdapter<T> packages/universal/search/src/adapters/opensearch-adapter.ts L38-L60 Verbatim
OpenSearchConfig, AwsSigningConfig packages/universal/search/src/adapters/opensearch-adapter.ts L12-L23 Verbatim
MeiliSearchAdapter<T> packages/universal/search/src/adapters/meilisearch-adapter.ts L35-L305 Verbatim
MeiliSearchConfig packages/universal/search/src/types/search.types.ts L150-L164 Verbatim

Requirements with no source line

Requirement Justification
'search:indexing' and 'search:result' lifecycle hooks (in addition to upstream 'search:querying') Genesis-wide CLAUDE.md rule 3 (enterprise-grade modularity) requires a complete intercept surface. Upstream only exposes 'search:querying'; downstream consumers (knowledge-graph, content-pipeline) need to rewrite indexed documents (e.g. inject embeddings) and post-process result sets (e.g. attach permission metadata). Symmetric naming with the existing hook avoids API drift.
QueryBuilder.must / should / mustNot / range / prefix / fuzzy / geo / boost / scoringFunction methods Upstream QueryBuilder only ships filter() and term(). The proposal task description explicitly requires these clauses (must / should / must_not / range / term / prefix / fuzzy / geo). Adding them on the builder side keeps a typed DSL while preserving every adapter's freedom to translate filters into its native query language.
bulkIndex and ensureSchema optional methods on SearchEngine<TDoc> Bulk indexing is a documented BaseElasticAdapter capability (upstream L149-L166) but is not part of the universal interface. Genesis policy requires the universal contract to expose every supported optimisation so callers can migrate engines without code changes; both methods default-throw with SEARCH.NOT_IMPLEMENTED so existing engines (e.g. MemorySearchEngine) compile unchanged.
AbortSignal threaded through every SearchClient and SearchEngine async method enterprise/search.md L43: "Backpressure & cancellation — AbortSignal everywhere async". Upstream lacks signal support on the client-orchestration surface.
Zod schemas for SearchClientConfig, ElasticBaseConfig, ElasticAuth, MeiliSearchConfig, AwsSigningConfig, FacetRequest, FieldBoost, every Analyzer config enterprise/search.md L37: "Zod schemas for all public types". Upstream validates through TypeScript only.
New SEARCH.PROVIDER_* / SEARCH.HOOK_HANDLER_FAILED / SEARCH.ANALYZER_INVALID / SEARCH.NOT_IMPLEMENTED / SEARCH.AWS_SIGNING_FAILED / SEARCH.BULK_INDEX_PARTIAL_FAILURE codes Genesis-wide policy (CLAUDE.md §Workflow rule 2) that every thrown error route through @suaveplan/error. Upstream throws HttpError / TimeoutError / RateLimitError from @suaveplan/error directly; genesis wraps them with stable SEARCH.* codes so consumers can pattern-match without depending on the foundation error types.
SearchPipeline.process accepts AbortSignal Long synonym graphs / stemming chains can dominate request latency on large queries; allowing abort midway through analysis matches the rest of the package.

Verification approach

Each spec delta requirement carries a **Verified by:** line pointing at concrete it(...) titles. The test files that will exist after this change ships:

  • packages/platform/search/src/client/search-client.test.ts — index / search / find / suggest round-trip against an in-memory engine; defaultLimit applied; fieldBoosts and scoringFunction re-rank results; removeMany and bulk index path
  • packages/platform/search/src/client/search-client.spec-conformance.test.ts — searchHooks.hook('search:querying') rewrites the query before the engine sees it; 'search:indexing' rewrites documents; 'search:result' post-processes; AbortSignal aborts mid-flight; injected MemorySearchEngine produces deterministic order
  • packages/platform/search/src/query-builder/query-builder.test.ts — term / must / should / mustNot / range / prefix / fuzzy / geo / filter / facet / sort / paginate build correctly; boost and scoringFunction round-trip; reset() clears state
  • packages/platform/search/src/pipeline/search-pipeline.test.ts — LowercaseAnalyzer / StemmingAnalyzer / StopWordAnalyzer / SynonymAnalyzer each tested in isolation; SearchPipeline walks analyzers in order; custom analyzer registers
  • packages/platform/search/src/pipeline/search-pipeline.spec-conformance.test.ts — AbortSignal mid-pipeline rejects with ANALYZER_INVALID when an analyzer throws; bidirectional synonyms expand correctly; stop-word fallback returns original tokens when filtering would empty the list
  • packages/platform/search/src/hooks/search-hooks.test.ts — 'search:querying' / 'search:indexing' / 'search:result' lifecycle bus tested with multiple handlers; handler-priority order respected; unsubscribe removes record
  • packages/platform/search/src/hooks/search-hooks.spec-conformance.test.ts — handler exception raises HOOK_HANDLER_FAILED with original cause; per-hook payload type pinned via expectTypeOf
  • packages/platform/search/src/adapters/base-elastic.test.ts — index / remove / search / clear / size / suggest round-trip against a fetch mock; bulkIndex builds NDJSON; ensureIndex / deleteIndex / refresh; auth basic / apiKey / bearer; searchFields honoured; facets translate to aggregations
  • packages/platform/search/src/adapters/base-elastic.spec-conformance.test.ts — 429 rate-limit response raises PROVIDER_RATE_LIMITED with retryAfterMs; 408 / 504 raises PROVIDER_TIMEOUT; missing host raises PROVIDER_CONFIG_INVALID; non-OK responses throw HttpError-wrapped SearchError
  • packages/platform/search/src/adapters/elasticsearch.test.ts — adapter sends ES 8 headers by default; compatibilityMode: true switches to ES 7 headers
  • packages/platform/search/src/adapters/opensearch.test.ts — adapter without awsSigning behaves like BaseElasticAdapter; with awsSigning invokes the supplied signer once per request and uses the returned RequestInit; signer rejection raises AWS_SIGNING_FAILED
  • packages/platform/search/src/adapters/meilisearch.test.ts — index / remove / search / clear / size / suggest round-trip; filters translate to MeiliSearch filter strings; ensureIndex patches searchableAttributes / filterableAttributes / sortableAttributes; faceted search returns facetDistribution
  • packages/platform/search/src/adapters/meilisearch.spec-conformance.test.ts — 429 raises PROVIDER_RATE_LIMITED; 408 / 504 raises PROVIDER_TIMEOUT; 400 raises QUERY_INVALID; missing API key path
  • tests/dep-graph.test.ts (cross-package) — @suaveplan/search declares @suaveplan/events as a runtime dep; the package still has zero browser-only deps; subpath count matches feature module count

Migration impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/search to enumerate. As of grep -rl "@suaveplan/search" packages/ --include="package.json" (2026-05-02), the only match is the package's own package.json — there are no in-repo dependents on genesis yet.

  • (no in-repo dependents) — additive only; existing barrel exports (MemorySearchEngine, createMemorySearchEngine, SEARCH_CODES, SearchError, bm25Score, tfIdfScore, tokenize, simpleStem, normalizeToken, buildTrigrams, trigramSimilarity, every Zod schema, every type alias) are preserved unchanged. New exports (SearchClient, QueryBuilder, SearchPipeline, four analyzers, searchHooks, BaseElasticAdapter, ElasticsearchAdapter, OpenSearchAdapter, MeiliSearchAdapter, every new type alias) are additive.

The SearchEngine<TDoc> interface gains two optional methods (bulkIndex and ensureSchema) with default-throw stubs in the abstract class so MemorySearchEngine compiles unchanged. Consumers that depended on the old SearchEngine shape continue to compile because the new methods are optional.

For foundation-package safety, a bunx turbo build typecheck test --filter=...@suaveplan/search sweep runs in §6 of tasks.md.

Parity Checklist

Capability Upstream Migrated Notes
MemorySearchEngine (BM25 + trigram + tokenizer) yes yes Already on genesis
SearchEngine<TDoc> interface yes yes Already on genesis; gains optional bulkIndex / ensureSchema in this change
SearchClient<T> orchestrator yes yes (this change) AbortSignal threaded through all async methods
QueryBuilder fluent DSL yes yes (this change) Adds must / should / mustNot / range / prefix / fuzzy / geo / boost / scoringFunction
SearchPipeline analyzer chain yes yes (this change) Verbatim
LowercaseAnalyzer yes yes (this change) Verbatim
StemmingAnalyzer (Porter-like) yes yes (this change) Verbatim
StopWordAnalyzer (configurable) yes yes (this change) Verbatim
SynonymAnalyzer (one-way / bidirectional) yes yes (this change) Verbatim
searchHooks typed event bus yes yes (this change) Adds 'search:indexing' and 'search:result'
FacetDistribution / FacetValue / FacetRequest types yes yes (this change) Verbatim
FieldBoost / ScoringFunction types yes yes (this change) Verbatim
SearchSuggestion / SuggestOptions yes yes (this change) Verbatim
BaseElasticAdapter<T> yes yes (this change) Verbatim
ElasticsearchAdapter<T> yes yes (this change) Verbatim
OpenSearchAdapter<T> (with optional AWS sigv4) yes yes (this change) AwsSigningConfig is pluggable; no AWS-SDK runtime dep
MeiliSearchAdapter<T> yes yes (this change) Verbatim
Algolia adapter no 🚫 (out of scope) Not in upstream; will land as @suaveplan/search-algolia once a tenant requests it
Typesense adapter no 🚫 (out of scope) Not in upstream; query shape doesn't map onto Elastic family base class
Vector / dense retrieval no 🚫 (out of scope) Not in upstream; lands as @suaveplan/search-vector

No upstream capabilities are dropped.

Reference in New Issue View Git Blame Copy Permalink