Initial Commit
This commit is contained in:
74
openspec/changes/sitemap-completion/proposal.md
Normal file
74
openspec/changes/sitemap-completion/proposal.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Sitemap completion — extensions, incremental builder, gzip compressor
|
||||
|
||||
**Package**: `@suaveplan/sitemap`
|
||||
**Status**: PROPOSED
|
||||
**Type**: feature + spec-conformance
|
||||
|
||||
## Why
|
||||
|
||||
`@suaveplan/sitemap` (`packages/api/sitemap/`) currently ships only the core builder, index-builder, parser, and URL utilities. Four upstream surfaces are absent: the Google image-sitemap extension, the Google video-sitemap extension, the incremental on-disk builder, and the gzip compressor. Without them, callers must re-implement Google-spec XML by hand (image / video) or stream URLs through the in-memory builder until they hit Google's 50,000-URL / 50 MiB hard cap. All four legacy modules already exist (`packages/universal/sitemap/src/extensions/image-sitemap.ts` 111 L; `extensions/video-sitemap.ts` 182 L; `incremental/incremental-builder.ts` 213 L; `compression/sitemap-compressor.ts` 91 L) and pass the genesis quality bar — porting them closes the parity gap.
|
||||
|
||||
## What changes
|
||||
|
||||
### `@suaveplan/sitemap`
|
||||
|
||||
- ADDED: `SitemapImageExtension` — `addImage(urlEntry, image)` mutates a `SitemapUrl` to include a `<image:image>` block per Google's image-sitemap spec; supports `{ caption, location, license, title }`.
|
||||
- ADDED: `SitemapVideoExtension` — `addVideo(urlEntry, video)` mutates a `SitemapUrl` to include a `<video:video>` block; supports `{ thumbnail, title, description, contentLoc, playerLoc, duration, expiration, rating, viewCount, publicationDate, familyFriendly, requiresSubscription, uploader, live }`.
|
||||
- ADDED: `IMAGE_NS` / `VIDEO_NS` namespace constants and Zod schemas `SitemapImageExtensionSchema`, `SitemapVideoExtensionSchema`.
|
||||
- ADDED: `IncrementalSitemapBuilder` class — append-only builder that streams URLs in batches to disk through an injectable `WritableStorageLike` adapter; preserves URL order; rolls over to a new sitemap file at 50,000 URLs OR 50 MiB (whichever first); emits a `SitemapIndex` entry per rollover.
|
||||
- ADDED: `IncrementalSitemapBuilderOptions` interface — `outputDir`, `baseFilename` (default `sitemap`), `urlsPerSitemap` (default 50_000, max 50_000), `maxBytes` (default 50 *1024* 1024), `storage` (`WritableStorageLike`), `compress` (boolean — when true, each rollover output is gzipped).
|
||||
- ADDED: `compressSitemap(xml: string): Promise<Uint8Array>` and `compressSitemapIndex(xml: string): Promise<Uint8Array>` async helpers — gzip with valid `gzip` magic bytes (`1f 8b`), DEFLATE compression level 6 (default), ISO-8859-1 OS field, mtime stamped from injectable `Clock`.
|
||||
- ADDED: `decompressGzipSitemap(bytes: Uint8Array): Promise<string>` round-trip helper for tests.
|
||||
- ADDED: `SITEMAP_CODES` extensions — `IMAGE_VALIDATION_FAILED`, `VIDEO_VALIDATION_FAILED`, `INCREMENTAL_ROLLOVER_FAILED`, `COMPRESSION_FAILED` registered via existing `defineCodes` block.
|
||||
- ADDED: subpath exports `./extensions/image-sitemap`, `./extensions/video-sitemap`, `./incremental/incremental-builder`, `./compression/sitemap-compressor`.
|
||||
- MODIFIED: root `src/index.ts` re-exports the new extensions, builder, and compressor helpers.
|
||||
|
||||
## Out of scope
|
||||
|
||||
- **News sitemap extension** (`<news:news>` block) — separate change. The Google news-sitemap is a different schema with publication-name / publication-language fields and a strict 30-day TTL window; folding it into this change dilutes review.
|
||||
- **Mobile sitemap extension** — deprecated by Google in 2015; do not port.
|
||||
- **Streaming gzip via `CompressionStream`** — keep the synchronous `fflate` path for now; once the gzip surface is stable a follow-up change can swap to a `WritableStream` pipeline.
|
||||
- **`SitemapBuilder.compress()` instance method** — out of scope to keep the core builder side-effect free; callers gzip the output explicitly via `compressSitemap`.
|
||||
|
||||
## Source mapping
|
||||
|
||||
This change derives from `../suaveplan/enterprise/sitemap.md` and the upstream package source.
|
||||
|
||||
| Requirement | Source file | Source lines | Notes |
|
||||
|---|---|---|---|
|
||||
| `SitemapImageExtension` + Zod schema | `packages/universal/sitemap/src/extensions/image-sitemap.ts` | L1-L111 | Verbatim port; existing `SitemapImageSchema` in `schemas/sitemap.schema.ts` reused |
|
||||
| `SitemapVideoExtension` + Zod schema | `packages/universal/sitemap/src/extensions/video-sitemap.ts` | L1-L182 | Verbatim port; existing `SitemapVideoSchema` reused |
|
||||
| `IncrementalSitemapBuilder` | `packages/universal/sitemap/src/incremental/incremental-builder.ts` | L1-L213 | Verbatim port; replace `Date.now()` and `process.env.HOME` paths with injectable `Clock` and `WritableStorageLike` |
|
||||
| `compressSitemap` / `compressSitemapIndex` / `decompressGzipSitemap` | `packages/universal/sitemap/src/compression/sitemap-compressor.ts` | L1-L91 | Verbatim port |
|
||||
|
||||
### Requirements with no source line
|
||||
|
||||
| Requirement | Justification |
|
||||
|---|---|
|
||||
| Injectable `Clock` for gzip mtime field | AGENTS.md determinism rule (no `Date.now()` in hot paths). Upstream stamps `Date.now()`, breaking deterministic test fixtures. |
|
||||
| Injectable `WritableStorageLike` adapter for `IncrementalSitemapBuilder` | AGENTS.md §11 ("shared types belong in `@suaveplan/types`"). Upstream writes directly via `fs.promises.writeFile`, coupling the builder to Node. |
|
||||
| `INCREMENTAL_ROLLOVER_FAILED` / `COMPRESSION_FAILED` error codes | AGENTS.md §13 row 3. Upstream throws plain `Error`. |
|
||||
|
||||
## Verification approach
|
||||
|
||||
Each requirement carries a `**Verified by:**` line in the spec delta.
|
||||
|
||||
- `packages/api/sitemap/src/extensions/image-sitemap/image-sitemap.test.ts` — covers Image extension scenarios
|
||||
- `packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.test.ts` — covers Video extension scenarios
|
||||
- `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.test.ts` — covers append, rollover at 50k URLs, rollover at 50 MiB, index entry emission
|
||||
- `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.spec-conformance.test.ts` — pins injected `WritableStorageLike` contract and Clock determinism
|
||||
- `packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.test.ts` — covers gzip round-trip, magic-byte assertions, mtime determinism
|
||||
|
||||
## Impact
|
||||
|
||||
### Affected dependents
|
||||
|
||||
Run `bunx turbo ls --filter=...@suaveplan/sitemap` to enumerate.
|
||||
|
||||
- No dependents (verified by `grep -rl "@suaveplan/sitemap" /home/ubuntu/web/genesis/packages/ --include="package.json"` → only `packages/api/sitemap/package.json` itself).
|
||||
|
||||
Pure additive change.
|
||||
|
||||
## Migration impact
|
||||
|
||||
None — additive only. Existing `SitemapBuilder`, `SitemapIndexBuilder`, `parseSitemap`, `parseSitemapIndex` calls compile unchanged. The `SitemapUrl` type already carries optional `images?: SitemapImage[]` and `videos?: SitemapVideo[]` fields (in `schemas/sitemap.schema.ts`); the new extensions populate those fields rather than introducing a new shape.
|
||||
140
openspec/changes/sitemap-completion/specs/sitemap/spec.md
Normal file
140
openspec/changes/sitemap-completion/specs/sitemap/spec.md
Normal file
@@ -0,0 +1,140 @@
|
||||
<!-- markdownlint-disable MD024 -->
|
||||
|
||||
# Capability Spec Delta: sitemap (sitemap-completion)
|
||||
|
||||
**Package**: `@suaveplan/sitemap`
|
||||
**Status**: PROPOSED
|
||||
|
||||
> Every requirement below carries a `**Verified by:**` line at authoring time. (See `openspec/AGENTS.md` Section 13.)
|
||||
|
||||
## ADDED Requirements
|
||||
|
||||
### Requirement: Image-sitemap extension
|
||||
|
||||
The package SHALL expose a `SitemapImageExtension` class with `addImage(urlEntry, image)` that mutates the given `SitemapUrl` to include a `<image:image>` block per Google's image-sitemap spec. Each image SHALL support `loc` (required), `caption`, `geoLocation`, `title`, and `license`. Up to 1,000 images per URL SHALL be permitted (Google's hard cap); the 1,001st throws.
|
||||
|
||||
#### Scenario: addImage appends to images array in order
|
||||
|
||||
- **WHEN** `addImage(urlEntry, { loc: "https://example.com/a.jpg" })` is called twice with different `loc`s on the same `urlEntry`
|
||||
- **THEN** `urlEntry.images` is `[{ loc: ".../a.jpg" }, { loc: ".../b.jpg" }]` in insertion order
|
||||
|
||||
#### Scenario: full optional set survives Zod validation
|
||||
|
||||
- **WHEN** `addImage(urlEntry, { loc, caption, geoLocation, title, license })` is called with all fields populated
|
||||
- **THEN** the resulting `urlEntry.images[0]` round-trips through `SitemapImageSchema.parse` without loss
|
||||
|
||||
#### Scenario: 1,001st image throws IMAGE_VALIDATION_FAILED
|
||||
|
||||
- **WHEN** 1,001 calls to `addImage` are issued for the same `urlEntry`
|
||||
- **THEN** the 1,001st throws `SitemapError` with code `SITEMAP.IMAGE_VALIDATION_FAILED` and `context.imageCount === 1001`
|
||||
|
||||
**Verified by:** `packages/api/sitemap/src/extensions/image-sitemap/image-sitemap.test.ts::"appends images in insertion order"` AND `packages/api/sitemap/src/extensions/image-sitemap/image-sitemap.test.ts::"round-trips full optional set"` AND `packages/api/sitemap/src/extensions/image-sitemap/image-sitemap.test.ts::"throws IMAGE_VALIDATION_FAILED at 1001 images"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Video-sitemap extension
|
||||
|
||||
The package SHALL expose a `SitemapVideoExtension` class with `addVideo(urlEntry, video)` that mutates the given `SitemapUrl` to include a `<video:video>` block per Google's video-sitemap spec. Each video SHALL support `thumbnail` (required), `title` (required), `description` (required), `contentLoc` OR `playerLoc` (at least one required), `duration`, `expiration`, `rating`, `viewCount`, `publicationDate`, `familyFriendly`, `requiresSubscription`, `uploader`, and `live`.
|
||||
|
||||
#### Scenario: video without contentLoc or playerLoc rejected
|
||||
|
||||
- **WHEN** `addVideo(urlEntry, { thumbnail, title, description })` is called with neither `contentLoc` nor `playerLoc`
|
||||
- **THEN** it throws `SitemapError` with code `SITEMAP.VIDEO_VALIDATION_FAILED` and `context.field === "contentLoc|playerLoc"`
|
||||
|
||||
#### Scenario: rating range enforced (0.0 to 5.0)
|
||||
|
||||
- **WHEN** `addVideo` is called with `rating: 5.5`
|
||||
- **THEN** it throws `SitemapError` with code `SITEMAP.VIDEO_VALIDATION_FAILED` and `context.field === "rating"`
|
||||
|
||||
#### Scenario: duration range enforced (1 to 28800 seconds)
|
||||
|
||||
- **WHEN** `addVideo` is called with `duration: 0`
|
||||
- **THEN** it throws `SitemapError` with code `SITEMAP.VIDEO_VALIDATION_FAILED` and `context.field === "duration"`
|
||||
|
||||
#### Scenario: full video round-trips through schema
|
||||
|
||||
- **WHEN** `addVideo` is called with every optional field populated
|
||||
- **THEN** `urlEntry.videos[0]` round-trips through `SitemapVideoSchema.parse` without loss
|
||||
|
||||
**Verified by:** `packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.test.ts::"rejects video missing both contentLoc and playerLoc"` AND `packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.test.ts::"enforces rating range"` AND `packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.test.ts::"enforces duration range"` AND `packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.test.ts::"round-trips full optional set"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Incremental on-disk sitemap builder
|
||||
|
||||
The package SHALL expose an `IncrementalSitemapBuilder` class that streams URLs through an injectable `WritableStorageLike` adapter, preserving insertion order, rolling over to a new sitemap file at 50,000 URLs OR 50 MiB (whichever first), and emitting one `SitemapIndex` entry per rollover. A `finalize()` method SHALL flush the in-progress sitemap, write the assembled index file, and return the index path.
|
||||
|
||||
#### Scenario: append + finalize produces single sitemap when under cap
|
||||
|
||||
- **WHEN** 100 URLs are appended and `finalize()` is called
|
||||
- **THEN** the storage adapter receives exactly 2 writes: `sitemap-1.xml` (the URLs) and `sitemap.xml` (the index pointing at `sitemap-1.xml`)
|
||||
|
||||
#### Scenario: rollover at exactly 50,000 URLs
|
||||
|
||||
- **WHEN** 50,001 URLs are appended (49,999 → 50,000 → 50,001) and `finalize()` is called
|
||||
- **THEN** the storage adapter receives writes for `sitemap-1.xml` (URLs 1-50000), `sitemap-2.xml` (URL 50001), and `sitemap.xml` (the index with 2 entries)
|
||||
|
||||
#### Scenario: rollover at 50 MiB
|
||||
|
||||
- **WHEN** URLs are appended whose cumulative XML byte size crosses 50 MiB before reaching 50,000 URLs
|
||||
- **THEN** the builder rolls over at the URL whose append would exceed 50 MiB; that URL goes into `sitemap-2.xml`, not `sitemap-1.xml`
|
||||
|
||||
#### Scenario: finalize is idempotent
|
||||
|
||||
- **WHEN** `finalize()` is called twice in succession
|
||||
- **THEN** the second call returns the same index path AND the storage adapter does not receive any additional writes (no double-emit)
|
||||
|
||||
#### Scenario: rollover error is structured
|
||||
|
||||
- **WHEN** the storage adapter throws on a rollover write
|
||||
- **THEN** the builder re-throws as `SitemapError` with code `SITEMAP.INCREMENTAL_ROLLOVER_FAILED` and `context.cause` carrying the original error
|
||||
|
||||
**Verified by:** `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.test.ts::"appends and finalizes a single sitemap when under cap"` AND `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.test.ts::"rolls over at exactly 50000 URLs"` AND `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.test.ts::"rolls over at 50 MiB byte cap"` AND `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.spec-conformance.test.ts::"finalize is idempotent"` AND `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.spec-conformance.test.ts::"wraps storage errors as INCREMENTAL_ROLLOVER_FAILED"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: Incremental builder honours injected Clock and storage
|
||||
|
||||
The package SHALL accept an injectable `Clock` from `@suaveplan/types` (default: system clock) for the gzip mtime field and any timestamps emitted in OTel metrics, and an injectable `WritableStorageLike` (no default — required).
|
||||
|
||||
#### Scenario: injected clock controls gzip mtime
|
||||
|
||||
- **WHEN** `compress: true` is set and a fake `Clock` returning `1_700_000_000_000` is injected
|
||||
- **THEN** the gzip header for each rollover output carries mtime `1_700_000_000` (seconds, gzip header convention)
|
||||
|
||||
#### Scenario: storage adapter receives all expected paths
|
||||
|
||||
- **WHEN** 60,000 URLs are appended with `outputDir: "/tmp/x"` and `baseFilename: "sm"`
|
||||
- **THEN** the adapter sees writes to `/tmp/x/sm-1.xml`, `/tmp/x/sm-2.xml`, and `/tmp/x/sm.xml` (the index) — exactly three writes
|
||||
|
||||
**Verified by:** `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.spec-conformance.test.ts::"injected clock controls gzip mtime"` AND `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.spec-conformance.test.ts::"storage adapter receives expected paths"`
|
||||
|
||||
---
|
||||
|
||||
### Requirement: gzip compression helpers
|
||||
|
||||
The package SHALL expose async helpers `compressSitemap(xml: string): Promise<Uint8Array>`, `compressSitemapIndex(xml: string): Promise<Uint8Array>`, and `decompressGzipSitemap(bytes: Uint8Array): Promise<string>`. The compressed output SHALL begin with the gzip magic bytes `1f 8b`. The mtime header SHALL come from an injectable `Clock` (default: `() => 0` for deterministic test fixtures).
|
||||
|
||||
#### Scenario: compress produces gzip magic bytes
|
||||
|
||||
- **WHEN** `compressSitemap("<urlset/>")` is awaited
|
||||
- **THEN** the first two bytes of the result are `0x1f 0x8b`
|
||||
|
||||
#### Scenario: round-trip preserves XML byte-for-byte
|
||||
|
||||
- **WHEN** an arbitrary XML string is passed through `compressSitemap` then `decompressGzipSitemap`
|
||||
- **THEN** the output equals the input string
|
||||
|
||||
#### Scenario: compression failure surfaces as structured error
|
||||
|
||||
- **WHEN** the underlying gzip implementation rejects the input (e.g. injected mock throws)
|
||||
- **THEN** the helper throws `SitemapError` with code `SITEMAP.COMPRESSION_FAILED` and `context.cause` carrying the original error
|
||||
|
||||
#### Scenario: deterministic mtime under default clock
|
||||
|
||||
- **WHEN** `compressSitemap("<urlset/>")` is called twice with the default clock (`() => 0`)
|
||||
- **THEN** both outputs are byte-for-byte identical
|
||||
|
||||
**Verified by:** `packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.test.ts::"emits gzip magic bytes"` AND `packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.test.ts::"round-trips XML byte-for-byte"` AND `packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.test.ts::"wraps gzip errors as COMPRESSION_FAILED"` AND `packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.test.ts::"produces deterministic output under default clock"`
|
||||
|
||||
<!-- markdownlint-enable MD024 -->
|
||||
101
openspec/changes/sitemap-completion/tasks.md
Normal file
101
openspec/changes/sitemap-completion/tasks.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# Tasks — Sitemap completion
|
||||
|
||||
> Every checkbox MUST reflect work actually done in this change. Do not retroactively tick.
|
||||
|
||||
## 0. Pre-flight (non-negotiable)
|
||||
|
||||
- [ ] 0.1 Confirm clean working tree (`git status` reports nothing) and no in-flight worktrees with uncommitted work (`git worktree list` then `git status` in each)
|
||||
- [ ] 0.2 If any worktrees exist under `.claude/worktrees/`, account for each one: still-active agent, work-pending-merge, or stale-and-prune. Stale worktrees MUST be removed (`git worktree remove`) before opening a new one — code that has been lost twice on main is the failure mode this guards against.
|
||||
- [ ] 0.3 Confirm the package set in `MANIFEST.md` matches `find packages -maxdepth 3 -name package.json` count. Drift here means a prior change archived without regenerating the manifest.
|
||||
|
||||
## Audit Note (2026-05-02)
|
||||
|
||||
Status: **DEFERRED**. `packages/api/sitemap/src/` only contains `core`, `errors`, `schemas`, `utils` — no `extensions/`, `incremental/`, or `compression/` directories. SitemapImageExtension, SitemapVideoExtension, IncrementalSitemapBuilder, and compressSitemap helpers are all absent. 0% of work is done.
|
||||
|
||||
## 1. Spec authoring
|
||||
|
||||
- [ ] 1.1 Read `../suaveplan/enterprise/sitemap.md` end-to-end
|
||||
- [ ] 1.2 Read `openspec/specs/sitemap/spec.md` (if it exists) — currently absent; this change introduces the canonical capability spec for the four new surfaces
|
||||
- [ ] 1.3 Read `openspec/AGENTS.md` Sections 13 and 14
|
||||
- [ ] 1.4 Draft `proposal.md` from `openspec/templates/proposal.md`
|
||||
- [ ] 1.5 Fill `## Source mapping` table with upstream line ranges
|
||||
- [ ] 1.6 Draft `specs/sitemap/spec.md` from `openspec/templates/spec-delta.md`
|
||||
- [ ] 1.7 Every requirement carries a `**Verified by:**` line
|
||||
- [ ] 1.8 `openspec change validate sitemap-completion` exits with no errors
|
||||
|
||||
## 2. Implementation — image extension
|
||||
|
||||
- [ ] 2.1 Port `SitemapImageExtension` from `packages/universal/sitemap/src/extensions/image-sitemap.ts` (L1-L111) into `packages/api/sitemap/src/extensions/image-sitemap/image-sitemap.ts`
|
||||
- [ ] 2.2 Reuse existing `SitemapImageSchema` from `packages/api/sitemap/src/schemas/sitemap.schema.ts` for input validation
|
||||
- [ ] 2.3 Throw `SitemapError` with code `IMAGE_VALIDATION_FAILED` on Zod failure
|
||||
- [ ] 2.4 Sibling `.md` (≥200 words, basic + advanced examples)
|
||||
|
||||
## 3. Implementation — video extension
|
||||
|
||||
- [ ] 3.1 Port `SitemapVideoExtension` from `packages/universal/sitemap/src/extensions/video-sitemap.ts` (L1-L182) into `packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.ts`
|
||||
- [ ] 3.2 Reuse existing `SitemapVideoSchema` for input validation
|
||||
- [ ] 3.3 Throw `SitemapError` with code `VIDEO_VALIDATION_FAILED` on Zod failure
|
||||
- [ ] 3.4 Sibling `.md`
|
||||
|
||||
## 4. Implementation — incremental builder
|
||||
|
||||
- [ ] 4.1 Port `IncrementalSitemapBuilder` from `packages/universal/sitemap/src/incremental/incremental-builder.ts` (L1-L213) into `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.ts`
|
||||
- [ ] 4.2 Replace direct `fs.promises.writeFile` with injectable `WritableStorageLike` from `@suaveplan/types`
|
||||
- [ ] 4.3 Replace `Date.now()` with injectable `Clock`
|
||||
- [ ] 4.4 Implement rollover at `urlsPerSitemap` (default 50_000) OR `maxBytes` (default 50 *1024* 1024) — whichever first
|
||||
- [ ] 4.5 Emit one `SitemapIndex` entry per rollover; carry final index assembly through `finalize()`
|
||||
- [ ] 4.6 When `compress: true`, pipe each rollover output through `compressSitemap` and append `.gz` extension
|
||||
- [ ] 4.7 Add `INCREMENTAL_ROLLOVER_FAILED` to `SITEMAP_CODES` via existing `defineCodes` call
|
||||
- [ ] 4.8 Sibling `.md`
|
||||
|
||||
## 5. Implementation — gzip compressor
|
||||
|
||||
- [ ] 5.1 Port `compressSitemap` / `compressSitemapIndex` / `decompressGzipSitemap` from `packages/universal/sitemap/src/compression/sitemap-compressor.ts` (L1-L91) into `packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.ts`
|
||||
- [ ] 5.2 Use `fflate` (already in catalog) for gzip; verify gzip magic bytes `1f 8b` in output
|
||||
- [ ] 5.3 Stamp the gzip header `mtime` field from injectable `Clock` (defaults to `() => 0` for deterministic test fixtures)
|
||||
- [ ] 5.4 Add `COMPRESSION_FAILED` to `SITEMAP_CODES`
|
||||
- [ ] 5.5 Sibling `.md`
|
||||
|
||||
## 6. Wiring
|
||||
|
||||
- [ ] 6.1 Update `packages/api/sitemap/src/index.ts` to re-export the four new modules and their types
|
||||
- [ ] 6.2 Add subpath exports `./extensions/image-sitemap/image-sitemap`, `./extensions/video-sitemap/video-sitemap`, `./incremental/incremental-builder/incremental-builder`, `./compression/sitemap-compressor/sitemap-compressor` to `packages/api/sitemap/package.json`
|
||||
- [ ] 6.3 Update root `README.md` `## Modules` section to link the four new co-located `.md`s
|
||||
|
||||
## 7. Tests
|
||||
|
||||
- [ ] 7.1 `packages/api/sitemap/src/extensions/image-sitemap/image-sitemap.test.ts` — adds image, multiple images, all optional fields, Zod rejection
|
||||
- [ ] 7.2 `packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.test.ts` — adds video, all optional fields, contentLoc / playerLoc mutual exclusion handling, Zod rejection
|
||||
- [ ] 7.3 `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.test.ts` — append URLs in batches; rollover at 50k URLs; rollover at 50 MiB; final `finalize()` produces a valid sitemap index XML
|
||||
- [ ] 7.4 `packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.spec-conformance.test.ts` — injected `WritableStorageLike` receives the right paths; injected `Clock` controls mtime; `compress: true` produces gzip-magic output
|
||||
- [ ] 7.5 `packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.test.ts` — gzip round-trip; magic bytes `1f 8b` present; mtime deterministic; `decompressGzipSitemap` recovers the original XML byte-for-byte
|
||||
- [ ] 7.6 Every `it(...)` carries `{ timeout: 5000 }` (30000 for the rollover-at-50k test, which appends 50_000 URLs)
|
||||
|
||||
## 8. Documentation
|
||||
|
||||
- [ ] 8.1 Sibling `.md` for every new implementation `.ts` (≥200 words, basic + advanced examples, full API reference)
|
||||
- [ ] 8.2 README links the four new `.md`s under `## Modules`
|
||||
|
||||
## 9. Pre-archive verification (the 5-gate triple)
|
||||
|
||||
- [ ] 9.1 `bun run check` — Biome 0/0/0 across the monorepo
|
||||
- [ ] 9.2 `bunx turbo lint typecheck test --filter=@suaveplan/sitemap` — green
|
||||
- [ ] 9.3 `bunx turbo test:coverage --filter=@suaveplan/sitemap` — 100/100/100/100
|
||||
- [ ] 9.4 `bun scripts/openspec-conformance.ts @suaveplan/sitemap` — exit 0, no `error` findings
|
||||
- [ ] 9.5 `bunx turbo build --filter=@suaveplan/sitemap` — green
|
||||
|
||||
## 10. Pre-archive 7-row checklist (AGENTS.md Section 13)
|
||||
|
||||
- [ ] 10.1 Generics threaded; `expectTypeOf` assertions on `IncrementalSitemapBuilderOptions` literal-type fields
|
||||
- [ ] 10.2 Zod schemas wired at every public boundary (`SitemapImageSchema`, `SitemapVideoSchema`, `IncrementalSitemapBuilderOptionsSchema`)
|
||||
- [ ] 10.3 New error codes thrown from at least one production path (`IMAGE_VALIDATION_FAILED`, `VIDEO_VALIDATION_FAILED`, `INCREMENTAL_ROLLOVER_FAILED`, `COMPRESSION_FAILED`)
|
||||
- [ ] 10.4 Strict-mode tested: rollover at exactly 50,000 URLs vs 49,999; rollover at exactly 50 MiB vs 50 MiB - 1 byte
|
||||
- [ ] 10.5 OTel counters / histograms emitted (`sitemap.incremental.rollover_count`, `sitemap.compression.bytes_in`, `sitemap.compression.bytes_out`); asserted with a real in-memory `MeterProvider`
|
||||
- [ ] 10.6 Lifecycle dedup: `IncrementalSitemapBuilder.finalize()` is idempotent — calling it twice produces the same index XML and does not double-emit storage writes
|
||||
- [ ] 10.7 No retroactive tick — every box reflects work done in this change
|
||||
|
||||
## 11. Archive
|
||||
|
||||
- [ ] 11.1 Fold spec delta into `openspec/specs/sitemap/spec.md` (creating it if absent)
|
||||
- [ ] 11.2 Move `openspec/changes/sitemap-completion/` → `openspec/archive/<yyyy-mm-dd>-sitemap-completion/`
|
||||
- [ ] 11.3 Refresh `MANIFEST.md`
|
||||
Reference in New Issue
Block a user