SuavePlan/suaveplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
suaveplan/openspec/changes/sitemap-completion/proposal.md
admin fbc70f96e6 Initial Commit
2026-05-27 23:29:53 +01:00

6.7 KiB
Raw Permalink Blame History

Sitemap completion — extensions, incremental builder, gzip compressor

Package: @suaveplan/sitemap Status: PROPOSED Type: feature + spec-conformance

Why

@suaveplan/sitemap (packages/api/sitemap/) currently ships only the core builder, index-builder, parser, and URL utilities. Four upstream surfaces are absent: the Google image-sitemap extension, the Google video-sitemap extension, the incremental on-disk builder, and the gzip compressor. Without them, callers must re-implement Google-spec XML by hand (image / video) or stream URLs through the in-memory builder until they hit Google's 50,000-URL / 50 MiB hard cap. All four legacy modules already exist (packages/universal/sitemap/src/extensions/image-sitemap.ts 111 L; extensions/video-sitemap.ts 182 L; incremental/incremental-builder.ts 213 L; compression/sitemap-compressor.ts 91 L) and pass the genesis quality bar — porting them closes the parity gap.

What changes

@suaveplan/sitemap

  • ADDED: SitemapImageExtension — addImage(urlEntry, image) mutates a SitemapUrl to include a <image:image> block per Google's image-sitemap spec; supports { caption, location, license, title }.
  • ADDED: SitemapVideoExtension — addVideo(urlEntry, video) mutates a SitemapUrl to include a <video:video> block; supports { thumbnail, title, description, contentLoc, playerLoc, duration, expiration, rating, viewCount, publicationDate, familyFriendly, requiresSubscription, uploader, live }.
  • ADDED: IMAGE_NS / VIDEO_NS namespace constants and Zod schemas SitemapImageExtensionSchema, SitemapVideoExtensionSchema.
  • ADDED: IncrementalSitemapBuilder class — append-only builder that streams URLs in batches to disk through an injectable WritableStorageLike adapter; preserves URL order; rolls over to a new sitemap file at 50,000 URLs OR 50 MiB (whichever first); emits a SitemapIndex entry per rollover.
  • ADDED: IncrementalSitemapBuilderOptions interface — outputDir, baseFilename (default sitemap), urlsPerSitemap (default 50_000, max 50_000), maxBytes (default 50 1024 1024), storage (WritableStorageLike), compress (boolean — when true, each rollover output is gzipped).
  • ADDED: compressSitemap(xml: string): Promise<Uint8Array> and compressSitemapIndex(xml: string): Promise<Uint8Array> async helpers — gzip with valid gzip magic bytes (1f 8b), DEFLATE compression level 6 (default), ISO-8859-1 OS field, mtime stamped from injectable Clock.
  • ADDED: decompressGzipSitemap(bytes: Uint8Array): Promise<string> round-trip helper for tests.
  • ADDED: SITEMAP_CODES extensions — IMAGE_VALIDATION_FAILED, VIDEO_VALIDATION_FAILED, INCREMENTAL_ROLLOVER_FAILED, COMPRESSION_FAILED registered via existing defineCodes block.
  • ADDED: subpath exports ./extensions/image-sitemap, ./extensions/video-sitemap, ./incremental/incremental-builder, ./compression/sitemap-compressor.
  • MODIFIED: root src/index.ts re-exports the new extensions, builder, and compressor helpers.

Out of scope

  • News sitemap extension (<news:news> block) — separate change. The Google news-sitemap is a different schema with publication-name / publication-language fields and a strict 30-day TTL window; folding it into this change dilutes review.
  • Mobile sitemap extension — deprecated by Google in 2015; do not port.
  • Streaming gzip via CompressionStream — keep the synchronous fflate path for now; once the gzip surface is stable a follow-up change can swap to a WritableStream pipeline.
  • SitemapBuilder.compress() instance method — out of scope to keep the core builder side-effect free; callers gzip the output explicitly via compressSitemap.

Source mapping

This change derives from ../suaveplan/enterprise/sitemap.md and the upstream package source.

Requirement Source file Source lines Notes
SitemapImageExtension + Zod schema packages/universal/sitemap/src/extensions/image-sitemap.ts L1-L111 Verbatim port; existing SitemapImageSchema in schemas/sitemap.schema.ts reused
SitemapVideoExtension + Zod schema packages/universal/sitemap/src/extensions/video-sitemap.ts L1-L182 Verbatim port; existing SitemapVideoSchema reused
IncrementalSitemapBuilder packages/universal/sitemap/src/incremental/incremental-builder.ts L1-L213 Verbatim port; replace Date.now() and process.env.HOME paths with injectable Clock and WritableStorageLike
compressSitemap / compressSitemapIndex / decompressGzipSitemap packages/universal/sitemap/src/compression/sitemap-compressor.ts L1-L91 Verbatim port

Requirements with no source line

Requirement Justification
Injectable Clock for gzip mtime field AGENTS.md determinism rule (no Date.now() in hot paths). Upstream stamps Date.now(), breaking deterministic test fixtures.
Injectable WritableStorageLike adapter for IncrementalSitemapBuilder AGENTS.md §11 ("shared types belong in @suaveplan/types"). Upstream writes directly via fs.promises.writeFile, coupling the builder to Node.
INCREMENTAL_ROLLOVER_FAILED / COMPRESSION_FAILED error codes AGENTS.md §13 row 3. Upstream throws plain Error.

Verification approach

Each requirement carries a **Verified by:** line in the spec delta.

  • packages/api/sitemap/src/extensions/image-sitemap/image-sitemap.test.ts — covers Image extension scenarios
  • packages/api/sitemap/src/extensions/video-sitemap/video-sitemap.test.ts — covers Video extension scenarios
  • packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.test.ts — covers append, rollover at 50k URLs, rollover at 50 MiB, index entry emission
  • packages/api/sitemap/src/incremental/incremental-builder/incremental-builder.spec-conformance.test.ts — pins injected WritableStorageLike contract and Clock determinism
  • packages/api/sitemap/src/compression/sitemap-compressor/sitemap-compressor.test.ts — covers gzip round-trip, magic-byte assertions, mtime determinism

Impact

Affected dependents

Run bunx turbo ls --filter=...@suaveplan/sitemap to enumerate.

  • No dependents (verified by grep -rl "@suaveplan/sitemap" /home/ubuntu/web/genesis/packages/ --include="package.json" → only packages/api/sitemap/package.json itself).

Pure additive change.

Migration impact

None — additive only. Existing SitemapBuilder, SitemapIndexBuilder, parseSitemap, parseSitemapIndex calls compile unchanged. The SitemapUrl type already carries optional images?: SitemapImage[] and videos?: SitemapVideo[] fields (in schemas/sitemap.schema.ts); the new extensions populate those fields rather than introducing a new shape.

Reference in New Issue View Git Blame Copy Permalink