diff --git a/.ai-specs/DATA_MODEL_SPEC.md b/.ai-specs/DATA_MODEL_SPEC.md new file mode 100644 index 0000000..33b5296 --- /dev/null +++ b/.ai-specs/DATA_MODEL_SPEC.md @@ -0,0 +1,246 @@ +# Data Model Specification + +## CId (Canonical Identifier) System + +### Format Rules + +CIds are identifiers for saints, feasts, and liturgical elements. They follow these routing rules: + +| CId Pattern | Target File | Examples | Notes | +|-------------|-------------|----------|-------| +| `01XX-01ZZ` | `01.json` | `010101`, `014567` | January saints | +| `02XX-02ZZ` | `02.json` | `023456`, `027890` | February saints | +| ... | ... | ... | ... | +| `12XX-12ZZ` | `12.json` | `123456`, `127890` | December saints | +| Other CIds | `misc.json` | `772426`, `373`, `868` | Non-month-prefixed | + +### Routing Logic +```typescript +function bundleFileForCid(cid: string): string { + return /^(0[1-9]|1[0-2])\d+/.test(cid) + ? `${cid.substring(0, 2)}.json` + : 'misc.json'; +} +``` + +## Menaion Bundle Structure + +### Format Definition +```json +{ + "MM-DD": [ + { "id": "CId" }, + { "id": "CId" } + ] +} +``` + +**Key Points**: +- Each date key: "MM-DD" format (Julian calendar) +- Array of objects with only "id" field +- CIds correspond to actual commemorations on that date +- One per language: `menaion-bundle.json`, `{lang}-menaion-bundle.json` + +### Example +```json +{ + "01-01": [ + { "id": "010101" }, + { "id": "772426" }, + { "id": "010102" } + ] +} +``` + +## Life Entry Structure + +### Format Definition +```json +{ + "CId": { + "name": { + "nominative": "Прп. Сильвестра Печерского (XII)", + "short": "Прп. Сильвестра Печерского", + "genitive": "Прп. Сильвестра Печерского" + }, + "life": { + "id": "minei-0101-772426", + "copyright": "Четьи-Минеи свт. Димитрия Ростовского", + "text": "
Complete life text in HTML...
" + } + } +} +``` + +### Required Fields +- **`CId`**: String key (the canonical identifier) +- **`name.nominative`**: Required field (validated by validate-data.ts) + +### Optional Fields +- **`name.short`**: Short name variant +- **`life.text`**: HTML content (only written when empty) +- **`life.id`**: Source identifier (e.g., "minei-0101-772426") +- **`life.copyright`**: Copyright string + +## Service Template Structure + +### Hierarchical JSON Format +```json +{ + "ROYALHOURS": { + "0": { + "text": "In the name of the Father...", + "type": "0" + } + }, + "MATINS": { + "kathisma": { + "text": "Kathisma content..." + } + } +} +``` + +### Conditional Logic Markers +- **P-prefix nodes**: Optional content that may be included +- **Var/I-prefix nodes**: Conditional selection logic +- **Plain nodes**: Required content always included + +## Chetyi-Minei Source Format + +### Format Definition +```json +{ + "MM-DD": [ + { + "title": "Память святого X", + "html": "Content paragraphs...
" + } + ] +} +``` + +### Key Characteristics +- Generated by `convert-mines.ts` from EPUB source +- Stored in `scripts/output/minei.json` (gitignored, 33MB) +- Titles and HTML are carefully extracted and parsed +- Footnotes collected and appended as "Примечания:" blocks + +## File Organization Rules + +### Lives Files +- **Location**: `static/data/{lang}/lives/` +- **Files**: + - `01.json` through `12.json` (month-prefixed CIds) + - `misc.json` (non-month-prefixed CIds) +- **Routing**: Based on first two digits of CId + +### Menaion Files +- **Location**: `static/data/{lang}/menaion/` +- **Files**: `01-01.json` through `12-31.json` (daily entries) +- **Purpose**: Daily liturgical readings by date + +### Service Templates +- **Location**: `static/data/{lang}/services/` +- **Shared**: `static/data/shared/services/templates/` + +### Calendar Data +- **Location**: `static/data/{lang}/calendar/` and `static/data/shared/calendar/` +- **Purpose**: Movable feast calculations and metadata + +## Data Invariants + +### CId Consistency Rules +- Same CId must have consistent name across all languages +- Bundle CIds must have corresponding lives entries +- CIds that exist in bundle must have lives entries +- CIds with lives entries must exist in bundle for their date + +### File Organization Invariants +- No month file contains CIds with different month prefix +- `misc.json` contains all CIds outside the 01-12 prefix range +- All data files are tracked in `static/data/` + +### Content Validation Invariants +- All lives entries must have `name.nominative` field +- HTML content in `life.text` must be properly escaped +- JSON files must be parseable (no syntax errors) +- File routing must match CId prefixes consistently + +## Common Data Mistakes to Avoid + +### CId Pattern Errors +- ⚠️ Don't create CIds with invalid prefixes +- ⚠️ Don't put CIds in wrong month files +- ⚠️ Don't create duplicate CIds with different meanings + +### JSON Structure Errors +- ⚠️ Don't omit required fields (`name.nominative`) +- ⚠️ Don't nest structures incorrectly (service templates) +- ⚠️ Don't mix CId numbers and strings inconsistently +- ⚠️ Don't leave trailing commas in JSON + +### File Organization Mistakes +- ⚠️ Don't place month-prefixed CIds in misc.json +- ⚠️ Don't create inconsistent JSON indentation +- ⚠️ Don't mix languages in incorrect directories + +## Examples + +### Valid CId Examples +- `010101` → `01.json` (January saint) +- `1253` → `12.json` (December saint) +- `772426` → `misc.json` (non-prefixed saint) + +### Invalid CId Examples +- `00501` → Should not exist (prefix `00` → misc) +- `13xxx` → Should not exist (prefix `13` → misc) + +### File Routing Examples +- CId `1253` routes to `12.json` +- CId `772426` routes to `misc.json` +- CId `010101` routes to `01.json` + +### Bundle Entry Examples +```json +{ + "01-15": [ + { "id": "010201" }, + { "id": "772455" } + ] +} +``` + +### Life Entry Examples +```json +{ + "772426": { + "name": { + "nominative": "Прп. Сильвестра Печерского (XII)", + "short": "Прп. Сильвестра Печерского" + }, + "life": { + "id": "minei-0101-772426", + "copyright": "Четьи-Минеи свт. Димитрия Ростовского", + "text": "Complete life content...
" + } + } +} +``` + +## Current Project State + +### Coverage Statistics +- Total CIds: 3046 +- Lives with text: 1190 (39% coverage) +- Empty lives: 1856 (61% remain) + +### Data Files +- Total validated files: 274 +- Main source files: 6 Lives bundle files (misc, 01-12) +- Service template files: Multiple per language + +### Recent Changes +- Bundle fix: 868 Иаков Боровицký moved from 05-22 to 10-23 +- Enrichment: 64 manual fixes added to enrich-lives.ts +- Coverage increase: 1009→1190 lives with text (191 new) \ No newline at end of file diff --git a/.ai-specs/ENRICHMENT_WORKFLOW.md b/.ai-specs/ENRICHMENT_WORKFLOW.md new file mode 100644 index 0000000..27d46d5 --- /dev/null +++ b/.ai-specs/ENRICHMENT_WORKFLOW.md @@ -0,0 +1,257 @@ +# Enrichment Workflow for AI Agents + +## Quick Start Guide + +Run enrichment in safe mode first: +```bash +# Dry run to see matches +npx tsx scripts/enrich-lives.ts + +# Write changes when confident +npx tsx scripts/enrich-lates.ts --write +``` + +## Common Enrichment Mistakes + +### Auto-Matching Pitfalls +- ⚠️ Don't ignore low scores when a clear match exists +- ⚠️ Don't skip review phase entirely — need to examine unmatched entries carefully +- ⚠️ Don't assume auto means "correct" — scores ≥0.6 can still be wrong-saint +- ⚠️ Don't overlook `already-filled` status — some matches hit CIds already filled + +### Manual Fixes Errors +- ⚠️ Don't create substrings that match multiple source titles on same date +- ⚠️ Don't use substrings with spaces that appear in multiple contexts +- ⚠️ Don't forget about OCR artifacts in source titles (Latin `a`, `[1]` markers) +- ⚠️ Don't mismatch case forms: account for case differences in substrings + +### Collision Issues +- ⚠️ Don't expect first chapter to have a heading (this is by design) +- ⚠️ Don't count chapters by counting `` tags alone (verify by checking content) +- ⚠️ Don't append to CIds that already have life.text without checking collision behavior +- ⚠️ Don't assume all multi-source CIds follow same pattern + +### Validation Skipping +- ⚠️ Don't skip `npx tsx scripts/validate-data.ts` — it catches real errors +- ⚠️ Don't skip `npx tsc --noEmit` — type errors indicate real problems +- ⚠️ Don't skip `npm run build` — build failures need investigation +- ⚠️ Don't assume "all files valid" without actually running it + +## Decision Points for AI Agents + +### When to Add MANUAL_FIXES +- When score is 0.35-0.6 and you can identify the correct target +- When source title contains distinctive substring not in other titles on same date +- When auto-matcher picks wrong saint (e.g., different homonym on same date) +- When target CId exists but is listed on wrong calendar date (bundle bug) +- When narrative maps to known CId but has empty life (can be enriched) + +### When to Leave Unmatched +- When source is a narrative or homily without saint association +- When no appropriate CId exists in the bundle for that saint +- When target CId already has life (no data improvement possible) +- When cross-date narratives have no clear target CId + +### When to Use `--month` Flag +- When working on enrichment for a specific period only +- When testing changes without affecting entire dataset +- When troubleshooting issues in a particular month + +## Quality Checkpoints + +### After Manual Fixes Addition +- Verify substring uniqueness per date (use verification script) +- Check that target CIds exist and are empty +- Re-run dry run to verify improvement +- Check for potential new conflicts with existing fixes + +### After Writing Changes +- Verify coverage increased appropriately +- Check that collision targets have correct chapter count +- Confirm only intended files were written +- Verify no unintended side effects on other files + +### Before Committing +- Run full validation: `npx tsx scripts/validate-data.ts` +- Type check: `npx tsc --noEmit` +- Build verify: `npm run build` +- Check git status to confirm intended files staged + +## Troubleshooting Common Issues + +### Unexpected Unmatched Entries +**Problem**: Good match exists but score < 0.35 +- **Solution**: Add MANUAL_FIX with distinctive substring +- **Check**: Verify target CId exists and is empty +- **Alternative**: Check if bundle has correct CIds on that date + +### No Auto Matches on Date +**Problem**: All entries on a date show unmatched +- **Solution**: Check menaion-bundle.json for correct CIds on that date +- **Alternative**: May be narrative without target CId → leave unmatched + +### Target Already Filled +**Problem**: Manual fix shows `already-filled` in report +- **Solution**: Fix targeting or remove fix (no data change) +- **Check**: Verify CId already has life.text (no improvement possible) + +### Wrong Match Applied +**Problem**: Good score applied to wrong CId +- **Solution**: Add MANUAL_FIX with more distinctive substring +- **Verify**: Check score distribution for potential collisions + +### Collision Issues +- ⚠️ **First chapter unheaded**: This is expected behavior by design +- ⚠️ **Expected behavior**: First source writes without heading, subsequent sources append +- **Check**: Verify total chapters match expected count for CId + +### Strange JSON Syntax Errors +- **Problem**: Parse errors in data files after modification +- **Solution**: Check JSON syntax, fix trailing commas, quote issues +- **Verify**: Run `validate-data.ts` to get specific error location + +### Build Size Changes +- **Problem**: Unexpected file size increases/decreases +- **Solution**: Check for malformed content, duplicate data, formatting changes +- **Check**: Compare before/after file sizes in git diff + +## Verification Commands + +### Coverage Tracking +```bash +# Check coverage statistics +python3 -c " +import json, glob, os +total=has=0 +for fp in glob.glob('static/data/ru/lives/*.json'): + data=json.load(open(fp)) + for cid,entry in data.items(): + total+=1 + if isinstance(entry.get('life'), dict) and entry.get('life',{}).get('text'): + has+=1 +print(f'Coverage: {has}/{total} ({has*100//total}%) lives with text') +" +``` + +### Collision Chapter Count Check +```bash +# Check collision targets have correct chapter counts +python3 -c " +import json, re +misc = json.load(open('static/data/ru/lives/misc.json')) +for cid in ['493', '1674', '257401']: # Known collision targets + text = misc[cid]['life']['text'] + headings = re.findall(r'([^<]+)', text) + print(f'{cid}: {len(headings)} chapters') +" +``` + +### Find Specific Pattern in Data +```bash +npx tsx scripts/tools/search-texts.ts "Захарии" --lang cu --type menaion +``` + +### Get Statistics by Language +```bash +npx tsx scripts/tools/stats.ts --lang ru +``` + +## Recent Context Updates + +### Known Fixed Issues +- 868 Иаков Боровицký moved from 05-22 to 10-23 (bundle mis-mapping correction) +- Сретение→373 added (02-02 Сказание about Presentation) +- Захарий Печерский→822 added (03-24 miracle story) +- Анна пророчица→2396 added (08-28 prophetess Anna) + +### Current Unmatched Floor +- 27 entries remain unmatched (all narratives/homilies or already-filled targets) +- These are at the floor for Chetyi-Minei source — no recoverable matches available + +## File Routing Quick Reference + +### Lives File Routing +- Files: `static/data/ru/lives/01.json` through `12.json` for month-prefixed CIds +- File: `misc.json` for non-month-prefixed CIds +- Route: `bundleFileForCid(cid: string):` checks `/^(0[1-9]|1[0-2])\d+/.test(cid)` + +### Bundle File Organization +- Main bundle: `static/data/cu/menaion-bundle.json` (authoritative calendar) +- Language overrides: `{lang}-menaion-bundle.json` +- Format: Date keys (MM-DD) with arrays of CId objects + +## Next Steps After Completing Enrichment + +### Quality Assurance +- Verify collision targets have correct chapter counts +- Check that file sizes are reasonable after writing +- Confirm coverage increased as expected + +### Documentation Updates +- Update DATA_FORMAT.md or DATA_RUS.md if schema changed +- Update coverage statistics in relevant documentation +- Add new manual fixes to enrichment documentation if needed + +### Commit Sequence +```bash +# 1. Validate, typecheck, build all passing +npx tsx scripts/validate-data.ts && npx tsc --noEmit && npm run build + +# 2. Stage all changes +git add scripts/enrich-lives.ts scripts/output/lives-match-report.json static/data/ru/lives/ + +# 3. Commit with description +git commit -m "enrich [N] more ru lives: [description]" +``` + +## Coverage Tracking + +### Current Status +- Total CIds: 3046 +- Lives with text: 1190 (enriched from Chetyi-Minei) +- Unmatched minei chapters: 27 (floor: narratives, no-target, already-filled) +- Files validated: 274 files (all passing) + +### Recent Enrichment Work +- Baseline: 1009 lives with text (from prior work) +- Latest commits added 181 lives (121 + 57 + 3) +- Manual fixes added: 214 total in enrich-lives.ts +- Bundle corrections applied: 868 date fix + +### Progress Tracking +- Review band: 127 entries resolved, 0 remaining +- Unmatched band: 94 → 30 → 27 (final floor) +- Coverage: 1009→1130→1190 lives (181 net increase) + +## Example Workflow for Adding Manual Fixes + +### Step 1: Identify Problematic Match +``` +Date: 03-24 +Source: "Воспоминание о чуде в Печерском monastery" → score 0.12 +Candidate: 722 (Прп. Евстратий Печерский, LIFE) - wrong saint +Target: 822 (Прп. Захарий Печерский, EMPTY) - correct saint +``` + +### Step 2: Create Fix +```typescript +'03-24|воспоминание о чуде': '822' +``` + +### Step 3: Verify Uniqueness +```bash +# Check if substring is unique on date 03-24 +python3 -c " +import json, re +m=json.load(open('scripts/output/minei.json')) +titles=[t['title'] for t in m['03-24']] +cnt=sum(1 for t in titles if 'воспоминание о чуде' in re.sub(r'\s+',' ',t['title']).lower()) +print(f'Count: {cnt}') +" +``` + +### Step 4: Apply and Verify +```bash +npx tsx scripts/enrich-lives.ts # Check new counts +npx tsx scripts/validate-data.ts # Validate all files +``` \ No newline at end of file diff --git a/.ai-specs/PROJECT_CONTEXT.md b/.ai-specs/PROJECT_CONTEXT.md new file mode 100644 index 0000000..07f4b42 --- /dev/null +++ b/.ai-specs/PROJECT_CONTEXT.md @@ -0,0 +1,124 @@ +# Project Context for AI Agents + +## Quick Project Overview + +**Ponomar** is a Progressive Web App for Orthodox Church calendar, liturgical services, and Bible readings. + +- **Technology**: TypeScript + Vite + Tailwind CSS v4 +- **Data**: JSON-based liturgical data organized by language (en, cu, el, fr, zh, ru) +- **Architecture**: Client-side PWA with static JSON data, no backend database +- **Core Functionality**: Calendar display, service assembly, Bible reading, saint lives + +## Key Architectural Patterns + +### Data Organization +- **Language-based structure**: `static/data/{lang}/` (en, cu, el, fr, zh, ru) +- **Bundle system**: menaion-bundles, lives bundles, service templates +- **File routing**: CIds with `01`-`12` prefix → month files, else `misc.json` + +### Core Scripts +- `convert-all.ts` — Master conversion script +- `enrich-lives.ts` — Fuzzy matching and enrichment pipeline +- `validate-data.ts` — Main validator (checks 274 files) +- `convert-minei.ts` — Chetyi-Minei EPUB to JSON conversion +- `migrate-data.ts` — Data normalization + +## Recent Work Context + +### Completed Enrichment +- **Chetyi-Minei import**: 1154 chapters processed, 1190/3046 lives enriched +- **Manual fixes**: 214 manual corrections in `enrich-lives.ts` +- **Bundle fixes**: 868 Иаков Боровицký corrected from 05-22 to 10-23 +- **Coverage**: 39% of lives have texts (source from Chetyi-Minei) + +### Validation Status +- All data files validated: 274 files checked +- TypeScript compiles cleanly +- Build succeeds without errors + +## Critical Invariants + +- **Never overwrite existing `life.text`** — enrichment only fills empty entries +- **File routing is strict**: CId prefix determines file location +- **Bundle CIds must exist**: Menaion CIds require corresponding lives entries +- **Static data is tracked**: `static/data/` in git (except `scripts/output/minei.json`) +- **Validation must pass**: Changes must pass `validate-data.ts`, `tsc --noEemit`, `npm run build` + +## Key Script Quick Reference + +```bash +# Development +npm run dev # Start dev server +npm run build # Production build + +# Data operations +npm run convert # Convert XML to JSON +npm run migrate # Normalize data format + +# Validation +npx tsx scripts/validate-data.ts # Main validator +npx tsc --noEmit # Type checking +npm run build # Build verification + +# Enrichment +npx tsx scripts/enrich-lives.ts # Dry run enrichment +npx tsx scripts/enrich-lives.ts --write # Apply enrichment +``` + +## File Structure Highlights + +``` +src/ +├── core/ # Domain logic (calendar, paschalion, service assembly) +├── ui/ # View components +└── main.ts # Entry point + +scripts/ +├── convert-*.ts # XML→JSON conversion pipeline +├── validate-*.ts # Comprehensive validation suite +└── enrich-lives.ts # Enrichment with manual fixes + +static/data/ +├── cu/ # Church Slavonic (authoritative calendar) +├── ru/ # Russian (enriched lives) +├── en/ # English +└── shared/ # Cross-language resources +``` + +## Important Notes + +### Pipeline Dependencies +- **Chetyi-Minei source**: EPUB conversion requires `convert-minei.ts` +- **Enrichment workflow**: Requires minei.json output + menaion-bundle.json +- **Validation chain**: Each commit must pass validation → typecheck → build + +### File System Constraints +- `scripts/output/minei.json` is gitignored (33MB source file) +- `static/data/` is fully tracked in git +- Lives files are written by month (01.json-12.json) or misc.json + +## Common Tools Reference + +- **Search**: `npx tsx scripts/tools/search-texts.tsПреподобный отец наш Сильвестр...
" + } + } +} +``` +**Structure**: Object keyed by CId (string), each containing: +- `name` object with case variants (`nominative`, `short`, `genitive`, etc.) +- `life` object (optional) with `id`, `copyright`, and `text` fields + +### Calendar Entry +```json +{ + "pascha": { "year": 2025, "date": "04-20" }, + "fixed": [ + { "date": "01-01", "rank": 5, "celebration": "Circumcision" } + ] +} +``` +**Structure**: Contains movable feast calculations and fixed calendar data. + +### Service Template Entry +```json +{ + "ROYALHOURS": { + "0": { + "text": "In the name of the Father...", + "type": "0" + } + } +} +``` +**Structure**: Hierarchical service structure with text nodes and conditional logic. + +--- + +## File Routing Rules + +### Lives File Routing +Lives bundles are routed based on CId prefix: + +| CId Pattern | Target File | Examples | +|-------------|--------------|----------| +| `01XX-01ZZ` | `01.json` | `010101`, `014567` | +| `02XX-02ZZ` | `02.json` | `023456`, `027890` | +| ... | ... | ... | +| `12XX-12ZZ` | `12.json` | `123456`, `127890` | +| Other CIds | `misc.json` | `772426`, `373`, `2396` | + +**Examples:** +- `010101` → `01.json` +- `1253` → `12.json` +- `772426` → `misc.json` +- `373` → `misc.json` + +### Bundle File Naming +- Main bundle: `menaion-bundle.json` (per language) +- Language overrides: `{lang}-menaion-bundle.json` +- No prefix-based routing for bundles (single file per language) + +--- + +## Data Components Breakdown + +### Menaion Bundles +**Purpose**: Authoritative date → CId mapping for calendar rendering + +**Structure**: Date-keyed JSON with arrays of CIds +```json +{ + "01-01": [{ "id": "010101" }, { "id": "772426" }], + "01-02": [{ "id": "010201" }, { "id": "772455" }] +} +``` + +**Coverage**: 366 Julian calendar days × multiple entries per day +**File Size**: ~50-100KB per language +**Usage**: Calendar view determines which saints to display on each day + +### Lives Bundles +**Purpose**: Saint biographies and lives for enrichment + +**Structure**: CId-keyed JSON with name variants and optional life text +```json +{ + "772426": { + "name": { + "nominative": "Прп. Сильвестра Печерского (XII)", + "short": "Прп. Сильвестра Печерского" + }, + "life": { + "id": "minei-0101-772426", + "copyright": "Четьи-Минеи свт. Димитрия Ростовского", + "text": "Complete life text in HTML...
" + } + } +} +``` + +**Coverage**: 3046 total CIds, 1190 with life.text enriched +**Routing**: Prefix-based (01.json-12.json + misc.json) +**Usage**: Lives display on saint detail pages + +### Calendar Data +**Purpose**: Movable feast calculations and calendar metadata + +**Components**: +- `shared/calendar/triodion.json` — Pre-Lent through Holy Week +- `shared/calendar/pentecostarion.json` — Pascha through All Saints +- `{lang}/calendar/` — Language-specific calendar data + +**Key Features**: +- Paschalion calculations +- Movable feast dating +- Fasting period definitions +- Holiday and commemoration metadata + +### Service Templates +**Purpose**: Structured liturgical service data with conditional logic + +**Organization**: +``` +services/ +├── octoecheos/ # Eight tones × days +├── triodion/ # 37 Lenten sections +├── pentecostarion/ # 50 Paschal days +├── menaion/ # 366 daily entries + 34 feasts +├── canons/ # Paraclete, Octoechos, Great Canon +└── shared/templates/ # Reusable service components +``` + +**Structure**: Hierarchical JSON with text nodes and conditional prefixes +- **P-prefix nodes**: Optional content that may be included +- **Var/I-prefix nodes**: Conditional selection logic +- **Plain nodes**: Required content always included + +### Bible Metadata +**Purpose**: Bible version mappings and reading pericopes + +**Structure**: Version identifiers and pericope mappings across 13 Bible versions +- Church Slavonic +- Russian Synodal +- KJV +- Brenton (Septuagint) +- Vulgate +- Other versions + +### Prayer Rules & Commands +**Purpose**: Daily cycle structures and prayer rule definitions + +**Components**: +- **Prayer Rules**: Daily reading schedules +- **Commands**: Service command structures +- **Horologion**: Hours, Kathismata, Compline, etc. + +### Fasting Rules +**Purpose**: Period definitions and fasting exceptions + +**Structure**: JSON with fasting periods, exceptions, and detailed rules +- Pre-Lent periods +- Great Lent +- Fast-free periods +- Fish/strict fasting days + +--- + +## Data Management Tools + +### Conversion Pipeline (`scripts/convert-*.ts`) + +#### Master Script +**`convert-all.ts`** — Orchestrates all conversion steps +```bash +npm run convert +``` + +Converts XML data from Java Ponomar sources to JSON format. + +#### Individual Converters + +**`convert-lives.ts`** — Saint lives XML → JSON conversion +- Input: `ponomar/Ponomar/languages/{lang}/xml/lives/*.xml` +- Output: `{lang}/lives/{01.json|misc.json}` bundles +- Features: + - XML parsing with attribute extraction + - Name case normalization + - Grammar data extraction + - Service data extraction + - Scripture reference extraction + - Bundle generation by month prefix + +**`convert-calendar.ts`** — Calendar metadata conversion +- Input: XML calendar data +- Output: JSON calendar structures +- Features: + - Movable feast calculations + - Fixed feast mappings + - Period definitions + +**`convert-services.ts`** — Service template conversion +- Input: XML service definitions +- Output: JSON service templates +- Features: + - Hierarchical structure preservation + - Conditional logic extraction + - Text node normalization + +**`convert-bible-meta.ts`** — Bible metadata extraction +- Input: XML bible metadata +- Output: JSON version mappings and pericopes + +**`convert-fasting.ts`** — Fasting rules conversion +- Input: XML fasting definitions +- Output: JSON fasting rules and periods + +**`convert-commands.ts`** — Command structure conversion +- Input: XML command definitions +- Output: JSON command structures + +**`convert-service-rules.ts`** — Service rule extraction +- Input: XML service rule definitions +- Output: JSON service rules + +### Migration Tools (`scripts/migrate-data.ts`) + +**Purpose**: Normalize data format across versions and languages + +**Operations:** +1. Strip `@_` prefixes from lives name keys (all languages) +2. Remove `SId` from menaion entries (keep id = CId) (all languages) +3. Rename service directories to lowercase (cu, el, fr, zh/Hans, zh/Hant, ru) +4. Flatten Octoecheos structure (cu, fr, zh/Hans, zh/Hant) +5. Convert `TEXT` → `text` in command files +6. Move shared services into `templates/` subdirectory +7. Normalize name key casing (Nominative → nominative, Short → short) +8. Migrate merged calendar files (triodion.json, pentecostarion.json) + +**Usage:** +```bash +npm run migrate # Apply migrations +npm run migrate:dry # Preview changes +``` + +### Validation Suite (`scripts/validate-*.ts`) + +#### Main Validator +**`validate-data.ts`** — Comprehensive data validation +```bash +npx tsx scripts/validate-data.ts +``` +**Validates:** +- Life entry structure (required fields) +- Lives bundle format (CId keys, name objects) +- Calendar data integrity +- Service template structure +- Prayer format compliance +- Command node structure +- Commemoration entries +- Fasting rule definitions + +**Output:** File count, errors, warnings + +#### Specialized Validators +**`validate-i18n.ts`** — Internationalization completeness +```bash +npx tsx scripts/validate-i18n.ts +``` +Checks coverage across languages (en/ru/cu). + +**`validate-tabs.ts`** — Service tab/template consistency +```bash +npx tsx scripts/validate-tabs.ts +``` +Validates that service tabs match template structures. + +**`validate-menaion.ts`** — Menaion data validation +```bash +npx tsx scripts/validate-menaion.ts +``` +Validates menaion entries against calendar rules. + +**`validate-prayer-rule.ts`** — Prayer rule structure validation +```bash +npx tsx scripts/validate-prayer-rule.ts +``` +Validates prayer rule format and completeness. + +**Period-specific Validators:** +- `validate-triodion.ts` — Triodion period validation +- `validate-pentecostarion.ts` — Pentecostarion validation +- `validate-holy-week.ts` — Holy Week validation +- `validate-great-canon.ts` — Great Canon validation +- `validate-great-compline.ts` — Great Compline validation +- `validate-libcs.ts` — Library content validation + +### Utility Tools (`scripts/tools/`) + +**`search-texts.ts`** — Full-text search across data files +```bash +npx tsx scripts/tools/search-texts.ts