Token rules

• Token cannot be divided into parts
• Tokens must be separated by whitespace
  • unless the language syntax allows them to be adjacent (like () or keywords).

Examples of valid adjacency

if(x==y) { return; } // No space needed between (), {}, or operators

Tokens That Don’t Require Whitespace

These punctuation tokens can appear directly next to other tokens:

• ( ) — parentheses
• { } — braces
• [ ] — brackets
• , ; : — separators
• . -> :: — member access
• Operators (+, -, *, etc.)