Testing Error Recovery in JSON Parsing Components

JSON (JavaScript Object Notation) is a ubiquitous data format for exchanging information. Parsing JSON, the process of converting a JSON string into a usable data structure (like a JavaScript object or array), is a fundamental operation in many applications. While standard libraries provide robust parsers, developers sometimes build custom JSON parsing components, particularly in scenarios requiring specific optimizations, streaming capabilities, or when working with constrained environments.

A critical, yet often overlooked, aspect of building any parser, including a JSON parser, is error handling and recovery. A parser needs to gracefully handle malformed input, report errors accurately, and ideally, attempt to recover from errors to find more errors or provide a partially parsed result where possible.

Why Error Recovery Matters in JSON Parsing

While strict JSON compliance is the goal, real-world scenarios often involve imperfect data:

• Manual Editing: Users might manually edit configuration files or data structures in JSON format and introduce typos.
• Faulty Data Sources: External systems or APIs might occasionally send malformed JSON due to bugs or network issues.
• Streaming/Partial Data: In streaming scenarios, data might be truncated or corrupted mid-stream.
• Developer Tooling: Parsers in linters, formatters, or IDEs benefit greatly from error recovery to provide multiple error messages instead of stopping on the first syntax issue.

A parser that simply stops and throws an error on the very first syntax violation can be frustrating. Effective error recovery aims to continue parsing after an error, allowing for more comprehensive error reporting.

Common JSON Parsing Errors

JSON has a relatively simple grammar, but there are many ways to break its rules:

• Syntax Errors:
  • Missing commas between array elements or object properties.
  • Missing colons between object keys and values.
  • Unclosed brackets [ or braces {.
  • Mismatched quotes or unescaped special characters in strings.
  • Invalid number formats (e.g., leading zeros, missing fractional part).
  • Use of single quotes instead of double quotes for strings or keys.
  • Trailing commas.
• Structural Errors:
  • A key-value pair outside of an object.
  • An extra value after the root element (JSON must have a single root value).
• Lexical Errors:
  • Invalid tokens (e.g., `true` written as `trie`).
  • Unexpected characters.

Strategies for Error Recovery

Different strategies can be employed to handle errors and attempt recovery during parsing. The complexity of implementation varies.

Panic Mode Recovery

This is the simplest strategy. When an error is detected, the parser discards input tokens until it finds a "synchronization token" – a token that is likely to appear after a grammatical construct. For JSON, potential synchronization tokens might be ,, ], or }.

Example: Parsing [1, 2 invalid 3, 4]. The parser reads [, 1, ,, 2, invalid. It sees invalid, which is unexpected after 2. In panic mode, it discards invalid and looks at the next token, 3. Still unexpected (missing comma or end of array). Discard 3. Look at ,. Ah, a comma! This could be a synchronization token. It then expects a value after the comma (which is 4). It might report an error about "unexpected token 'invalid'" and possibly "unexpected token '3'". It might successfully parse [1, 2, 4], but the recovery skipped 3 entirely, which might not be desired.

While easy to implement, panic mode can skip large portions of input, potentially missing subsequent errors or producing a significantly altered parse tree.

// Inside a parsing function like parseArray or parseObject
try {
  // ... parse expected tokens ...
} catch (error) {
  reportError(error); // Report the error at the current position

  // Panic mode recovery attempt: Skip tokens until a potential sync token
  const syncTokens = [TokenType.Comma, TokenType.BracketClose, TokenType.BraceClose, TokenType.EOF];
  while (currentToken && !syncTokens.includes(currentToken.type)) {
    eat(currentToken.type); // Consume the unexpected token
  }
  // After the loop, the parser might be at a potential recovery point
  // or at the end of the input.
}

Phrase-Level Recovery

This strategy attempts to fix the error locally by inserting or deleting a small number of tokens to make the input sequence conform to a valid production rule. This requires more specific logic for different types of expected tokens.

Example: Parsing {"name": "Alice" "age": 30}The parser reads {, "name", :, "Alice". It then expects a , or }. It sees "age". A phrase-level recovery might diagnose "missing comma before 'age'" and conceptually insert a comma, then continue parsing , "age": 30 as a new property.

This can provide better error messages and potentially recover more of the parse structure than panic mode, but it's more complex to design and implement recovery logic for every potential error point in the grammar.

while (currentToken.type === TokenType.String) {
  const key = parseString() as string;
  // Check for missing colon
  if (currentToken.type !== TokenType.Colon) {
    reportError("Missing colon after object key.");
    // Attempt recovery: assume colon was meant to be there, don't eat current token
    // If next token is value, parse it. This is a simplified approach.
    if (isValueStartToken(currentToken.type)) { // isValueStartToken checks if token can start a value
       obj[key] = parseValue(); // Try to parse value without eating colon
    } else {
       // Cannot recover, maybe use panic mode or skip this pair
       reportError("Could not recover from missing colon.");
       // ... further error handling / skipping ...
       break; // Exit loop or try to sync
    }
  } else {
      eat(TokenType.Colon); // Consume the colon
      obj[key] = parseValue(); // Parse the value
  }

  // Check for missing comma or closing brace
  if (currentToken.type === TokenType.Comma) {
    eat(TokenType.Comma);
  } else if (currentToken.type !== TokenType.BraceClose) {
    reportError("Expected comma or closing brace in object.");
    // Attempt recovery: Maybe assume a comma was missing and continue
    // This might require looking ahead or making assumptions.
    // A simple approach might just break or try panic mode here.
    // For example, if next token is a string, assume missing comma and continue loop:
    if (currentToken.type === TokenType.String) {
       reportError("Assuming missing comma.");
       // continue; // loop will check currentToken again
    } else {
       // Cannot recover easily, break or sync
       break;
    }
  }
  // If we reach here and currentToken is BraceClose, the loop condition handles it.
}

Error Productions

This is a more formal approach where the grammar itself is augmented with special "error productions" that explicitly describe common error patterns. The parser, built using a grammar-based tool, recognizes these error patterns and triggers associated recovery actions.

Example Error Production for a missing comma in an object:Object ::= "{" ( String ":" Value ( "," String ":" Value )* | ErrorObjectBody )? "}"ErrorObjectBody ::= String ":" Value // Handles a key-value pair where a comma was expected before it

This method integrates error handling deeply into the parser's structure but requires designing a specific error grammar and often using parser generator tools, which might be overkill for a simple JSON parser.

Testing Error Recovery

Simply checking if the parser throws an error isn't enough when testing error recovery. You need to test:

• Error Reporting: Are errors reported accurately (correct type of error, location - line/column)?
• Recovery Success: Does the parser successfully continue parsing after the error?
• Subsequent Error Detection: Can the parser find multiple errors in a single malformed input?
• Output After Recovery: If the parser produces a partial result, is it meaningful or at least predictable? (This is less common for typical JSON parsers but relevant for linters/formatters).
• No Infinite Loops: Does the parser terminate even on severely malformed input?

Generating Test Cases

Creating test cases for error recovery requires systematically introducing errors into valid JSON structures:

• Single Errors: Introduce one specific type of error at different locations (start, middle, end of arrays, objects, strings).

  Examples of single errors:

  // Missing comma
  {"name": "Alice" "age": 30}
  
  // Unclosed array
  [1, 2, 3
  
  // Invalid value type in array
  ["apple", banana, "cherry"] // 'banana' is not a valid JSON token
  
  // Colon instead of comma
  {"a": 1 : "b": 2}
  
  // Trailing comma
  [1, 2, 3,]
  
  // Extra content after root
  {"data": true} extra

• Multiple Errors: Combine several single errors in one input string. This tests if the recovery from the first error allows the parser to encounter the second.

  Example with multiple errors:

  [ {"a": 1 "b": 2} invalid, 3, ] extra // Errors: missing comma after 1, invalid token 'invalid', trailing comma after 3, extra content after array.

• Edge Cases: Test errors involving empty structures {}, [], deeply nested structures, very long strings, large numbers, etc.
• Invalid Characters/Tokens: Feed the parser input with completely foreign characters or sequences that don't belong in JSON.

Assertions and Verification

For each error test case, you need to assert:

• The parser throws an error (or reports one if using a system that collects errors).
• The error message is informative and indicates the type of error and its location.
• (If applicable) The parser continued and reported other errors, or successfully parsed a subsequent part of the input.
• (If applicable) The resulting data structure matches the expected outcome after recovery (this is often tricky and depends heavily on the recovery strategy).

Using snapshot testing can be helpful for verifying the exact error output (message, location, type) for a large suite of malformed inputs.

Conclusion

Building a JSON parsing component with robust error recovery is significantly more complex than building one that simply fails on the first error. It requires careful design of error handling logic, potentially involving panic mode, phrase-level corrections, or even formal error productions. More importantly, it demands a thorough testing strategy with comprehensive test cases covering various single and multiple error scenarios. By investing time in error recovery and its testing, you can create parsing components that are more resilient, user-friendly (through better error messages), and capable of handling the messy realities of real-world data.