Handling Forbidden Characters in JSON Property Names

JSON (JavaScript Object Notation) has become a ubiquitous data interchange format, used everywhere from API responses to configuration files. While JSON is generally flexible, it does impose some restrictions on property names that can cause confusion and errors for developers.

In this article, we'll explore the rules governing JSON property names, common issues with forbidden characters, and practical strategies for working with challenging property names across different environments.

JSON Property Name Rules and Restrictions

Official JSON Specification

According to the RFC 8259 (the JSON specification), a property name must be a string. The spec states:

While the specification doesn't explicitly forbid specific characters within property names, it does require that property names be valid strings enclosed in double quotes. This has some important implications.

Characters That Must Be Escaped

Within JSON property names, certain characters must be escaped with a backslash:

• \" - Double quote
• \\ - Backslash
• \b - Backspace
• \f - Form feed
• \n - New line
• \r - Carriage return
• \t - Tab

{
  "normal_key": "value",
  "key with\"quotes\"": "value",
  "line\nbreak": "value",
  "tab\tcharacter": "value"
}

Unicode Characters in Property Names

JSON property names can include Unicode characters, either directly or using Unicode escape sequences:

{
  "café": "value with direct Unicode",
  "caf\u00E9": "value with escaped Unicode"
}

Common Problems with JSON Property Names

1. Using Reserved Words or Special Characters

While not forbidden by JSON itself, using characters like dots, spaces, or characters that have special meaning in programming languages can cause issues when working with JSON data:

{
  "first.name": "John",
  "last name": "Doe",
  "user-id": 12345
}

// JavaScript
const user = JSON.parse(jsonString);

// These won't work as expected:
console.log(user.first.name); // Error: name is not defined
console.log(user.last name);  // Syntax error
console.log(user.user-id);    // Will perform subtraction!

2. Starting with Numbers or Special Characters

Property names that start with numbers or special characters can cause issues in some programming languages and frameworks:

{
  "123property": "value",
  "$special": "value",
  "#hashtag": "value"
}

3. Control Characters and Invisible Characters

Control characters (ASCII 0-31) can be particularly problematic, as they may not be visible in editors but can cause parsing errors:

// Invalid JSON - contains ASCII control character in property name
{
  "title\u0007with_bell": "value"
}

4. Empty Property Names

While technically valid in JSON, empty property names can cause issues in many programming environments:

{
  "": "value for empty key"
}

Environment-Specific Considerations

JavaScript/TypeScript

In JavaScript, object properties that contain special characters need to be accessed using bracket notation:

const data = {
  "normal-key": "value1",
  "key.with.dots": "value2",
  "123numeric": "value3"
};

// Won't work:
console.log(data.normal-key);      // Interpreted as data.normal - key
console.log(data.key.with.dots);   // Tries to access nested properties

// Correct approach:
console.log(data["normal-key"]);
console.log(data["key.with.dots"]);
console.log(data["123numeric"]);

Python

In Python, dictionary keys with special characters work similarly to JavaScript:

import json

data = json.loads('{"special.key": "value", "123": "numeric key"}')

# Access with bracket notation
print(data["special.key"])
print(data["123"])

Databases

When storing JSON in databases, property names may face additional restrictions:

• MongoDB allows almost any character in field names but reserves field names starting with $ and containing . for special operations
• PostgreSQL's JSONB type allows most characters but accessing them may require special handling in SQL queries
• SQL Server's JSON functions may have limitations when property names contain certain characters

Strategies for Handling Problematic Property Names

1. Use Safe Naming Conventions

The simplest approach is to avoid problematic characters altogether by following safe naming conventions:

• Use only letters, numbers, and underscores
• Start property names with a letter
• Avoid spaces and special characters
• Use consistent conventions (camelCase, snake_case, etc.)

2. Property Name Transformation

When working with external systems that produce problematic property names, transform them to safer alternatives:

// JavaScript example: Transform problematic keys
function transformKeys(obj) {
  if (typeof obj !== 'object' || obj === null) return obj;
  
  if (Array.isArray(obj)) {
    return obj.map(item => transformKeys(item));
  }
  
  return Object.entries(obj).reduce((result, [key, value]) => {
    // Replace dots, spaces, etc. with underscores
    const safeKey = key.replace(/[.\s-]/g, '_');
    result[safeKey] = transformKeys(value);
    return result;
  }, {});
}

// Usage
const safeData = transformKeys(problematicData);

3. Use Property Mapping

Create explicit mappings between problematic property names and their safe alternatives:

// Define mapping
const propertyMap = {
  'first.name': 'firstName',
  'last name': 'lastName',
  'user-id': 'userId'
};

// Apply mapping
function mapProperties(data, mapping) {
  const result = {};
  
  for (const [originalKey, mappedKey] of Object.entries(mapping)) {
    if (originalKey in data) {
      result[mappedKey] = data[originalKey];
    }
  }
  
  return result;
}

const mappedData = mapProperties(originalData, propertyMap);

4. Custom Accessor Functions

Implement helper functions to safely access properties with problematic names:

// JavaScript example
function getNestedProperty(obj, path) {
  // Handle path as dot notation like "user.personal.first.name"
  return path.split('.').reduce((current, part) => 
    current && current[part] !== undefined ? current[part] : undefined, obj);
}

// Usage
const firstName = getNestedProperty(data, "user.personal.first.name");

JSON Schema Validation for Property Names

When designing APIs or data models, you can use JSON Schema to enforce rules about property names:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "propertyNames": {
    "pattern": "^[a-zA-Z][a-zA-Z0-9_]*$"
  },
  "additionalProperties": {
    "type": "string"
  }
}

Edge Cases and Special Scenarios

1. Reserved Words in Different Languages

Be cautious about using language-specific reserved words as property names:

{
  "class": "premium",    // Reserved in many languages
  "function": "admin",   // JavaScript reserved word
  "import": true,        // Reserved in multiple languages
  "select": "all"        // SQL reserved word
}

2. Numeric Property Names

Numeric keys are valid in JSON but may behave unexpectedly in different environments:

{
  "0": "zero",
  "1": "one",
  "2": "two"
}

// In JavaScript:
const data = JSON.parse(jsonString);
console.log(data[0]); // "zero" - works as expected
console.log(data["0"]); // "zero" - also works

// But in some environments, numeric keys might be treated differently
// or might be automatically sorted in numeric order

3. JSON in URL Parameters

When including JSON in URL parameters, properties with certain characters need special URL encoding:

// Original JSON
{
  "search query": "example term"
}

// URL encoded (issues with both the property name and value)
https://api.example.com/search?filter=%7B%22search%20query%22%3A%22example%20term%22%7D

Conclusion

While JSON property names offer considerable flexibility, they can create challenges when they contain special characters, spaces, or other problematic elements. By understanding the rules and limitations, and by implementing appropriate strategies for handling such property names, you can work effectively with JSON data from various sources.

When you have control over the JSON structure, the best approach is to follow safe naming conventions that work well across all environments. When working with external data sources, implement transformation or mapping strategies to convert problematic property names into safer alternatives.

Remember that the goal is to create JSON that is not only syntactically valid but also practical for use in your specific application context. Tools like Offline Tools' JSON Formatter can help validate your JSON and identify potential issues with property names before they cause problems in your application.