JSON Property Inheritance in Multi-Stage Deployments

Managing application configuration across different deployment environments (like development, staging, and production) is a common challenge. Databases URLs, API keys, feature flags, logging levels, and external service endpoints often vary between stages. While environment variables are a popular method, using configuration files, particularly JSON, offers structure and version control benefits.

When using JSON for configuration, you often find yourself repeating many settings that are common across all environments, with only a few properties differing. This is where the concept of JSON property inheritance becomes incredibly useful.

The Problem with Repetitive Configuration

Imagine you have a configuration file for your application:

{
  "app": {
    "name": "My App (Dev)",
    "version": "1.0.0"
  },
  "database": {
    "host": "localhost",
    "port": 5432,
    "user": "dev_user",
    "password": "dev_password",
    "database": "myapp_dev"
  },
  "api": {
    "baseUrl": "http://localhost:3000/api/v1",
    "timeout": 5000
  },
  "logging": {
    "level": "debug",
    "format": "json"
  },
  "features": {
    "betaEnabled": true,
    "darkMode": true
  }
}

{
  "app": {
    "name": "My App",
    "version": "1.0.0"
  },
  "database": {
    "host": "prod.mydb.com",
    "port": 5432,
    "user": "prod_user",
    "password": "PROD_SECURE_PASSWORD",
    "database": "myapp_prod"
  },
  "api": {
    "baseUrl": "https://api.myapp.com/v1",
    "timeout": 10000
  },
  "logging": {
    "level": "info",
    "format": "json"
  },
  "features": {
    "betaEnabled": false,
    "darkMode": true
  }
}

Notice the significant overlap between `dev.json` and `prod.json`. The `app.version`, `database.port`, `logging.format`, and `features.darkMode` are the same. If you need to update `app.version` or add a new common setting, you have to do it in multiple files. This repetition violates the DRY (Don't Repeat Yourself) principle, increases the risk of errors (forgetting to update one file), and makes configuration harder to maintain.

What is JSON Property Inheritance?

JSON property inheritance, in this context, is a strategy where you define a base configuration and then create environment-specific configuration files that "inherit" from the base, only overriding or adding the properties that differ. This is typically achieved by merging JSON objects.

The core idea is a layered approach:

1. Base Configuration: Contains all the common settings shared across all environments.
2. Environment-Specific Configuration: Contains only the settings that are different or new for a particular environment.
3. Merging Process: At application startup or build time, the environment-specific configuration is merged on top of the base configuration. Properties in the environment file overwrite properties in the base file if they exist in both. New properties in the environment file are added.

Implementing Inheritance with Layered Files

Let's refactor the previous example using a base file and environment-specific overrides.

{
  "app": {
    "name": "My App",
    "version": "1.0.0"
  },
  "database": {
    "port": 5432,
    "user": "default_user",
    "password": "default_password"
  },
  "api": {
    "timeout": 10000
  },
  "logging": {
    "level": "info",
    "format": "json"
  },
  "features": {
    "betaEnabled": false,
    "darkMode": true,
    "newFeature": false
  },
  "commonSetting": "This is a setting common to all environments"
}

{
  "app": {
    "name": "My App (Dev)"
  },
  "database": {
    "host": "localhost",
    "user": "dev_user",
    "password": "dev_password",
    "database": "myapp_dev"
  },
  "api": {
    "baseUrl": "http://localhost:3000/api/v1",
    "timeout": 5000 // Override timeout
  },
  "logging": {
    "level": "debug" // Override logging level
  },
  "features": {
    "betaEnabled": true // Override betaEnabled
  },
  "devSpecificSetting": "This setting only exists in development"
}

{
  "database": {
    "host": "prod.mydb.com",
    "user": "prod_user",
    "password": "PROD_SECURE_PASSWORD",
    "database": "myapp_prod"
  },
  "api": {
    "baseUrl": "https://api.myapp.com/v1"
    // Use the base timeout of 10000
  },
  // Use the base logging level of info
  "features": {
    "newFeature": true // Enable new feature in production
  }
  // Use the base commonSetting
}

To get the final configuration for an environment, you perform a deep merge of the base JSON object with the environment-specific JSON object. A deep merge means that nested objects are also merged property by property, rather than the environment object simply replacing the base object entirely.

For example, merging `dev.json` onto `base.json` would result in:

{
  "app": {
    "name": "My App (Dev)", // Overridden
    "version": "1.0.0"      // Inherited from base
  },
  "database": {
    "host": "localhost",     // Added
    "port": 5432,            // Inherited from base
    "user": "dev_user",      // Overridden
    "password": "dev_password", // Overridden
    "database": "myapp_dev"  // Added
  },
  "api": {
    "baseUrl": "http://localhost:3000/api/v1", // Added
    "timeout": 5000          // Overridden
  },
  "logging": {
    "level": "debug",        // Overridden
    "format": "json"         // Inherited from base
  },
  "features": {
    "betaEnabled": true,     // Overridden
    "darkMode": true,        // Inherited from base
    "newFeature": false      // Inherited from base
  },
  "commonSetting": "This is a setting common to all environments", // Inherited from base
  "devSpecificSetting": "This setting only exists in development"   // Added
}

Handling Different Data Types During Merge

Merging objects is straightforward, but you need to define how other data types are handled:

• Primitive Types (strings, numbers, booleans, null): The value in the environment file simply replaces the value in the base file.
• Objects: Deep merge is usually the desired behavior, merging properties from the environment object into the base object.
• Arrays: This is where it gets tricky and depends on your needs. Common strategies include:
  • Replace: The array in the environment file completely replaces the array in the base file.
  • Concatenate: Elements from the environment array are appended to the elements from the base array.
  • Merge by Key: If the array contains objects with unique identifiers (like an `id` or `name` property), merge objects with matching keys and add non-matching ones. This is complex and less common for simple config.

  Typically, the "replace" strategy for arrays is the simplest and often sufficient for configuration files.

Tooling and Libraries

Implementing the merge logic yourself is possible, but using a well-tested library is often better. Many programming languages and ecosystems have libraries for deep merging objects.

For example, in Node.js/TypeScript, you might use libraries like `lodash.merge` or `deepmerge`.

// Using a hypothetical deepMerge function

import deepMerge from 'deepmerge'; // Example library import

const baseConfig = require('./base.json');
const envConfig = require('./${NODE_ENV}.json'); // Dynamically load env file

const finalConfig = deepMerge(baseConfig, envConfig);

// finalConfig is now the merged object ready to use
console.log(finalConfig);

Some configuration management libraries might also provide built-in support for layering or inheritance from multiple files or sources.

Advantages

• DRY (Don't Repeat Yourself): Reduces redundancy by keeping common settings in one place.
• Maintainability: Easier to update common settings or add new ones across all environments.
• Readability: Environment-specific files are smaller and show only the differences, making it clear what changes between stages.
• Reduced Errors: Less copy-pasting means fewer chances of introducing typos or inconsistencies.
• Version Control Friendly: Changes to configuration are explicit in smaller, difference-focused files.

Disadvantages and Considerations

• Requires Merge Logic: You need a process (tooling or code) to perform the merge before the configuration can be used.
• Complexity with Arrays: Deciding and implementing the array merge strategy can add complexity.
• Debugging: Sometimes tracking down the final value of a deeply nested property requires looking at multiple files and understanding the merge order.
• Sensitive Data: Passwords and API keys should ideally not be stored directly in these files, especially if they are committed to public repositories. Environment variables or secrets management systems are better suited for sensitive data, potentially overriding values from the JSON merge.

Integration with Deployment Pipelines

In a multi-stage deployment pipeline, the merging process typically happens as part of the build or deployment step for each environment. The build script determines the target environment (e.g., based on a `NODE_ENV` variable) and then performs the merge, creating a final, flattened configuration object that the application can easily load.

This ensures that the application code itself doesn't need complex logic to figure out its configuration; it just loads the already merged, environment-specific config file.

Conclusion

JSON property inheritance using layered configuration files and a merging process is an effective pattern for managing differences across development, staging, and production environments. It promotes the DRY principle, improves maintainability, and makes configuration easier to understand by highlighting only the environment-specific overrides. While it introduces a merging step into your workflow, the benefits in terms of reduced errors and improved clarity often outweigh this overhead, especially for applications with non-trivial configuration. Remember to combine this technique with secure practices for handling sensitive information.