Lisp Dialects and JSON Interoperability

Lisp, with its iconic S-expressions, might seem worlds apart from JSON, the prevalent data interchange format of the web. Lisp structures are deeply nested lists and atoms, while JSON relies on key-value objects and ordered arrays. However, in practice, Lisp systems frequently need to interact with the outside world, and that world speaks JSON.

Fortunately, the inherent flexibility and powerful data manipulation capabilities of Lisp make integrating with JSON not only possible but often quite elegant. This page explores how various Lisp dialects handle the challenge of parsing JSON into native Lisp data structures and serializing Lisp data into JSON strings.

Why JSON in Lisp?

The primary reason is interoperability. Lisp applications often need to:

• Communicate with web services (REST APIs, etc.) that exchange data in JSON.
• Process configuration files or data dumps stored in JSON format.
• Generate JSON output for client-side applications or other systems.

Mapping JSON data into structures that are idiomatic to the Lisp dialect being used is key to making this interaction seamless.

Mapping JSON to Lisp Data Structures

JSON defines a few fundamental data types: objects, arrays, strings, numbers, booleans (true/false), and null. Lisp dialects have their own equivalents, and the mapping typically follows these conventions:

• JSON Object (``): Maps to a key-value store. Common Lisp often uses association lists (alists) or hash tables. Scheme might use alists or records. Clojure prominently uses maps. The keys (JSON strings) can be mapped to Lisp strings or keywords.
• JSON Array (`[]`): Maps to an ordered collection. Lisp lists or vectors are common representations.
• JSON String (`"..."`): Maps directly to Lisp strings.
• JSON Number (`123`, `4.5`): Maps to Lisp numbers (integers, floats, rationals, etc.).
• JSON Boolean (`true`, `false`): Maps to the dialect's boolean equivalents (e.g., `T`/`NIL` in Common Lisp, `#t`/`#f` in Scheme, `true`/`false` in Clojure).
• JSON Null (`null`): Maps to the dialect's null/nil representation (e.g., `NIL` in Common Lisp, `null` object or specific value in Scheme, `nil` in Clojure).

The choice of mapping (e.g., alist vs. hash table for objects, lists vs. vectors for arrays, string keys vs. keyword keys) often depends on the specific Lisp library being used and performance considerations. Clojure's strong support for persistent maps with keyword keys makes it particularly well-suited for working with JSON, often mapping {"name": "Alice"} to {:name "Alice"}.

Parsing JSON in Different Lisp Dialects

Parsing involves taking a JSON string and converting it into the corresponding Lisp data structures. Most dialects rely on external libraries for robust JSON parsing.

Common Lisp: Using cl-json (Example)

cl-json is a popular library for Common Lisp. By default, it often maps JSON objects to association lists or hash tables and arrays to lists. You can often configure the mapping.

(ql:quickload "cl-json") ; Load the library (if not already loaded)

(let ((json-string "{\"name\": \"Alice\", \"age\": 30, \"isStudent\": false}"))
  (json:decode-json-from-string json-string))

;; Possible output (association list, string keys):
;; (("name" . "Alice") ("age" . 30) ("isStudent" . NIL))

(let ((json-string "[10, \"hello\", true, null]"))
  (json:decode-json-from-string json-string))

;; Possible output (list):
;; (10 "hello" T NIL)

Scheme: Using json-p (Example - Syntax may vary by implementation)

Schemes have various libraries like json-p (part of SRFI 180). Mapping conventions can differ.

; Assuming json-p is available and imported
(import (json-p))

(let ((json-string "{\"city\": \"London\", \"population\": 9000000}"))
  (json-read (open-string-input-port json-string)))

;; Possible output (e.g., Scheme record type or alist):
;; #<json-object ("city" . "London") ("population" . 9000000)>

(let ((json-string "[\"apple\", \"banana\", \"cherry\"]"))
  (json-read (open-string-input-port json-string)))

;; Possible output (e.g., Scheme vector):
;; #("apple" "banana" "cherry")

Clojure: Using clojure.data.json or cheshire (Example)

Clojure's built-in data structures (maps, vectors) align well with JSON. Libraries like clojure.data.json or the faster cheshire are commonly used. Keys are often mapped to Clojure keywords by default.

(require '[clojure.data.json :as json])

(let [json-string "{\"product\": \"Laptop\", \"price\": 1200.50, \"inStock\": true}"]
  (json/read-json json-string))

;; Output (Clojure map with keyword keys by default):
;; {:product "Laptop" :price 1200.5 :inStock true}

(require '[clojure.data.json :as json])

(let [json-string "[{\"id\": 1}, {\"id\": 2}]"]
  (json/read-json json-string))

;; Output (Clojure vector of maps):
;; [{:id 1} {:id 2}]

Serializing Lisp Data to JSON

Serialization is the reverse process: converting Lisp data structures back into a JSON string. Libraries provide functions to handle this, respecting the standard JSON format.

Common Lisp: Using cl-json (Example)

(ql:quickload "cl-json")

(let ((lisp-data '(("user" . "Bob") ("isActive" . T) ("roles" "admin" "editor"))))
  (json:encode-json-to-string lisp-data))

;; Output (string):
;; "{\"user\":\"Bob\",\"isActive\":true,\"roles\":[\"admin\",\"editor\"]}"

Scheme: Using json-p (Example)

(import (json-p))

; Assuming a Scheme record or alist representing a JSON object
(let ((scheme-data '#<json-object ("id" . 101) ("items" . #("itemA" "itemB"))>))
  (with-output-to-string (lambda () (json-write scheme-data))))

;; Output (string):
;; "{\"id\":101,\"items\":[\"itemA\",\"itemB\"]}"

Clojure: Using clojure.data.json or cheshire (Example)

Clojure's maps (often with keyword keys) and vectors map naturally to JSON objects and arrays.

(require '[clojure.data.json :as json])

(let [clj-map {:firstName "Jane" :lastName "Doe" :age 25}]
  (json/write-json clj-map))

;; Output (string - requires capturing *out* or using a writer):
;; "{\"firstName\":\"Jane\",\"lastName\":\"Doe\",\"age\":25}"

(let [clj-vector ["red" "green" "blue"]]
  (json/write-json clj-vector))

;; Output (string):
;; "[\"red\",\"green\",\"blue\"]"

Key Considerations and Challenges

• Data Type Fidelity: Ensuring that Lisp numbers map correctly to JSON numbers (e.g., dealing with integers vs. floats, large numbers).
• Key Representation: Deciding whether JSON string keys should map to Lisp strings or keywords (Clojure often prefers keywords, Common Lisp/Scheme often use strings in alists/hash tables). Consistency is important.
• Null/Nil Mapping: Correctly handling the conversion between JSON's null and the dialect's representation of nothingness (NIL, nil, specific object).
• Error Handling: Robustly handling malformed JSON input during parsing.
• Performance: For high-throughput applications, the performance of the JSON library can be critical. Some libraries are significantly faster than others (e.g., cheshire in Clojure vs. clojure.data.json).
• Streaming vs. In-Memory: For very large JSON documents, streaming parsers that process the data chunk by chunk without loading the entire structure into memory are necessary. Some libraries offer this.

Conclusion

Despite their syntactical differences, Lisp dialects and JSON coexist peacefully thanks to well-developed libraries. These libraries provide the essential functions for transforming JSON data into native Lisp structures and vice-versa. While the specific data structure mapping might vary slightly between dialects and libraries, the underlying principles of parsing and serialization are standard. Leveraging these tools allows Lisp applications to effectively participate in the modern data landscape, exchanging information seamlessly with systems that rely on JSON.