Are Unknown JSON Keys a Valid Approach to Sending Data?

Question

There a lot's of SO questions asking "how" to get unknown JSON keys from a JSON object, and I've done it plenty of times, but after starting to use JSON schema to start designing API request and responses, I am starting to question if having unknown or even dynamic keys that the JSON consumer has to be smart enough to know how to use.

For example, assume you are designing a simple graphing API for a bar graph. You could tell your consumer that you are just going to send them data in this format:

"data": {
  "Sunday": 1,
  "Monday": 2, 
  "Tuesday": 3,
  "Wednesday": 4,
  "Thursday": 5,
  "Friday": 6,
  "Saturday": 7
}

Technically, this is enough to create a graph where the labels on the x-axis are the keys in this JSON object, and the numbers are the scalars graphed on the y-axis. It's also technically dynamic, as you could create a whole new graph with the same pattern:

"data": {
  "Jack": 1,
  "Jill": 2,
  "Alice" 3
}

While JSON allows for flexibility, I'm beginning to question the usability of this. It feels like the equivalent of treating variable names in a Java class as a way of sending information.

An alternative approach might be to define a datapoint schema or something like this:

"data": [
  {
    "category": "Jack",
    "value": 1
  },
  {
    "category": "Jill",
    "value": 2
  },
  {
    "category": "Alice",
    "value": 3
  }
]

Additionally, this seems more extensible as well, as additional values can be added to the schema. However, it there is probably less marshaling and it's probably a bit easier to just use the unknown keys approach.

It just got me wondering, I have used unknown keys on the past, but they suddenly don't seem like the best approach.

1. Are using unknown keys an intended use case for JSON? Or just a side affect that is being taken advantage of?
2. Are there any best practices that promote or discourage the use of unknown keys?
3. If there are any best practices for using unknown keys, under what conditions are they recommended?

Answer 1

Are using unknown keys an intended use case for JSON? Or just a side affect that is being taken advantage of?

In the case of your first two examples, it's the latter. The data is being put into a JSON object, which is an unordered structure. This lack of order makes these two bits of JSON equivalent, even if the one on the right doesn't lay out the bars on your chart as you intended:

"data": {           "data": {
  "Jack": 1,          "Alice": 3,
  "Jill": 2,          "Jack": 1,
  "Alice" 3           "Jill": 2
}                   }

You could write a JSON parser that will preserve the order of the members, but anything else processing it before you see it isn't likely to do the same. (ECMA 404 says nothing about order or uniqueness, but most parsers represent JSON objects as dictionaries that force this behavior.)

If there are any best practices for using unknown keys, under what conditions are they recommended?

The best practice is to use them as they were intended, which is in situations where you need to look up a value by name:

{
    "ssh": 22,
    "smtp": 25,
    "dns": 53,
    "http": 80
}

The implications of a structure that stores values by key are that the keys are unique and order doesn't matter. If you don't know what the keys are ahead of time, you can still answer questions like "what are the keys?" by iterating over them and, having answered that, use what you learned to fetch the values.

Your third example is the correct way to structure the data. The array imposes order on the bars in your graph. The (unordered) objects inside have members whose order doesn't matter because anything using them will be looking for them by name.

Answer 2

JSON can be used quite well to represent three basic data structures: Arrays, Dictionaries, and C-style structs. Arrays are represented by JSON arrays, Dictionaries and structs are represented by JSON "objects" (the JSON standard calls dictionaries "objects"). The only difference is that when representing structs, you often have many, many dictionaries with the same set of keys.

Your "alternative approach" just makes matters more difficult. Most languages come with a standard library that makes lookup in an array slow, and lookup in a dictionary fast. So your alternative approach either turns a dictionary into an array, wasting space and more importantly wasting speed, or forces the client developer to extract the dictionary manually from the JSON array (and in that cases, the redundancy cause you trouble - what if your array contains a dictionary with different keys? )

The argument "it's more extensible" is wrong. A dictionary consists of key-value pairs. The values can be dictionaries (this time dictionaries containing structs).