Losing precision when reading json file

I have thousands of json files that I would like to add a new field name and values to without altering the precision of values that are already in the file. Currently I have the json files being read, add a new field, and then overwritten - but this method removes trailing zeros. Is there a way to just add a new field to a structure in the json file without reading in the data? That would be so much simpler and eliminate my issue of change in precision.

I saw the JSONLab toolbox on the file exchange that allows you to specify the float format when saving a file, but my issue is that not all of the values have the same precision so hardcoding in a specific format would not be ideal. I saw another potential solution using containers.Map, but unsure how to go about that in my case. Any ideas/suggestions would be helpful.

The original json file looks something like the following:

{

"someField": 1.0,

"timestamp": "Tue Jul 20 17:37:56 2021",

"aStructure": {

"something": 1000.0,

"aSubStructure": {

"test": {

"hello": 1234.5678,

"thanks": "some characters are fun, not all",

"scale": 0.0,

"another": 4e-05

}

}

},

"anotherStructure": {

"someSubField": 1.0,

"nope": "ABC00"

}

}

After running the below (generalized):

for i = 1:length(metaTab.name)
    [fid]=fopen(filename{i}); % filename is a table with thousands of paths to *.json files
    raw = fread(fid,inf); % this seems to be where I lose precision
    str = char(raw');
    fclose(fid);
    old_meta{i} = jsondecode(str);
    
    old_meta{i}.anotherStructure.someNewField = 0.75;
    new_meta{i} = jsonencode(old_meta{i},'PrettyPrint',true);
    
    fid = fopen(filename,'w');
    fprintf(fid, '%s', new_meta{i});
    fclose(fid);
end

The new json file looks like the following:

{

"someField": 1,

"timestamp": "Tue Jul 20 17:37:56 2021",

"aStructure": {

"something": 1000,

"aSubStructure": {

"test": {

"hello": 1234.5678,

"thanks": "some characters are fun, not all",

"scale": 0,

"another": 4e-5

}

}

},

"anotherStructure": {

"someSubField": 1,

"nope": "ABC00",

"someNewField": 0.75

}

}

Thanks!

[Les Beckham]

I don't understand your concern.

For example, the new value of aStructure.something (1000) is exactly the same as the original value (1000.0), numerically. The only number in your original file that has significant digits beyond the decimal point (aStructure.aSubstructure.test.hello) has retained all of those significant digits So, you have the exact same numerical values represented in the new file.

If you could provide an example where you are actually losing precision in the numerical data, perhaps we would see that there is actually a question to be answered here. As it stands, I maintain that there is no problem.

[Nikki]

The zeros after the decimal point are significant. In my scenario, these values have units and the zeros indicate the resolution of which they were measured.

[dpb]

The original file had to have been written to the desired precision specified to have included the precision digits; if it's imperative to repeat that, then you'll have to ascertain how many digits of precision to write when you read the file to reproduce it.

I don't use JSON enough to be at all familiar with it, but the alternative to actually determiningt that precision if it would accept it would be to read/.write the values a strings instead -- then you'd have the specific format.

[Walter Roberson]

https://en.wikipedia.org/wiki/JSON

Numbers in JSON are agnostic with regard to their representation within programming languages. While this allows for numbers of arbitrary precision to be serialized, it may lead to portability issues. For example, since no differentiation is made between integer and floating-point values, some implementations may treat 42, 42.0, and 4.2E+1 as the same number, while others may not. The JSON standard makes no requirements regarding implementation details such as overflow, underflow, loss of precision, rounding, or signed zeros, but it does recommend expecting no more than IEEE 754 binary64 precision for "good interoperability".

Answer 1

[Walter Roberson]

So, if you are treating 10 and 10.0 as being different numbers, then you are relying on implementation dependencies that are outside the scope of JSON encoding, and cannot expect programs to have built-in support for compatibility with your non-standard needs.

You should probably do one of two things:

1. Include a "precision" field for each numeric value that indicates explicitly the number of significant digits; OR
2. Write the numbers as strings.

[Rik]

I was about to post this as the answer myself, so I moved it.

I would suggest the second option. If you don't insert the new fields as text, you will have to write your own JSON parser. That isn't very hard (I should know, I implemented my own), but worth avoiding if you can.