Access Violation - There is deep copy?

[frbitten]

Hi
I use the library in a very complex application with numerous threads and json assignment operations.
I did all the mutex control to avoid competition between threads when accessing the same json object. But I'm getting lots of "access violation" at random times during execution.

Where I investigated it appears that the memory that some internal pointers of the JSON object point to has been removed. That's why I suspect that when making an assignment (operator =) the values ​​are not set in a completely new object but rather some pointers to the data are passed on. Therefore, when removing the original object, the pointer memory is removed.

Is there any way I can do a "deep copy" to ensure everything has been duplicated and be able to test whether this solves my problem and confirms my theory?

Thanks

[nlohmann]

Can you provide an example? I am not sure what to comment on this. We run the tests with ASAN, so I don't believe there are dangling pointers.

[frbitten]

I can't emulate it in a simple example. The system is very complex with more than 15 threads and a lot of exchange of json objects, readings and writings.
So I don't consider it a bug, it's just a suspicion. Because I have a lock on all writing and reading and I still have problems.

That's why the question about a deep copy, at the moment I changed the code to do a dump and then a parse (I didn't want this option due to performance). I will continue testing in this manner to confirm the suspicion. Sometimes you need to run it for a few days for the problem to happen.

[frbitten]

My suspicion is as follows: I have an object where one of its attributes is json, this object has a GetJson method that returns the json attribute. The object is controlled by mutex, but when returning the json from the method it appears that the returned json and the json of the object point to the same memory area. And I have problems with that. I don't rule out a bug in my software, but as I've been suffering from this problem for some time and have already closed all possible ports, I'm only left with something internal to the json.

[frbitten]

I stopped using direct assignment (method return) and started dumping and parsing again and my problems stopped.
Some pointer in the JSON object is being passed in the assignment, but I couldn't identify which one exactly.

nlohmann::json getValue()
{
    const std::lock_guard<std::mutex> lock(this->varValMutex);
    return nlohmann::json::parse(this->varVal.dump());
}

I used the update method to set the value.

void setValue(nlohmann::json value)
{
    const std::lock_guard<std::mutex> lock(this->varValMutex);
    if (value.empty())
    {
        varVal.clear();
    }
    else
    {
        varVal.update(value);
    }
}

Now as a rule I will avoid using
nlohmann::json json=json2;

[frbitten]

@nlohmann

The problem has reduced a lot with the changes mentioned, but it is still happening.
Here are some prints with the error and the pointer data. If you have any suggestions on how to analyze to find the problem. I don't have time at the moment to study the entire library to be able to analyze a possible cause.

If you notice in the images some internal pointers of the JSON objects are invalid (0xdddddddddd)
I don't have a way to reproduce the problem, it happens randomly during execution and in some cases it takes almost 3 days of uninterrupted execution to happen.

[gregmarr]

0xdd is the pattern used to fill dead memory in a debug build. Something likely deleted the json objects in one thread while another thread was iterating the object tree.

[frbitten]

That's my conclusion too. But how is an internal JSON pointer removed and the rest remains valid? That's why I suspect that some internal pointer is being shared via the assignment operator.

[gregmarr]

It could be use after free or double delete of some other pointer that happened to be pointing to that memory. I think you're going to need to use something like asan or valgrind to find this error.

[frbitten]

The problem is caused by the use in multiple threads of a list of objects that have a JSON object as one of their attributes.
I controlled the threads so they wouldn't access the list at the same time and the problem stopped happening. I still don't understand why the error occurs in the JSON library and not in other parts of the same object.

I'm going to close the issue as it's probably not a library problem.