How to gtest Linux-PAM client code?

[joseph-reynolds]

I am new to gtest and looking for guidance on how to approach testing my new Linux-PAM client code. This is new C++ code which uses the usual pam_start(), pam_authenticate(), etc. functions (ref: https://linux.die.net/man/3/pam, https://linux.die.net/man/8/pam) together with a new PAM conversation function and a new PAM config file (ref: https://linux.die.net/man/5/pam.d).

Disclaimers: I am not trying to use gtest to test my new PAM config file. I do not need to test a new PAM service module. I am not looking for help using PAM; the code was tested without using gtest, and now I want gtests. Also, I realize that gtest may not be a good fit for Linux-PAM applications.

I looked in this project, github.com/linux-pam, stackoverflow, and elsewhere but did find anyone using gtest with Linux-PAM.

I read the GoogleTest docs and have clear ideas how to proceed with wrapping mocking, and faking. But there is one question about the basic structure of the tests. Should I have

1. One set of tests which calls the client functions, fakes the PAM library functions (pam_start, pam_authenticate, etc.), and really invokes the conversation function, or
2. One set of tests for the client function and another set of tests for the conversation function.

I sketched out the mocks and fakes I would need. Discussion (numbering as above).

1. Given that the client code and its conversation function necessarily go together, faking and mocking the pam library functions makes the most sense to me. The code under test would flow from the client function, through the mocked and faked PAM library, to the conversation function (multiple invocations), and back to the client function. (The faked PAM library would not read the PAM config files; it would simply call malloc and free and the conversation function as called for.)
2. It also makes sense to test each piece individually (the client code and the conversation function). For client functions, mock and fake just enough of the PAM library functions for testing. And for the conversation function, write code to invoke it multiple times as if called from the PAM library functions. But this fails to capture the interaction between the client code and its conversation function, instead relying on tedious fakes.

So I am looking for pointers or advice.