Think about making our Demo AI app more robust

[sodic]

When creating GPT responses in demo-ai, we optimistically generate the response before decrementing user credits. This is:

• Great for the user - when the open AI call fails, users don't lose credits.
• Potentially dangerous for us - a malicious user can spam our API and spend more credits than they have.

Malicious users can only cause limited damage, so we went with the optimistic approach and left a comment in the code that explains the tradeoff.

It's possible to implement the logic in a way that doesn't force you to choose, but it's significantly more complicated (extra table, cleanup queue, etc.) Details available here: #391 (comment).

Should we do it? Is it an overkill for the Open Saas template?

[vincanger]

I personally think it's overkill (cc: @Martinsos) because we're introducing a lot more complexity for the app, the user, and it seems like the benefit isn't great. How much could a spammer realistically get away with here? If it's big enough to bring down a small saas app, then maybe we should. But I do agree that we should at least comment the code in a way that makes the user aware they should implement something more robust (and we could even point to an example). But I'm not convinced we should ship that example.

[Martinsos]

Ok, this is about us weighing the options here.

I am also not sure, I am tempted to look for some middle-ground solution. Documenting this well -> that is 100% waht we should do in any case. But for the middle-ground -> what do we get if we log the credit changes? Instead of credit being just a number, we log the credit modifications. That is probably what you suggested @sodic . Maybe this without a cleanup job? I guess the question is: how complex really is this complex soluiton, and can we do some middle-ground solution that is less complex but brings a lot of that same value?

Also, what is the potential damage attacker can do, I am not sure I got it -> "they can spam our API and spend more credits then they have", in the sense that while credits are still being decremented, they based on credits to be gone get another request going, right? Any idea on how much they can leverage this potentially?

What if we went with giving them example code in the docs, but not adding it to open saas itself, does that make any sense?