ChatGPT and other Generative AI tools are now changing the world, offering great ways to enhance products and increase productivity. It’s therefore a wise choice to think if such tools could be integrated in your product. If you find a use case for Generative AI use, you will need to use an API like OpenAI’s GPT API, which costs money per token.

I have the feeling that some users may actually use those tools far too much, that could cost you more than other users. The question is, how would you make sure you have a predictable net income, and you don’t make a loss on some users using these systems extensively? Would you put a cap per user or would you try to do something more intelligent like a user-base budget that would dynamically change per user depending on the availability of that budget? Meaning, if 90% of your users are not using the AI tools much, you have plenty of resources to allow those users who do need them to use them with virtually no limit.