Hmm, would it really be more expensive? vast.ai can be extremely cheap.

But I would also be interested in the topic as a whole. I think you would first have to calculate this very precisely. e.g. what is the scope of an average user request? Not many users will fill the entire context window with every request. If we had or could estimate an average value, we would derive how many tokens/second an economically efficient GPU works with and extrapolate that to the price of 1 million tokens.

But there are other important factors as well:

• Which country is the hardware located in? Electricity prices can also be extremely different from country to country.

• How much did the operator have to pay for all his hardware? As a bulk buyer, you almost always get better prices, regardless of the sector.

• Does he perhaps also operate his own photovoltaic systems and, if so, to what extent?

• It is also important to remember that not every product leads directly to financial profits. If you have enough capital and can afford it over a certain period of time, you may deliberately consider doing a loss-making business in order to eliminate the competition. This way one could hope to gain reach and popularity with customers, who would then buy other products in the future. (See OpenAI and ChatGPT).