What are Tokens?
Tokens are units of text used by OpenAI for natural language processing. Tokens can vary in length, ranging from individual characters to entire words or phrases. For example, the sentence "I love using Indeemo!" consists of seven tokens: ["I", "love", "using", "Ind”, “eem”, “o", "!"].
Essentially, a token is a chunk of text that the AI breaks regular words / phrases into in order to analyse data.
Current Token Limits
The reason we count the number of tokens in every Prompt is that OpenAI currently imposes an 32,000 token limit per prompt for their GPT 4 model.
This 32,000 token limit includes both:
1. The Prompt input (the Indeemo captions / notes / transcripts / OCR text)
AND
2. The Result you get back from the AI.
For example, if all of the text in the Responses you submit in your Prompt amounts to 32,000 tokens, you won’t have any tokens available for the AI Result that you request from OpenAI.
However, if your prompt amounts to 2,000 tokens, then you will have 30,000 tokens available for the Result that comes back from OpenAI.
We expect this limit to be removed by OpenAI in the near future, but in the interim, working within the 32,000 token limit may require you to be tactical in how you structure your Prompts and modify what Responses you select.
Please find more on this here: Submitting a prompt