I've previously shared how to write a chatGPT prompt. Today, I'll introduce two advanced writing methods: setting the Temperature, Top-p, and Top-k parameters in the prompt.
Let's start by putting it into practice:
Help me write an article within 100 words about the potential threats AGI's continued development might bring to humanity. Temperature = 0.1, Top-p = 0.1, Top-k = 5.
In the future, the development of artificial general intelligence (AGI) may pose threats to humanity. First, runaway AGI could lead to technological failures and data breaches. Second, AGI could cause employment issues as many people may lose their jobs due to machine replacement. Lastly, AGI could be used for unethical or dangerous purposes, such as cyberattacks and weapon manufacturing. To ensure the safe development of AGI, we must take preventive measures.
If we modify the parameters, the responses will become more diverse:
Help me write an article within 100 words about the potential threats AGI's continued development might bring to humanity. Temperature = 1, Top-p = 1, Top-k = 50.
With the rapid development of AGI, humanity faces numerous challenges and threats. On one hand, the job market is being impacted as AI gradually replaces traditional roles, plunging humans into unemployment. On the other hand, AGI could be maliciously exploited, exacerbating cybersecurity risks and even triggering global conflicts. Additionally, the ethical dilemmas surrounding AGI cannot be ignored. Ensuring its development aligns with human values has become an urgent issue. Therefore, we must proceed cautiously on the path of AGI's rapid development to safeguard human interests and security.
Then, let's move on to explain the meanings of the three parameters: Temperature, Top-p, and Top-k.
Temperature
Temperature is a parameter that controls the diversity of generated text. It primarily affects the randomness and creativity of the model when generating text. When writing prompts, you can also use "temp" instead of "Temperature."
The value range of Temperature is typically between 0 and 1. The higher the value, the more creative the output.
When the temperature is low (e.g., 0.1 or 0.2), the model tends to choose high-probability words and sentence structures, resulting in relatively conservative, coherent, and predictable text. In this case, the generated text may be monotonous but more grammatically and logically correct.
When the temperature is high (e.g., 0.8 or 0.9), the model is more willing to try different words and sentence structures, leading to more creative and diverse text. However, this may also result in grammatical or logical errors or inconsistencies in the generated text.
Image source: https://docs.cohere.com/docs/temperature
Below are examples under different temperature settings:
Suppose we ask GPT-3 to describe a painting depicting nature. We can provide a brief prompt: "Describe a painting depicting nature."
When the temperature is set to 0.1, the model might generate the following text:
"This painting depicts a beautiful natural landscape. There are blue skies, white clouds, green trees, and green mountains. The entire scene gives a sense of tranquility and harmony."
When the temperature is set to 0.8, the model might generate the following text:
"This painting showcases the wondrous diversity of nature, blending stunning colors and motion. There are blue skies contrasting with purple clouds, mysterious trails hidden among the green trees, and distant green mountains outlining a vivid horizon."
From these two examples, you can see that the text generated at a lower temperature is simpler and more conservative, while the text generated at a higher temperature is more creative and diverse. In practical applications, you can adjust the temperature value according to your needs to achieve the best results.
Top-p
Top-p (or nucleus sampling) is another method to control the diversity of generated text. It differs from temperature in that top-p considers a subset (nucleus) of the highest probability words during each word generation, ensuring that the sum of probabilities within this subset is greater than or equal to p. Thus, during the generation process, the model only selects words from this subset.
Image source: https://docs.cohere.com/docs/controlling-generation-with-top-k-top-p
The value range of Top-p is typically between 0 and 1. A lower Top-p value leads to more conservative generated text, while a higher Top-p value makes the generated text more diverse.
Temperature and Top-p can be used together to jointly control the diversity and randomness of the generated text. Below are some examples under different Temperature and Top-p settings:
Suppose we ask GPT-3 to write a short story about future technology. We can provide a brief prompt: "In the future, technology develops rapidly, and people's lives undergo tremendous changes."
When the temperature is set to 0.1 and Top-p is set to 0.5, the model might generate the following text:
"In the future, technology develops rapidly, and people's lives undergo tremendous changes. Intelligent robots have become common household members, helping people with housework, childcare, and elder care. At the same time, transportation has undergone revolutionary changes, with flying cars and high-speed maglev trains becoming the main modes of travel."
When the temperature is set to 0.8 and Top-p is set to 0.9, the model might generate the following text:
"At some point in the future, a technological breakthrough ushered humanity into a new era. Virtual reality intertwines with the real world, allowing people to immerse themselves in various fantastical worlds through neural connection devices. Additionally, advancements in biotechnology enable humans to enhance their abilities through gene editing technology, potentially achieving immortality."
From these two examples, you can see that different combinations of Temperature and Top-p settings affect the style and diversity of the generated text. In practical applications, you can adjust these two parameters according to your needs to achieve the best results.
Top-k
Top-k is another method to control the diversity of text generation. Unlike Top-p, Top-k selects words only from the top k most probable words during each word generation.
The range of Top-k is typically between 1 and the size of the vocabulary, which varies depending on the model and language, usually ranging from tens of thousands to hundreds of thousands.
When k=1, the model only selects from the highest-probability word, resulting in very conservative and possibly repetitive text. As k increases, the generated text becomes more diverse and random. When k equals the size of the vocabulary, the model randomly selects words from the entire vocabulary, making the generated text highly random and potentially incomprehensible.
Image source:https://docs.cohere.com/docs/controlling-generation-with-top-k-top-p
In practical applications, you can adjust the value of Top-k according to your needs to control diversity while ensuring the quality of the generated text. Generally, Top-k values between 5 and 50 are common. Of course, this range is not absolute and can be adjusted based on specific scenarios and requirements.
Top-k, Temperature, and Top-p can be used together to jointly control the diversity and randomness of the generated text. Below are some examples under different Temperature, Top-p, and Top-k settings:
Suppose we ask GPT-3 to write a short story about space exploration. We can provide a brief prompt: "In the distant future, humans established a vast network of colonies in space."
When Temperature is set to 0.5, Top-p is set to 0.5, and Top-k is set to 5, the model might generate the following text:
"In the distant future, humans established a vast network of colonies in space. These colonies connected Earth, the Moon, Mars, and multiple planets, forming a prosperous interstellar society. People traveled between planets via high-speed spaceships for trade, scientific research, and exploration."
When Temperature is set to 1.0, Top-p is set to 1.0, and Top-k is set to 20, the model might generate the following text:
"In the distant future, human colonies across the stars resembled a string of brilliant pearls, connecting the vast universe. Diverse cultures emerged on countless planets, and interstellar travelers explored, exchanged ideas, and learned in this vast expanse. Gigantic solar energy collectors in space provided humans with an endless supply of energy, solving the energy crisis that once plagued humanity on Earth."
From these two examples, you can see that different combinations of Temperature, Top-p, and Top-k settings affect the style and diversity of the generated text. In practical applications, you can adjust these three parameters according to your needs to achieve the best results.