Tag: GPT

Trying the Realtime Prompting Guide for GPT-Realtime: Experiments with Vietnamese Voice Input

Posted on by hello@scuti

Try Realtime Prompting Guide for GPT-Realtime

1.Introduction

OpenAI’s Realtime API enables the creation of interactive voice experiences with ultra-low latency. Instead of waiting for a full text input, the model can “listen” to a user while they are still speaking and respond almost instantly. This makes it a powerful foundation for building voice assistants, audio chatbots, automated customer support, or multimodal creative applications.

To get the best results, writing a clear and well-structured prompt is essential. OpenAI published the Realtime Prompting Guide as a playbook for controlling model behavior in spoken conversations.

References:

2.What is GPT-Realtime

GPT-Realtime is a model/API designed to handle continuous audio input and provide rapid responses. Its key features include:

  • Real-time speech-to-text recognition.

  • Robust handling of noisy, cut-off, or unclear audio.

  • Customizable reactions to imperfect audio, such as asking for repetition, clarifying, or continuing in the user’s language.

  • Support for detailed prompting to ensure safe, natural, and reliable responses.

3.Overview of the Prompting Guide

The Realtime Prompting Guide outlines seven best practices for writing system prompts for voice agents:

1. Be precise, avoid conflicts.
Instructions must be specific and consistent. For example, if you say “ask again when unclear,” don’t also instruct the model to “guess when unsure.”

2. Use bullet points instead of paragraphs.
Models handle lists better than long prose.

3. Handle unclear audio.
Explicitly instruct what to do when input is noisy or incomplete: politely ask the user to repeat and only respond when confident.

4. Pin the language when needed.
If you want the entire conversation in one language (e.g., English only), state it clearly. Otherwise, the model may switch to mirror the user.

5. Provide sample phrases.
Include example greetings, clarifications, or closing lines to teach the model your desired style.

6. Avoid robotic repetition.
Encourage varied phrasing for greetings, confirmations, and closings to keep interactions natural.

7. Use capitalization for emphasis.
For example: “IF AUDIO IS UNCLEAR, ASK THE USER TO REPEAT.”

4.Prompt Examples

Sample Prompt A – Avoid conflicts, be clear

SYSTEM: – Always speak clearly and respond in the same language the user is speaking. – If the user’s audio is unclear, noisy, partial, or unintelligible, politely ask them to repeat. – Never assume what was said if you did not hear it clearly. – Use short, varied phrases; avoid repeating the same sentence. – Provide helpful, concise responses.

USER: “Hello, can you help me with my internet issue?”

ASSISTANT: (responds according to the prompt, asks for clarification if needed)

Sample Prompt B – Handling unclear audio

SYSTEM: – If the audio is not clear, ask: “I’m sorry, I didn’t catch that. Could you please repeat?” – Only respond when you are confident you understood the user’s request. – Continue conversation in the same language as the user. – Use bullet points for clarity.

USER: “Um… internet…” (noisy, unclear audio)

ASSISTANT: (follows the system instructions)

Sample Prompt C – Keep a natural style, avoid repetition

SYSTEM: – Do not respond with the same phrase twice. – Vary greetings, confirmations, closings. – Aim for a warm, helpful tone. – If the user ends the conversation, say something like: “If there’s anything else you need, feel free to ask.”

USER: “Thank you, that’s all.”

ASSISTANT: “You’re welcome! Glad I could help. Take care!” (or another variation)

5.Experiments

For my testing, I deliberately used Vietnamese speech to see how the model would react in different situations.

First Test: Speaking Unclear Without a Prompt

To begin, I tested what would happen if I spoke unclearly in Vietnamese without providing any system prompt.

For example, I said:

USER: “Ư… mệnh của mình không tỏa lại” (unclear Vietnamese speech)

The model responded in Indonesian, saying:

ASSISTANT: “Tentu, aku di sini untuk membantu. Coba ceritakan dulu apa yang lagi kamu rasakan atau alami. Kita bisa cari solusinya bareng-bareng.”

This shows that when no system prompt is defined, the model may guess or switch languages unpredictably, instead of asking for clarification.

Second Test: Adding a System Prompt

Next, I added a system prompt to guide the model’s behavior when the audio is unclear:

SYSTEM:
- If the audio is not clear, ask: “I’m sorry, I didn’t catch that. Could you please repeat?”
- Only respond when you are confident you understood the user’s request.
- Continue conversation in the same language as user.
- Use bullet points for clarity.

Then I spoke unclearly in Vietnamese again, for example:

USER: “Um… internet…” (spoken quietly, unclear audio)

This time, the model followed the system instructions and politely asked me to repeat. Sometimes, it even suggested that I try saying a simple test sentence so it could better check whether my voice was coming through clearly.

This shows how a well-written system prompt can prevent the model from making random guesses or switching languages, ensuring a more reliable and natural conversation flow.

Third Test: Singing to the Model

Finally, I experimented by singing in Vietnamese to see how the model would react.

The model listened carefully and was able to understand the lyrics and emotional tone of my singing. However, when I asked it to repeat the lyrics back to me, it refused due to copyright restrictions.

This shows that while GPT-Realtime can analyze and comment on songs — such as summarizing their meaning, describing the mood, or suggesting new lines in a similar style — it cannot reproduce lyrics verbatim. In practice, this means you should not expect the model to sing or echo back copyrighted content.

6.Conclusion

GPT-Realtime provides smooth and natural voice interactions with minimal latency. However, its effectiveness depends heavily on the prompt.

Key takeaways:

  • Always write a clear, bullet-pointed system prompt.
  • Define explicit behavior for unclear audio.
  • Control language use and discourage robotic repetition.
  • Respect copyright limitations: the model will not repeat lyrics verbatim but can summarize or create new content.

The Realtime Prompting Guide is a practical resource for building high-quality voice agents that are both natural and safe.

Installing and Using GPT-OSS 20B Locally with Ollama

Posted on by hello@scuti

In this document, we will explore how to install and run GPT-OSS 20B — a powerful open-weight language model released by OpenAI — locally, with detailed instructions for using it on a Tesla P40 GPU.

1. Quick Introduction to GPT-OSS 20B

  • GPT-OSS 20B is an open-weight language model from OpenAI, released in August 2025—the first since GPT-2—under the Apache 2.0 license, allowing free download, execution, and modification.

  • The model has about 21 billion parameters and can run efficiently on consumer machines with at least 16 GB of RAM or GPU VRAM.

  • GPT-OSS 20B uses a Mixture-of-Experts (MoE) architecture, activating only a subset of parameters (~3.6B) at each step, saving resources and energy.

  • The model supports chain-of-thought reasoning, enabling it to understand and explain reasoning processes step by step.

2. Hardware & Software Preparation

Hardware requirements:

  • RAM or VRAM: minimum 16 GB (can be system RAM or GPU VRAM).

  • Storage: around 12–20 GB for the model and data.

  • Operating system: macOS 11+, Windows, or Ubuntu are supported.

  • GPU (if available): Nvidia or AMD for acceleration. Without a GPU, the model still runs on CPU but very slowly.

Software options:

  • Ollama: the simplest method; quick installation with a convenient CLI.

  • LM Studio: a graphical interface, suitable for beginners.

  • Transformers + vLLM (Python): flexible for developers, integrates well into open-source pipelines.

3. How to Run GPT-OSS 20B with Ollama (GPU Tesla P40)

3.1 Goal and Timeline

  • Goal: successfully run GPT-OSS 20B locally using Ollama, leveraging the Tesla P40 GPU (24GB VRAM).

  • Timeline: the first setup takes about 15–20 minutes to download the model. After that, launching the model takes only a few seconds.

3.2 Environment Preparation

  • GPU: Tesla P40 with 24GB VRAM, sufficient for GPT-OSS 20B.

  • NVIDIA Driver: version 525 or higher recommended. In the sample logs, CUDA 12.0 works fine.

  • RAM: minimum 16GB.

  • Storage: at least 20GB free space; the model itself takes ~13GB plus cache.

  • Operating system: Linux (Ubuntu), macOS, or Windows. The following example uses Ubuntu.

3.3 Install Ollama

The fastest way:

curl -fsSL https://ollama.com/install.sh | sh

Or manually (Linux):

curl -LO https://ollama.com/download/ollama-linux-amd64.tgz
sudo tar -C /usr -xzf ollama-linux-amd64.tgz

Start the Ollama service:

OLLAMA_HOST=0.0.0.0:8888 ollama serve

When the log shows listening on [::]:8888, the server is ready.

3.4 Download GPT-OSS 20B

Open a new terminal and run:

OLLAMA_HOST=0.0.0.0:8888 ollama pull gpt-oss:20b

The first download is about 13GB. When the log shows success, the model is ready.

3.5 Run the Model

Start the model and try chatting:

OLLAMA_HOST=0.0.0.0:8888 ollama run gpt-oss:20b

Example:

>>> hi
Hello! 👋 How can I help you today?

3.6 Verify GPU Usage

Run:

nvidia-smi

Result: the Tesla P40 (24GB) consumes around 12–13GB VRAM for the process /usr/bin/ollama. The Ollama log also shows “offloading output layer to GPU” and “llama runner started in 8.05 seconds”, proving the model is running on GPU, not CPU.

3.7 Monitor API and Performance

Ollama exposes a REST API at http://127.0.0.1:8888.
Common endpoints include /api/chat and /api/generate.

Response times:

  • Short prompts: about 2–10 seconds.

  • Long or complex prompts: may take tens of seconds to a few minutes.

4. Conclusion

You have successfully run GPT-OSS 20B on a Tesla P40. The initial model download takes some time, but afterward it launches quickly and runs stably. With 24GB VRAM, the GPU can handle the large model without overload. While long prompts may still be slow, it is fully usable for real-world experiments and local project integration.

© 2015 - 2025 Scuti. All rights reserved.