Achei uma IA

Por favor, habilite o JavaScript para usar esta aplicação.

, '', text) return text.strip() json_generator = StructuredOutputGenerator(pipe, tokenizer) print(\"\\n Example 1: Entity Extraction\") print(\"-\" * 50) entity_schema = { \"name\": \"string\", \"type\": \"string (person/company/place)\", \"description\": \"string (1-2 sentences)\", \"key_facts\": [\"list of strings\"] } entity_result = json_generator.generate_json( \"Extract information about: Tesla, Inc.\", entity_schema ) if entity_result[\"success\"]: print(json.dumps(entity_result[\"data\"], indent=2)) else: print(f\"Error: {entity_result['error']}\") print(\"\\n\\n Example 2: Recipe Generation\") print(\"-\" * 50) recipe_schema = { \"name\": \"string\", \"prep_time_minutes\": \"integer\", \"cook_time_minutes\": \"integer\", \"servings\": \"integer\", \"difficulty\": \"string (easy/medium/hard)\", \"ingredients\": [{\"item\": \"string\", \"amount\": \"string\"}], \"steps\": [\"string\"] } recipe_result = json_generator.generate_json( \"Create a simple recipe for chocolate chip cookies\", recipe_schema ) if recipe_result[\"success\"]: print(json.dumps(recipe_result[\"data\"], indent=2)) else: print(f\"Error: {recipe_result['error']}\") We build more advanced generation controls by introducing configurable reasoning effort and a structured JSON output workflow. We define different effort modes to vary how deeply the model reasons, how many tokens it uses, and how detailed its answers are during inference. We also create a JSON generation utility that guides the open-weight model toward schema-like outputs, cleans the returned text, and retries when the response is not valid JSON. Copy Code Copied Use a different Browser print(\"\\n\" + \"=\" * 70) print(\" PART 6: Multi-turn Conversations with Memory\") print(\"=\" * 70) class ConversationManager: \"\"\" Manages multi-turn conversations with context memory. Implements the Harmony format pattern used by gpt-oss. \"\"\" def __init__(self, pipeline, tokenizer, system_message: str = None): self.pipe = pipeline self.tokenizer = tokenizer self.history = [] if system_message: self.system_message = system_message else: self.system_message = \"You are a helpful, friendly AI assistant. Remember the context of our conversation.\" def chat(self, user_message: str, max_new_tokens: int = 300) -> str: \"\"\"Send a message and get a response, maintaining conversation history.\"\"\" messages = [{\"role\": \"system\", \"content\": self.system_message}] messages.extend(self.history) messages.append({\"role\": \"user\", \"content\": user_message}) output = self.pipe( messages, max_new_tokens=max_new_tokens, do_sample=True, temperature=0.8, top_p=1.0, pad_token_id=self.tokenizer.eos_token_id, ) assistant_response = output[0][\"generated_text\"][-1][\"content\"] self.history.append({\"role\": \"user\", \"content\": user_message}) self.history.append({\"role\": \"assistant\", \"content\": assistant_response}) return assistant_response def get_history_length(self) -> int: \"\"\"Get number of turns in conversation.\"\"\" return len(self.history) // 2 def clear_history(self): \"\"\"Clear conversation history.\"\"\" self.history = [] print(\" Conversation history cleared.\") def get_context_summary(self) -> str: \"\"\"Get a summary of the conversation context.\"\"\" if not self.history: return \"No conversation history yet.\" summary = f\"Conversation has {self.get_history_length()} turns:\\n\" for i, msg in enumerate(self.history): role = \" User\" if msg[\"role\"] == \"user\" else \" Assistant\" preview = msg[\"content\"][:50] + \"...\" if len(msg[\"content\"]) > 50 else msg[\"content\"] summary += f\" {i+1}. {role}: {preview}\\n\" return summary convo = ConversationManager(pipe, tokenizer) print(\"\\n Multi-turn Conversation Demo:\") print(\"-\" * 50) conversation_turns = [ \"Hi! My name is Alex and I'm a software engineer.\", \"I'm working on a machine learning project. What framework would you recommend?\", \"Good suggestion! What's my name, by the way?\", \"Can you remember what field I work in?\" ] for turn in conversation_turns: print(f\"\\n User: {turn}\") response = convo.chat(turn) print(f\" Assistant: {response}\") print(f\"\\n {convo.get_context_summary()}\") print(\"\\n\" + \"=\" * 70) print(\" PART 7: Streaming Token Generation\") print(\"=\" * 70) from transformers import TextIteratorStreamer from threading import Thread import time def stream_response(prompt: str, max_tokens: int = 200): \"\"\" Stream tokens as they're generated for real-time output. \"\"\" messages = [{\"role\": \"user\", \"content\": prompt}] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, return_tensors=\"pt\" ).to(model.device) streamer = TextIteratorStreamer( tokenizer, skip_prompt=True, skip_special_tokens=True ) generation_kwargs = { \"input_ids\": inputs, \"streamer\": streamer, \"max_new_tokens\": max_tokens, \"do_sample\": True, \"temperature\": 0.8, \"top_p\": 1.0, \"pad_token_id\": tokenizer.eos_token_id, } thread = Thread(target=model.generate, kwargs=generation_kwargs) thread.start() print(\" Streaming: \", end=\"\", flush=True) full_response = \"\" for token in streamer: print(token, end=\"\", flush=True) full_response += token time.sleep(0.01) thread.join() print(\"\\n\") return full_response print(\"\\n Streaming Demo:\") print(\"-\" * 50) streamed = stream_response( \"Count from 1 to 10, with a brief comment about each number.\", max_tokens=250 ) We move from single prompts to stateful interactions by creating a conversation manager that stores multi-turn chat history and reuses that context in future responses. We demonstrate how we maintain memory across turns, summarize prior context, and make the interaction feel more like a persistent assistant instead of a one-off generation call. We also implement streaming generation so we can watch tokens arrive in real time, which helps us understand the model’s live decoding behavior more clearly. Copy Code Copied Use a different Browser print(\"\\n\" + \"=\" * 70) print(\" PART 8: Function Calling / Tool Use\") print(\"=\" * 70) import math from datetime import datetime class ToolExecutor: \"\"\" Manages tool definitions and execution for gpt-oss. \"\"\" def __init__(self): self.tools = {} self._register_default_tools() def _register_default_tools(self): \"\"\"Register built-in tools.\"\"\" @self.register(\"calculator\", \"Perform mathematical calculations\") def calculator(expression: str) -> str: \"\"\"Evaluate a mathematical expression.\"\"\" try: allowed_names = { k: v for k, v in math.__dict__.items() if not k.startswith(\"_\") } allowed_names.update({\"abs\": abs, \"round\": round}) result = eval(expression, {\"__builtins__\": {}}, allowed_names) return f\"Result: {result}\" except Exception as e: return f\"Error: {str(e)}\" @self.register(\"get_time\", \"Get current date and time\") def get_time() -> str: \"\"\"Get the current date and time.\"\"\" now = datetime.now() return f\"Current time: {now.strftime('%Y-%m-%d %H:%M:%S')}\" @self.register(\"weather\", \"Get weather for a city (simulated)\") def weather(city: str) -> str: \"\"\"Get weather information (simulated).\"\"\" import random temp = random.randint(60, 85) conditions = random.choice([\"sunny\", \"partly cloudy\", \"cloudy\", \"rainy\"]) return f\"Weather in {city}: {temp}°F, {conditions}\" @self.register(\"search\", \"Search for information (simulated)\") def search(query: str) -> str: \"\"\"Search the web (simulated).\"\"\" return f\"Search results for '{query}': [Simulated results - in production, connect to a real search API]\" def register(self, name: str, description: str): \"\"\"Decorator to register a tool.\"\"\" def decorator(func): self.tools[name] = { \"function\": func, \"description\": description, \"name\": name } return func return decorator def get_tools_prompt(self) -> str: \"\"\"Generate tools description for the system prompt.\"\"\" tools_desc = \"You have access to the following tools:\\n\\n\" for name, tool in self.tools.items(): tools_desc += f\"- {name}: {tool['description']}\\n\" tools_desc += \"\"\" To use a tool, respond with: TOOL: <tool_name> ARGS: <arguments as JSON> After receiving the tool result, provide your final answer to the user.\"\"\" return tools_desc def execute(self, tool_name: str, args: dict) -> str: \"\"\"Execute a tool with given arguments.\"\"\" if tool_name not in self.tools: return f\"Error: Unknown tool '{tool_name}'\" try: func = self.tools[tool_name][\"function\"] if args: result = func(**args) else: result = func() return result except Exception as e: return f\"Error executing {tool_name}: {str(e)}\" def parse_tool_call(self, response: str) -> tuple: \"\"\"Parse a tool call from model response.\"\"\" if \"TOOL:\" not in response: return None, None lines = response.split(\"\\n\") tool_name = None args = {} for line in lines: if line.startswith(\"TOOL:\"): tool_name = line.replace(\"TOOL:\", \"\").strip() elif line.startswith(\"ARGS:\"): try: args_str = line.replace(\"ARGS:\", \"\").strip() args = json.loads(args_str) if args_str else {} except json.JSONDecodeError: args = {\"expression\": args_str} if tool_name == \"calculator\" else {\"query\": args_str} return tool_name, args tools = ToolExecutor() def chat_with_tools(user_message: str) -> str: \"\"\" Chat with tool use capability. \"\"\" system_prompt = f\"\"\"You are a helpful assistant with access to tools. {tools.get_tools_prompt()} If the user's request can be answered directly, do so. If you need to use a tool, indicate which tool and with what arguments.\"\"\" messages = [ {\"role\": \"system\", \"content\": system_prompt}, {\"role\": \"user\", \"content\": user_message} ] output = pipe( messages, max_new_tokens=200, do_sample=True, temperature=0.7, pad_token_id=tokenizer.eos_token_id, ) response = output[0][\"generated_text\"][-1][\"content\"] tool_name, args = tools.parse_tool_call(response) if tool_name: tool_result = tools.execute(tool_name, args) messages.append({\"role\": \"assistant\", \"content\": response}) messages.append({\"role\": \"user\", \"content\": f\"Tool result: {tool_result}\\n\\nNow provide your final answer.\"}) final_output = pipe( messages, max_new_tokens=200, do_sample=True, temperature=0.7, pad_token_id=tokenizer.eos_token_id, ) return final_output[0][\"generated_text\"][-1][\"content\"] return response print(\"\\n Tool Use Examples:\") print(\"-\" * 50) tool_queries = [ \"What is 15 * 23 + 7?\", \"What time is it right now?\", \"What's the weather like in Tokyo?\", ] for query in tool_queries: print(f\"\\n User: {query}\") response = chat_with_tools(query) print(f\" Assistant: {response}\") print(\"\\n\" + \"=\" * 70) print(\" PART 9: Batch Processing for Efficiency\") print(\"=\" * 70) def batch_generate(prompts: list, batch_size: int = 2, max_new_tokens: int = 100) -> list: \"\"\" Process multiple prompts in batches for efficiency. Args: prompts: List of prompts to process batch_size: Number of prompts per batch max_new_tokens: Maximum tokens per response Returns: List of responses \"\"\" results = [] total_batches = (len(prompts) + batch_size - 1) // batch_size for i in range(0, len(prompts), batch_size): batch = prompts[i:i + batch_size] batch_num = i // batch_size + 1 print(f\" Processing batch {batch_num}/{total_batches}...\") batch_messages = [ [{\"role\": \"user\", \"content\": prompt}] for prompt in batch ] for messages in batch_messages: output = pipe( messages, max_new_tokens=max_new_tokens, do_sample=True, temperature=0.7, pad_token_id=tokenizer.eos_token_id, ) results.append(output[0][\"generated_text\"][-1][\"content\"]) return results print(\"\\n Batch Processing Example:\") print(\"-\" * 50) batch_prompts = [ \"What is the capital of France?\", \"What is 7 * 8?\", \"Name a primary color.\", \"What season comes after summer?\", \"What is H2O commonly called?\", ] print(f\"Processing {len(batch_prompts)} prompts...\\n\") batch_results = batch_generate(batch_prompts, batch_size=2) for prompt, result in zip(batch_prompts, batch_results): print(f\"Q: {prompt}\") print(f\"A: {result[:100]}...\\n\") We extend the tutorial to include tool use and batch inference, enabling the open-weight model to support more realistic application patterns. We define a lightweight tool execution framework, let the model choose tools through a structured text pattern, and then feed the tool results back into the generation loop to produce a final answer. We also add batch processing to handle multiple prompts efficiently, which is useful for testing throughput and reusing the same inference pipeline across several tasks. Copy Code Copied Use a different Browser print(\"\\n\" + \"=\" * 70) print(\" PART 10: Interactive Chatbot Interface\") print(\"=\" * 70) import gradio as gr def create_chatbot(): \"\"\"Create a Gradio chatbot interface for gpt-oss.\"\"\" def respond(message, history): \"\"\"Generate chatbot response.\"\"\" for user_msg, assistant_msg in history: messages.append({\"role\": \"user\", \"content\": user_msg}) if assistant_msg: messages.append({\"role\": \"assistant\", \"content\": assistant_msg}) messages.append({\"role\": \"user\", \"content\": message}) output = pipe( messages, max_new_tokens=400, do_sample=True, temperature=0.8, top_p=1.0, pad_token_id=tokenizer.eos_token_id, ) return output[0][\"generated_text\"][-1][\"content\"] demo = gr.ChatInterface( fn=respond, title=\" GPT-OSS Chatbot\", description=\"Chat with OpenAI's open-weight GPT-OSS model!\", examples=[ \"Explain quantum computing in simple terms.\", \"What are the benefits of open-source AI?\", \"Tell me a fun fact about space.\", ], theme=gr.themes.Soft(), ) return demo print(\"\\n Creating Gradio chatbot interface...\") chatbot = create_chatbot() print(\"\\n\" + \"=\" * 70) print(\" PART 11: Utility Helpers\") print(\"=\" * 70) class GptOssHelpers: \"\"\"Collection of utility functions for common tasks.\"\"\" def __init__(self, pipeline, tokenizer): self.pipe = pipeline self.tokenizer = tokenizer def summarize(self, text: str, max_words: int = 50) -> str: \"\"\"Summarize text to specified length.\"\"\" messages = [ {\"role\": \"system\", \"content\": f\"Summarize the following text in {max_words} words or less. Be concise.\"}, {\"role\": \"user\", \"content\": text} ] output = self.pipe(messages, max_new_tokens=150, temperature=0.5, pad_token_id=self.tokenizer.eos_token_id) return output[0][\"generated_text\"][-1][\"content\"] def translate(self, text: str, target_language: str) -> str: \"\"\"Translate text to target language.\"\"\" messages = [ {\"role\": \"user\", \"content\": f\"Translate to {target_language}: {text}\"} ] output = self.pipe(messages, max_new_tokens=200, temperature=0.3, pad_token_id=self.tokenizer.eos_token_id) return output[0][\"generated_text\"][-1][\"content\"] def explain_simply(self, concept: str) -> str: \"\"\"Explain a concept in simple terms.\"\"\" messages = [ {\"role\": \"system\", \"content\": \"Explain concepts simply, as if to a curious 10-year-old. Use analogies and examples.\"}, {\"role\": \"user\", \"content\": f\"Explain: {concept}\"} ] output = self.pipe(messages, max_new_tokens=200, temperature=0.8, pad_token_id=self.tokenizer.eos_token_id) return output[0][\"generated_text\"][-1][\"content\"] def extract_keywords(self, text: str, num_keywords: int = 5) -> list: \"\"\"Extract key topics from text.\"\"\" messages = [ {\"role\": \"user\", \"content\": f\"Extract exactly {num_keywords} keywords from this text. Return only the keywords, comma-separated:\\n\\n{text}\"} ] output = self.pipe(messages, max_new_tokens=50, temperature=0.3, pad_token_id=self.tokenizer.eos_token_id) keywords = output[0][\"generated_text\"][-1][\"content\"] return [k.strip() for k in keywords.split(\",\")] helpers = GptOssHelpers(pipe, tokenizer) print(\"\\n Helper Functions Demo:\") print(\"-\" * 50) sample_text = \"\"\" Artificial intelligence has transformed many industries in recent years. From healthcare diagnostics to autonomous vehicles, AI systems are becoming \"\"\" print(\"\\n1⃣ Summarization:\") summary = helpers.summarize(sample_text, max_words=20) print(f\" {summary}\") print(\"\\n2⃣ Simple Explanation:\") explanation = helpers.explain_simply(\"neural networks\") print(f\" {explanation[:200]}...\") print(\"\\n\" + \"=\" * 70) print(\" TUTORIAL COMPLETE!\") print(\"=\" * 70) print(\"\"\" You've learned how to use GPT-OSS on Google Colab! WHAT YOU LEARNED: ✓ Correct model loading (no load_in_4bit - uses native MXFP4) ✓ Basic inference with proper parameters ✓ Configurable reasoning effort (low/medium/high) ✓ Structured JSON output generation ✓ Multi-turn conversations with memory ✓ Streaming token generation ✓ Function calling and tool use ✓ Batch processing for efficiency ✓ Interactive Gradio chatbot KEY TAKEAWAYS: • GPT-OSS uses native MXFP4 quantization (don't use bitsandbytes) • Recommended: temperature=1.0, top_p=1.0 • gpt-oss-20b fits on T4 GPU (~16GB VRAM) • gpt-oss-120b requires H100/A100 (~80GB VRAM) • Always use trust_remote_code=True RESOURCES: GitHub: https://github.com/openai/gpt-oss Hugging Face: https://huggingface.co/openai/gpt-oss-20b Model Card: https://arxiv.org/abs/2508.10925 Harmony Format: https://github.com/openai/harmony Cookbook: https://cookbook.openai.com/topic/gpt-oss ALTERNATIVE INFERENCE OPTIONS (for better performance): • vLLM: Production-ready, OpenAI-compatible server • Ollama: Easy local deployment • LM Studio: Desktop GUI application \"\"\") if torch.cuda.is_available(): print(f\"\\n Final GPU Memory Usage:\") print(f\" Allocated: {torch.cuda.memory_allocated() / 1e9:.2f} GB\") print(f\" Reserved: {torch.cuda.memory_reserved() / 1e9:.2f} GB\") print(\"\\n\" + \"=\" * 70) print(\" Launch the chatbot by running: chatbot.launch(share=True)\") print(\"=\" * 70) We turn the model pipeline into a usable application by building a Gradio chatbot interface and then adding helper utilities for summarization, translation, simplified explanation, and keyword extraction. We show how the same open-weight model can support both interactive chat and reusable task-specific functions inside a single Colab workflow. We end by summarizing the tutorial, reviewing the key technical takeaways, and reinforcing how GPT-OSS can be loaded, controlled, and extended as a practical open-weight system. In conclusion, we built a comprehensive hands-on understanding of how to use GPT-OSS as an open-source language model rather than a black-box endpoint. We loaded the model with the correct inference path, avoiding incorrect low-bit loading approaches, and worked through important implementation patterns, including configurable reasoning effort, JSON-constrained outputs, Harmony-style conversational formatting, token streaming, lightweight tool use orchestration, and Gradio-based interaction. In doing so, we saw the real advantage of open-weight models: we can directly control model loading, inspect runtime behavior, shape generation flows, and design custom utilities on top of the base model without depending entirely on managed infrastructure. Check out the Full Code Implementation . Also, feel free to follow us on Twitter and don’t forget to join our 130k+ ML SubReddit and Subscribe to our Newsletter . Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post A End-to-End Coding Guide to Running OpenAI GPT-OSS Open-Weight Models with Advanced Inference Workflows appeared first on MarkTechPost .","datePublished":"2026-04-18T03:39:46+00:00","dateModified":"2026-06-28T05:45:22.056444+00:00","image":"https://s.w.org/images/core/emoji/17.0.2/72x72/1f527.png","publisher":{"@type":"Organization","name":"Achei uma IA","url":"https://acheiumaia.com.br"}}