Building with LLMs

Text Generation

Build text generation applications with LLMs including structured outputs and streaming.

Text Generation Capabilities

Modern LLMs can:

  • Answer questions
  • Summarize documents
  • Translate text
  • Write creative content
  • Analyze sentiment
  • Extract structured data
  • Generate code

Key Parameters

  • Temperature: Controls randomness (0 = deterministic, 1 = creative)
  • Max tokens: Maximum response length
  • Top-p (nucleus sampling): Only sample from top p% of probability mass
  • Stop sequences: Tokens that end generation

Streaming

For better user experience, stream the response token by token rather than waiting for the full response.

Structured Outputs

Force the model to output valid JSON or structured data for easier programmatic processing.

Example

python
import anthropic
import json

client = anthropic.Anthropic()

# Basic generation
def generate_text(prompt, temperature=1.0, max_tokens=500):
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=max_tokens,
        messages=[{"role": "user", "content": prompt}]
    )
    return response.content[0].text

# Summarization
def summarize(text, length="brief"):
    prompt = f"Summarize the following text in a {length} paragraph:\n\n{text}"
    return generate_text(prompt)

# Sentiment analysis
def analyze_sentiment(text):
    prompt = f"""Analyze the sentiment of this text. Return ONLY a JSON object with fields:
    - sentiment: "positive", "negative", or "neutral"
    - confidence: float between 0 and 1
    - key_phrases: list of key phrases

    Text: {text}"""

    response = generate_text(prompt)
    # Parse JSON from response
    try:
        return json.loads(response)
    except:
        return {"error": "Could not parse response", "raw": response}

# Streaming for better UX
def stream_response(prompt):
    with client.messages.stream(
        model="claude-3-5-sonnet-20241022",
        max_tokens=1000,
        messages=[{"role": "user", "content": prompt}]
    ) as stream:
        for text in stream.text_stream:
            print(text, end="", flush=True)  # print as it arrives
    print()  # newline at end

# Text classification
def classify_text(text, categories):
    categories_str = ", ".join(categories)
    prompt = f"""Classify this text into exactly one of these categories: {categories_str}

Text: {text}

Respond with ONLY the category name."""
    return generate_text(prompt, temperature=0).strip()

# Test
print(classify_text(
    "This Python tutorial explains list comprehensions.",
    ["Technology", "Sports", "Politics", "Entertainment"]
))
Try it yourself — PYTHON