deblan-mirror/go-twig
1
0
Fork 0
mirror of https://github.com/semihalev/twig.git synced 2026-03-14 13:55:46 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add memory profiling and benchmarking tools

Browse source 
- Create comprehensive benchmarks for memory allocation analysis
- Implement a memory analysis utility to identify allocation hotspots
- Add command-line profiling tool for different template complexities
- Create analysis script to generate detailed memory reports
- Add benchmarking documentation with optimization strategy

These tools will help identify critical areas for optimization
in implementing a zero-allocation rendering path.

🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
semihalev 2025-03-11 23:39:49 +03:00
commit 741ef0dd81
5 changed files with 1056 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

88
BENCHMARKING.md Normal file
View file

@ -0,0 +1,88 @@
# Memory Allocation Analysis for Zero-Allocation Rendering
This guide explains how to use the memory profiling tools provided to identify and optimize memory allocation hotspots in the Twig template engine.
## Running the Memory Analysis
To run a comprehensive memory analysis and generate reports:
```bash
./scripts/analyze_memory.sh
```
This script will:
1. Run benchmarks with memory allocation tracking
2. Generate heap allocation profiles
3. Analyze the profiles to identify allocation hotspots
4. Create a comprehensive report in the `reports/` directory
## Interpreting the Results
The analysis generates several files:
- `reports/memory_optimization_report.md` - Main report with allocation analysis and recommendations
- `reports/benchmark_results.txt` - Raw benchmark results with memory statistics
- `reports/top_allocations.txt` - Top memory allocation sources from pprof
- `reports/heap.prof` - Heap allocation profile that can be analyzed with `go tool pprof`
- `reports/simple_profile.txt`, `reports/medium_profile.txt`, `reports/complex_profile.txt` - Profile results by template complexity
## Using Individual Profiling Tools
### Running Benchmarks with Memory Stats
```bash
go test -run=^$ -bench=. -benchmem ./memory_profile_test.go
```
This command runs all benchmarks and reports allocations per operation.
### Generating a Heap Profile
```bash
go test -run=^$ -bench=BenchmarkRenderComplexTemplate -benchmem -memprofile=heap.prof ./memory_profile_test.go
```
### Analyzing the Heap Profile
```bash
go tool pprof -alloc_space heap.prof
```
Common pprof commands:
- `top` - Show top allocation sources
- `list FunctionName` - Show line-by-line allocations in a function
- `web` - Open a web visualization of the profile
### Using the Profiling Tool
For targeted profiling of specific template complexity:
```bash
go run cmd/profile/main.go -complexity=3 -iterations=1000 -memprofile=complex.prof
```
Options:
- `-complexity` - Template complexity level (1=simple, 2=medium, 3=complex)
- `-iterations` - Number of template renders to perform
- `-memprofile` - Output file for memory profile
- `-cpuprofile` - Output file for CPU profile (optional)
## Zero-Allocation Implementation Strategy
Based on the profile results, implement optimizations in this order:
1. **Object Pooling** - Implement pools for all temporary objects
2. **String Operations** - Optimize string handling to avoid allocations
3. **Context Management** - Improve context creation, cloning, and cleanup
4. **Expression Evaluation** - Minimize allocations in expression execution
5. **Buffer Management** - Reuse output buffers with proper pooling
## Testing Your Optimizations
After each optimization, run the memory benchmarks again to verify the reduction in allocations:
```bash
go test -run=^$ -bench=BenchmarkRender -benchmem ./memory_profile_test.go
```
The goal is to see zero (or minimal) allocations per operation in the `allocs/op` column.

302
cmd/profile/main.go Normal file
View file

@ -0,0 +1,302 @@
// Package main provides a simple utility to run memory profiling on the Twig template engine
package main
import (
"bytes"
"flag"
"fmt"
"log"
"os"
"runtime"
"runtime/pprof"
"strconv"
"time"
"github.com/semihalev/twig"
)
func main() {
// Command-line flags
cpuProfile := flag.String("cpuprofile", "", "write cpu profile to file")
memProfile := flag.String("memprofile", "", "write memory profile to file")
complexityLevel := flag.Int("complexity", 2, "template complexity level (1-3)")
iterations := flag.Int("iterations", 1000, "number of template renders to perform")
flag.Parse()
// CPU profiling if requested
if *cpuProfile != "" {
f, err := os.Create(*cpuProfile)
if err != nil {
log.Fatal("could not create CPU profile: ", err)
}
defer f.Close()
if err := pprof.StartCPUProfile(f); err != nil {
log.Fatal("could not start CPU profile: ", err)
}
defer pprof.StopCPUProfile()
}
// Run the appropriate benchmark based on complexity level
var totalTime time.Duration
var totalAllocBytes uint64
var numGC uint32
// Record memory stats before
var memStatsBefore runtime.MemStats
runtime.ReadMemStats(&memStatsBefore)
switch *complexityLevel {
case 1:
totalTime = runSimpleTemplateBenchmark(*iterations)
case 2:
totalTime = runMediumTemplateBenchmark(*iterations)
case 3:
totalTime = runComplexTemplateBenchmark(*iterations)
default:
log.Fatalf("Invalid complexity level: %d (must be 1-3)", *complexityLevel)
}
// Record memory stats after
var memStatsAfter runtime.MemStats
runtime.ReadMemStats(&memStatsAfter)
// Calculate allocation statistics
totalAllocBytes = memStatsAfter.TotalAlloc - memStatsBefore.TotalAlloc
numGC = memStatsAfter.NumGC - memStatsBefore.NumGC
// Report results
fmt.Printf("=== Twig Memory Profiling Results ===\n")
fmt.Printf("Complexity Level: %d\n", *complexityLevel)
fmt.Printf("Total Iterations: %d\n", *iterations)
fmt.Printf("Total Time: %v\n", totalTime)
fmt.Printf("Time per Iteration: %v\n", totalTime/time.Duration(*iterations))
fmt.Printf("Total Memory Allocated: %d bytes\n", totalAllocBytes)
fmt.Printf("Memory per Iteration: %d bytes\n", totalAllocBytes/uint64(*iterations))
fmt.Printf("Number of GCs: %d\n", numGC)
// Memory profiling if requested
if *memProfile != "" {
f, err := os.Create(*memProfile)
if err != nil {
log.Fatal("could not create memory profile: ", err)
}
defer f.Close()
runtime.GC() // Get up-to-date statistics
if err := pprof.WriteHeapProfile(f); err != nil {
log.Fatal("could not write memory profile: ", err)
}
}
}
// runSimpleTemplateBenchmark runs a benchmark with a simple template
func runSimpleTemplateBenchmark(iterations int) time.Duration {
engine := twig.New()
err := engine.RegisterString("simple", "Hello, {{ name }}!")
if err != nil {
log.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"name": "World",
}
startTime := time.Now()
for i := 0; i < iterations; i++ {
var buf bytes.Buffer
template, _ := engine.Load("simple")
err := template.RenderTo(&buf, context)
if err != nil {
log.Fatalf("Error rendering template: %v", err)
}
}
return time.Since(startTime)
}
// runMediumTemplateBenchmark runs a benchmark with a medium complexity template
func runMediumTemplateBenchmark(iterations int) time.Duration {
engine := twig.New()
templateContent := `
<div class="profile">
<h1>{{ user.name }}</h1>
<p>Age: {{ user.age }}</p>
{% if user.bio %}
<div class="bio">{{ user.bio|capitalize }}</div>
{% else %}
<div class="bio">No bio available</div>
{% endif %}
<ul class="skills">
{% for skill in user.skills %}
<li>{{ skill.name }} ({{ skill.level }})</li>
{% endfor %}
</ul>
</div>
`
err := engine.RegisterString("medium", templateContent)
if err != nil {
log.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"user": map[string]interface{}{
"name": "John Doe",
"age": 30,
"bio": "web developer and open source enthusiast",
"skills": []map[string]interface{}{
{"name": "Go", "level": "Advanced"},
{"name": "JavaScript", "level": "Intermediate"},
{"name": "CSS", "level": "Beginner"},
{"name": "HTML", "level": "Advanced"},
},
},
}
startTime := time.Now()
for i := 0; i < iterations; i++ {
var buf bytes.Buffer
template, _ := engine.Load("medium")
err := template.RenderTo(&buf, context)
if err != nil {
log.Fatalf("Error rendering template: %v", err)
}
}
return time.Since(startTime)
}
// runComplexTemplateBenchmark runs a benchmark with a complex template
func runComplexTemplateBenchmark(iterations int) time.Duration {
engine := twig.New()
// Base template
baseTemplate := `
<!DOCTYPE html>
<html>
<head>
<title>{% block title %}Default Title{% endblock %}</title>
{% block styles %}
<style>
body { font-family: Arial, sans-serif; }
</style>
{% endblock %}
</head>
<body>
<header>{% block header %}Default Header{% endblock %}</header>
<main>{% block content %}Default Content{% endblock %}</main>
<footer>{% block footer %}© {{ "now"|date("Y") }} Sample Site{% endblock %}</footer>
</body>
</html>
`
err := engine.RegisterString("base", baseTemplate)
if err != nil {
log.Fatalf("Error registering template: %v", err)
}
// Macro template
macroTemplate := `
{% macro renderProduct(product) %}
<div class="product">
<h3>{{ product.name }}</h3>
<p>{{ product.description|capitalize }}</p>
<div class="price">{{ product.price|format("$%.2f") }}</div>
{% if product.tags %}
<div class="tags">
{% for tag in product.tags %}
<span class="tag">{{ tag }}</span>
{% endfor %}
</div>
{% endif %}
</div>
{% endmacro %}
`
err = engine.RegisterString("macros", macroTemplate)
if err != nil {
log.Fatalf("Error registering template: %v", err)
}
// Page template
pageTemplate := `
{% extends "base" %}
{% import "macros" as components %}
{% block title %}{{ page.title }} - {{ parent() }}{% endblock %}
{% block styles %}
{{ parent() }}
<style>
.product { border: 1px solid #ddd; padding: 10px; margin-bottom: 10px; }
.price { font-weight: bold; color: #c00; }
.tag { background: #eee; padding: 2px 5px; margin-right: 5px; }
</style>
{% endblock %}
{% block header %}
<h1>{{ page.title }}</h1>
<nav>
{% for item in navigation %}
<a href="{{ item.url }}">{{ item.text }}</a>
{% if not loop.last %} | {% endif %}
{% endfor %}
</nav>
{% endblock %}
{% block content %}
<div class="products">
<h2>Product List ({{ products|length }} items)</h2>
{% for product in products %}
{{ components.renderProduct(product) }}
{% endfor %}
{% set total = 0 %}
{% for product in products %}
{% set total = total + product.price %}
{% endfor %}
<div class="summary">
<p>Total products: {{ products|length }}</p>
<p>Average price: {{ (total / products|length)|format("$%.2f") }}</p>
<p>Price range: {{ products|map(p => p.price)|sort|first|format("$%.2f") }} - {{ products|map(p => p.price)|sort|last|format("$%.2f") }}</p>
</div>
</div>
{% endblock %}
`
err = engine.RegisterString("page", pageTemplate)
if err != nil {
log.Fatalf("Error registering template: %v", err)
}
// Create a complex context with various data types
products := make([]map[string]interface{}, 20)
for i := 0; i < 20; i++ {
products[i] = map[string]interface{}{
"id": i + 1,
"name": "Product " + strconv.Itoa(i+1),
"description": "This is product " + strconv.Itoa(i+1) + " with detailed information.",
"price": 15.0 + float64(i)*1.5,
"tags": []string{"tag1", "tag2", "tag3"}[0:1+(i%3)],
}
}
context := map[string]interface{}{
"page": map[string]interface{}{
"title": "Product Catalog",
},
"navigation": []map[string]interface{}{
{"url": "/", "text": "Home"},
{"url": "/products", "text": "Products"},
{"url": "/about", "text": "About"},
{"url": "/contact", "text": "Contact"},
},
"products": products,
}
startTime := time.Now()
for i := 0; i < iterations; i++ {
var buf bytes.Buffer
template, _ := engine.Load("page")
err := template.RenderTo(&buf, context)
if err != nil {
log.Fatalf("Error rendering template: %v", err)
}
}
return time.Since(startTime)
}

138
memory_analysis.go Normal file
View file

@ -0,0 +1,138 @@
// This file contains a utility function to generate a memory allocation report
package twig
import (
"fmt"
"os"
"runtime/pprof"
"sort"
"strings"
)
// RunMemoryAnalysis runs a comprehensive memory analysis on the template engine
// and generates a report of allocation hotspots.
func RunMemoryAnalysis() error {
// Create a memory profile output file
f, err := os.Create("twig_memory.pprof")
if err != nil {
return fmt.Errorf("failed to create memory profile: %v", err)
}
defer f.Close()
// Write a memory profile with detailed allocation info
if err := pprof.WriteHeapProfile(f); err != nil {
return fmt.Errorf("failed to write memory profile: %v", err)
}
// Generate a report based on the memory profile
reportFile, err := os.Create("twig_memory_report.txt")
if err != nil {
return fmt.Errorf("failed to create report file: %v", err)
}
defer reportFile.Close()
// Header information
fmt.Fprintf(reportFile, "# TWIG MEMORY ALLOCATION REPORT\n\n")
fmt.Fprintf(reportFile, "This report shows memory allocation hotspots in the Twig template engine.\n")
fmt.Fprintf(reportFile, "Optimizing these areas will help achieve a zero-allocation rendering path.\n\n")
// Instructions for viewing the profile
fmt.Fprintf(reportFile, "## How to View the Profile\n\n")
fmt.Fprintf(reportFile, "Run this command to analyze the memory profile:\n")
fmt.Fprintf(reportFile, "```\ngo tool pprof -alloc_space twig_memory.pprof\n```\n\n")
fmt.Fprintf(reportFile, "Common commands in the pprof interface:\n")
fmt.Fprintf(reportFile, "- `top`: Shows the top allocation sources\n")
fmt.Fprintf(reportFile, "- `list FunctionName`: Shows line-by-line allocations in a function\n")
fmt.Fprintf(reportFile, "- `web`: Opens a web browser with a visualization of the profile\n\n")
// Benchmarking instructions
fmt.Fprintf(reportFile, "## Benchmark Results\n\n")
fmt.Fprintf(reportFile, "Run this command to see allocation statistics:\n")
fmt.Fprintf(reportFile, "```\ngo test -run=^$ -bench=. -benchmem ./memory_profile_test.go\n```\n\n")
// Generate tables for common allocation sources
generateAllocationTables(reportFile)
// Recommendation section
fmt.Fprintf(reportFile, "## Optimization Recommendations\n\n")
fmt.Fprintf(reportFile, "Based on common patterns in template engines, consider these areas for optimization:\n\n")
recommendations := []string{
"**Context Creation**: Pool and reuse RenderContext objects",
"**String Concatenation**: Replace with direct WriteString to output buffers",
"**Expression Evaluation**: Eliminate intermediate allocations during evaluation",
"**Filter Chain Evaluation**: Reuse filter result objects",
"**Map Creation**: Pre-size maps and reuse map objects where possible",
"**String Conversions**: Use allocation-free ToString implementations for common types",
"**Buffer Management**: Pool and reuse output buffers",
"**Node Creation**: Extend the node pool to cover all node types",
"**Slice Allocations**: Pre-allocate slices with expected capacity",
}
for _, rec := range recommendations {
fmt.Fprintf(reportFile, "- %s\n", rec)
}
// Implementation strategy
fmt.Fprintf(reportFile, "\n## Implementation Strategy\n\n")
fmt.Fprintf(reportFile, "1. **Start with high-impact areas**: Focus on the top allocation sources first\n")
fmt.Fprintf(reportFile, "2. **Implement pools for all temporary objects**: Especially RenderContext objects\n")
fmt.Fprintf(reportFile, "3. **Optimize string operations**: String handling is often a major source of allocations\n")
fmt.Fprintf(reportFile, "4. **Review all map/slice creations**: Pre-size collections where possible\n")
fmt.Fprintf(reportFile, "5. **Incremental testing**: Benchmark after each optimization to measure impact\n\n")
fmt.Fprintf(reportFile, "## Final Notes\n\n")
fmt.Fprintf(reportFile, "Remember that some allocations are unavoidable, especially for dynamic templates.\n")
fmt.Fprintf(reportFile, "The goal is to eliminate allocations in the core rendering path, prioritizing the\n")
fmt.Fprintf(reportFile, "most frequent operations for maximum performance impact.\n")
return nil
}
// Helper function to generate allocation tables for common sources
func generateAllocationTables(w *os.File) {
// String Handling Table
fmt.Fprintf(w, "### String Operations\n\n")
fmt.Fprintf(w, "| Operation | Allocation Issue | Optimization |\n")
fmt.Fprintf(w, "|-----------|-----------------|-------------|\n")
stringOps := [][]string{
{"String Concatenation", "Creates new strings", "Use WriteString to buffer"},
{"Substring Operations", "Creates new strings", "Reuse byte slices when possible"},
{"String Conversion", "Boxing/unboxing", "Specialized ToString methods"},
{"Format/Replace", "Creates intermediate strings", "Write directly to output buffer"},
}
for _, op := range stringOps {
fmt.Fprintf(w, "| %s | %s | %s |\n", op[0], op[1], op[2])
}
fmt.Fprintf(w, "\n")
// Context Operations Table
fmt.Fprintf(w, "### Context Operations\n\n")
fmt.Fprintf(w, "| Operation | Allocation Issue | Optimization |\n")
fmt.Fprintf(w, "|-----------|-----------------|-------------|\n")
contextOps := [][]string{
{"Context Creation", "New map allocations", "Pool and reuse context objects"},
{"Context Cloning", "Copying maps", "Selective copying or copy-on-write"},
{"Variable Lookup", "Interface conversions", "Type-specialized getters"},
{"Context Merging", "Map copies for scope", "Prototype or linked contexts"},
}
for _, op := range contextOps {
fmt.Fprintf(w, "| %s | %s | %s |\n", op[0], op[1], op[2])
}
fmt.Fprintf(w, "\n")
// Node Operations Table
fmt.Fprintf(w, "### Node Operations\n\n")
fmt.Fprintf(w, "| Node Type | Allocation Issue | Optimization |\n")
fmt.Fprintf(w, "|-----------|-----------------|-------------|\n")
nodeOps := [][]string{
{"Expression Nodes", "New node for each evaluation", "Pool and reuse node objects"},
{"Filter Nodes", "Chained filter allocations", "Intermediate result pooling"},
{"Loop Nodes", "Iterator allocations", "Reuse loop context and iterators"},
{"Block Nodes", "Block context allocations", "Pool block contexts"},
}
for _, op := range nodeOps {
fmt.Fprintf(w, "| %s | %s | %s |\n", op[0], op[1], op[2])
}
fmt.Fprintf(w, "\n")
}

368
memory_profile_test.go Normal file
View file

@ -0,0 +1,368 @@
package twig
import (
"bytes"
"testing"
)
// BenchmarkRenderSimpleTemplate benchmarks rendering a simple template with just variables
func BenchmarkRenderSimpleTemplate(b *testing.B) {
engine := New()
err := engine.RegisterString("simple", "Hello, {{ name }}!")
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"name": "World",
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("simple")
_ = template.RenderTo(&buf, context)
}
}
// BenchmarkRenderComplexTemplate benchmarks rendering a template with conditionals, loops, and filters
func BenchmarkRenderComplexTemplate(b *testing.B) {
engine := New()
templateContent := `
<div class="container">
<h1>{{ title|upper }}</h1>
{% if showHeader %}
<div class="header">Welcome, {{ user.name }}!</div>
{% endif %}
<ul class="items">
{% for item in items %}
<li>{{ item.name }} - {{ item.price|format("$%.2f") }}</li>
{% endfor %}
</ul>
{% set total = 0 %}
{% for item in items %}
{% set total = total + item.price %}
{% endfor %}
<div class="total">Total: {{ total|format("$%.2f") }}</div>
</div>
`
err := engine.RegisterString("complex", templateContent)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"title": "Product List",
"showHeader": true,
"user": map[string]interface{}{
"name": "John",
},
"items": []map[string]interface{}{
{"name": "Item 1", "price": 10.5},
{"name": "Item 2", "price": 15.0},
{"name": "Item 3", "price": 8.75},
{"name": "Item 4", "price": 12.25},
{"name": "Item 5", "price": 9.99},
},
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("complex")
_ = template.RenderTo(&buf, context)
}
}
// BenchmarkRenderMacros benchmarks rendering a template with macro definitions and calls
func BenchmarkRenderMacros(b *testing.B) {
engine := New()
templateContent := `
{% macro input(name, value='', type='text') %}
<input type="{{ type }}" name="{{ name }}" value="{{ value|e }}">
{% endmacro %}
{% macro form(action, method='post') %}
<form action="{{ action }}" method="{{ method }}">
{{ _self.input('username', user.username) }}
{{ _self.input('password', '', 'password') }}
{{ _self.input('submit', 'Login', 'submit') }}
</form>
{% endmacro %}
{{ _self.form('/login') }}
`
err := engine.RegisterString("macros", templateContent)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"user": map[string]interface{}{
"username": "johndoe",
},
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("macros")
_ = template.RenderTo(&buf, context)
}
}
// BenchmarkRenderInheritance benchmarks rendering a template with inheritance
func BenchmarkRenderInheritance(b *testing.B) {
engine := New()
// Base template
baseTemplate := `
<!DOCTYPE html>
<html>
<head>
<title>{% block title %}Default Title{% endblock %}</title>
</head>
<body>
<header>{% block header %}Default Header{% endblock %}</header>
<main>{% block content %}Default Content{% endblock %}</main>
<footer>{% block footer %}Default Footer{% endblock %}</footer>
</body>
</html>
`
err := engine.RegisterString("base", baseTemplate)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
// Child template
childTemplate := `
{% extends "base" %}
{% block title %}{{ pageTitle }} - {{ parent() }}{% endblock %}
{% block header %}
<h1>{{ pageTitle }}</h1>
{{ parent() }}
{% endblock %}
{% block content %}
{% for item in items %}
<div class="item">{{ item }}</div>
{% endfor %}
{% endblock %}
`
err = engine.RegisterString("child", childTemplate)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"pageTitle": "Products Page",
"items": []string{
"Product 1",
"Product 2",
"Product 3",
},
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("child")
_ = template.RenderTo(&buf, context)
}
}
// BenchmarkRenderFilters benchmarks heavy use of filter chains
func BenchmarkRenderFilters(b *testing.B) {
engine := New()
templateContent := `
{% set text = "Hello, this is some example text for filtering!" %}
<p>{{ text|upper|trim }}</p>
<p>{{ text|lower|replace("example", "sample")|capitalize }}</p>
<p>{{ text|split(" ")|join("-")|upper }}</p>
<p>{{ text|length }}</p>
<p>{{ '2023-05-15'|date("Y-m-d") }}</p>
<p>{{ 123.456|number_format(2, ".", ",") }}</p>
<p>{{ ['a', 'b', 'c']|join(", ")|upper }}</p>
<p>{{ text|slice(7, 10)|capitalize }}</p>
<p>{{ text|replace({"example": "great", "text": "content"}) }}</p>
<p>{{ text|default("No text provided")|upper }}</p>
`
err := engine.RegisterString("filters", templateContent)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("filters")
_ = template.RenderTo(&buf, nil)
}
}
// BenchmarkRenderWithLargeContext benchmarks rendering with a large context
func BenchmarkRenderWithLargeContext(b *testing.B) {
engine := New()
templateContent := `
<ul>
{% for user in users %}
<li>{{ user.id }}: {{ user.name }} ({{ user.email }})
<ul>
{% for role in user.roles %}
<li>{{ role }}</li>
{% endfor %}
</ul>
</li>
{% endfor %}
</ul>
`
err := engine.RegisterString("large_context", templateContent)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
// Create a large context with 100 users
users := make([]map[string]interface{}, 100)
for i := 0; i < 100; i++ {
users[i] = map[string]interface{}{
"id": i + 1,
"name": "User " + string(rune(65+i%26)),
"email": "user" + string(rune(65+i%26)) + "@example.com",
"roles": []string{"User", "Editor", "Admin", "Viewer"}[0:1+(i%4)],
}
}
context := map[string]interface{}{
"users": users,
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("large_context")
_ = template.RenderTo(&buf, context)
}
}
// BenchmarkContextCloning benchmarks the RenderContext cloning operation
func BenchmarkContextCloning(b *testing.B) {
engine := New()
// Create a base context with some data
baseContext := NewRenderContext(engine.environment, map[string]interface{}{
"user": map[string]interface{}{
"id": 123,
"name": "John Doe",
"roles": []string{
"admin", "editor", "user",
},
},
"settings": map[string]interface{}{
"theme": "dark",
"notifications": true,
"language": "en",
},
"items": []map[string]interface{}{
{"id": 1, "name": "Item 1"},
{"id": 2, "name": "Item 2"},
{"id": 3, "name": "Item 3"},
},
}, engine)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Create a clone of the context
clonedCtx := baseContext.Clone()
clonedCtx.Release() // Return to pool after use
}
baseContext.Release() // Clean up
}
// BenchmarkExpressionEvaluation benchmarks various expression evaluations
func BenchmarkExpressionEvaluation(b *testing.B) {
engine := New()
// Register a simple template with different expression types
templateContent := `
{{ 1 + 2 * 3 }}
{{ "Hello " ~ name ~ "!" }}
{{ items[0] }}
{{ user.name }}
{{ items|length > 3 ? "Many items" : "Few items" }}
{{ range(1, 10)|join(", ") }}
`
err := engine.RegisterString("expressions", templateContent)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"name": "World",
"user": map[string]interface{}{
"name": "John",
"age": 30,
},
"items": []string{"a", "b", "c", "d", "e"},
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("expressions")
_ = template.RenderTo(&buf, context)
}
}
// BenchmarkStringOperations benchmarks string manipulation operations
func BenchmarkStringOperations(b *testing.B) {
engine := New()
// Register a template with various string operations
templateContent := `
{{ " Hello, World! "|trim }}
{{ text|replace("o", "0") }}
{{ text|upper }}
{{ text|lower }}
{{ text|capitalize }}
{{ text|slice(7, 5) }}
{{ text|split(", ")|join("-") }}
{{ "%s, %s!"|format("Hello", "World") }}
`
err := engine.RegisterString("string_ops", templateContent)
if err != nil {
b.Fatalf("Error registering template: %v", err)
}
context := map[string]interface{}{
"text": "Hello, World!",
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var buf bytes.Buffer
template, _ := engine.Load("string_ops")
_ = template.RenderTo(&buf, context)
}
}

160
scripts/analyze_memory.sh Executable file
View file

@ -0,0 +1,160 @@
#!/bin/bash
# Create output directory
mkdir -p reports
echo "=== Running Memory Allocation Analysis for Twig Template Engine ==="
echo ""
# Run standard benchmarks with memory statistics
echo "Step 1: Running benchmarks with memory allocation reporting..."
go test -run=^$ -bench=BenchmarkRender -benchmem ./memory_profile_test.go | tee reports/benchmark_results.txt
# Generate allocation profile
echo ""
echo "Step 2: Generating heap allocation profile..."
go test -run=^$ -bench=BenchmarkRenderComplexTemplate -benchmem -memprofile=reports/heap.prof ./memory_profile_test.go
# Run heap profile analysis and save top allocations
echo ""
echo "Step 3: Analyzing allocation profile..."
go tool pprof -text -alloc_space reports/heap.prof > reports/top_allocations.txt
echo ""
echo "Step 4: Generating detailed memory profile report..."
# Run with different template complexities
echo " - Profiling simple templates..."
go run cmd/profile/main.go -complexity=1 -iterations=1000 -memprofile=reports/simple.prof > reports/simple_profile.txt
echo " - Profiling medium templates..."
go run cmd/profile/main.go -complexity=2 -iterations=1000 -memprofile=reports/medium.prof > reports/medium_profile.txt
echo " - Profiling complex templates..."
go run cmd/profile/main.go -complexity=3 -iterations=1000 -memprofile=reports/complex.prof > reports/complex_profile.txt
# Generate flamegraph (requires go-torch if available)
if command -v go-torch &> /dev/null
then
echo ""
echo "Step 5: Generating flamegraph visualization..."
go-torch -alloc_space reports/heap.prof -file reports/allocations_flamegraph.svg
fi
# Compile the comprehensive report
echo ""
echo "Step 6: Compiling final report..."
cat > reports/memory_optimization_report.md << 'EOF'
# Twig Template Engine Memory Optimization Report
## Summary
This report analyzes memory allocation patterns in the Twig template engine to identify areas for implementing a zero-allocation rendering path.
## Benchmark Results
```
EOF
cat reports/benchmark_results.txt >> reports/memory_optimization_report.md
cat >> reports/memory_optimization_report.md << 'EOF'
```
## Top Allocation Sources
The following are the top functions allocating memory during template rendering:
```
EOF
head -20 reports/top_allocations.txt >> reports/memory_optimization_report.md
cat >> reports/memory_optimization_report.md << 'EOF'
```
## Memory Profile by Template Complexity
### Simple Templates
EOF
grep -A 10 "Memory" reports/simple_profile.txt >> reports/memory_optimization_report.md
cat >> reports/memory_optimization_report.md << 'EOF'
### Medium Templates
EOF
grep -A 10 "Memory" reports/medium_profile.txt >> reports/memory_optimization_report.md
cat >> reports/memory_optimization_report.md << 'EOF'
### Complex Templates
EOF
grep -A 10 "Memory" reports/complex_profile.txt >> reports/memory_optimization_report.md
cat >> reports/memory_optimization_report.md << 'EOF'
## Key Allocation Hotspots
Based on the profiling data, these areas should be prioritized for optimization:
1. **String Operations** - String concatenation, substring operations, and conversions
2. **Context Creation** - Creating and copying RenderContext objects
3. **Map Allocations** - Temporary maps created during rendering
4. **Slice Allocations** - Dynamic arrays for node collections
5. **Expression Evaluation** - Temporary objects created during expression processing
6. **Buffer Management** - Output buffer allocations
7. **Function/Filter Calls** - Parameter passing and result handling
## Optimization Strategies
### String Operations
- Replace string concatenation with direct writes to io.Writer
- Use pooled byte buffers instead of creating new strings
- Implement specialized ToString methods to avoid allocations for common types
### Context Handling
- Implement pooling for RenderContext objects
- Create a linked-context mechanism instead of copying values
- Preallocate and reuse context maps
### Map and Slice Allocations
- Preallocate maps and slices with known capacities
- Reuse map and slice objects from pools
- Avoid unnecessary copying of collections
### Expression Evaluation
- Pool expression evaluation result objects
- Optimize common expression patterns with specialized handlers
- Reduce intermediate allocations in expression trees
### Implementation Plan
1. Start with the highest allocation areas first
2. Implement object pooling for all major components
3. Create specialized non-allocating paths for common operations
4. Revise string handling to minimize allocations
5. Optimize hot spots in critical rendering code paths
## Next Steps
1. Implement object pools for all identified allocation sources
2. Create benchmarks to validate each optimization
3. Develop specialized string handling utilities
4. Optimize context handling and cloning
5. Enhance expression evaluation to minimize allocations
EOF
echo ""
echo "Report generation complete. See reports/memory_optimization_report.md for results."
echo ""