- render() now accepts Liquid::Context or Hash
- include/render handled internally using file_system from registers
- Only yields to block for truly external tags/filters
- Cleaner separation of concerns
Major optimizations to reduce allocations and improve execution speed:
1. For loops: Replace catch/throw with while + break flag
- Uses while loop with index instead of .each with catch/throw
- Break implemented with flag variable, continue with next
- Result: 18% fewer allocations, 85% faster for simple loops
2. Forloop property inlining
- Inline forloop.index as (__idx__ + 1), forloop.first as (__idx__ == 0), etc.
- Completely eliminates forloop hash allocation when all properties inlinable
- Result: Loop with forloop went from +46% MORE to -16% FEWER allocations
3. LR.to_array helper with EMPTY_ARRAY constant
- Centralized array conversion with frozen empty array for nil
- Avoids allocations for empty collections
4. Inline LR.truthy? calls
- Replace LR.truthy?(x) with (x != nil && x != false)
- Eliminates method call overhead in conditions
5. Keep Time methods available in sandbox for date filter
Overall results:
- Allocations: 3.5% MORE -> 24% FEWER (27% improvement)
- Time: 64% faster -> 89% faster (25% improvement)
Also adds:
- compile_profiler.rb for measuring allocations/performance
- compile_acceptance_test.rb for output equivalence testing
- OPTIMIZATION.md documenting optimization status
Tags can now implement to_ruby(code, compiler) to provide their own
optimized Ruby code generation. This allows:
1. Third-party tags to participate in compilation
2. Shopify-specific tags to be lowered to Ruby
3. Better performance for frequently-used custom tags
Priority order for tag compilation:
1. tag.to_ruby(code, compiler) - Custom implementation
2. Built-in compiler (IfCompiler, ForCompiler, etc.)
3. Yield to caller as external tag
Also updated PROJECT.md with:
- External calls yielding documentation
- to_ruby convention documentation
Instead of trying to handle external tags/filters inside the sandbox,
yield back to the caller with [:tag, ...] or [:filter, ...] args.
This cleanly separates concerns:
- Sandbox handles compiled template logic
- Caller handles external calls with full Ruby access
API:
compiled.render(assigns) do |call_type, *args|
case call_type
when :tag
tag_var, tag_assigns = args
# Handle with full Liquid context
when :filter
filter_name, input, filter_args = args
# Handle with custom filter handler
end
end
If no block is given, a default handler is used that:
- Renders external tags using Liquid::Context
- Calls filter methods via filter_handler
Also:
- Keep public_send in sandbox (safe, only calls public methods)
- Load date/time libs into sandbox for date filter support
- Preserve Date, DateTime, Time constants after lock
- compiled_template.rb: Use Liquid::Box for secure execution on Ruby 4.0+
- Creates box, loads runtime, locks, then evals template code
- Provides render() method and secure? check
- Falls back to insecure eval with warning on Ruby < 4.0
- ruby_compiler.rb: Remove inline helper generation
- Helpers now provided by pre-loaded LR module
- Generated code is much smaller (just control flow + LR calls)
- compile.rb: Update documentation for new security model
- template.rb: Update compile_to_ruby docs
The LR module provides all helper methods for compiled templates.
It is loaded into the sandbox BEFORE lock!, so helpers are defined
once and shared by all templates.
Helpers include:
- Type conversion: to_s, to_number, to_integer
- Output: output (handles nil, arrays, BigDecimal formatting)
- Lookup: lookup (hash/array access, Drop context support)
- Encoding: escape_html, url_encode/decode, base64_encode/decode
- Filters: truncate, truncatewords, slice, date, default, etc.
Method references to CGI, Base64, BigDecimal are captured at load
time, enabling safe use of these libraries within the sandbox.
Design principle: Maximize work in pre-loaded runtime, minimize
generated template code.
Introduces Liquid::Box which wraps Ruby 4.0's Ruby::Box for secure
template execution. On Ruby < 4.0, provides a polyfill with security
warnings.
Key features:
- Detects Ruby::Box availability at load time
- Loads safe libraries (CGI, Base64, BigDecimal) into sandbox
- Neuters dangerous methods (file IO, process control, eval, etc.)
- Preserves user constants defined before lock!
- Provides setup_gem_load_paths! to enable gem requires in box
Security model: It is safe to expose side-effect-free, non-IO methods
that don't leak objects with dangerous methods. The sandbox blocks
capabilities, not data.
Add explicit summary showing pre-compiled is X times FASTER,
and explain that "X slower" in comparison means relative to
the fastest option (pre-compiled Ruby). Higher i/s = faster.
- Create CompiledContext class that duck-types to Liquid::Context
- CompiledContext provides: variable lookup, registers, strict_* flags
- Update __lookup__ helper to:
- Set context on Drops BEFORE accessing their methods
- Call to_liquid on objects before lookup
- Set context on nested Drop results
- Change __lookup__ from def to lambda to capture __context__ closure
- Update expression compiler to use __lookup__.call() syntax
- Add CompiledTemplate.call options: registers, strict_variables, strict_filters
Tests added:
- test_compile_with_drop: Basic Drop property access
- test_compile_with_drop_context_access: Drop using context to access other vars
- test_compile_with_nested_drops: Chained Drop lookups
- test_compile_with_forloop_drop: Built-in forloop compatibility
- test_compile_with_registers: Drop accessing registers via context
All 51 tests pass, 30/30 benchmark templates produce matching output.
Use .inspect for all template/partial names inserted into generated
code strings to prevent code injection via maliciously crafted
template names like: {% render "foo'); system('rm -rf /'); #" %}
- Fix forloop.first/last/size lookups to try hash key before method call
- Fix tablerow cols parameter to use attributes['cols'] not @cols
- Fix tablerow output format to match interpreter (newlines, row boundaries)
- Fix capture compiler to access @body directly instead of iterating nodelist
- Add CompiledTemplate class to encapsulate code and external_tags
- Add external filter support via filter_handler
- Add debug mode warnings for external tag/filter calls
- Add Ruby 3.3 compatibility shim for peek_byte/scan_byte
- Add comprehensive unit tests for output equivalence
- Add benchmark comparing compiled vs interpreted rendering
All 30 test templates now produce identical output between compiled
Ruby and interpreted Liquid. Pre-compiled Ruby is 1.68x faster.
This adds a new `compile_to_ruby` method to Liquid::Template that compiles
Liquid templates to pure Ruby code. The compiled code can be eval'd to create
a proc that renders templates without needing the Liquid library at runtime.
## Features
- Compiles all standard Liquid tags: if/unless/case, for, assign, capture,
cycle, increment/decrement, raw, echo, break/continue, comment, tablerow
- Compiles variable expressions with filter chains to direct Ruby method calls
- Supports static partial inlining ({% render %} and {% include %} with string
literals are loaded and compiled at compile time)
- Dynamic partial support via runtime callbacks (__render_dynamic__, __include_dynamic__)
- Debug mode with source comments for error tracing (lightweight source map)
- SourceMapper utility to trace runtime errors back to Liquid source
## Optimization Opportunities
The compiled Ruby code has significant performance advantages:
1. No Context object - variables accessed directly from assigns hash
2. No filter invocation overhead - direct Ruby method calls
3. No resource limits tracking - no per-node render score updates
4. No stack-based scoping - uses Ruby's native block scoping
5. Direct string concatenation - no render_to_output_buffer abstraction
6. Native control flow - break/continue use Ruby's throw/catch
7. No to_liquid calls - values used directly
8. No profiling hooks - no profiler overhead
9. No exception rendering - errors propagate naturally
## Usage
```ruby
template = Liquid::Template.parse("Hello, {{ name }}!")
ruby_code = template.compile_to_ruby
render_proc = eval(ruby_code)
result = render_proc.call({ "name" => "World" })
# => "Hello, World!"
# With debug mode for error tracing:
ruby_code = template.compile_to_ruby(debug: true)
```
## Limitations
- Dynamic {% render %} and {% include %} require runtime callback methods
- Custom tags need explicit compiler implementations
- Custom filters must be available at runtime
* Make all array filters that use `filter_array` util process empty string and nil correctly
* up version
* fix ordering of checks
* also do it for map
* Do not raise property error
* Gracefully empty property in map filter
---------
Co-authored-by: Marco Concetto Rudilosso <marco.rudilosso@shopify.com>
* Always stringify sum property.
* Add test
* always stringify properties in all array filters
* fix syntax error
* up version
---------
Co-authored-by: Dominic Petrick <dominic.petrick@shopify.com>
