744 Commits

Author SHA1 Message Date
Tobi Lutke
c5a44be104
perf: optimize compiled template allocations and performance
Major optimizations to reduce allocations and improve execution speed:

1. For loops: Replace catch/throw with while + break flag
   - Uses while loop with index instead of .each with catch/throw
   - Break implemented with flag variable, continue with next
   - Result: 18% fewer allocations, 85% faster for simple loops

2. Forloop property inlining
   - Inline forloop.index as (__idx__ + 1), forloop.first as (__idx__ == 0), etc.
   - Completely eliminates forloop hash allocation when all properties inlinable
   - Result: Loop with forloop went from +46% MORE to -16% FEWER allocations

3. LR.to_array helper with EMPTY_ARRAY constant
   - Centralized array conversion with frozen empty array for nil
   - Avoids allocations for empty collections

4. Inline LR.truthy? calls
   - Replace LR.truthy?(x) with (x != nil && x != false)
   - Eliminates method call overhead in conditions

5. Keep Time methods available in sandbox for date filter

Overall results:
- Allocations: 3.5% MORE -> 24% FEWER (27% improvement)
- Time: 64% faster -> 89% faster (25% improvement)

Also adds:
- compile_profiler.rb for measuring allocations/performance
- compile_acceptance_test.rb for output equivalence testing
- OPTIMIZATION.md documenting optimization status
2025-12-31 13:24:33 -04:00
Claude
520e86b8c8
Add Liquid Drop support for compiled templates
- Create CompiledContext class that duck-types to Liquid::Context
- CompiledContext provides: variable lookup, registers, strict_* flags
- Update __lookup__ helper to:
  - Set context on Drops BEFORE accessing their methods
  - Call to_liquid on objects before lookup
  - Set context on nested Drop results
- Change __lookup__ from def to lambda to capture __context__ closure
- Update expression compiler to use __lookup__.call() syntax
- Add CompiledTemplate.call options: registers, strict_variables, strict_filters

Tests added:
- test_compile_with_drop: Basic Drop property access
- test_compile_with_drop_context_access: Drop using context to access other vars
- test_compile_with_nested_drops: Chained Drop lookups
- test_compile_with_forloop_drop: Built-in forloop compatibility
- test_compile_with_registers: Drop accessing registers via context

All 51 tests pass, 30/30 benchmark templates produce matching output.
2025-12-31 15:21:30 +00:00
Claude
7be2922f91
Fix compiler output equivalence and add comprehensive tests
- Fix forloop.first/last/size lookups to try hash key before method call
- Fix tablerow cols parameter to use attributes['cols'] not @cols
- Fix tablerow output format to match interpreter (newlines, row boundaries)
- Fix capture compiler to access @body directly instead of iterating nodelist
- Add CompiledTemplate class to encapsulate code and external_tags
- Add external filter support via filter_handler
- Add debug mode warnings for external tag/filter calls
- Add Ruby 3.3 compatibility shim for peek_byte/scan_byte
- Add comprehensive unit tests for output equivalence
- Add benchmark comparing compiled vs interpreted rendering

All 30 test templates now produce identical output between compiled
Ruby and interpreted Liquid. Pre-compiled Ruby is 1.68x faster.
2025-12-31 14:58:06 +00:00
Claude
e0f856fd11
Add Liquid to Ruby compiler for optimized template rendering
This adds a new `compile_to_ruby` method to Liquid::Template that compiles
Liquid templates to pure Ruby code. The compiled code can be eval'd to create
a proc that renders templates without needing the Liquid library at runtime.

## Features

- Compiles all standard Liquid tags: if/unless/case, for, assign, capture,
  cycle, increment/decrement, raw, echo, break/continue, comment, tablerow
- Compiles variable expressions with filter chains to direct Ruby method calls
- Supports static partial inlining ({% render %} and {% include %} with string
  literals are loaded and compiled at compile time)
- Dynamic partial support via runtime callbacks (__render_dynamic__, __include_dynamic__)
- Debug mode with source comments for error tracing (lightweight source map)
- SourceMapper utility to trace runtime errors back to Liquid source

## Optimization Opportunities

The compiled Ruby code has significant performance advantages:

1. No Context object - variables accessed directly from assigns hash
2. No filter invocation overhead - direct Ruby method calls
3. No resource limits tracking - no per-node render score updates
4. No stack-based scoping - uses Ruby's native block scoping
5. Direct string concatenation - no render_to_output_buffer abstraction
6. Native control flow - break/continue use Ruby's throw/catch
7. No to_liquid calls - values used directly
8. No profiling hooks - no profiler overhead
9. No exception rendering - errors propagate naturally

## Usage

```ruby
template = Liquid::Template.parse("Hello, {{ name }}!")
ruby_code = template.compile_to_ruby
render_proc = eval(ruby_code)
result = render_proc.call({ "name" => "World" })
# => "Hello, World!"

# With debug mode for error tracing:
ruby_code = template.compile_to_ruby(debug: true)
```

## Limitations

- Dynamic {% render %} and {% include %} require runtime callback methods
- Custom tags need explicit compiler implementations
- Custom filters must be available at runtime
2025-12-31 14:21:58 +00:00
Marco Concetto Rudilosso
c34dd812c5 Fully revert calling to_s on filter array 2025-04-14 17:36:45 +02:00
Marco Concetto Rudilosso
f676e699a4 rollback changes to tests 2025-04-14 11:24:01 +02:00
Marco Concetto Rudilosso
b459eb656f remove test 2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso
c6dcf3e714 add test 2025-04-14 11:17:46 +02:00
Marco Concetto Rudilosso
4dae678c63 Revert "Stringify properties before filtering (#1929)"
This reverts commit f5d6a36574b7ddf5e1a00395f553a0f872fb9f16.
2025-04-14 11:17:46 +02:00
Ian Ker-Seymer
dbe709c3bf
Use to_liquid_value in uniq filter (#1948)
* Use to_liquid_value in uniq filter

* Bump version to 5.8.4
2025-04-09 15:00:01 -04:00
Ian Ker-Seymer
2b75bfaff4
Fix regression when using empty/nil properties with array filters (#1944)
* Make all array filters that use `filter_array` util process empty string and nil correctly

* up version

* fix ordering of checks

* also do it for map

* Do not raise property error

* Gracefully empty property in map filter

---------

Co-authored-by: Marco Concetto Rudilosso <marco.rudilosso@shopify.com>
2025-04-04 11:27:29 -04:00
Marco Concetto Rudilosso
aa1640035f
Always stringify properties in all array filters (#1936)
* Always stringify sum property.

* Add test

* always stringify properties in all array filters

* fix syntax error

* up version

---------

Co-authored-by: Dominic Petrick <dominic.petrick@shopify.com>
2025-03-19 13:50:01 -04:00
Ian Ker-Seymer
f5d6a36574
Stringify properties before filtering (#1929) 2025-03-17 17:43:57 -04:00
Guilherme Carreiro
6453a0ea48 Implement nodelist in the Doc tag so it may be visited 2025-02-26 13:14:39 +01:00
Guilherme Carreiro
f643af4bac Update the implementation to make {% doc %} as strict as {% raw %} 2025-02-20 12:37:09 +01:00
Guilherme Carreiro
ae8a0a86ac Remove misleading unit test (thank you, @EvilGenius13) 2025-02-20 12:37:09 +01:00
Guilherme Carreiro
b439d0da53 Update {% doc %} to no longer support nested tags (as {% comment %} does) 2025-02-20 12:37:09 +01:00
Guilherme Carreiro
16592cfb8f Add support to LiquidDoc with the new {% doc %} tag 2025-02-20 12:37:09 +01:00
Chris AtLee
550135c0b9 Raise SyntaxError on invalid UTF8 strings in lexer/tokenizer 2025-02-11 14:23:15 -05:00
Guilherme Carreiro
f9454d8cf3 Fix array filters to not support nested properties 2025-01-31 13:53:17 +01:00
Guilherme Carreiro
bf1419b8ac Apply the same fix for find_index and has 2025-01-24 08:39:01 +01:00
Guilherme Carreiro
5718c4cee2 Fix the find filter to return nil when filtering empty arrays 2025-01-24 08:39:01 +01:00
Ian Ker-Seymer
6372289ba3
Ensure we use InputIterator#each when in join filter (#1898) 2025-01-16 11:36:36 -05:00
Ian Ker-Seymer
0ec52a40b5
Use Liquid::Utils.to_s for join filter (#1897) 2025-01-16 11:21:58 -05:00
Ian Ker-Seymer
74af735f0e
Allow for custom < Hash classes to override #to_s (#1896) 2025-01-16 11:16:13 -05:00
Ian Ker-Seymer
4b65a28722
Implement logic for stringify Hashes to keep compat with 3.4 (#1892)
* Exploring

* Bump to v5.6.5

---------

Co-authored-by: Dominic Petrick <dominic.petrick@shopify.com>
2025-01-15 16:36:53 -05:00
Guilherme Carreiro
6909570f8e
Add find, find_index, has, and reject filters to arrays (#1869)
* Add reject filter #1573

* Add deep search for filter taking in properties #1749

* Update branch with main

* Add `find`, `find_index`, `has`, and `reject` filters to arrays

* Refactor: avoid usage of public_send

---------

Co-authored-by: Anders Søgaard <andershagbard@gmail.com>
Co-authored-by: Anders Søgaard <9662430+andershagbard@users.noreply.github.com>
2025-01-14 10:33:31 +01:00
Michael Go
58777bcd93 float has to start with a digit 2025-01-13 17:13:27 -04:00
Michael Go
a07ae90523 use Ruby Hash as default expression cache 2025-01-10 15:07:57 -04:00
Michael Go
0b8e30b819 update unit test to be compatible with new source_location format 2025-01-10 14:37:54 -04:00
Michael Go
97cada61f9 remove Expression2 strscan workaround 2025-01-10 14:23:57 -04:00
Michael Go
d48708ae46 skip expression caching with Expression1 2025-01-07 14:32:46 -04:00
Michael Go
42d822bda9 users can provide optional expression cache 2025-01-07 14:32:46 -04:00
Michael Go
04bd9dbe90 refactor Lexer to only take StringScanner 2025-01-07 14:32:46 -04:00
Michael Go
7c592c1c00 store StringScanner in ParseContext and reuse it through parsing 2025-01-07 14:32:45 -04:00
Michael Go
002e4caea7 refactor Lexer to be static class function 2025-01-07 14:32:07 -04:00
Michael Go
7c8a269b4d fix cycle tag not resetting 2025-01-07 14:32:07 -04:00
Michael Go
0b1dc295ff fix quirky negative sign expression markup parsing 2025-01-07 14:32:07 -04:00
Michael Go
8ecb703d4d use StringScanner to improve Expression Parsing and Tokenizer 2025-01-07 14:32:06 -04:00
Bahar Pourazar
a0a4307e7d Fix bug in tokenizer with nil value 2024-12-17 15:34:13 +00:00
Ian Ker-Seymer
fdd8c714b2
Stop testing against liquid-c (#1868)
* Stop testing against `liquid-c`

* Bump to `v5.6.0.rc2`
2024-12-11 12:23:50 -05:00
Ian Ker-Seymer
63583ffe5b
Write one value at a time for array variables (#1863)
* Write one value at a time for array variables

* Handle recursive array
2024-12-11 10:16:58 -05:00
Michael Go
c77ff68573 clean up all warnings by using new Environment 2024-11-04 16:15:05 -04:00
Ian Ker-Seymer
b0cba0bfd2
Remove Liquid.cache_classes option (#1847) 2024-11-04 14:41:56 -05:00
Michael Go
06f44226c0
Merge pull request #1846 from Shopify/env-propgating
propagate Environment on new Context creation
2024-11-04 15:33:50 -04:00
Michael Go
3ed54bfdf9 propagate Environment on new Context creation 2024-11-04 15:32:07 -04:00
Michael Go
29986d3704 remove TagRegistry 2024-11-04 15:22:56 -04:00
Michael Go
ffce6de8bb avoid using StringScanner eos 2024-10-30 13:45:10 -03:00
Michael Go
f00670cb01 refactor lexer unit test 2024-10-30 13:44:36 -03:00
Michael Go
f6a3e25e2e fix parsing quirky incomplete expressions 2024-10-30 13:44:35 -03:00