Repository navigation
Find violations in ERB templates - #46
Merged
Merged
Conversation
ERB templates were replaced with a placeholder before parsing, so `count` returned 0 for their violations and every `sources*` method left them out. The same happened to the other files packwerk parses as Ruby (.builder, .gemspec, .ru, Gemfile, Rakefile). - Extract the Ruby from ERB templates with herb, blanking the HTML in place so line numbers still match the template. Context lines still come from the template itself. - Parse every file type that packwerk parses as Ruby. - Warn when a file can't be parsed. Files that can't be parsed, or that have no code, now have no references instead of raising a TypeError. Fixes #1
…be parsed - Search the root of a file's AST too. A template whose only Ruby is a bare constant, like `<%= Billing::Invoice %>`, had no references. - Include the line and message of the first syntax error in the "Could not parse" warning. In ERB templates the line is the template line, so the problem can be found without seeing the extracted Ruby. - Cover .rake, non-HTML ERB, missing files, files packwerk doesn't parse, and warning only once per file.
The line number was used as a 0-based index, so the window ran from 2 lines before the reference to 4 after, and the label didn't match the lines shown. For a reference in the first 2 lines of a file, the start went negative and wrapped to the end of the file, showing unrelated lines or none. ERB partials often reference constants near the top, so this would come up a lot with ERB support. Also pass the offsets through from Violations#sources_with_contexts, which ignored them.
`all` passes on an empty array, so the example also passed on main, where ERB templates had no contexts at all.
Closed
6 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1
Problem
FileCache#get_filereplaced every file that isn't.rbor.rakewith the placeholder string"x"before parsing. So for a violation in an ERB template,countreturned 0 and everysources*method left it out, with no warning. The same happened to the other files packwerk parses as Ruby:.builder,.gemspec,.ru,GemfileandRakefile.Approach
ERB templates go through
Herb.extract_ruby, which blanks out everything except the Ruby code in place, so line numbers still match the template. Each tag ends with;, so two tags on one line stay separate statements, and ERB comments and<%%escapes are dropped. The rest of the pipeline (receiver chains, anonymization, line numbers) works unchanged, andsources_with_contextsstill shows lines from the template itself.Why herb: packwerk 3.x parses ERB with
better_html, which is deprecated in favor of herb, and packwerk's move away from it (Shopify/packwerk#447) uses the sameHerb.extract_rubycall. Packwerk's current ERB parser couldn't be reused anyway, because it discards source positions.Other changes:
RUBY_FILEnow matches every file type packwerk parses as Ruby, fromPackwerk::Parsers::Factory.Could not parse <file>:<line> (<syntax error>), so its violations are not includedand contributes no references. In an ERB template the line is the template line. Previously that raised a SorbetTypeError, as did a file with no code at all (e.g. only comments). Files with no code now have no references and no warning.Violation::RUBY_FILEand its "does not play nicely with ERB files" comment.<%= Billing::Invoice %>) is now found. The search skipped the root of the AST.sources_with_contextsshows the right lines. It used the 1-based line number as a 0-based index, so the window ran from 2 lines before to 4 after, and for a reference on line 1 or 2 the start went negative and wrapped to the end of the file. ERB partials often reference constants near the top, so this would come up a lot.Violations#sources_with_contextsalso passes itsstart_offset:andend_offset:through now; it ignored them.New dependency
herb >= 0.9: 0.9.0 added theextract_rubykeyword options used here, and the new specs pass on 0.9.0 as well as 0.11.0. The options are passed explicitly rather than relying on herb's pre-1.0 defaults.sorbet/rbi/gems/herb@0.11.0.rbi.Testing
spec/query_packwerk/file_cache_spec.rbcovers references and their template line numbers in an ERB template (a block helper, two tags on one line, a comment followed by code on the same line, an escaped tag), non-HTML ERB, a constant that is a template's only code, files with no code, missing files, a syntax error (warned about once per file), each Ruby file type, and a file packwerk doesn't parse.spec/query_packwerk/violation_spec.rbcoverscountandsources_with_contextsfor a violation in an ERB template, including the lines shown around each reference and a reference on the first line.violations_spec.rbcovers the offsets passed toViolations#sources_with_contexts.main. The other 5 pin behavior that already worked (.rake, missing files, a template with no Ruby,Gemfile.lock). Flipping any of the threeextract_rubyoptions, or removing the no-AST fallback, fails at least one.package_todo.ymlfiles now resolves to at least one reference (previously none), and every reported line number points at the template line containing the reported source.bundle exec rspec(90 examples),rubocopandsrb tcpass. The new specs also pass locally on Ruby 3.3 and 3.4.