[bash] commands after HEREDOC delimiter are highlighted like the HEREDOC

[aeschli]

From @dexwerx on January 5, 2017 20:59

• VSCode Version: 1.8.1
• OS Version: Windows 7 x64

Steps to Reproduce:

1. Create a Shell Script
2. Paste

# grab field types from ORACLE
sqlplus -s user/pass << EOD | sed '/^$/d' | awk 'NR>2{print}' > temp.txt
SET PAGESIZE 0;
SET FEEDBACK OFF;
DESCRIBE $1;
EOD

everything after the end of heredoc token EOD is highlighted the same as the rest of the heredoc.

Copied from original issue: microsoft/vscode#18192

[infininight]

Not seeing this issue in TextMate, the regex here looked reasonably straightforward so nothing jumps out as being the issue. My first suspicion is a different in regex engine syntax but I know basically nothing about how vsCode is implemented language-wise.

[mkhl]

I believe that there are two issues here:

1. The code as pasted doesn’t highlight anything as a heredoc. The grammar doesn’t allow whitespace in between << and the heredoc token, although it should. I believe we’ve recently fixed this for generic redirections but not for heredocs.

2. With heredoc highlighting fixed (or when removing the space before EOD), the remainder of the line starting the heredoc is also treated as a heredoc, although it shouldn’t. Fixing this is more involved and I’m still thinking about how best to do this.

   A short sketch of what I’m thinking:
   Our current rules describe a heredoc as beginning and ending at the token.
   Instead, the beginning token could open a scope with two rules:

   1. begin: lookbehind at the heredoc operator and token, end: newline, include: $self and possibly a rule matching backslash-escaped newlines (to prevent those from ending this scope)
   2. begin: lookbehind at newline (or lookahead at anything), end: lookahead at heredoc token
      The second rule would provide the actual heredoc scope.
      I don’t think child rules can access their parents’ matches, so the actual heredoc token wouldn’t be available there. We could try to treat anything like possible heredoc tokens, relying on the fact that if the parent’s end fails the second rule’s begin would match.

   This problem also seems to be present in the Ruby bundle.

[sorbits]

There is also the possibility of matching the (start) heredoc token and then capture the rest of the line, and for the captures part of the rule, include $self, this would avoid the nested scope, but it would not be possible to handle escaped newlines after the heredoc token.

[dexwerx]

annual bump!

[IvanRoman]

I was wondering if there is any update on this. I can see this issue on Ubuntu 18.04.1 with VS Code v. 1.27.2

[NorseGaud]

2019 bump!!! This is extremely annoying.

On Mojave.

[Mist-Hunter]

Same problem. 2020 bump, just started to get on the vscode bus, but I use heredoc's allot..

[Mist-Hunter]

Same problem. 2020 bump, just started to get on the vscode bus, but I use heredoc's allot..

FYI, quoting EOT seems to fix this. Example.

No quotes, not working

Quotes (double below, but single also works) working. Using double due presence of variables

[Mist-Hunter]

Update to myself. The quotes didn't work, they actually broke variable expansion. Randomly though I discovered that you need a space after EOT. Bash doesn't care, but VSCode does. To be fair, many pages do show the spaces: https://linuxize.com/post/bash-heredoc/ ,so even though they aren't needed by bash, I'm not sure this is a bug.

This may be well known by others, but I'm guessing this bug wouldn't be here if everyone knew.

cat < makes VS Code sad

cat < gets highlighted properly and works!

[jannek-aalto]

Still broken in vscode 1.49.2 for ksh space-trimming heredoc syntax '<<#word'.

FOO=$(cat <<#EOT
  foo
  bar
EOT
)