Josh Peng b9a362a185 Improve Markdown code block tokens (#17591) ...

* Improve Markdown code block tokens

Code blocks without a language weren't tokenized.
Code blocks didn't have their ending ``` punctuation tokenized.
Code blocks used to only have one token. Now each block has the following tokens available for syntax highlighters:
- Starting and ending ``` punctuations
- Code block's language setting
- Code snippet

* Variable whitespace for MD code block ``` token

Allow for variable amount of whitespacing before ``` code blocks

* Reorder raw blocks

Raw blocks were preventing tokenizing as languaged blocks. Putting them on bottom resolves this.

* Fix MD block detection when following paragraph

Used to require a new line inbetween ``` code blocks and preceding paragraph text.

* Prevent broken language grammar leaks in MD fences

Prevents leaks in MD code fences while also capturing the closing fence punctuations.

* Update Markdown tokenizer test file

2017-01-03 10:03:16 -08:00

..

markdown.tmLanguage

Improve Markdown code block tokens (#17591)

2017-01-03 10:03:16 -08:00