So far I have two types of tokenizers: #markup and #source, the #markup supports HTML, XML, SVG, etc. and #source is generic tokenizer for all sources that use Java/JS/C/C++/D/TIScript alike syntax. CSS falls to that category with some adjustments ( name token in CSS can contain ‘-‘ ).
Yet, the tokenizer supports stacked mode: you can declare “islands” of different syntax. This way I am parsing CSS-inside-HTML and script-inside-HTML.
andrew, are you using the native tokenizer of TIScript?
So far I have two types of tokenizers: #markup and #source, the #markup supports HTML, XML, SVG, etc. and #source is generic tokenizer for all sources that use Java/JS/C/C++/D/TIScript alike syntax. CSS falls to that category with some adjustments ( name token in CSS can contain ‘-‘ ).
Yet, the tokenizer supports stacked mode: you can declare “islands” of different syntax. This way I am parsing CSS-inside-HTML and script-inside-HTML.
well I note it also tokening CSS… good work