Tokenizer + ::mark() = syntax colorizer

Here is selfie of syntax (tiscript) colorizer – the text below is a full source code of syntax highlighting routine.
The code has colorized itself:

syntax colorizer
syntax colorizer

Can your browser do that in 40 lines of code?

And here are styles that define style of tokens:

plaintext > text::mark(number) { color: brown; }
plaintext > text::mark(number-unit) { color: brown; }
plaintext > text::mark(string) { color: teal; }
plaintext > text::mark(keyword) { color: blue; }
plaintext > text::mark(symbol) { color: brown; }
plaintext > text::mark(literal) { color: brown; }
plaintext > text::mark(comment) { color: green; }

Easy, no?

And even shorter selfie, colorizer wrapped as an aspect component (referenced from colorizer.css):

Object match feature in Sciter

In Sciter I’ve added so called object match feature to built-in like operator and switch/like statement.

Let’s imagine that we have some sequence of objects/data like:

      var shoppingCartData = [
        { product:"Apple", price: 0.95, calories:52 },
        { product:"Orange", price: 0.75, calories:47 },
        { product:"Toothbrush", price: 1.75 },
        { product:"Pear", price: 1.25, calories:57 },      
        { product:"Soap", price: 2.75 },

And we would like to do something separately for food items and other purchased goods. With the new switch/like addition that will simply look like this:

for( var item in shoppingCartData ) 
     like { price: Float, calories: Integer } :  
           // appears as a food item, do something with it
     like { price: Float } : 
           // not a food item
     case "deleted" :
           // item was deleted, skip it  
           // unknown thing, is it just a present? 

Same pattern matching was added to the like expression:

if( item like { product: String, price: Float } )
    stdout.println( item, "is a pricey product" );

Hope this will make life easier in some cases.

display:none is considered harmful

Quite often in HTML/CSS based UIs we need to hide/show some elements in runtime.

Probably the most widespread (and the worst) solution is to set display:none CSS property programmatically.

That sounds quite easy at the first glance but solves only half of the problem – it hides – removes the element from rendering tree. But what you would do when you need to show element that was previously hidden that way? Setting it to display:block is very wrong. Far not all elements have display:block model by default in HTML.  <img>, <input>, <select>, <textarea>, etc are display:inline-block. <table> and all its children have special display:table**** values. Setting them to display:block will lead you to quite surprising results.

Therefore in order to use display:none in runtime you will need to know original display value of the element. Not quite convenient and maintainable. Imagine that your web designer (probably sitting on other continent) decided to replace some elements horizontally by declaring display: table-cell on them. You will get problems hiding/showing such elements by switching display: none <-> block.

To overcome such a problem jQuery for example implements its hide() method by storing existing display value to special property named "olddisplay". And restores the display from that property when you need to show the element. That approach is far from ideal either for obvious reasons.

Better solution

Better solution is to use attribute "hidden". So instead of setting CSS property you will need to add attribute "hidden" to the element when you need to hide it and remove that attribute when you will want to show the element. In order this to work you will need to add single and simple rule to your stylesheet:

[hidden] { display:none !important; }

If some elements need to be hidden by default you will use hidden attribute in your markup, for example as:

<button id="ok-button" hidden>OK</button>

In Sciter you can define some virtual property, e.g. "shown", to all DOM elements (in some common "utilities" module):

/* CSS must have [hidden] { display:none } rule in order this to work */
property Element.shown(v) 
  get return != "none";
  set this.attributes["hidden"] = v ? undefined : true;

and use it as

var someEl = ...

someEl.shown = true; // to show
someEl.shown = false; // to hide

The solution is not free from obvious drawbacks though:

  1. you need that special [hidden] { display:none; } rule to be present in your CSS and
  2. In all cases when you need to hide/show the element you should do it either through that special "shown" property or explicitly by removing "hidden" attribute from the element.


Since Sciter version you can use visibility:none; in CSS to exclude the element from rendering. Exactly in the same way as display:none;. It has exactly the same effect.

As the visibility property is orthogonal to the dsiplay you can safely assign "none" and "visible" property to it without affecting display model of the element.

Therefore, the visibility property in Sciter can accept following values:

  • none – the element is excluded from rendering tree in the same way as display:none, that is Sciter specific;
  • hidden – the element is in rendering tree, takes space but not rendered;
  • collapse – the element is in rendering tree, takes space but its height or width is collapsed to zero. E.g. in flow:horizontal container visibility:collapse child will take space vertically but rendering width will be set to zero. In Sciter this will work for any elements but in standard CSS only for table rows for some unknown to me reason.
  • visible – default value, element is rendered normally.

And here is modified version of the shown scripting property defined above:

property Element.shown(v) 
  get return this.isVisible;
  set = v ? "visible" : "none";

10 years road to Sciter

Last year it was a jubilee of my BlockNote editor – it is 10 years old now. Quite a serious age for software product I think.


BlockNote, in particular its HTML WYSIWYG editing engine, was the very first version of H-SMILE core that works now inside the Sciter. H-SMILE core was used in other software products, for example in EverNote. In fact I was one of four developers who implemented excellent idea of Stepan Pachikov (EverNote idea-monger and founder) and created first public versions of EverNote.

In parallel and after that I was designing various UIs and editing components. WordProcessor engine that I’ve created for Top Producer Systems (The company). That WordProcessor still works in their flagship product AFAIK. Current stack of Web technologies still does not provide facilities for page oriented WYSIWYG editing so there is nothing to replace WordProcessor yet. I’ve got quite interesting experience implementing text layout and editing algorithms there. Ideas from there have found their way in Sciter code in one way or another.

At some point I’ve got an impression that C++ is not the best language for UI programming. In particular GC as a memory management mechanism is significantly better suitable for typical UI tasks – many small objects with quite non-trivial ownership graphs. I’ve started looking on Java but having dependency on installed Java VM didn’t look right at the moment. So I’ve created so called j-SMILE JavaVM (based on SuperWaba) that was tailored specifically for UI purposes – designed to be combined into executable of the applications together with bytecodes it is running. J-SMILE demo (buggy but works somehow) and sources of class files are available here.
J-SMILE screenshot.

In J-SMILE I started experimenting with HTML as a UI layout manager and sinking/bubbling event propagation mechanism used in Web browsers. These mechanisms were proven to be highly useful for desktop UI purposes too.

Pretty much at the same time with my J-SMILE development Sun-Microsoft “Java Wars” had begun. And this immediately put the end to my J-SMILE development. I realized that I have no chances on this front and started looking for other options.

Some time later D-language raised in the sky and gave me real hope for the best ever language-behind-UI. GCable, natively compileable so fast and with pretty sufficient functional features. So I’ve started porting J-SMILE to it under the name Harmonia framework.

Harmonia testbed application
Harmonia testbed application

For those who are interested to see Harmonia in action: Harmonia demo, Harmonia sources and class tree.

D is really very cool and convenient language but after some time I’ve decided to stop Harmonia development for the following reasons:

  1. It was time when D-language was changing very frequently. Pretty much each nightly build was changing core grammar. So Harmonia development was similar to shooting a moving target. Far from being normal development process.
  2. Having each and every object manageable (garbage collectible) is far from being optimal. HTML DOM tree is a large set of small objects. All these small objects are flooding manageable heap making GC cycles very CPU intense. That’s not about Harmonia only by the way – I believe Android suffers from this problem too when its UI freezes noticeably.
  3. I’ve got very strong feeling that each part of UI framework shall use its own memory management and ownership principles that are optimal for the role they are playing. HTML DOM tree has very regular structure with clear one parent – many children ownership graph. Life cycle of DOM elements (read “UI elements”) is also quite deterministic. There is absolutely no need for GC if to speak about HTML/CSS.
  4. But code-behind-UI – code that does UI automation in “on-click-here-push-item-there-and-collapse-panel-over-there” fashion has to be manageable. GC there is the must – ownership graph is unknown upfront and frequently contains loops.

So I reverted back to C++ and created pure DOM and rendering implementation – HTMLayout. And started looking on embeddable languages that can be used with it as “language behind UI”. Here are languages that I considered for that purpose:

  1. Ruby and Python – were rejected almost immediately. HTML friendly language shall have syntax that is not dependent on tabs, line feeds, etc. For quite many reasons. And yet Python was using C/C++ heap for object allocations, UI objects lifecycle require compacting GC – objects created and destroyed frequently.
  2. Lua – pretty much for the same syntax reasons. But it was a good candidate. I even started creating JS syntax alike compiling front-end for its VM. But discovered in process that Lua’s object/class model is too rudimentary I would say.
  3. JavaScript – as a natural choice was always there. But its prototyping schema, “hackish” way of doing classes there and that famous “automatic semicolon injection” problem were stopping me each time I was trying to consider it. Only ECMAScript version 6 has started to appear solid enough. But that’s just recently.

Needless to say that my investigation ended up in creation of first C-SMILE and then TIScipt. TIScript is almost JavaScript (JavaScript++ if you wish) but with modular system (namespaces) and classes. It also got built-in persistence. And it inherits base syntax and runtime library from JavaScript.

HTMLayout combined with TIScript became Sciter as you see it now.

Oh, and in parallel with all this I was working as UI Architect and team lead at Top Producer Systems ( company) on various desktop UI, Web and mobile applications projects. And yet participated in HTML5 and CSS3 specification development as invited expert in W3C HTML Working Group.

Sciter. Declarative behavior assignment by CSS: ‘prototype’ and ‘aspect’ properties

From the very beginning Sciter supported declarative scripting class assignment to DOM elements.

If you want all div.some-widget elements in your document to behave in some special way (be subclassed) then you will need:

1. in script to declare

class SomeWidget : Behavior {
  function attached() { ... } // called with 'this' set to the DOM element 
                              // being subclassed on DOM construction
  function detached() { ... }
  // ... SomeWidget specific methods here

2. and in CSS to declare

div.some-widget { behavior:SomeWidget; } 
/* or if SomeWidget id declared in other file: */
div.some-widget { SomeWidget url(some-widget.tis); }

After that all div elements having class=”some-widget” will have SomeWidget class assigned to them. I have explained this mechanism 8 years ago (time is passing, yes) here.

This works reliably and is quite convenient.

The only problem with behaviors/prototypes – at any given moment of time particular DOM element can have one and only one scripting class. JavaScript and TIScript do not support multiple inheritance.

Imagine that you have multiple functions in script aimed to configure some DOM element to serve some specific functionality.
In other words each of such functions adds its own aspect (partial functionality) to the element it is called on. Like:

function addClickHandlerFor(element) {
  // adds onClick handler defined by "click" element attribute in html
  element.on("click", function() { 
    var attrClick = element.attributes["click"];  
    element.eval(attrClick); // evaluate the expression

You can have set of such functions configuring different aspects of elements/behaviors.

This works in principle but you will need to call such functions explicitly for all elements for which you will need such configurations. And don’t forget to call them it for content created dynamically (non-trivial task by itself).

Considered all these I have introduced new ‘aspect’ Sciter specific CSS property.

The aspect CSS property

It is declared as

  aspect: "function name" [ url(of-function-implementation-file) ];

Where "function name" is fully qualified name of the “aspect” function that is aimed to configure/setup some additional functionality on the element. And the url() is file name where it is defined.

Principles of aspect handling:

The “aspect” function is an ordinary tiscript function that gets called

  1. with this set to the DOM element satisfying the CSS rule.
  2. strictly once per life time of the DOM element.

And yet, the aspect CSS property uses non-standard inheritance – if the element has multiple matching rules with the aspect defined the used aspect is a list of all aspects. Thus if you have have these rules (example taken from the Plus engine):

[click] { aspect:"Plus.Click"; }
[dblclick] { aspect:"Plus.DblClick"; }

and the element defined in markup as

<b id="clickable" click="..." dblclick="...">text</b>

It will get two calls – Plus.Click() and Plus.DblClick() for it. As if you have the following in your CSS:

#clickable { aspect:"Plus.Click" "Plus.DblClick"; }

The aspect mechanism is actively used by Plus ( /samples/+plus/ ) and Lang ( /samples/+lang/ ) engines in Sciter SDK.
Plus provides declarative data binding facilities “a la” AngularJS and Lang is about i18n support.

[tiscript] ‘this’ and ‘this super’ function arguments

Each function in JavaScript and TIScript gets implicit argument named this.

So when you call method of some object as then foo object is passed to the bar(param) function as a value of this argument. And the param will get value of 1.

All of us who are practicing JS, Python, Ruby, etc. know about that ‘this’ thing pretty well.

But what shall you do when you have inner function and want to access ‘this’ value of outer function? The only way in JavaScript for that is to declare other variable with distinct name and assign ‘this’ to it: var that = this;.

To cover such code patterns I’ve introduced in TIScript “super this” concept, so we have following implicit variables in any function:

  • this – standard this variable;
  • this super – standard this variable of outer function;
  • this super super – this variable of outer-outer function;
  • etc.

Here is an example that outputs “6” in standard output:

class Test {

  function this(data) { // constructor = data;   // instance field 

  function Outer(arg1) {
    // this - hidden argument, local variable 
    // arg1 - argument, local variable

    function Inner(arg2)  {
      // this - hidden argument, local variable 
      // arg2 - argument, local variable
      // arg1 - outer variable - outer argument
      // this super - outer variable - outer 'this' argument
        (this super).data   // 1 
        + arg1              // 2
        + arg2              // 3
    return Inner;

var test = new Test(1);

var innerFunc = test.Outer(2);

stdout.println( innerFunc(3) );

Model-View-Whatever, the Plus engine for Sciter.


I would say that human history is a history of reinventing "wheels" of various kinds.

This time we see concept of data binding reincarnated with slightly pathetic name Model-View-Controller. Of course, as many people as many meaning they give to the MVC abbreviation but, nevertheless, it is all around basic idea of data binding – you have data (the Model these days) declaratevily bound with UI "controls" (the View). I believe Microsoft’s VisualBasic 4 and its IDE was the very first usable implementation of the idea. There was no Controller concept at that moment so their implementation was quite limiting – while you can implement 90% of your data editing needs using simple declarations you will spend 90% of your design time fighting with the rest of 10% of needed functionality.

The Plus framework for Sciter.

The Plus framework you can find in Sciter SDK is quite compact (400 LOC) and relatively simple implementation of that old data binding concept with controller means.

Note, the Plus is not an attempt to solve every html/css/script UI problem as AngularJS does. It is just a data binding mechanism with the concept of @observing functions (controllers in my interpretation).


Model in Plus interpretation is some tiscript namespace object that contains data variables (and optionally functions) to be bound with particular container in HTML DOM.

For example if you declare this script:

namespace Data {
  var correspondent = "world"; // variable to be bound

and corresponding markup:

<section model="Data">
   Whom to greet: <input name="correspondent"> ?
   <p>The greeting: Hello <output name="correspondent">!</p>

and include in your document "plus.css" file you will get live data binding between Data.correspondent data variable and two DOM elements: two ways with input[name=correspondent] and one way (only view) binding with output[name=correspondent]. So when you type something in that input you will see the data also rendered in output element.  To see this alive load sdk/samples/+plus/demos/0-basic-variable-binding.htm in sciter.exe from its SDK.

The model and name DOM attributes.

Note that <section> element contains model="Data" attribute. It instructs the Plus engine to establish data binding between content of this section and members of namsepace Data {} in script. Name of the bound namespace can be any suitable, not just Data.

Any DOM element inside that section[model] may have name attribute defined. On binding initialization the Plus will try to find data element in the model with that name and if such data variable is found it will made two or one way (for <output> elements) binding between .value of that DOM element and the data variable. The name can be compound – may contain ‘.‘ (dot)-separated list of names. This way  you can bind DOM elements with object fields inside the model:

namespace Contact {
  var name = { first: "Albert", last: "Einshtein" };
  var phone = "....";

and markup:

<form model="Contact" id="contact-details"> 
  <label for="name.first">First name></label> <input name="name.first">
  <label for="name.last">Last name></label> <input name="name.last">

Celsius to Fahrenheit convertor.

Controllers, the @observing function decorator.

File plus.tis (the Plus engine implementation) contains declaration of function decorator named @observing. With that decorator you can define functions that will be triggered (called by the engine) when variable(s) they are observing change.

As an example let’s define simple Celsius to Fahrenheit conversion tool that should work in two directions – when you define celcius value it will calculate its fahrenheit representation. And vice versa. Something similar to the form on the right:

First we will define our Data namespace:

      include "../plus.tis"; // model below uses @observing decorator defined in plus.tis  
      namespace Data // our model
        var celsius = 0; 
        var fahrenheit = 32;
        // this function is observing 'celsius' and calculates 'fahrenheit'
        @observing "celsius"
          function toFahrenheit() {
            fahrenheit = celsius * 9 / 5 + 32;
        // this function is observing 'fahrenheit' and calculates 'celsius'
        @observing "fahrenheit"
          function toCelcius() {
            celsius = (fahrenheit - 32) * 5 / 9;

Note two functions above: function toFahrenheit() is observing celcius variable. When celcius variable will change, for example as a result of changes in <input|number(celsius)> field, the toFahrenheit() function will be triggered and will set value of fahrenheit variable. As we have another input bound with the fahrenheit variable:

<body model="Data">
  <p><input|number(celsius)>°C and <input|number(fahrenheit)>°F</p>

we will see in it results of calculation. This works in both directions – from fahrenheit to celcius and from celcius to fahrenheit.

To see this alive load sdk/samples/+plus/demos/1-basic-function-binding.htm sample in sciter.exe.

That’s it for now. In the next article I’ll explain use of repeatable attribute to bind arrays of objects with rpepatable sections and other samples. If you don’t want to wait check other samples in sdk/samples/+plus/demos/ folder of the SDK. They are self descriptive.

TIScript vs JavaScript.Harmony (ECMAScript 6), Part I

Variables, block scope

As we know local variables in JavaScript are declared in function scope. That creates many problems and surprises for people who came to JS from other languages using the same Java/C/C++ notation.

Consider sample that follows. Here we create array of 10 functions, each of them shall return idx value it was created with.

var farr = [];
for(var i = 0; i < 10; ++i) {
  var idx = i; 
  farr[idx] = function() { return idx; }

If you call functions from the farr array in JavaScript you will get these results:

console.log( farr[5]() ); // -> 9
console.log( farr[2]() ); // -> 9

Surprise? I think so.

And if you run the same code in TIScript you will get results you expect:

stdout.println( farr[5]() ); // -> 5
stdout.println( farr[2]() ); // -> 2

That’s because TIScript uses block scope – all variables declared inside some block { ... } are local to that block.

To "fix" that initial design mistake (IMO) authors of ECMAScript.6 introduced new keyword let that allows to define variables in block scope, so for that sample to work properly you will do this:

var farr = [];
for(var i = 0; i < 10; ++i) {
  let idx = i; // note the 'let' here. idx is in block scope now. 
  farr[idx] = function() { return idx; }

Not that perfect of course but at least something.


TIScript from the beginning supported const keyword – declaration of read only variables:

const mask = "*.tis";

Any attempt to change value of the mask generates an error in TIScript: for local constants – at compile time and for global constants – at runtime.

In JavaScript ‘const’ is introduced in ECMAScript.6 with exactly same meaning.

Default parameters

From the very beginning TIScript supports definition of default values of function parameters:

function person( name = "anonymous", age = null ) {
   return { name: name, 
            age:age };

We can call such function without any parameters:

var p = person(); // ->  "anonymous", p.age -> null

ECMAScript.6 introduces exactly the same feature – parameters may have default values in JavaScript when browsers will catch up ECMAScript.6.

Rest parameters

In TIScript when you define function as this:

function foo( a, b, r.. ) { stdout.println(a,b,r) }

and call it as:

foo( 1,2,3,4 );

The r variable will be set to an array [3,4] and a and b will get 1 and 2 values correspondingly.

ECMAScript.6 uses slightly different syntax for the same feature:

function foo( a, b, ...r ) { console.log(a,b,r) } 

Array comprehensions

ECMAScript.6 officially introduces feature named "array comprehensions" – mechanism that allows to declare/initialize arrays using kinda declarative syntax. Array comprehensions came to JS from Python I believe. While in Python notation they probably look OK but in JS syntax the comprehensions are not readable most of the time.

Let’s say we have an array:

var numbers = [1, 2, 3, 21, 22, 30];

and we want to create another array that will have only even numbers. With the comprehensions we can define it as:

var evens = [i for (i of numbers) if (i % 2 === 0)]; 

Not quite readable in my opinion.

In TIScript I would write it as:

var evens = numbers.filter( :i: i % 2 === 0 );

– shorter and so more readable.

TIScript has no array comprehensions at the moment and probably will not have them until I’ll see really good use of them.

Promises/A+ implementation in Sciter2

The Promises, as a concept, is generalization of callback mechanism. This pattern is quite popular these days so Sciter2 SDK contains now (sdk/samples/+promise/) pretty simple (60 lines of code) implementation of the Promises.

The promise is an object that:

  1. maintins list/chain of callback function pairs [onsuccess:function, onfailure:function] by providing .then(onsuccess, onfailure) method;
  2. promise object provides the way to "execute" the chain, either succes or failure callbacks (if an error occurs);
  3. each callback function in the chain receives input (parameters) from output (return [values]) of previous callback in the chain.

To create the promise in Sciter simply do this:

var oath = promise();

The promise() function and promise object

The promise() function in my implementation returns function/object that has .then() method defined on it. So to attach callback functions to the promise you will do this:

oath.then( function( data ) { return [data+1] } ) // #1
    .then( function( data ) { return [data+2] } ) // #2
    .then( function( data ) { stdout.println("success:", data)}, // #3 
           function( reason ) { stderr.println("error:", reason)} );

Now we have promise in variable oath that has three onsuccess functions assigned to it.

When time comes for the promise to be fulfilled, our code will do it by invoking the promise (as it is a function) with its first parameter set to true and with additional parameters that will be passed to the first callback in the chain:

oath(true, 1);

This will call first callback with 1 in data. It will return 1 + 1 -> 2.
That 2 value will be passed to second callback that will return 2 + 2 -> 4.
And finally last callback will just do println:


To reject the promise we just need to call it with first parameter set to false:

oath(false, "something went wrong!");

This will call our sole onerror callback and we will get:

error: something went wrong!

The promise.when() function, parallel execution

The promise has also defined static function promise.when(...)  that accepts list of promises and return another promise that will be fullfilled/rejected when all input promises will be completed.

function printBandC(b,c) { stdout.println(b,c) }

var BandC = 
    promise.when( self.request(#get-json, urlB),
                  self.request(#get-json, urlC)).then(printBandC);

There are quite many articles about the subject, just google for “Promises JavaScript”

Here is the full source of promise.tis module:

Continue reading “Promises/A+ implementation in Sciter2”

Q.tis – micro port of jQuery for Sciter.

I’ve published today Sciter with q.tis – micro-port of essential jQuery features.
Here is the list of supported functions.

It is just enough to put include "t.tis"; in your code and any existing DOM function that returns array of elements will
“automagically” produce the q-collection.

In my implementation I am using the fact that functions like Element.selectAll("selector") return array object that is instanceof ElementList.
ElementList as any class is extensible in run-time. So I’ve just added bunch of function ElementList.jqueryMethod() {} to bring that functionality to the Sciter.

The beauty of the approach is that you can use as the q() function (analog of $() in jQuery) as Sciter’s standard $$() to write something like this:

var itemsWithLinks = $$(ul.topics > li).$has(a:link);

to get list of list items that have <a>’s inside.
The same but in classic style and without use of “stringizers”:

var itemsWithLinks = q("ul.topics > li").has("a:link");

You may also find handy jquery-alike event handling. Methods element.on(),, and element.trigger() are available as for q-collections as for DOM element instances.