vladimir piskarev's blog

Musings on the Eclipse Handly project and software development in general

Announcing Handly 0.3 Release

It is with great pleasure that I announce the on-time availability of the Handly 0.3 release, symbolically coinciding with Eclipse Mars. This release includes major new features, such as an outline framework, integration with text editors (in addition to preexisting integration with Xtext editor), a real-world exemplary implementation (a simplified Java model), and a number of API enhacements. This release is primarily focused on bringing more value to early adopters while further evaluating the Handly core framework.


Handly is a relatively new project at Eclipse started with an initial contribution from 1C. In brief, it can be described as a framework for handle-based models — with an emphasis on code models that render Eclipse workspace from a programming language angle.

Because the rest of the IDE usually depends on the code model, the quality of the design and implementation of that model is of the utmost importance. Traditionally, such models were built either entirely from scratch or by forking and modifying preexisting models. The traditional process required much effort, was tedious and error-prone. The resulting models were effectively silos with a completely isolated API, which prevented a possibility of developing reusable IDE components, although the models did seem to have certain traits in common.

The Handly project begs to differ with the traditional approach. It provides a set of flexible building blocks that help developers create handle-based models similar in design principles to the JDT Java model. The framework imposes almost no restrictions on the shape of the models or on the languages modeled. The uniform API makes it possible to develop generic IDE components dealing with such models — some common UI components are already provided by Handly. The intent is to come up with a really nice design in this problem area through an open and transparent development process primarily driven by community feedback. For more information, please visit the project’s website.

Despite the “incubating technology” status of Handly, there already are three known major adopters (in chronological order): 1C:Enterprise Development Tools, erlide (in yet-to-be-released version 0.30), and Codasip Studio — two commercial products and an esteemed open-source IDE. We are very grateful for their vote of confidence to the project, active participation and valuable feedback. With this release, we hope to encourage further adoption and receive broader feedback.

Some design discussions with early adopters are already taking place on the project’s forum, but also (as it stands, chiefly) on the handly-dev mailing list, with potentially even more interesting discussions soon to follow.

So if you are interested in this project and would like to have a say in it, I encourage you to engage as early as possible by playing around with this release and giving feedback.

Thank you!

Handly 0.3 : Integration with TextFileBuffer-based editors

Besides providing out-of-the-box integration with Xtext editor, Handly can also support other editors. In particular, it allows for integration with TextFileBuffer-based editors. The following notes are intended as forward pointers only; full-blown editor implementation is beyond the scope of this article.

To integrate a TextFileBuffer-based editor with working copy functionality, you’ll need to subclass the editor’s document provider from the class SourceFileDocumentProvider (which extends the TextFileDocumentProvider).

For example:

public class FooFileDocumentProvider
    extends SourceFileDocumentProvider
    public FooFileDocumentProvider()
        super(new FooElementForEditorInputFactory());
        // ...

    // ...

You’ll also need to implement a Handly-based reconciler for the editor:

public class FooReconciler
    extends HandlyReconciler
    public FooReconciler(ITextEditor editor, IWorkingCopyManager manager)
        super(editor, manager);

    protected void addElementChangeListener(IElementChangeListener listener)

    protected void removeElementChangeListener(IElementChangeListener listener)

The reconciler is connected to the editor via a source viewer configuration:

public class FooSourceViewerConfiguration
    extends TextSourceViewerConfiguration
    private final ITextEditor editor;
    private final IWorkingCopyManager manager;

    public FooSourceViewerConfiguration(IPreferenceStore preferenceStore,
        ITextEditor editor, IWorkingCopyManager manager)
        this.editor = editor;
        this.manager = manager;

    public IReconciler getReconciler(ISourceViewer sourceViewer)
        if (editor != null && editor.isEditable())
            return new FooReconciler(editor, manager);
        return null;

Then, you can wire it all together in the editor:

public class FooEditor
    extends AbstractDecoratedTextEditor
    protected void initializeEditor()
        FooFileDocumentProvider provider =
        setSourceViewerConfiguration(new FooSourceViewerConfiguration(
            getPreferenceStore(), this, provider));

Note that the functionality described in this article is only available since Handly 0.3, which will be released tomorrow (June 24th). The initial API should be considered provisional. Feedback can be directed to the project’s forum or right to the developer mailing list (the list requires subscription).

Handly 0.3 : API enhacements

Handly 0.3 includes a number of API enhancements; some of them are breaking changes. Here I would like to focus on the most important ones:

ISourceElement: remove hard dependency on ISourceFile (Bug 461262)

Source elements no longer have the getSourceFile() method. Such method, if necessary or desired for a specific model, can be provided by the model implementor, probably in model-specific terms.

The problem with this method was that it had been originally specified to return the source file containing the source element and could not return null, assuming, thus, that source elements are always contained in a source file in the workspace. It was realized that this assumption didn’t hold for models like the JDT Java model that support source attachment for “binary modules” such as class files. For example, in the Java model, both ICompilationUnit and IClassFile consist of pretty much the same elements: ITypes, IMethods and IFields, which all extend the ISourceReference interface (that roughly corresponds to ISourceElement in Handly).

Fortunately, this design flaw was fixed in Handly 0.3. Unfortunately, a few breaking changes had to be introduced to make it possible. As a rule, we try to preserve backward compatibility, but in this case it was not feasible, and it is hoped that future benefits should outweight present costs.

Here’s the complete list of breaking changes related to this redesign around source elements:

  • ISourceElement#getSourceFile() method got removed
  • ISourceElementInfo#getSnapshot() may now return null (if no source is available)
  • ISourceElementInfo#getFullRange() and #getIdentifyingRange() may now return null, but may no longer return a NULL_RANGE (see below)
  • TextRange#isNull() and #NULL_RANGE got removed. The special source range with a -1 offset and a 0 length was a JDT legacy that I did not readily identified as such. It would be very painful to deprecate it (JDT couldn’t): that’s why we had to do it as soon as possible. For a bit of history, see bug 130161.

Deprecate ISourceFileFactory (Bug 464588)

The interface ISourceFileFactory has been superseded, for all intents and purposes, by IElementForEditorInputFactory, and may be removed in a future release. The new interface is more generic and capable. There is little point in having an imperfect API except for backward compatibility: ISourceFileFactory was allowed (and intended) to be implemented by clients.

So, if you have previously implemented ISourceFileFactory (and, for Xtext languages, bound it in the Xtext UI module for the language, as was required by Handly/Xtext integration), please consider implementing (and binding it in the UI module, for Xtext languages) the new interface IElementForEditorInputFactory instead. A typical implementation might look like this.

Handly 0.3 : Outline framework and quick outline support

A major new feature of the upcoming 0.3 release of Handly is an outline framework (org.eclipse.handly.ui.outline package). Only a part of it depends on the handle-based model API provided by Handly; much of the code is fairly generic and can be reused in other contexts, thanks to a multi-layered design. Handly 0.3 also includes quick outline support (org.eclipse.handly.ui.quickoutline package), which nicely complements the outline framework. A detailed description of both facilities is beyond the scope of this note, but the Basic Example (org.eclipse.handly.examples.basic.ui) in Handly SDK now uses the outline framework and offers quick outline in the Foo editor; it might be a good place to start. The API is fairly well documented (in Javadocs) and is expected to be quite easy to use.

As a teaser, compare the from-scratch previous implementation of the outline page in the Basic Example with the corresponding new implementation that uses the outline framework. The new implementation is both richer in ability and less in size. And you can easily do a lot more with it: this more advanced implementation provides problem marker decoration and a user-toggleable filter with just a few additional lines of code.

The initial API should be considered provisional. I would like to invite potential adopters to take a look and leave a feedback; as always, it would be much easier to fix any discovered deficiences early on. Feedback can be directed to the project’s forum or right to the developer mailing list (the list requires subscription).

Handly 0.3 : Example model for Java

A highlight of the upcoming Handly 0.3 release (just a few days away) is a Java model example (org.eclipse.handly.examples.javamodel*), which demonstrates a Handly-based model for the Java language along the lines of the JDT Java model.

Back in Handly 0.1, a basic exemplary implementation was provided: a complete Handly-based model for a simple Xtext-based language, with the outline page and navigator view implemented atop the model. Intentionally contrived to be simple so as not to explain a lot of stuff that isn’t Handly-related, but at the same time sufficient for demonstrating many design and implementation aspects, this example provided the basis for a later multi-part tutorial, which has received some acclaim from early adopters.

It was obvious, though, that a more advanced example was really needed. Such an example would be called for to demonstrate how to implement some of the more advanced features with Handly and, ideally, spur further Handly adoption.

In particular, the following requirements for this example had been gathered via the project’s forum:

  • “Real-world”;
  • Demonstrate how to deal with a classpath-like concept (e.g. source folders);
  • Exemplify model integration with a custom text editor (i.e. non-Xtext).

After a discussion with early adopters, it was decided that an example model for Java would fit the bill. It could exemplify all the features mentioned above and would be sufficiently real-world (despite the inevitable simplifications vs. the full-fledged JDT implementation). Besides, adopters already know Java (and probably JDT internals too), so it sounded like a useful common ground. Also, JDT provides some important services (like parsing) that could be reused by the example code.

So, I set to work on the Java model example. The outcome of this work for Handly 0.3 is the core model and a bunch of tests. There is no corresponding UI yet. As one whould expect from an example, the model is somewhat contrived. In particular:

  • Only classpath entries of kind CPE_SOURCE, with paths of length 1 are considered (i.e. only source folders that are direct children of the project resource);
  • Inclusion/exclusion filters are ignored — everything is included;
  • Only default output location is supported, with path of length 1;
  • Initializers and local/anonymous types are not represented in the model.

Some of these constraints may be relaxed in the future (especially based on community feedback).

The following UI parts are expected to appear in a subsequent release:

  • Example Java editor, with Handly-based reconciling and outline;
  • Example Java navigator view.

It is hoped that this example will be useful to Handly adopters. In particular, I tried to craft the series of commits carefully to show how one can grow the model organically. For instance, it can be seen that at one point there was no effective model cache/delta processor, only a bare model structure that, however, was fully functional and testable. Only after the model structure was fully fleshed out, the model cache/delta processor were implemented. This is the way I would recommend for developing a Handly-based model.

Feedback is most welcome and can be directed to the project’s forum or right to the developer mailing list (the list requires subscription).

On M-theory and H-models, or can EMF fit every nail

Those readers who are interested in modern physics may know about M-theory that seems to be gathering a lot of hype in recent years. It is the latest toy of theoretical physicists, which many think could become the ultimate “theory of everything”. It is a subject of ongoing debate, however, whether there actually can be a single unified theory capable of explaining everything or, instead, it is a set of theories, each describing different aspects of the universe (with some overlap). And if one adopts model-dependent realism, it doesn’t really matter :-)

In Eclipse world, EMF has been steadily gaining traction. Rightfully, EMF is such a nice, flexible tool that many problems start looking like a nail. But can an entire IDE (like JDT) be essentially EMF-based? Xtext might lead many to believe so, but let’s take a closer look and see how JDT is structured from the modeling point of view. After all, it reflects thirty years of experience!

It may be surprising that there are at least three different models in JDT, each describing different aspects of the IDE (with some overlap) and having very distinctive characteristics of its own:

  • Java model

    I like to think of the Java model as the IDE ground (and glue). This model is the common foundation for many core services as well as all views and actions in JDT. One can also think of it as a Java-oriented projection of the Eclipse workspace.

    The Java model is a handle-based model (H-model for short). Its elements are handles: value objects that act like a key to a model element. This gives it a number of special properties that are very useful for a large-scale in-memory model of the whole IDE:

    • Stable (elements to which a reference can be kept, e.g. to show in a view)
    • Light weight (model non exhaustive, no resolved information kept)
    • Lazily populated, LRU cache (doesn’t hold on resources)
    • Eventually consistent (need not be consistent all the time, allows “dirty reads”)
    • Thread safe without imposing much locking (fosters high concurrency)

    That’s quite unique combination of properties, if you think of it. It’s not a mere coincidence that the underlying workspace resource model is also an H-model (with slightly different properties).

  • Abstract Syntax Tree (AST)

    Each AST node represents a Java source code construct, such as a name, type, expression, statement, or declaration. The parser parses a string containing Java source code and returns an AST for it.

    It is important to note that the AST is really a tree (no cross-references). Since ASTs tend to consume a lot of memory, JDT holds only one AST, for the currently active editor, reconciling it as necessary. Clients should not keep references to elements of the AST.

  • Bindings

    When an AST is created by the parser, there is an option to also create bindings. A binding represents a named entity in the Java language, such as a package, type, field, method, constructor, or local variable.

    The world of bindings (an environment) provides an interconnected picture of the structure of the program as seen from the compiler’s point of view. This is the only JDT model with graph structure, i.e. cross-references. One can think of it as the semantic model of the language.

It seems only one of those models (Bindings) can be naturally represented with EMF. Another (AST) might fit EMF, with some caveats. The remaining one (Java model) is not a good fit for EMF at all. (Beg to differ? You are welcome to leave a comment. I may be missing something.) It looks like “M-theory” for Eclipse-based IDEs (where ‘M’, in this case, stands for Modeling) would indeed include a set of models with different goals, properties, and requirements to the underlying implementation technology.

In particular, the common needs of H-models like the Java model are to be addressed in Handly, an Eclipse technology project that is about to turn one year old. Handly (at its core) supplies basic building blocks that help developers create their own H-models.

A nice bonus is that many useful IDE components (micro-frameworks) could be implemented atop the common API for H-models provided by Handly — there must be a considerable potential for unification. As an example, the upcoming Handly 0.3 release, scheduled to coincide with Eclipse Mars, will provide an outline framework (including quick outline support) for H-models. The outline functionality is already in place, actually, in the project’s git repository and latest I-builds.

The vision behind Handly is not that of an all-encompassing IDE framework with its many inherent assumptions. Rather, it envisages a set of loosely-coupled micro-frameworks around H-models — that common IDE ground. If you are interested in giving Handly a try, there’s a multi-part tutorial on GitHub. Beware though: if you are new to H-models, it takes some getting used to. But if you are keen to know what principles make JDT (or other Eclipse-based IDEs) tick as a seamless whole (the *I*DE), I think it might be worth it.

Of course, Handly doesn’t prescribe a specific implementation technology for the other IDE models. They could be EMF-based (like in Xtext; in fact, Handly already includes a layer of integration with Xtext) or something else; it doesn’t matter much when one adopts model-dependent realism!

Announcing Handly 0.2.1 Release

I am pleased to announce the availability of the Handly 0.2.1 release. This service release addresses a few issues found in the 0.2 version.