vladimir piskarev's blog

Musings on the Eclipse Handly project and software development in general

Announcing Handly 0.5

I am very pleased to announce the availability of the Handly 0.5 release, a true “2.0” version in spirit. This release introduces an entirely new design that gives the implementor of a Handly-based model complete control over the model API. Among other things, this should make it possible to use Handly for (re-)implementing handle-based models where the model API is a given, just as in the case of a preexisting model API that needs to be preserved for backward compatibility. Many of the core APIs have been revised in this release to make Handly even more flexible and robust.

New and Noteworthy
Migration Guide
Downloads

Any feedback would be greatly appreciated.

Many thanks to all who contributed to this release:

  • Ondrej Ilcik, head of IDE Team at Codasip, has kindly contributed a navigator view for the Java model example, which made its way into this release. This is the first significant (~ 4K LOC) contribution to the project by a non-committer. Codasip Studio is one of the earliest adopters of Handly, and it is really great to see that a major adopter becomes a significant contributor. Also, Ondrej has been so kind as to share a great success story of Codasip Studio and Handly, which is all the more valuable as the first success story published by a Handly adopter.
  • Vlad Dumitrescu, the project lead of erlide, has been actively participating in discussions on the project’s mailing list. His earlier feedback helped inspire the new design introduced in this release, and he has also contributed great ideas about restructuring the project’s web page to make it more readable.
  • Peter Gribanov, 1C, has contributed a success story describing how 1C:Enterprise Development Tools, the earliest adopter of the project, is using Handly.

Thank you for contributing!

Handle-based models with Handly

Summary
Handle-based models are all around us in the Eclipse IDE: on top of the programming language agnostic Platform resource model, language-specific code-centric models of the underlying workspace form a basis for Eclipse Java Development Tools (JDT), C/C++ Development Tooling (CDT), and other Eclipse-based development tools. Traditionally, such models were built either entirely from scratch or by forking and modifying preexisting models. Handly, an incubating technology project at Eclipse, begs to differ with the traditional approach. At its core, it can be described as a class library for handle-based models, with special emphasis on code-centric models of the Eclipse workspace. This article, intended as a short introduction to the problem area, is hoped to raise awareness of the project within the wider Eclipse community. The article is supplemented with a slide deck.

By Vladimir Piskarev (1C), Project Lead, Handly
December 9, 2015


It goes without saying that handle-based models play an important role in the Eclipse IDE, with the workspace resource model being the foremost example.

A handle, in this context, is an object that acts like a key to a model element and has the following principal characteristics:

  • immutable, equals by value
  • can define behavior of the element, but doesn’t keep any element state beyond the key information
  • can refer to non-existing elements

Handle-based models employ a variation of the handle/body idiom, where clients have direct access only to handles to model elements, while the element state beyond the key information is stored separately in an internal body. Such design has a number of important properties:

  • handles are stable, you can freely keep references to them
  • handles are lightweight but can be rich in behavior
  • bodies can be computed on demand and virtualized

This makes handle-based models highly scalable and perfectly suited to presenting in Eclipse views such as Project Explorer, Search, Outline, etc.

The workspace resource model is, of course, programming language agnostic, so Eclipse development tools such as JDT (Java Development Tools) and CDT (C/C++ Development Tooling) are based on language-specific handle-based models that define a code-centric view on the underlying workspace and support navigation down to structural elements inside source files.

Traditionally, such models were built either entirely from scratch or by forking and modifying preexisting models. The process was tedious and error-prone, and the resulting models were effectively silos with a completely isolated API, which prevented a possibility of developing reusable IDE components around those models, although the models did seem to have certain traits in common.

The Handly project (eclipse.org/handly) begs to differ with the traditional approach. Basically, it provides a unified architecture and a set of flexible building blocks that help create handle-based models similar in design patterns and principles to the JDT Java model. This reduces programming effort, fosters software reuse and provides opportunities for interoperability.

When designing the new API, we attempted to distill the truly fundamental abstractions and aimed at a deeper introspection on what a common language infrastructure should include for ‘the model part’, and what would be better left to the model implementor. The framework imposes almost no restrictions on the shape of attainable models or on the languages modeled. At the same time, the API is rich enough to enable such features as comprehensive working copy support, including out-of-the-box integration of the working copy facility with the Xtext editor and support for integrating other source editors.

The core framework defines common protocols for elements of handle-based models, with special emphasis on code-centric models of the Eclipse workspace. It expects model elements to implement certain basic interfaces such as IHandle, ISourceElement and ISourceFile and provides skeletal implementations of these basic interfaces to minimize the effort required to implement the model. Naturally, the model implementor is free to define model specific interfaces such as IFooProject, IFooFile and IFooFunction and make model elements implement those interfaces in addition to the basic interfaces.

The provided implementation gives a lot of freedom when resolving handles to find the corresponding body. For example, using supplied building blocks a model can maintain an LRU cache of element bodies to set a cap on its memory footprint.

The core framework also defines common interfaces for change notifications in a handle-based model such as IHandleDelta and IElementChangeEvent and provides complete implementations of these interfaces that can be used as they stand or extended as circumstances warrant. It also includes infrastructure interfaces and implementations that provide essential support for the model such as buffer and cache management.

Of course, the devil is in the detail, and there’s more to it than that, but in a nutshell that’s it. Please refer to the System Overview (written in early 2014, but still relevant) for a more detailed discussion of the core framework.

The common protocols established by the core framework make it possible to develop generic IDE components dealing with Handly-based models. Some common UI components such as an outline framework, quick outline and working set support are already supplied by the project both as a proof of concept and as a way to deliver more value to our adopters.

Handly has been about two years in development at Eclipse. The 0.4 version has just been released, with many API improvements, model adaptation facility, and other significant enhancements. Handly is successfully used in two commercial products, with quite positive feedback from the early adopters. We have provided a comprehensive getting started guide and a number of exemplary implementations including a Java model example.

In the 0.5 release, tentatively scheduled to coincide with Eclipse Neon, we’ll try to stabilize the core API, in preparation for a later 1.0 release. The problem is that we’re lacking broader community feedback and participation, which would be required in order to truly finalize the API and graduate from the Incubation Phase.

Involving the Eclipse IDE community seems to be a tough question, if history is any guide… Development Tools Factorization is apparently a long-standing problem in Eclipse. There are, of course, pragmatic reasons for that. A lack of resources along with backward compatibility concerns immediately come to mind, among other things.

The situation is quite unfortunate, however. The factorization problem is important and will become increasingly important as more *DT projects are created: Eclipse is not just a Java IDE. This puzzle can only be solved together, and I believe that Handly might serve as a first piece, in some way or another. The project’s success crucially depends on more significant community involvement and suitable joining of forces.

I hope that this short introduction will raise awareness of the project within the wider Eclipse community. Any help or feedback would be greatly appreciated. Thank you very much for your attention!

Watch the slides

Announcing Handly 0.4 Release

On behalf of all who contributed, I am pleased to announce the availability of the Handly 0.4 release. This release is focused on API quality and includes many significant API improvements. It also includes notable new features (like the model adaptation facility), additional exemplary implementations, and many enhancements.

New and Noteworthy
Downloads

Any feedback would be greatly appreciated.

Handly 0.3.2 released

I am pleased to announce the availability of the Handly 0.3.2 release. This service release addresses an issue in Handly/Xtext integration that affected Handly 0.3 and 0.3.1 when used together with Xtext 2.8.3 or later.

Handly 0.3.1 released

I am pleased to announce the availability of the Handly 0.3.1 release. This service release addresses issues found in the 0.3 version.

Handly — Support for legacy models

You might ask… Handly looks interesting, but can it be of any use if I already have a (handle-based) code model with a well-established API? (Or, perhaps, you’d like to use Handly in a greenfield, but are a bit uncomfortable about the leakage of Handly API to your model’s clients.)

If this sort of questions makes you delay considering Handly adoption, perhaps you should wait no longer. Handly can support legacy models while still providing the benefits of a uniform handle-based API. Now when the dust settled after the Handly 0.3 release, let me show you how.

Although Handly imposes almost no restrictions on the model shape or on the languages modeled, it still requires that model elements implement some of the provided interfaces such as IHandle, ISourceElement and ISourceFile, which might be enough to break API compatibility with an existing model. However, a handle-based design makes it straightforward to implement lightweight adapter elements to the existing model API and expose this fully backwards compatible adapter model to clients. As long as those adapter elements can adapt themselves to underlying Handly-based model elements, they can be fully supported by the generic IDE infrastructure built on top of the uniform Handly API. The adapter model does incur some additional overhead in both time and space, but the approach is architecturally sound and in many cases the overhead should be manageable.

To make it more concrete and as a proof of concept, I have implemented a subset of the JDT Java model API atop the example model for Java and pushed it to the javamodel.jdt branch in the project’s repository. (I’m hesitant about merging it to master as I think it can unnecessarily complicate the Java model example. However, I’m open to reconsidering this decision based on the community feedback.)

Once the adapter model had been in place, it was very easy to use existing JDT UI classes to build an almost fully functional Java Navigator. I said almost because, since the adapter model was based on the deliberately contrived example model, many of its methods just throw UnsupportedOperationException. However, navigation does work, and you can see all the usual actions in the context menu (although it is not recommended to actually use those actions since, as mentioned above, the underlying adapter model is incomplete and not all operations are supported; you can still use the JDT’s Package Explorer for performing the actions). I must say that implementing such an adapter to the JDT Java model API was an exciting (if a bit short) experience!

In the end, it’s all about loose coupling. Your model API may depend on Handly, which can be convenient in many cases but is not strictly required: instead, you can treat your Handly-based model as an implementation detail, exposing only the adapter model to clients. Generic IDE components (such as those already provided by Handly) may depend on the uniform Handly API, but can know nothing about specifics of a model. Finally, there may be (in theory, at least) alternative implementations of the Handly API in addition to the one provided by the project itself.

Support for legacy models as well as overall API quality are going to be important themes on our way to the Handly 0.4 release. As always, feedback is most welcome and can be directed to the project’s forum or right to the developer mailing list.

Announcing Handly 0.3 Release

It is with great pleasure that I announce the on-time availability of the Handly 0.3 release, symbolically coinciding with Eclipse Mars. This release includes major new features, such as an outline framework, integration with text editors (in addition to preexisting integration with Xtext editor), a real-world exemplary implementation (a simplified Java model), and a number of API enhacements. This release is primarily focused on bringing more value to early adopters while further evaluating the Handly core framework.

Downloads

Handly is a relatively new project at Eclipse started with an initial contribution from 1C. In brief, it can be described as a framework for handle-based models — with an emphasis on code models that render Eclipse workspace from a programming language angle.

Because the rest of the IDE usually depends on the code model, the quality of the design and implementation of that model is of the utmost importance. Traditionally, such models were built either entirely from scratch or by forking and modifying preexisting models. The traditional process required much effort, was tedious and error-prone. The resulting models were effectively silos with a completely isolated API, which prevented a possibility of developing reusable IDE components, although the models did seem to have certain traits in common.

The Handly project begs to differ with the traditional approach. It provides a set of flexible building blocks that help developers create handle-based models similar in design principles to the JDT Java model. The framework imposes almost no restrictions on the shape of the models or on the languages modeled. The uniform API makes it possible to develop generic IDE components dealing with such models — some common UI components are already provided by Handly. The intent is to come up with a really nice design in this problem area through an open and transparent development process primarily driven by community feedback. For more information, please visit the project’s website.

Despite the “incubating technology” status of Handly, there already are three known major adopters (in chronological order): 1C:Enterprise Development Tools, erlide (in yet-to-be-released version 0.30), and Codasip Studio — two commercial products and an esteemed open-source IDE. We are very grateful for their vote of confidence to the project, active participation and valuable feedback. With this release, we hope to encourage further adoption and receive broader feedback.

Some design discussions with early adopters are already taking place on the project’s forum, but also (as it stands, chiefly) on the handly-dev mailing list, with potentially even more interesting discussions soon to follow.

So if you are interested in this project and would like to have a say in it, I encourage you to engage as early as possible by playing around with this release and giving feedback.

Thank you!