Handle-based models with Handly
by Vladimir Piskarev
Summary
Handle-based models are all around us in the Eclipse IDE: on top of the programming language agnostic Platform resource model, language-specific code-centric models of the underlying workspace form a basis for Eclipse Java Development Tools (JDT), C/C++ Development Tooling (CDT), and other Eclipse-based development tools. Traditionally, such models were built either entirely from scratch or by forking and modifying preexisting models. Handly, an incubating technology project at Eclipse, begs to differ with the traditional approach. At its core, it can be described as a class library for handle-based models, with special emphasis on code-centric models of the Eclipse workspace. This article, intended as a short introduction to the problem area, is hoped to raise awareness of the project within the wider Eclipse community. The article is supplemented with a slide deck.
By Vladimir Piskarev (1C), Project Lead, Handly
December 9, 2015
It goes without saying that handle-based models play an important role in the Eclipse IDE, with the workspace resource model being the foremost example.
A handle, in this context, is an object that acts like a key to a model element and has the following principal characteristics:
- immutable, equals by value
- can define behavior of the element, but doesn’t keep any element state beyond the key information
- can refer to non-existing elements
Handle-based models employ a variation of the handle/body idiom, where clients have direct access only to handles to model elements, while the element state beyond the key information is stored separately in an internal body. Such design has a number of important properties:
- handles are stable, you can freely keep references to them
- handles are lightweight but can be rich in behavior
- bodies can be computed on demand and virtualized
This makes handle-based models highly scalable and perfectly suited to presenting in Eclipse views such as Project Explorer, Search, Outline, etc.
The workspace resource model is, of course, programming language agnostic, so Eclipse development tools such as JDT (Java Development Tools) and CDT (C/C++ Development Tooling) are based on language-specific handle-based models that define a code-centric view on the underlying workspace and support navigation down to structural elements inside source files.
Traditionally, such models were built either entirely from scratch or by forking and modifying preexisting models. The process was tedious and error-prone, and the resulting models were effectively silos with a completely isolated API, which prevented a possibility of developing reusable IDE components around those models, although the models did seem to have certain traits in common.
The Handly project (eclipse.org/handly) begs to differ with the traditional approach. Basically, it provides a unified architecture and a set of flexible building blocks that help create handle-based models similar in design patterns and principles to the JDT Java model. This reduces programming effort, fosters software reuse and provides opportunities for interoperability.
When designing the new API, we attempted to distill the truly fundamental abstractions and aimed at a deeper introspection on what a common language infrastructure should include for ‘the model part’, and what would be better left to the model implementor. The framework imposes almost no restrictions on the shape of attainable models or on the languages modeled. At the same time, the API is rich enough to enable such features as comprehensive working copy support, including out-of-the-box integration of the working copy facility with the Xtext editor and support for integrating other source editors.
The core framework defines common protocols for elements of handle-based models, with special emphasis on code-centric models of the Eclipse workspace. It expects model elements to implement certain basic interfaces such as IHandle, ISourceElement and ISourceFile and provides skeletal implementations of these basic interfaces to minimize the effort required to implement the model. Naturally, the model implementor is free to define model specific interfaces such as IFooProject, IFooFile and IFooFunction and make model elements implement those interfaces in addition to the basic interfaces.
The provided implementation gives a lot of freedom when resolving handles to find the corresponding body. For example, using supplied building blocks a model can maintain an LRU cache of element bodies to set a cap on its memory footprint.
The core framework also defines common interfaces for change notifications in a handle-based model such as IHandleDelta and IElementChangeEvent and provides complete implementations of these interfaces that can be used as they stand or extended as circumstances warrant. It also includes infrastructure interfaces and implementations that provide essential support for the model such as buffer and cache management.
Of course, the devil is in the detail, and there’s more to it than that, but in a nutshell that’s it. Please refer to the System Overview (written in early 2014, but still relevant) for a more detailed discussion of the core framework.
The common protocols established by the core framework make it possible to develop generic IDE components dealing with Handly-based models. Some common UI components such as an outline framework, quick outline and working set support are already supplied by the project both as a proof of concept and as a way to deliver more value to our adopters.
Handly has been about two years in development at Eclipse. The 0.4 version has just been released, with many API improvements, model adaptation facility, and other significant enhancements. Handly is successfully used in two commercial products, with quite positive feedback from the early adopters. We have provided a comprehensive getting started guide and a number of exemplary implementations including a Java model example.
In the 0.5 release, tentatively scheduled to coincide with Eclipse Neon, we’ll try to stabilize the core API, in preparation for a later 1.0 release. The problem is that we’re lacking broader community feedback and participation, which would be required in order to truly finalize the API and graduate from the Incubation Phase.
Involving the Eclipse IDE community seems to be a tough question, if history is any guide… Development Tools Factorization is apparently a long-standing problem in Eclipse. There are, of course, pragmatic reasons for that. A lack of resources along with backward compatibility concerns immediately come to mind, among other things.
The situation is quite unfortunate, however. The factorization problem is important and will become increasingly important as more *DT projects are created: Eclipse is not just a Java IDE. This puzzle can only be solved together, and I believe that Handly might serve as a first piece, in some way or another. The project’s success crucially depends on more significant community involvement and suitable joining of forces.
I hope that this short introduction will raise awareness of the project within the wider Eclipse community. Any help or feedback would be greatly appreciated. Thank you very much for your attention!
I think I have read all you blog posts. I really appreciate the effort you make to explain your framework and your approach.
I am just a Java developer and I never worked on any “*DT” project… I am fascinated by those tools, but it also takes time to step into this domain. This is why I do not think that I can help you with your project.
I also guess that they are not enough resources to work on a reimplentation of JDT or CDT using handly. Maybe tooling for a new languages could be interesting (I have typescript in mind). Using the handly framework should also bring additional value… For example, when you have some tooling based on handly, how complicated it is to get an IDE based on JavaFX (reusing the codeeditor elements provided by e(fx)clipse project)
Hi jmini,
Thank you for reading all of this and for your valuable remarks. I really appreciate it.
There are numerous ways to contribute, ranging from leaving a comment just like you did, to joining the handly-dev mailing list and participating in design discussions there, to building on top of Handly (not only development tools but also other, higher-level frameworks), right through contributing directly to the project’s code base. If you ever happen to find a little time, going step-by-step through the getting started guide to Handly [1] might be a good way to learn how all the pieces fit together and to study some of the underlying design patterns that make many Eclipse development tools tick.
To be honest, I don’t know much about JavaFX or e(fx)clipse to make a worthwhile comment now, but thanks for the pointer: I think I need to take a look at that, it might be an interesting direction for the future.
We have been working on providing more value to adopters than “just” the core framework (out-of-the-box integration with the Xtext editor, an Outline framework, a number of exemplary implementations are a few examples), and will continue to do so (I think we could provide some support for the Search view, for instance), but understandably, we can only do so much, given the limited community participation. In this situation, there is no other reasonable way for me but to concentrate on the most important part: the core framework.
Regarding a possible reimplementation of existing *DT projects on top of a common language infrastructure, of which Handly might become a part in some form or another… It will not probably happen anytime soon. It is, of course, a complex problem, both technically and organizationally. But I’d like to articulate, once again, the significance of the problem, which will not become less important as time goes on and more *DT projects are created. Solving it would allow for sharing maintenance costs between project teams, probably to a great extent. All of the Eclipse IDE community could benefit from this.
To give you just one example, when conducting some scalability tests for Handly, we discovered that building large deltas for a model change notification was really slow [2]. We got it fixed in Handly and contributed a patch to JDT, where the code had been originally adapted from. But we cannot possibly provide a corresponding fix to each Eclipse *DT project, much less to non-Eclipse DT projects out there. It is also worth noting that merging the patch to JDT made it possible to discover a bug in the contributed code very soon, which was also lurking in Handly [3]. The bug has been fixed first in JDT and then in Handly code base. We can try to make a few general observations from this experience:
1. It was of mutual benefit both for Handly and for JDT.
2. The process was not very effective: it required much more effort and round trips than would have been necessary if we had a shared code base.
3. Other *DT projects were excluded and could not benefit from this valuable exchange.
And one more thing. Forking and modifying JDT code may be fine, if you can afford it. But even if you can, it doesn’t mean that you should. We created Handly for the sake of art as much as anything else. But it cannot deliver on its promise in isolation. The future of the project depends in a very real sense on participation of the Eclipse IDE community.
Thanks,
Vladimir
[1] https://github.com/pisv/gethandly/wiki
[2] https://bugs.eclipse.org/bugs/show_bug.cgi?id=443813
[3] https://bugs.eclipse.org/bugs/show_bug.cgi?id=456060