System Search - Finding Files Anywhere

Now that CDT 5.0 is out the door I'm starting to review some of the work the Carbide team has done over the past year and pick out things that might make good contributions to the next CDT release.

As we talk to C++ developers who have moved to our Eclipse based tools we occasionally ask them if they are working in Carbide all of the time or still need to their old IDE environment around. The old IDE is usually Visual Studio or CodeWarrior and we found that people were switching back so they could use them to search for and in a collection of files. C++ developers often need to work with files strewn all over the file system so the regular File Search in Eclipse, which only searches files in the workspace, doesn't do enough for them.

Of course there is a bug for this but we quickly came to the conclusion that we couldn't easily modify or extend standard File Search so we created a System Search that would let you look at files anywhere in the file system.


This should look pretty familiar except for the field at the bottom that lets you pick a location in the file system to search. The results are displayed in an adaptation of the standard search results view.

We've gotten a lot of very positive customer feedback but people have also been quick to ask for more features. There is also some more work we need to do before contributing this to CDT: a little more code cleanup is needed and more testing on multiple platforms.

If you want to play around with System Search the code is attached to the bug.

Austin Eclipse DemoCamp

I'm back at my desk now trying to stay awake after a coma inducing lunch of Stubb's BBQ following the Austin DemoCamp. A bunch of Nokia people from our office joined Chris Aniszczyk and people from IBM, Freescale, and others to look at a few demos and talk about Ganymede. (That's Chris pictured along with John Cortell and Glenn Meter).

Despite developing Eclipse plug-ins all day I haven't spent much time looking at the new PDE stuff in Ganymede so it was great to talk to Chris about it first hand. I talked a bit about CDT and how we develop Carbide.

The session was also something of a reunion of former Metrowerks people: the team that developed the CodeWarrior tools was bought by Motorola years ago which then spun them out along with Freescale. The CodeWarrior for Symbian team then moved to Nokia and now develops Carbide.c++ at our office here next to the coma inducing BBQ. So a lot of people who created the CodeWarrior tools back then are working with Eclipse now. It was great to have a chance to visit with some old friends and make some new ones. I'm hoping we can continue to get the Eclipse people in Austin together now and again.

Annoucing Eclipse b4

Details were unveiled today about project b4, a new release of Eclipse focusing on the past. Project lead Paul Rubens said "We're especially interested in the Baroque era, but all the contributions to Eclipse over the centuries will be gathered in this release."

An example of the historical breadth of the project can be seen in a patch contributed by Roman Centurion Lucius Vorenus. "We were a little concerned about support as the author has been dead for over 2000 years, but his patch applied cleanly so we decided to go for it." said Rubens. "The patch introduced the ISlaveResource interface and all of his unit tests passed."

Backward compatibility is an important goal of b4. "The ruins in Cyrenaica have yielded some interesting early API inscribed in stone tablets. We plan to support them all, except the ones that were already broken when we found them."

Researchers have also deciphered a patch hidden in the ceiling of the Sistine Chapel. "We normally would only take a patch if it is attached to a bugzilla, but the sheer beauty of this one, all in the original Latin, is too good to pass up. Besides it's just an NPE fix."

The contributions to b4 often reflect the hazards of a time when many committers' work was interrupted by violence and disease. "Many of the project teams were decimated by the Black Death, and a lot of promising contributions are unfinished. They won't compile because they end with phrases like 'castle breached, the end is near' or simply 'Aaaaarrrrrrggghhh'."

Development of b4 will be completely open and transparent according to Rubens: "Certainly it will be open, just as Roman citizenship was open to anyone who lived in the city." He hastened to add "But that was later extended to most of Italy and then even to some barbarian tribes."

The final release date for b4 is still uncertain "There are several patches in the Vatican archives we want, but now the Pope is involved in the IP review process and we don't know how long it will take."

Carbide 1.3 End Game

We are in the last weeks of testing Carbide 1.3: the code is frozen, the test team is rechecking everything, and the beta testers are trying out the release candidate build. This is usually the time when some bug that has been around all along makes a surprise appearance in a really troubling way.

Last week we started getting some reports of the entire workbench locking up. People who were really beating on the tools all day said that often after a paste or save operation the UI would be unresponsive. Of course we couldn't reproduce the problem but we had two interesting clues: some people said the UI started responding again after a while and one beta tester sent in a screenshot of the frozen workbench.

Everyone has their own personal "time out" setting: the amount of time they are willing to wait for an unresponsive application to start working again before they kill it or restart their computer or chuck it over the cube wall. So that some people said the workbench came back to life after a while probably indicated we had some lengthly process blocking the UI thread, not that Carbide was frozen forever.

The screenshot gave us a few more clues: the progress indicator said the CDT indexer job was just starting and that the text the user was pasting hadn't yet appeared in the editor. I started trying to reproduce the problem by forcing a project to reindex and then repeatedly pasting code.

After a lot of pasting we found that the problem had nothing to do with the paste operation but with the use of the control key (as in ctrl-v to paste or ctrl-s to save). With the control key down CDT was asked to produce a hyperlink. Normally this happens quickly because the file has already been indexed and the results cached, but when it's dirty it proceeds to do some indexing on the UI thread which leaves the workbench completely unresponsive.

It seemed like the right thing to do in this case was to just not do the indexing if the cache was stale. We turned to the CDT team to confirm this and within a few hours we had changes in place to prevent this problem and a couple other related issues too.

So now I'm starting this week like I did the last: hoping Carbide 1.3 can slide out the door without any serious issues popping up.

Looking at DSF

Our next major release of Carbide.c++ will be based on Ganymede which will include the quickly maturing Debug Services Framework. The DSF team has been busy joining the framework with CDT 5.0 and beefing up the reference implementation that works with gdb on linux.

The Carbide.c++ debugger uses the CDI apis to let our debug engine provide services to the common C++ debug support in CDT. That has worked pretty well but there are lots of new things we would like to do that would be a lot easier using DSF so I'm going to start looking into bringing up our debugger in this new, more flexible, and highly asynchronous environment. I haven't looked at DSF in any depth since last September and there has been a lot of refactoring since then but now that things have settled down with M5 I'm going to dive back in.

Bug Hunting in Big D

The people in our beta group have been really positive while sending plenty of constructive criticism about things we need to improve. It's been very satisfying working with them to track down some of the bugs that escaped us last time. One was a report about debugger performance: some people said that when debugging they found that "stepping through code was really slow."

These kinds of problems can be really difficult to reproduce and fix because there are so many variables. Only a few people were reporting the problem and it wasn't consistent. We couldn't reproduce it at all.

A lot of things can impact debugger stepping performance including the speed of the debug connection and the number of variables displayed each time, but none seemed to be a factor in this case.

As people ran into the problem we started to get more specific information: we heard that not only was stepping in the debugger slow but the entire Eclipse environment became really slow. Then someone noticed that everything on their computer was slow: every application, menu etc.

Next we heard from a developer on one of Nokia's software teams in Dallas. He could reproduce the problem but it only happened when debugging a particular project. I setup the same project and tried the same steps but couldn't reproduce the problem.

A trip to Dallas from our office here in Austin can easily occupy a long day, much of it spent in traffic, so first we tried a screen sharing session. That confirmed I was following the same repro steps with the project but getting different results: my debug session hummed along while my Dallas colleague's ground to a halt with the Java VM process sucking up 80 - 90% of the cpu. But the most interesting thing was that the application he was debugging (a music player) was sending a huge amount of text to the console view. The entire time a song was playing the console view was flooded with debug output. After playing a few songs the audio began to skip and the big slow down began.

I pulled a few long songs from my music collection and while my console view was also flooded with various debug messages I didn't see any performance hit. Time for a road trip.

When I got to Dallas we quickly reproduced the problem and all the evidence pointed to the console output. The platform debug console has a setting that limits the amount of text in a console. My first theory was that we were overloading whatever method keeps the console under this limit. Maybe we were dumping in so much text it was thrashing about trying to keep the limit enforced?

Eclipse does so much cool stuff for you automatically I assumed the console limit worked that way too. But that's built into the debug platform's ProcessConsole and for various reasons ours is built on top of MessageConsole instead. So we didn't have any limit on the amount of text in a console. Over time all the text dumped into our console consumed more and more memory.

Limiting the amount of text in the console view was simple enough and has fixed the problem. I'm still not sure why the effect of the big memory leak on my Dallas colleague's computer was so severe while my similarly configured laptop wasn't bothered. In any event the whole experience shows how initial descriptions of bugs can be misleading and that you need work closely with the people using the tools all day in order to track down difficult to reproduce problems.

Carbide.c++ beta winding down

Our Carbide.c++ tools for Symbian OS development have spent the last several months in front of hundreds of people in our beta program. Now it's time to wrap things up and get ready to release. For this update we moved to version 3.3.1 of the platform and 4.0.3 of CDT, added lots of little enhancements, fixed hundreds of bugs, and posted about a dozen patches for CDT and the platform.

People really like the improvements to the indexer and code navigation tools in CDT and gave us lots of feedback on where they could be improved, especially support for C++ templates. All the positive response was tempered by continued frustration with the way projects have to be organized so we are still looking for solutions to that problem.

Symbian OS projects have their own build system and our support for it is now better integrated into the new CDT project model. Our debugger now takes better advantage of the contextual launch support in Eclipse 3.3 and we added a view for managing and debugging lots of executables.

Opening up our bugzilla to everyone in the beta program has really been helpful and brought our team a little closer to everyone using the tools. Making our nightly builds available has let us get fixes out to people fast for quick validation. Overall I think running our beta program much like an Eclipse project has let us make the right kinds of changes faster than we have before.