Time for an update on the state of Carbide, the Eclipse Debugger for C/C++ (EDC), Symbian, and Nokia. As usual all comments and analysis are my own.
We finally shipped Carbide 3.2 to the public! This is the first publicly available 3.x version of Carbide and the first new version seen outside of Nokia in over a year. It includes support for a new build system, a brand new on-device debugger built on top of EDC, and lots of usability improvements. The delay was a result of one of the many iterations of Nokia's strategy over the past few years: for much of 2010 and until February 11th of 2011 we were telling developers that our mobile platform was the Qt app framework, not Symbian. The Qt folks were busy building their own IDE (Qt Creator) and so it was thought that developers would be confused if they had Carbide available as well. So Carbide was designated a "platform tool" only for use by Nokia's thousands of internal developers.
Then on February 11th our new CEO announced that Nokia would be using Microsoft's Windows Phone 7 software in the future and Symbian would be phased out over the next couple years. So our team has ended up in a strange position: we need to keep Carbide up to a high standard to support Nokia's continuing Symbian efforts but we know the plug will be pulled at some point, probably within the next year. Nokia has built in some flexibility for it's Symbian roadmap and arranged a soft landing by working out a deal to transfer 3000 people to Accenture at the end of this year. Accenture will then contract us back to Nokia to do the remaining Symbian work.
Our team's chief contribution to the Eclipse community over the past 24 months has been the Eclipse Debugger for C/C++ (EDC), part of the CDT project. Faced with the need for a new C/C++ debugger for Carbide and unhappy with any of the available options we designed and built one from the ground up. But the ground was already pretty rich: we leveraged all the work done in CDT's Debug Services Framework (DSF), used the CDT language parser for expression support, and relied on the Target Communications Framework (TCF) for low level debug services.
The first version of EDC shipped with Helios but it was pretty raw as we hadn't yet shipped a Carbide product using it. Indigo EDC is a big improvement in stability, performance, and feature set and we got to fix a number of things we didn't get right on the first attempt. The two reference debuggers provided with EDC (Windows & Linux) still aren't near production quality but are great for introduction and testing and provide a good example of how to build a debugger around EDC.
As we developed EDC we found other tools people interested in the idea and it reflects valuable contributions from the CDT community. As Carbide winds down I hope it continues to offer some value to anyone moving beyond their legacy debugger. I'd like to continue improving it and the rest of CDT but that awaits a volatile future.
Monday, May 16, 2011
Thursday, March 17, 2011
Copying Memory
I've been starting to clear some usability issues out of our backlog and ran into one that seemed simple enough but eventually required a little more thought and attention.
When debugging native code it's common to want to inspect memory so the Eclipse debug platform offers a standard Memory view. The view can host a number of memory renderings, each offering a different way of viewing a chunk of memory.
The "traditional" memory rendering is popular with C/C++ developers because it works much like the memory views in other debuggers and shows a range of memory in both binary and text formats.
Our developer (I'll call him Tom because that's his real name) wanted to do something that seemed pretty simple: select a hex value, copy it to the clipboard, and then paste it into a variable in the Variables view. But the global "Copy" command wasn't hooked up to the memory rendering so it did nothing. Tom then discovered "Copy to Clipboard" in the context menu.
But the "Copy to Clipboard" command copied all of the selected data from the rendering: addresses, binary, and text to the clipboard so it looked like this:
0x7FFF5FBFED20 74736574 00676E69 996E019D 3CBE8497 5FBFED40 testing...n...¾<@í¿_
Tom had to paste this into a text editor, then select just the binary/hex part, and finally had something to paste to set the value of his variable. This sort of thing delights some people because they find a work-around and then have another story to inflict on their colleagues, but Tom had real work to do, expected better, and logged a bug. Our team worked on other stuff that seemed more important until last week when I went back to take another look.
First I looked to see if the rendering's support for "Copy" could just do the right thing by grabbing the address, the binary data, or the text based on which part had been clicked in last. While that would work in the narrow case it wouldn't leave any indication of what it might do if you have forgotten which area you clicked in last, and the ability to copy all three (addresses, binary, text) would have been lost.
There needed to be a more explicit way to say which portion of the data you wanted to copy. So I changed the context menu to provide all four options.
This would make life better for Tom but still wouldn't fix the "select the hex data and hit control/command - c" scenario. So it also keeps track of one choice as the default option that is used when fired by the global "Copy" command, which is now hooked up to the rendering. Tom will have to select "Binary" once and then Copy will work the way he expected.
When debugging native code it's common to want to inspect memory so the Eclipse debug platform offers a standard Memory view. The view can host a number of memory renderings, each offering a different way of viewing a chunk of memory.
The "traditional" memory rendering is popular with C/C++ developers because it works much like the memory views in other debuggers and shows a range of memory in both binary and text formats.
 |
| "Traditional" memory rendering from CDT |
 |
| Memory View in Xcode 4 |
 |
| Memory view from Visual Studio 2008 |
 |
| Memory window from CodeWarrior in 2001 |
Our developer (I'll call him Tom because that's his real name) wanted to do something that seemed pretty simple: select a hex value, copy it to the clipboard, and then paste it into a variable in the Variables view. But the global "Copy" command wasn't hooked up to the memory rendering so it did nothing. Tom then discovered "Copy to Clipboard" in the context menu.
0x7FFF5FBFED20 74736574 00676E69 996E019D 3CBE8497 5FBFED40 testing...n...¾<@í¿_
Tom had to paste this into a text editor, then select just the binary/hex part, and finally had something to paste to set the value of his variable. This sort of thing delights some people because they find a work-around and then have another story to inflict on their colleagues, but Tom had real work to do, expected better, and logged a bug. Our team worked on other stuff that seemed more important until last week when I went back to take another look.
First I looked to see if the rendering's support for "Copy" could just do the right thing by grabbing the address, the binary data, or the text based on which part had been clicked in last. While that would work in the narrow case it wouldn't leave any indication of what it might do if you have forgotten which area you clicked in last, and the ability to copy all three (addresses, binary, text) would have been lost.
There needed to be a more explicit way to say which portion of the data you wanted to copy. So I changed the context menu to provide all four options.
This would make life better for Tom but still wouldn't fix the "select the hex data and hit control/command - c" scenario. So it also keeps track of one choice as the default option that is used when fired by the global "Copy" command, which is now hooked up to the rendering. Tom will have to select "Binary" once and then Copy will work the way he expected.
Thursday, November 11, 2010
CDT-EDC Multi Context Debugging
If you are ever at a gathering of the Eclipse CDT community and the conversation starts to drag just casually ask about support for debugging multiple processes, cores, or contexts at the same time. You'll hear about a lot of important interrelated issues and that the committers have been chipping away at them for some time.
One of the nice things about creating the EDC C/C++ debugger in CDT was the opportunity to include ground up support for debugging many contexts at once. But while that's fine under the hood we immediately ran into an important usability issue in the Debug view.
When you launch a debug session you start with a launch configuration that tells the debugger what to do and you end up with a Launch in the Debug View that shows the result. The Launch is usually named after the launch configuration that created it:
When you want to debug something else you can use another launch configuration that will create another Launch in the Debug View:
People find this confusing if both the things you are debugging are running in the same target environment. They expect them to be grouped together in the Debug view and they expect the Launch at the root of the view to represent the overall environment (phone, PC, multi-core system etc.)
The EDC debugger deals with this by letting new launches join existing ones if they are part of the same target environment. For example if you debug a Windows app (Simple.exe) you will get a new debug session and Launch in the Debug View.
Debug a second Windows app (BlackFlag.exe) and it will join the existing one:
EDC uses a description and image of the target environment provided by the debugger implementation. In that case it was the Windows machine name. In this one it's the name of my phone:
Of course there are lots of other issues related to multi context debugging. To see how the CDT community is addressing these you can follow along with the CDT multi-core working group.
One of the nice things about creating the EDC C/C++ debugger in CDT was the opportunity to include ground up support for debugging many contexts at once. But while that's fine under the hood we immediately ran into an important usability issue in the Debug view.
When you launch a debug session you start with a launch configuration that tells the debugger what to do and you end up with a Launch in the Debug View that shows the result. The Launch is usually named after the launch configuration that created it:
When you want to debug something else you can use another launch configuration that will create another Launch in the Debug View:
People find this confusing if both the things you are debugging are running in the same target environment. They expect them to be grouped together in the Debug view and they expect the Launch at the root of the view to represent the overall environment (phone, PC, multi-core system etc.)
The EDC debugger deals with this by letting new launches join existing ones if they are part of the same target environment. For example if you debug a Windows app (Simple.exe) you will get a new debug session and Launch in the Debug View.
Debug a second Windows app (BlackFlag.exe) and it will join the existing one:
EDC uses a description and image of the target environment provided by the debugger implementation. In that case it was the Windows machine name. In this one it's the name of my phone:
Tuesday, September 7, 2010
CDT-EDC Where is the Exit?
Our team is working on our next generation of C/C++ development tools for Symbian, Carbide 3.0 (C3). These build on the Eclipse Debugger for C/C++ (EDC) that's part of the CDT project. Our test team reported a bug that raises an interesting question for developers: do you want the debugger to show you what's really happening even if it seems a bit confusing?
Here is the situation: you are debugging a C++ function that looks something like this:
Say you are stepping through this function in the debugger and you start at line 31. "value" happens to be 200 so the condition on line 31 fails. You step over and end up on line 36. Step over again to go to line 37. So far everything looks OK. Now you step again from line 37 and you might reasonably expect to exit the function. Instead you end up on line 34! This looks like a heinous bug but is a case of the compiler reorganizing things even in a normal debug build with optimizations off.
The compiler has built this function by putting all of the return/exit code in one place, in the middle of the function at line 34. At the end of the function, at line 37, instead of generating the exit stuff again it simply jumps to the address at line 34 and exits there. It dutifully and accurately records this bit of slight-of-hand into the symbol information and the EDC debugger shows you on line 34.
Should the EDC debugger try to detect this just to avoid confusion? Another perspective is that the debugger should show you exactly what's happening even if the result is sometimes puzzling. What do you think the EDC debugger should do?
Here is the situation: you are debugging a C++ function that looks something like this:
Say you are stepping through this function in the debugger and you start at line 31. "value" happens to be 200 so the condition on line 31 fails. You step over and end up on line 36. Step over again to go to line 37. So far everything looks OK. Now you step again from line 37 and you might reasonably expect to exit the function. Instead you end up on line 34! This looks like a heinous bug but is a case of the compiler reorganizing things even in a normal debug build with optimizations off.
The compiler has built this function by putting all of the return/exit code in one place, in the middle of the function at line 34. At the end of the function, at line 37, instead of generating the exit stuff again it simply jumps to the address at line 34 and exits there. It dutifully and accurately records this bit of slight-of-hand into the symbol information and the EDC debugger shows you on line 34.
Should the EDC debugger try to detect this just to avoid confusion? Another perspective is that the debugger should show you exactly what's happening even if the result is sometimes puzzling. What do you think the EDC debugger should do?
Thursday, June 17, 2010
Giant Symbols
In the midst of all that I was asked to help a developer who is porting a very large productivity app to Symbian. His problem involved using the debugger in Carbide 2.x and the size of the symbol files generated for his application. A symbol file is generated by the build tools and contains the information used by the debugger to make sense of your source code, variables, and most of the other stuff you see in a debug session.
With all the apps developed at Nokia we've seen a lot of large symbol files but this one was nearly a 1GB, almost double the size of our largest sample. Our current tools (Carbide 2.x) crashed when trying to open the big symbol file and couldn't easily be fixed to handle something that size.
This is one of the use cases we had in mind when we developed the new debugger in C3, the Eclipse Debugger for C/C++ (EDC), part of the Eclipse CDT project. So we ran our new big symbol file sample through the EDC symbol reader unit tests and everything worked quickly and smoothly. I gave the developer a beta build of C3 and once we worked through some setup issues on Windows 7 he was able to debug the app with the giant symbol file and see all the debug info.
Then he says "that's the symbol file for just one of our dlls, all together we actually have about 14GB of symbol data." So I think we have now put the EDC debugger in C3 through a least one kind of worst case scenario.
While all this was going on someone asked why the symbol files were so large to begin with. This big productivity app has been ported to lots of platforms with different tool chains and they never ended up creating such a large amount of symbol data. After some investigation they discovered a build flag called "remove_unneeded_entries" that, when enabled, dramatically reduces the size of the symbol data. We still have more testing to do but so far it looks like it really helps debugger performance with no loss of useful data.
Tuesday, March 30, 2010
Carbide, CDT & EDC at EclipseCon 2010
Had a great time at EclipseCon in Santa Clara last week. Some highlights:
Face time with our colleagues from Nokia, the Symbian Foundation, Eclipse and CDT communities. Ronnie King, Warren Paul, and I came from the Austin office and had a great time catching up with people we usually don't get to see in person very often. I missed EclipseCon last year but I heard attendance was up and organizations were able to send more people this time.
I really liked the session format: the reduced times seemed to really focus the presentations and kept a wide variety of material flowing throughout the day. I know the CDT session Doug and I did seemed to fly by quickly.
Had a great response to the new EDC (Eclipse Debugger for C/C++) that will be in Carbide 3.0 and our team is contributing to CDT 7.0. People liked the overall concept, features, and extensibility and I think we'll soon start seeing more contributions from the community.
Lots of discussions about usability and performance, both very important to us for Carbide. There is new work going on in the Eclipse & CDT communities that will help us smooth out some rough spots our Carbide users have had to put up with.
Visiting developers at the Nokia office in Mt. View. We love getting to shadow developers around and watch them work. Great unfiltered feedback. We had a chance to introduce them to some features in Carbide they didn't know about and help them through some difficult issues that are tough to understand unless you can look over the person's shoulder.
I really liked the session format: the reduced times seemed to really focus the presentations and kept a wide variety of material flowing throughout the day. I know the CDT session Doug and I did seemed to fly by quickly.
Had a great response to the new EDC (Eclipse Debugger for C/C++) that will be in Carbide 3.0 and our team is contributing to CDT 7.0. People liked the overall concept, features, and extensibility and I think we'll soon start seeing more contributions from the community.
Lots of discussions about usability and performance, both very important to us for Carbide. There is new work going on in the Eclipse & CDT communities that will help us smooth out some rough spots our Carbide users have had to put up with.
Visiting developers at the Nokia office in Mt. View. We love getting to shadow developers around and watch them work. Great unfiltered feedback. We had a chance to introduce them to some features in Carbide they didn't know about and help them through some difficult issues that are tough to understand unless you can look over the person's shoulder.
Wednesday, February 24, 2010
Aligning Carbide 3.0 with Eclipse Helios
Like most teams that build C/C++ tools on top of Eclipse & CDT we have been building Carbide with the latest platform release and a somewhat forked version of CDT. This made sense as most of our releases have shipped in the winter or spring, long after the last major Eclipse release. But with the next major release of Carbide due after Helios we’ve jumped on the opportunity to align more closely on Eclipse 3.6 and CDT 7.0. How closely? Well, we’ve just started creating our Carbide builds with 3.6M5 and the CDT 7.0 sources straight out of the eclipse.org CVS repository.
We hesitated to do this in the past because it means we effectively have the entire CDT team contributing to Carbide in real time: any one of the committers can break our build or introduce bugs. But we’re more confident now and our experience with the CDT community has been very good: people rarely break the build and the quality and review of contributions has been excellent.
And since we sync up with the latest platform and CDT eventually anyway is makes more sense to discover any issues at the moment when the work is developed and contributed, not months later. Now both our internal test team and the Carbide beta group can give the new platform and CDT stuff a workout before it’s released. This also lets us get rid of our internal copy of CDT we would occasionally sync with the public one. Trying to keep it updated was a pain and it created a temping way for us to make changes that weren’t really ready to go back into the community.
We hesitated to do this in the past because it means we effectively have the entire CDT team contributing to Carbide in real time: any one of the committers can break our build or introduce bugs. But we’re more confident now and our experience with the CDT community has been very good: people rarely break the build and the quality and review of contributions has been excellent.
And since we sync up with the latest platform and CDT eventually anyway is makes more sense to discover any issues at the moment when the work is developed and contributed, not months later. Now both our internal test team and the Carbide beta group can give the new platform and CDT stuff a workout before it’s released. This also lets us get rid of our internal copy of CDT we would occasionally sync with the public one. Trying to keep it updated was a pain and it created a temping way for us to make changes that weren’t really ready to go back into the community.
Subscribe to:
Posts (Atom)
