Eclipse Debugger for C/C++ (EDC)

I'm very happy to announce that the sources for EDC, the Eclipse Debugger for C/C++, are now in the Eclipse CVS repository. EDC is a new component of the CDT project that provides a complete architecture for C/C++ debugging within Eclipse/CDT.

The Carbide team started developing EDC after setting on a new course that would open up nearly all of our development tools to either the Eclipse or Symbian communities. We couldn't continue using closed source debug components that were becoming difficult to enhance and maintain so we needed something new. We looked at existing open source debug technologies but concluded they wouldn't meet our needs. With the rare opportunity to start from scratch we decided we wanted a debugger that was perfectly tuned with Eclipse, focused on C/C++ development, had ground-up support for multiple cores/contexts, and a modern accessible design that would easily support new features. We also wanted to do the development in the open, in the same community that has supported our efforts over the past five years.

Building EDC has been much easier than it would have been a few years ago. The advances in the debug platform, the creation of the Debug Services Framework (DSF) in CDT, and the development of the Target Communications Framework (TCF) all provided essential building blocks. EDC is working on top of and between these components,  filling in the gaps and building out all the pieces required for a full working debugger.

While we wanted to develop EDC in the open we also wanted to get enough of the basics working so that when we contributed the code people would be able to easily try it out. So while we worked on all of the basic things a debugger should do we tried to keep the CDT community informed about our progress. Now that the starter code is in we'll be making our continuing work more visible, moving discussions into the eclipse bugzilla from our internal one. There is still a lot of work to do but now we're in a position to do it in collaboration with the community.

I've started building information about EDC on the CDT wiki here. EDC is not yet part of the regular CDT build but I hope to add it soon.

CDT-EDC Sprinting to 10-14

We've kicked off another sprint of EDC development: Warren is refactoring the Dwarf symbol reader to support an general symbol reader API that can be used for other symbol formats. Steve is adding support for more variable types: enums, arrays etc. Stephen is beefing up the Dwarf variable location to handle a lot of variable storage types we hadn't covered yet. Ling is cleaning up some bits of the Windows and Linux reference debuggers so they can reuse the existing CDT launch configurations. Chad is working on support for attaching to a running process. David is adding general support for consoles, integrating the Eclipse console view and data from the TCF streams service. Tim and Vasili are working on general stabilization and unit tests. I'm working on the beginnings of scripting support, outlining a DOM and prototyping how you can automate the EDC debugger from outside of Eclipse.

CDT-EDC Breakpoints

Finally a break in our long hot Austin summer with some rain and slightly cooler temps last weekend. In the eight hot weeks since my last post our team has been adding a bunch of new stuff to the EDC debugger we're preparing to contribute to CDT:

Symbian Device Connectivity - In addition to the our reference implementations on Windows and Linux we're now also debugging on a variety of Nokia phones. This has included supporting a new TCF debug agent and adding support for ARM to the existing x86 support in EDC. After using the existing TCF file system service to download everything we also created a new TCF service for installing applications on Symbian phones.

Breakpoints - Software breakpoints are now supported in a couple ways. If the TCF agent implements support for the TCF breakpoints service we'll use it, otherwise support is implemented in the DSF breakpoints service by writing memory to insert break instructions.

Disassembly - EDC now includes disassemblers for x86 and ARM that are built on a common disassembly framework. The disassemblers are used to populate the Disassembly View and to assist other operations like stepping.

Stepping - Stepping over/in/out of a range of code is now working both for TCF agents that know how to step over a range and those that can't. The Symbols service is used to determine the range of instructions to step over and then passes that to the agent. But if the agent doesn't support range stepping then the Run Control service will use the Disassembly service to find all the branch instructions and then set temporary breakpoints to perform the step.

One reason the breakpoint support was really interesting to implement was the interaction between several of EDC's DSF services. To support source level breakpoints you need to be able to get an address in some code from a source file it was built with and you need to know when and where the code is loaded. In EDC this begins when the debug agent sends a TCF context suspended event when a new code module is loaded. EDC's DSF Run Control service handles the event by sending a module loaded event to the other DSF services. The Module service adds a new entry to it's module list and then tells the Breakpoint service about the new module. The Breakpoint service asks the Symbol service if symbols are available for the new module. If so the Symbol service determines if the module was built using any of the source files that contain breakpoints. The Breakpoint service then asks the Symbol and Module services to provide memory addresses for the breakpoints, then installs them. Finally it tells the Run Control service to resume execution of the suspended context and your program begins running again. None of this is visible to the user except for a subtle change in the breakpoint marker to indicate it was installed successfully.

CDT-EDC Variables

It's over 100F/37C in Austin today and we've settled in for the lengthy sweltering drought they call "summer" here. It seems pretty quiet, partly because our Finnish colleagues are all on holiday and because our team has been decimated by some sort of stomach bug that has kept people home sick for several days at a time. Now that I've recovered a bit it's time for an EDC update.

Variables, at least the simple ones, are showing up now in the EDC debugger:

It all starts when one of the stack frames is asked for a list of its local variables. The EDC Symbol service supplies the list and the name of each variable is turned into an expression for the DSF Expression service to evaluate. When EDC evaluates the expression (which is really just the variable name) it needs to get the value of the variable.

For most simple types this is done by getting the location of the variable and reading memory from that address. Yet getting a variable's location out of the symbol information (DWARF 2 in this case) can be a complex multi-step process. For example, the symbol info might say integer X is stored 14 bytes off the stack pointer which is based on a register value with maybe an offset thrown in depending on where you are in the function. But eventually you get to read an value of the correct size (based on the variable's type) and you have something to use for the value of the expression, which is then dropped into the Variables View.

Of course we have a lot more to do with variables: complex types, formatting, location info etc. but the basic framework is in place to build on.

CDT-EDC Expression Parsing

I was fortunate to spend some time working at home recently with Max. He's a good coworker: sleeps much of the day and can be trusted with any confidential info that floats by. Sure, there were the occasional insistent interruptions, but no more than I get at the office.

Last sprint I started working on variable display in the EDC debugger and this quickly unraveled in a few interesting directions. One involves parsing and evaluating C/C++ expressions. The DSF framework considers all variables to be expressions, so if you want to display the value of variable "i" then you need to implement the same API used for more complex expressions like "i + 2" or "*((ShapePtr*)i).fore_color". I could probably work around this just to splash some simple variable values up but eventually we would need to parse and evaluate expressions anyway.

I approached this with some dread: the expression engine in our legacy debugger back-end was created in the mid 90's based on an early version of the CodeWarrior C++ compiler's front-end. As the C++ language evolved and the debugger was ported to support an endless series of now deceased architectures and platforms it's expression engine was extensively poked, prodded and patched. Eventually I would try to gather up all of the expression issues for a release and work on them all together. It usually took me a day of staring at the code to remember how it worked and some CVS sleuthing to discover why various "adjustments" were made. Then I would attempt to fix everything I could without breaking anything. Afterward I would go work on something else and forget the whole experience until the next release.

But when I first talked to Doug Schaefer about developing a new debugger in CDT he wondered if the CDT language parser could be used by the debugger for expression parsing. The debugger in the JDT does this, it uses the Java language parser to break an expression into actionable opcodes and then has an evaluation engine that executes those to get a value.

I set out to try this in the CDT and so far it is working great! After some startup pointers from Ed, and Mike, and Markus I've got the CDT language parser chewing on various expressions I give it and then producing a stack full of actions (add, subtract, dereference, look up value etc.) that get executed to produce a value. This can be used a lot of places in a debugger: in the Expressions View, when you hover in the editor, and in the Variable View's details pane. There is a lot more to flesh out but it's nice to be able to leverage so much existing work in CDT and build something that will be much easier to maintain in the future.

CDT-EDC Source Mapping

One of the basic things a debugger needs to do is show you where you are in your source code when you stop at a breakpoint or exception or after stepping. When this happens an EDC debugger's TCF debug agent sends it an event that includes a memory address for the location. If there is no other information available you'll end up in the Debug View with a stack frame location that looks like this:


The next step is to use the EDC-DSF Modules service to determine if this address is in some code the debugger knows about. The Modules service keeps track of all the bits of code loaded in the target process. If you can find a module for the address then you can show something more interesting like this:


Finally the EDC debugger checks to see if debug symbols are available for that module. Debug symbols include a mapping table of address ranges to line numbers in your source code. The EDC-DSF Symbols service checks to see if this information is available. If it can find a source location for the address then you can show a complete description like this:



So that's what I got working last sprint using the Windows debugger and an executable built with gcc/mingw and Dwarf2 for testing. After you have a source file to work with the existing source mapping facilities in CDT can handle finding missing source files and remapping files to new locations. Our legacy debugger back-end does it's own source mapping and it was a problem to keep it in sync with the mapping we are already doing in CDT. The whole process is simple faster, and less error prone in the EDC debugger.

CDT-EDC Launching

The team mostly worked on launching EDC debug sessions last sprint. We polished up the Windows launch so terminate shuts everything down correctly and modified the debug agent to shut itself down if no one was connected to it for a short time There are several launch types in Carbide for Symbian debugging that we thouht we might have to clone for EDC. But we've given them the same treatment as Pawel gave the launches in CDT 6.0: defining one standard set of launch types but assigning multiple launch delegates to each one. I wasn't aware you could do this until Pawel implemented this for the CDT launches. It's great for Carbide because we can reuse our existing launch support and just implement an EDC launch delegate. We probably wouldn't ship Carbide with both CDI and EDC launches enabled for the same type, but it will be great for testing and development.

Traditionally most C/C++ launches synchronously start a process and then debugging begins. The simple EDC launches start a process asynchronously and then wait for a process created event from the debug agent. Later we'll support more complex launches: ones that debug multiple processes, target specific DLLs and wait for a process to load them, or simply let you look at stuff on the target platform before debugging anything.

CDT-EDC Baby Steps

Our team runs two week sprints of development activity in pretty standard Agile & Scrum fashion. That seems like a good interval to post updates on our work on the Eclipse Debugger for C/C++ (EDC).

While our intention is to develop EDC as an integral part of the CDT project we haven't committed any of it yet: we want to get more stuff working and the community is too busy getting CDT 6.0 ready to look at it much anyway until after Galileo ships.

I spent much of the last sprint doing bits of high level design work but got back into the code to finish up a story Warren Paul was working on until he left for the hospital.

There has a been a baby boomlet in the CDT committer community over the past several months. Our baby Max arrived back in February. Then Pawel Piech had a new one arrive about a month ago. And last sprint it was Warren's turn.










So I finished up work he was doing on how the EDC debugger connects to TCF agents. We wanted the clients of the agents, usually DSF services, to not know much about TCF concepts like Peers or Channels, but to just be able to ask for "a run control service for Windows debugging." We created a TCFServiceManager and an TCFAgentDescriptor extension point to hide all the details of launching and getting a TCF service from an agent. The extension contains a property list that describes the agent and a delegate that knows how to start it up. The service manager finds the descriptor that matchs the client's request, asks the delegate to launch the agent if it isn't already in the TCF Peer list, then opens a Channel and returns the requested service. The agents get to decide when to shut themselves down, usually when no one has been connected to them for a while.

When an EDC debug session starts we hook up the DSF services to whatever TCF services they need to use. So now the EDC Windows debug agent is started automatically when you start a debug session and shuts down shortly after the session terminates.

The EDC Windows TCF agent is written in C/C++ but we also wanted a reference TCF agent written in Java that can be run in Eclipse or in another instance of the VM. Ling Wang produced one in a previous sprint and just finished adapting it to talk to GDB Server for use by the EDC linux debugger. We'll do something similar for Symbian OS device debugging except it will talk to TRK, the debug agent that runs on the phone.

Each sprint we try to have a simple ramp-up story we can give to another member of the larger Carbide team to help bring them up to speed on EDC. This time it was adding support for writing register values to the EDC Windows debugger (reading them already worked). Stephen Chong came in cold having never seen DSF or TCF but knocked it out in a few days, complete with unit tests.

Instant API

One of the aspects of our current financial crisis I've found interesting is the ability of the US Federal Reserve bank to create additional money out of thin air. How does that work? Do they have some money supply control site where Ben Bernanke enters a number and hits the "Submit" button?

I was reminded of this during this month's CDT call when we discussed fixing some bugs that require making a few non breaking API changes. This is an issue because it's after the API freeze date but I think our underlying problem is how through the best intentions we've created a bunch of API from code that wasn't intended to be.

When I started working on CDT the definition of API was a little loose but mostly described interfaces in a public package that we designated API in the CDT docs. Over time we moved to tighten this up and then finally decided to use the same definition as the platform: roughly any interface or class in a non-internal package was to be considered API. Having the API tooling helped with this considerably and newer code like DSF that was designed with this in mind moved into CDT to set a good example.

But this also means that lots of code in CDT that was probably never intended to be API instantly became API. Sure, it probably could have been designed better in the first place or we could have moved it to internal packages or I could have thought about this more at the CDT Summit instead of watching for the afternoon cookies to be delivered. But now here we are needing to fix bugs and needing to make non breaking changes to do it. On the call I said I thought we should just go ahead and make the changes we need and work on a better plan for this next time.

We've just been through something like this with our Carbide sources. We had put all the stuff we thought of as public API into a special SDK plug-in. But as part of open sourcing Carbide for the Symbian Foundation we found ourselves needing to conform to the Eclipse platform definition of API. So we kept ourselves busy for several weeks refactoring things into internal packages so we wouldn't suddenly make a bunch of stuff API that was intended to be internal and private. Now that we've finished and we're testing the result we're finding that we've broken plug-ins people have written for Carbide that were using "non-API" stuff that we renamed. We're reevaluated what we need to make real published API now but it's a good lesson that people will do what they need to in order to get their plug-ins to work and worry about the consequences later on. After all no one wants to create a tool and say to their customers "It doesn't solve your problem or work the way you would expect, but we only used the published APIs!" I guess it's a convoluted form of API requirements gathering.

Logging & Tracing

Since I'm not at EclipseCon this week I trying to get some real work done here in Austin now that the crowds from SXSW have left town. I've been looking into the best way to build good diagnostic logging into the EDC debugger, mostly as a support tool. When people report problems that we can't reproduce we can ask them to create a log of debugger activity that can help us pinpoint the problem.

In Carbide we have a simple diagnostic log based on java.util.logging that works pretty well but then I noticed that Eclipse 3.5M6 includes a new Equinox tracing API. When your application normally keeps a log of activity it is probably best to use one of the various logging services to help manage it. But in this case I just want to be able to get more information to help debug problem situations. It looks like the new tracing API will be a nice way to do this: there is a simple way to trace function entry, exit, exceptions, and simple messages. Each trace call can be tied to a particular debug option so we should be able to turn on tracing for specific parts of the code without getting details about everything at once.

For example this traceEntry call includes the debug option for the EDC run control service and will log the properties passed in and on exit will log the new object created.

public ExecutionDMC contextAdded(Map properties) {
EDCDebugger.getDefault().getTrace().traceEntry(RUN_CONTROL_TRACE,properties);;
ThreadExecutionDMC newDMC = new ThreadExecutionDMC(this, properties);
getSession().dispatchEvent(new StartedEvent(newDMC), RunControl.this.getProperties());
EDCDebugger.getDefault().getTrace().traceExit(RUN_CONTROL_TRACE, newDMC);
return newDMC;
}

Eventually we'll need some UI to help people turn this on and off and maybe some tools to help them send the file or attach it to a bug.

Eclipse Debugger for C/C++ (EDC)

In my last post I talked about how the Eclipse C++ debugger landscape had changed and led us to want a new debugger that’s perfectly tuned to work with CDT/DSF and Eclipse. The existing work in the debug platform and CDT/DSF provides a nice set of services that include user interface elements, commands, and APIs for debug data access. What’s missing is the part of the debugger that knows how to interact with the thing you are debugging to actually get the memory and register values, do run control, provide variable values etc. Traditionally this has been provided by a debugger back-end of some sort: CDT includes an integration of DSF and gdb while other tools vendors have a similar debugger engine, often extracted from their legacy product.

When taking a fresh look at this it didn’t make much sense to create a monolithic debugger back-end and try to integrate it with CDT/DSF. DSF already provides a well designed service model, what’s missing is a complete set of services to flesh out a debugger. Extending the reach of the DSF services into things traditionally done by a debugger back-end would also let us design and develop most of it using the same language, tools, and environment as all of our other Eclipse work.

Yet some services still need to reach out to native code either to interact with a native debug API or to do something performance intensive like parsing symbol files. So we would need a way for our DSF services to talk to the native code that is likely running in a different process or on another device. Fortunately the DSDP/TM/TCF project solves exactly this problem. TCF is a protocol that lets an agent (usually in native code) provide a discoverable set of services to clients (in this case our DSF services). This lets our new DSF services do as much of the work as they can and then use the TCF protocol to talk to remote devices or native code as required.

This collection of DSF services and TCF agents will form the foundation for a complete C/C++ debugger in Eclipse. So we're calling this project the Eclipse Debugger for C/C++ (EDC).

Building a new solution that will let us provide powerful, stable, maintainable, and consistent debugging for the platforms we support (Symbian, Windows, Maemo) means developing DSF services and TCF agents that target them. We’ll end up with a core set of common EDC plug-ins and agents and others that add specific support for a device or operating system. Anything of general interest will be developed within the CDT community and we'll put the Symbian specific work in the Symbian Foundation.

I'll post more details as we develop stuff.

C/C++ Debugger Retrospective

We're doing some new work on the debugger in Carbide and for some perspective I'll take a look back at how the Eclipse C/C++ device debug landscape has changed in the past few years.

When we first began designing our Carbide.c++ tools for Symbian mobile development a few years ago there was a lot of new activity around C/C++ debugging in the Eclipse community: The CDT project offered a proven set of APIs for debugger integration (CDI), but as vendors continued to move their existing debuggers into new Eclipse based environments it was clear that more work needed to be done to meet the expectations of veteran C/C++ developers.

Our team began creating C/C++ tools for Symbian using the CodeWarrior IDE and when we designed a new generation of Eclipse based tools we decided to reuse its proven debugger engine and hook it up to CDT/CDI. While this let us quickly deploy a good C/C++ debugger as an initial step, we also carefully watched developments in the device debugging project (DSDP/DD) and the creation of the Debug Services Framework (DSF, now part of CDT). It was also exciting to see the many improvements resulting from the debug platform team’s engagement with the device debug community. We began developing a list of improvements we would like to make to Carbide's debugger that would be difficult to do without using the new work in the platform and DSF.

At one point I started prototyping an integration of our debugger engine and DSF. While I got a number of things working it also became apparent that it wasn’t a very good fit: DSF uses a highly asynchronous service model for supplying information about the debug session and our debug engine didn’t work that way. I was having to rig up a lot of stuff in a clumsy way to make it work with DSF. To move our debugger forward it was looking like we would not only need to move to DSF but also rework a lot of our debug engine.

Doing that work looked difficult because the engine code was pretty convoluted and much of it needed an overhaul. Also the engine was closed source licensed code and this would be a issue as we moved to open source all of Carbide along with the Symbian Foundation.

So that’s how we have moved from our initial legacy debugger integration with CDT/CDI to how it looks today: lots of promising development in the CDT and device debug community but we're not yet able to take advantage of much of it with our existing debugger. So we started thinking about something new that we can develop in the open and tune to work perfectly with DSF and Eclipse.

Not Going to EclipseCon

After making the trip the past several years I'm going to have to miss EclipseCon this time. I'd like to be going: I'll miss the sessions, BOFs, and most of all the informal conversations with all the interesting people in the Eclipse community.

But this year our Baby Max arrived a bit early and so I'll be home helping with diapers etc. Maybe I can catch up with everything on Planet Eclipse when I'm up in the small hours.

Carbide 2.0 Update

I've somehow let the past several months go by without posting anything so it's time to catch up. Our team launched Carbide 2.0 late last year. It's built on top of Eclipse 3.4 and CDT 5.0 with all of our tools for Symbian OS development built in. It was an unusual release to work on because a lot of the work went into supporting things that are not immediately apparent to someone moving up from Carbide 1,3, For example we added support for Qt on S60 but until October when the tech preview was available most people couldn't try it out. There is a new build system for Symbian that we now support but it isn't widely used yet. And as more teams at Nokia move their projects to Carbide we added support for stuff they use but isn't yet ready for the whole development community. Even many of the improvements in CDT 5.0 result in much better C++ code navigation, but are not big visible new features.

So some people were a little unimpressed at first ("why is this Carbide 2.0? It feels like 1.4") but later came back with much more positive feedback as they found the new work and improvements to existing features.

But one of the best changes came at the end when we decided that all editions of Carbide should be free to anyone. While we have always had a free edition it didn't include some higher end tools for debugging and analysis. This will help grow the Symbian developer community, and our team is thrilled that our code no longer has to do all that license checking.