I reviewed a paper a while back that described how a set of Java developers used the search tools in Eclipse to explore a code base. Having performed a number of studies involving very smart developers in addition to being a proficient Eclipse developer myself, I surmised that the developers were not very proficient users of Eclipse. What made them ineffective — and more important, how can a developer become more proficient?
In this study, the developers were asked answer questions that required finding elements within the source code for a system that corresponded to domain concepts.
The biggest clue for me as to their inexperience was that all the developers used the File Search and Java Search dialogs exclusively. Some apparently used only the File Search! I was so surprised that I contacted the paper authors to confirm (i) that it was not an imposed restriction, and (ii) that their traces hadn’t lumped the use of the search dialogs with other search mechanisms (they were separate). Although these dialogs are useful for grep-style searches, such as for finding a message property, they are nowhere near as quick to use as the shortcut-driven searches like Open Declaration (F3) and References > Workspace (CtrlShiftG). And no expert developer would only use the File Search dialog!
Further confirmation came from a breakdown of what the developers were searching for. Many of the searches involved searches to map domain concepts to classes and interfaces. No proficient developer would use the Java Search dialog to find a class; they would instead use the Open Type dialog. The Open Type dialog is reactive, providing immediate feedback; the Java Search would take at least 10 ten times as long.
Understanding the expertise of participants is essential for drawing conclusions and making recommendations. In this case, I suggested that the authors make recommendations about improving instruction for new or inexperienced developers.
So what indicators correlate with expertise?
From my observations, one mark of expertise is inversely correlated to mouse use: the more mouse usage, the more a n00b. Experts seek out the functions that can be performed at the speed of thought. Dialogs and mouse pointing are interruptions.
Another sign of expertise is knowing and tailoring advanced options. One of the first actions I take with a new Eclipse installation is to disable the “Show Approximate Matches” search option: when I do a references search, I want to see exactly the callers to that method. Why would I want thousands of potential matches cluttering the view? If I wanted ineactness, then I would use the Java Search dialog!
[In fact, I never use the JDT searches any more: I use my own tool, Ferret, that provides far more focussed queries. I’ve let it slide though, and its PDE searches are a bit broken in 3.7. I’ll try to fix those.]
What signs can you think of that distinguish novice from expert?
[Update: the paper in question is: J Starke, C Luce, J Sillito (2009). Searching and Skimming: An Exploratory Study. In Proc Intl Conf Software Maintenance (ICSM).]