Archive
Agile thinking for distributed teams
Recent work changes have led me to think more intensely about development methodologies, project management for small teams, and similar issues, and I’ve been reading more about Agile development. I am much more interested in the philosophical underpinnings, the values of Agile, the overall approach of Agile development, than I am in the specific practices that make up a particular “official” methodology such as XP or Scrum. So I was interested to see Brian Marick’s post on his Exploration Through Example blog entitled “What is Agile? — beats me, but I know it when I see it“.
To me at my current stage of thinking, the most important components of Agile development include:
- short release cycles with working software to the customer as often as practical
- safety, trust and openness in the development team
- frequent (almost constant) and informal interaction between members of the team
- open communication between the user and the developers
- transparency and visibility of information
- focus on solving the business issues
- adaptability
- emphasis on the practical over the ideal, but always pushing for incrementally better
- an attitude that fosters creative solutions, dedication to solving real problems, and ownership of the problems, process and solutions
I think that the Agile manifesto wraps a lot of this up. It embraces the following values:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Reponding to change over following a plan
Agile processes focus on adapting to real needs, rather than trying to predict them and then sticking to the predictions regardless of how accurate they turn out to be. They focus on meeting the needs of people over the needs of the development process itself.
There is a lot in the various methodologies that is good, but to me the philosophy for development espoused by these ideas is what its about. Methodologies need to adapt to the situation, just as other components of planning and process must.
Most of the information I have come across in the “formal” Agile methodologies completely overlook or dismiss distributed teams. This is just not well connected to the real world. Instead of dismissing those of us who live in distributed teams (and the numbers are growing), we need ideas about how to reach the goals of Agile development in the reality we live in. That mostly means finding ways to work closely together, to have transparency, code review, collaborative approaches to working and coding, and high levels of interaction and trust in teams that might be physically together only rarely.
For me, the biggest component to success over the long term in development projects is in the relationships–in trust and open communication–between the members of the development team, between the developers and users and customers, and between the development team and management. If those are missing, things can quickly fall apart. Building and maintaining these in distributed teams, and when the team is not local to the customer, is challenging. The real world many of us live and work in day to day, the team, the boss, and the customer might be literally an ocean away. In this world the values of Agile development that I outlined above hold just as true. Its just the charts scribbled on paper, pair programming on one computer, and in person stand up meetings that do not.
Drupal Text_Wiki Filter
On the old version of my site I had a page about a module I wrote for Drupal 4.7 that added a filter using the PEAR Text_Wiki package. At the time I wrote it, there were no other good options for wiki markup in Drupal filters. I used the module on several sites, but I didn’t release it on drupal.org because I didn’t intend to maintain or support it over the longer term.
Since then, other people have created modules to provide wiki markup and other wiki-like functionality to Drupal. There’s a PEAR Wiki Filter module, which does basically what my module did except is maintained and has versions for Drupal 5 and 6. There’s also the new Flexifilter, which looks very promising but doesn’t have a “stable” release yet. And of course the Wikitools module goes a long way toward integrating wiki-like functionality into Drupal well beyond just markup.
I’m not sure if I was picking a product specifically for wiki that I would choose Drupal. Drupal is an excellent CMS and community collaboration tool. However, wikis are really their own thing, and there is quite a bit that wiki software does normally that has to be pieced together back into Drupal, making it work differently than it really is designed to work. I think that Drupal is the leader in what it is good at. That doesn’t make it the solution for everything.
On the other hand, if you need to use Drupal for some other reason, or if you have a Drupal site that needs some wiki functionality, take a look at the modules I mentioned. I think they will do the job better than resurrecting my old module.
Zend Framework
Well, I’m continuing to look at Zend Framework. It has a lot of nice features, and a not insignificant learning curve to get what I would need out of it. Add to this that I don’t have time in my current work schedule to play with it, so I’m doing it in my “free” time. I’d like it to be quicker to figure out. Not that any particular piece of it is hard, there’s just lots of it and each part has to be unraveled and then plugged into what I’ve learned of the whole. I am currently working with 1.5, which is actually a Release Candidate at the moment, but earlier versions don’t include everything I need. Since I’m not going to be ready to use this in anything like a production app for a little while at least (assuming I end up with this approach), hopefully it will be a stable release by then.
Since I’m exploring ZF, I want to get as much out of it using ZF as I can, rather than combining it with other approaches. It might turn out that using a combination of resources is the best approach (as Wil Sinclair suggested in response to my last post on this), but I don’t want to use that as a crutch not to figure out what I can get out of ZF.
Some of the things I want to be able to do include:
- Modular controller layout – I want to separate the controller code for various types of things. ZF allows for this pretty well. It took me a little fiddling to figure this part out, but most of that was me trying to get it to do variations. There are some pretty good articles about how to do this in addition to the documentation.
- Modular model layout – I need to support different data sources, often of very different types. For example, the same application might need to interact with Postgres, MySQL, and a SOAP service. For Drupal I wrote a dataconnector module, and then a library for each different data source that belongs to this module. Much of what I had to do to get Drupal to do this sort of thing already exists in ZF. This is going to take more investigation to do in a way that is maintainable and easy to manage, but I haven’t had a chance to spend much time on it. Its possible that the directory layout of putting the model files in a shared directory is ok. Just don’t know yet.
- Form and input validation – I’m probably going to write a post about this issue by itself soon. Its what I’m looking at now, and I’m not sure I’ve figured out the way that I think this would work best in the ZF context. Zend_Form has the server side validation nicely packaged. Actually, I’d like to do three levels of validation/filtering: at the client (for usability), at the controller for business logic and the first level of security filtering (this is what Zend_Form does), and at the model (for data integrity and injection avoidance). I need to investigate how I want to do model level filtering. That’s actually the easiest, I just haven’t looked at it yet in the ZF context. The database interaction layer isn’t sufficient for filtering by itself because at that level we don’t know all the uses we might be making of the fields. Just doing it at the controller layer isn’t sufficient because we don’t know that all data interactions will pass through any given controller, and we don’t want to miss anything. The next piece I’m going to investigate, though, is tying something like the jQuery validation plugin into Zend_Form.
- I still haven’t gotten to XML_RPC and SOAP. There actually isn’t a stand-alone SOAP component yet, but the StrikeIron service component has a fairly basic SOAP base class.
- Some core issues like logging and error (exception) handling. These are fairly boring, so I’ll put them off as long as I can.
- Zend_Translate and Zend_Locale. I haven’t looked at these at all, but I need to. Like the previous bullet, this isn’t my favorite piece. I’m putting it here in part to remind myself not to skip it.
- PDF creation. At FH (my day job) we have some stuff using PDFlib’s PPS libraries. I haven’t even thought about this yet.
- I haven’t tackled AJAX for forms yet, although I don’t expect this to be a big issue from what I’ve seen of Zend_Form so far. In forms that are an integrated part of business process, I like forms that save each field in the background as it is filled out. I’ve used Dojo to do this in the past, but I think I’ll move it all to jQuery (largely because of Drupal’s influence). Not that there’s anything wrong with Dojo. Its a great tool kit. Its great to have such good choices.
There’s lots more to put on this list, but that will do for now. Tune in next week, same bat time, same bat channel. Or some time, some where, maybe.
Frameworks
Over the years I’ve written a number of web applications basically from scratch. I almost always use PEAR and other libraries, but for the most part I’ve always used my own code as the basic framework. Last year I started building a number of modules for Drupal, including some that automate business processes (such as inventory and producing PDF documents for brochures based on live business data). This is convenient because these modules fit nicely into larger intranet type sites for the organizations that use them. But Drupal isn’t really designed to be an application framework. Since it isn’t intended to be that, they apparently feel free to completely redesign key parts of the API between versions, which really makes upgrading expensive.
So more recently I’ve been looking again at a couple of the frameworks that are available for PHP, this time Solar and Zend Framework. I’ve looked at these before, along with Cake. I didn’t care for Cake, and ZF and Solar weren’t ready yet last time I looked.
Solar and ZF are much closer to ready for real use now than they were last time I looked closely at them. Over the weekend I built a few simple prototype level things from both Solar and ZF. The short of it is that I really like Solar. It is very easy, and I found it very intuitive. I shouldn’t be surprised, Paul M Jones has a history of writing good stuff. I had a site running in very short order with Solar.
I think that the main problem with Solar is the size of the community. I need a number of the features that Zend Framework provides, but aren’t available for Solar yet, like SOAP, XMLRPC, RSS, support for MS SQL server connections (I’d rather not need that, but I do), and others. I could write them for Solar, but most of what I need is already available from ZF.
Zend Framework doesn’t have the conceptual consistency and elegance of Solar, and I’m having a hard time piecing together the bits of it that I need. Its sort of like building your own Linux from scratch, maybe something every Linux geek should do at least a couple of times but not something you want to do when all you need is a working computer to get stuff done. When you just need to get work done you want a Ubuntu or OpenSuse to package it for you so you can just get to work.
In the end, ZF is more like PEAR on steroids than it is the framework I want to save me time in piecing something together. Still, I think its the closest to what I need, and it has the momentum and support behind it to have a rosier future. So I guess the best approach would be to build my own blank application from the pieces I will commonly use, with all of the components I commonly will need bundled in already, and archive that as an application template.
Or maybe I’ll try Solar some more.
Or maybe I’ll just keep cooking my own.
Test first or test later
A lot has been said about test driven development. While I am a proponent of testing, I am not a fan of test driven development. I could go on at length about it, but actually somebody else has already said it clearly, so here’s a link to what Paul M Jones said about it instead of boring everybody with a retelling here. I’m pretty sure he’s smarter than I am anyway.
David's Tech Blog 