Testing and the Open Source Value Proposition

One of the key concepts from Agile Development and Extreme Programming is to test early and often. I generally embrace this critical notion from both development disciplines with a slight amendment. My variation ? Test everywhere but effectively.

By “everywhere” I mean that every phase of the IT development process should have measures and tests designed to reduce as much as possible the key risks identified and associated with the project. So right up front in the feasibility and design stages, developers should be looking to identify and test project risks. One of the easiest comes out of this initial development process. Specifically, a measure of commitment of the various stakeholders is their willingness to commit resources, time and most importantly the right people to the project. If they cant do that then the essential consensus on the goals, scope, schedule and resource limits to the project is potentially in jeopardy. My immediate reaction is to adopt an Agile/Scrum phased set of working deliverables on short time horizons of 3-8 months maximum. The whole idea is to get greater commitment from the top by quarterly or semi-annual tangible and useful “can do” deliveries of modules.

By “effectively” I mean that any project group can efficaciously tell what tests and how often they need to be applied to reduce the most tension and uncertainty of knowing whether a particular phase, tool, and/or process is going to work properly. So the critical outputs from the Requirements, Design and Coding phases are risk analyses that identify not just the risks but also the earliest best possible tests to decide whether the risks can be reduced and or effectively handled by the evolving system.

Open Source software delivers such test benefits much better than proprietary software.

The reason is simple – the upfront cost and risk of the Open Source software is nil compared to proprietary software. People can try it out and stop early if it does not work to satisfaction. Further, to make money and/or retain users the Open Source vendor must deliver effective support, training, and consulting/mentoring. This means emphatically that the success of the open source software in meeting the users needs is paramount; not nearly so for the proprietary vendor. In fact the proprietary vendor not only gets their money upfront but also subjugates the user to a “no-real-help-or-responsibility-from-us” EULA agreement that clearly precludes any warranties as to reliability, security, and other measures of effective performance.

But proprietary vendors counter that the upfront fee creates competition for innovations and enhancements among vendors that is totally lacking in the open source field. Not so again, because the support process most often includes a roadmap for scheduled fixes, enhancement and improvements. Take a look at the roadmap for Pentahos Open Source BI stack for an example. So Open Source vendors have every reason to blue sky and dream as the proprietary – its necessary for their survival. In fact I would be recommending to Open Source vendors to be charging for the right to see some future betas. This would help to defray the development costs and would certainly attract those looking for a competitive advantage.

As IT matures into a highly competitive, zero-sum marketplace reduced to dueling monopolists, Open Source is the natural pendulum swing back to meeting users needs first.

(c)Jacques Surveyer 2005