Problem Domain Crossover

When technology crosses several different problem domains, it remains independent of a single customer's problem. Domain-crossing technology, if we can call it that, has no specific vocabulary and if there are any reusable key concepts to be found, they are so vague that developing a specific problem domain for the technology may be difficult. At this level of generality, special features that are introduced into the technology to focus on any particular problem are simply not going to translate well across domains.

If we were looking for a specific problem domain, then we have to ask what concepts are outside the scope of the domain? Where are the boundaries? In other words, if there is one specific problem domain for the technology, then what concepts should never be modeled in the domain, and why?

These are important questions to ask before introducing a new abstraction or conceptual model into your core code base, if your software can be used across problem domains. Whether it affects your service layer, your data model, or ends up deeply embedded at some other level, new abstractions may severely constrain future application of the technology in different spaces.

I always try to design for the general case in software development projects because one can never anticipate how technology will be applied in the future or where your organization may derive business value from its application elsewhere.

Software developed this way gets deployed as a cohesive solution in a customer environment, where a certain level of generality adds complexity and may become overwhelming to grok. Sometimes it takes a highly skilled technologist who looks way beyond the documentation to put all the pieces together into something that solves a specific set of customer problems.

The end result of this difficult exercise is inevitable: a collection of very loosely integrated pieces talking to one another, none of which individually point to a particular customer problem, but that once integrated and configured in production as something more specific, delivers what is needed.

The Ctrl tries to fit into this way of thinking about software. The language provides abstractions only to the extent that the timestamp of an event can be manipulated by statements in the language to lookup related events and compare event properties. That's pretty much it. Other language features supported are general programming constructs: if statements, variables, and for loops.

With these basic features, the Ctrl can be used to solve a myriad of complex problems involving the detection of patterns in event streams. For those cases in which a customer problem cannot be solved with basic features alone, a call-out mechanism is provided by the rules engine, to invoke external code. That external code remains isolated in the environment of the customer it is developed for, and never becomes part of the core.

In the next few iterations of the language, as we get some experience with customers, my goal is to add more CEP features without doing anything that may couple the language to any particular customer problem, since the language and the interpreter are part of the core.

This can be a struggle. Pressure to take shortcuts or to do something specialized for that special customer is difficult to ward off. But pushing back is a good thing to do because otherwise a heavy price is always left to be paid somewhere down the road when your generalized solution to many different problems is contaminated by special one-offs.

When shortcuts are taken, or something overly specialized makes its way into the core, future development on that core becomes more and more costly to do. This is a vicious cycle in software development that you always want to avoid getting trapped into, as explained by the debt metaphor.