From Practice to Pattern to Syntax

In the software industry, Design Patterns are widely recognized. It is a widely held assumption that these patterns require customization. Recently, I have been wondering if the philosophy that these elements need customization is correct.

The source of this philosophy is obvious as Christopher Alexander’s book The Timeless Way of Building is a key source of inspiration for the authors. Alexander believes strongly that one factor contributing to the “lifeless character” of modern building is the use of modular components, and the lack of “fit” that results.

While there are many parallels between the process of building beautiful and functional buildings and building software, there are certainly significant differences. As an industry, we have sacrificed potential, mostly in the form of execution cycles, for modular components, higher-level languages, operating systems, etc. The construction industry has also made sacrifices of potential for the cost benefits of modular components. However, in the case of construction, the sacrifice has been of quality, execution time.

For the construction industry there is no equivalent to execution cycles; construction time is a separate concept more akin to development time. Even if there were such a concept, it is unlikely the construction industry would have it in such excess as the software industry. As long as computing power continues to advance, there will be excess capacity to fuel these tradeoffs.

Can design patterns move from the realm of patterns into reusable modules? Better yet, can they move from patterns into reusable syntax? It is already happening, albeit slowly, and with little formal recognition. Recognition is low because the efforts do not completely implement the patterns. Many patterns have complexities that syntax cannot completely capture. Syntax however can get close. How close would it need be for the pattern itself become unnecessary?

I have not spent the time to think about every pattern and how this might happen, but my guess is, yes, it is possible. Similar to the sacrifice of many custom assembler algorithm patterns, for generic looping constructs, I believe the same will happen to today’s patterns. The Iterator looks to be the first casualty, as languages like C# and Ruby build in constructs that codify the pattern into a much easier to use, if slightly more restrictive sheath (yield). Delegates also change a many of the patterns, especially Observer.

This is not totally a new development either, as languages as far back as Smalltalk, implemented language features that made Proxies trivial. In truth, every pattern has a large degree of busy work involved, and the best languages in the future will likely be those that automate these elements. Some creative features might take the place of patterns, or the most common forms of each pattern might become codified in the language syntax.

One idea is better codification of the delegation-by-composition design. This would allow a member variable to proxy for a class’s external interface. The example below shows one possible syntax for this.

public interface IAsset
{
decimal purchasePrice {get;}
}

public class OwnedPlane : Plane, IAsset
{
IAsset interface by asset;
private bool operational;
public bool Operational { get { return isOperational; }}

public void bought(Corporation buyer)
{
asset = new CoporateAsset(buyer);
}

public void bought(Person buyer)
{
buyer = new PersonalAsset(buyer);
}
}

public class PropertyTaxCaluclator
{
public decimal TaxPlane(OwnedPlane plane)
{
return plane.purchasePrice * (plane.operational ? FAATaxRate : normalTaxRate);
}
}

You could have a tool generate code like below, in place of the line in bold, but when then this code must be maintained. Even if your tools can regenerate the code, it will still be a distraction to review. (Remember in a real example you might have dozens of properties.)

private IAsset asset;
public decimal purchasePrice { get { return asset.purchasePrice; } }

If you are not planning to modify the code, you are far better off with a simple concise statement that says exactly what you mean. There likely will be resistance to codification, as it will require some sacrifices. In addition, there is likely to be at least a few bad implementations that do more harm than good. In the end, I believe codification will win out, as it is simply the most succinct, direct, complete and meaningful implementation.

In order to achieve a higher level of abstraction, you must provide your tools with information about a higher level of intent. The only caveat is that the design of systems to capture that intent is hard, and often requires a progressive process that first accumulates knowledge in documentation like Design Patterns. Later, general-purpose implementations incorporate this knowledge to cover the most common cases. Eventually, however, the lessons learned from these preliminary steps should result in language constructs that replace the most common designs.

For very complex patterns, I imagine there will be more intermediary steps, such as implementations in tools like Software Factories. The software factory concept is something I have only recently begun to understand, but it shows a lot of promise as a bridge between knowledge and final implementation. The promise is actually much bigger than that, but that is best left for another day.

The name software factory sounds awfully scary, conjuring up visions of slave driver bosses, and rote development. The name is unfortunate, because this is quite far from the truth, intent and reality of software factories. If I might, I would offer up the alternatives of Framework or Pattern Factories. Not only do these names better represent actual abilities, but they also avoid the negative connotations of “software factory”.

A software factory cannot generate a complete application. A software factory can generate a complexly working framework, and can use further recipes to generate pattern implementations. Only when actual business logic supplements the generated code is an application complete.

The Next Gen Of Cars (I cant wait!!)

This is one of the enabling technlogies of the Transportation Concept that I blogged about a long time ago. Watch the video and you’ll understand the idea a bit better.

read more | digg story

Principles of Language Design – Semantic Capture

The time has come for programming languages to move away from only being a collection of commands for a computer to process. It is time for programming languages to become a method of expressing intent, in addition to the traditional role of providing executable commands to the processor.

The concept itself is not entirely new; the language Eiffel did it to some degree. Eiffel never really took off, but I do not think the ability to codify assertions was the reason.
Even if it was, today is a different day. One recent trend in software development is automated unit testing. Many unit tests are nothing more than a primitive expression of semantics. A language that allowed greater expression of intent, could allow tools to automatically validate or test most of what unit tests do today.

Unit tests are a great development, and will still be necessary, but they make more sense for higher-level tests than for simple assertions. Using unit tests for simple assertions requires repetitive behaviors, and usually occurs as a separate process. Automated validation of intention specifications can accomplish the same goal while remaining integrated to the developer process, and requires far less code to support. By remaining integrated to the developer process, developers know about problems much sooner, and will be able to fix them with much less overhead from communication, re-tasking, etc.

Many intentions can be validated algorithmically, and provide developers with compile time, or sometimes even intellisense time warnings.

Modeling is also very hot these days, and intention specification is a natural method to augment modeling tools and provide better reverse engineering and model synchronization capabilities.

There are not very many downsides to including capabilities like this in languages either. In fact, all of those that I can think of are only perceived downsides.

The one I expect to be the most common concern is the perceived conflict with conciseness. Conciseness if another principle of language design, but it should be applied in moderation. In addition, the rules of conciseness do not apply to intention specification features because intention specification adds necessary information. In fact, when intention specification replaces unit tests, the result is more concise code. Lastly, intention specification is (or should be) optional.

It is true that embedding intention specifications within code will add to the size, and when not necessary may complicate a readers task. However, usually they will add valuable content that would have been expressed in unstructured comment or disconnected documents. In addition, by providing set syntax for intention specification, IDE’s will be able to hide that information when a reader wants to ignore it.

Another perceived downside is that of inflexibility. This stems from the perception that intention specifications might be used to enforce rules. While it is possible to do this, rational language design will avoid requiring enforcement. It is far better to allow users to make the final decision, rather than the language designer. I actually plan to explain this principle in a future post, but for now, I will point to a good and bad example of how to protect users from mistakes.

On the good side is the implementation of pointers in C#. C# fully supports pointers, but requires users to accept very explicitly the risks involved by using the unsafe enclosure block.

On the bad side are the implementations of exceptions in both Java and C#. Java makes the mistake of requiring the exception to be handled or rethrown, which often encourages bad practices among developers, such as always rethrowing Exception. C# on the other hand reacted to this by removing the ability to specify exceptions entirely. The right solution lies in between these two extremes, where specification of Exceptions is possible, but not required.

Another Nail

EMI has announced they will open a free advertising supported legal peer-to-peer service later this year. You can read about the service on hypebot, so I will not rehash the whole story. One bit of commentary I thought I would add is to draw attention that Apple, unless it begins licensing its technology, cannot participate in services like this.

So, this is yet another expansion of the market rules, which Apple’s closed market strategy will have a hard time keeping up with. It is so odd to be making the same arguments against Apples business strategy that I made almost 15 years ago in relation to Windows vs. Mac. I wonder how it will turn out…

Why I Blog about Apple

It has become obvious to me, that people quite easily misunderstand my posts about Apple. They are not really about technology. Anything related to technology is really just supporting data. They are about two things.

The first is the iPod+iTunes monopoly, and the long-term negative effects that will result from supporting a closed system monopoly.

I am trying to convince people that we should be asking Apple to license FairPlay. This one is not about Apple vs. Microsoft. In fact if Apple did follow my advice and license FairPlay, it would be a sure fire kill shot for WMA. Why would any store or device prefer the relatively unused WMA over FairPlay? Simply put, they would not. Without the option to do this they settle for WMA.

In the long term, I think Apple will lose, not because WMA or Microsoft is better, but because the combination of the entire industry is better than Apple alone. By all appearances Apple will not realize this mistake until it is too late, and all of their potential partners are firmly bound to WMA and Microsoft. By being open with WMA licensing, Microsoft is slowly binding the entire industry (other than Apple) to their format, much as was done with MS-DOS and Windows.

I do not consider this process evil. In fact, the process itself is beneficial. The binding is a negative side effect, but the overall benefits of open licensing outweigh that side effect. I think it makes sense to try to mitigate this binding through intelligent regulation, but that is another topic for another day.

The second is about the Apple community’s assumption that the world would be better off with Apple having 80% OS market share, rather than Microsoft.

I really should not care much about this, considering that Apple’s basic business strategy is flawed in a way that will prevent them from ever having 80% OS market share. Unlike portable digital music, Apple does not have the opportunity any longer to grab market share in an emerging market. The OS market is far too mature for that to occur.

Despite this, I hear this argument so often that it does tend to get under my skin in a way that I feel a palpable need to refute it. I am not suggesting that Apple users need to stop using their Macs. If you like it, use it. There are some benefits to OSX.

However, be honest with yourselves and admit that most of those benefits derive from elements that would either disappear or be very negative if Apple had 80% market share, rather than 3%. Security is one example, and lack of conflicts with closed system hardware is another.

Also, be honest with yourself and admit that your biggest gripes come from these elements as well. A closed system with 3% market share will not have the same amount of compatibility with hardware or software, as the 80% system that is openly licensed.

If you really want to advocate a better dominant operating system, I suggest you choose one that is open. The Linux group, or better, BSD group, have a better argument for this then Apple does, because they are open. However, I think you do get a better user-experience through Apple than through Linux/BSD. Just because Apple is not Microsoft, does not make them Linux. Also, since I am sure this will come up, just because Apple uses BSD, does not make them BSD. OSX is as free as a man in Alcatraz, and as open as the ground beneath your feet.

I will however try to refrain in the future, as I think I have made my point in as clear as manner as I can manage.

WhatIf: Microsoft announces Windows Vista to be available only on XBOX.

Yes, the title is complete fiction; but what if it was true? An anti-trust suit would be filed, and Vista would never make it to market, and that would make sense.

So why is it the Microsoft hating Apple users seem to be suggesting this is Microsoft should be doing? No, they don’t say this literally, but the implication is there. Within the Apple community, the current hype is about how Apple’s closed system is superior to the PC system. If the suggestion is that closed systems are better, than should everyone be creating them, including Apples competitors, including Microsoft?

Personally, I think they are wrong, but that is the suggestion that comes across loud and clear. I think they are wrong because the closed system has been defeated several times already. In fact, this has a lot to do with Microsoft’s initial rise to dominance. Not only is there the obvious defeat of the original Macintosh operating system, but the mainframe market as well. Mainframes often are not considered a direct competitor to MS-DOS systems, but it is certainly true that all microcomputer architectures were in competition with mainframe architecture.

But I digress, the point is MS-DOS became so popular because it did something very few others were even trying to do, and that was open the hardware-software loop, so that when a developer wrote code he could move it to different hardware. There were a few others trying to do this, but most were not much different from MS-DOS (like CP/M86). Microsoft however succeeded, creating the IBM compatible market with IBM’s assistance. The only other effort to achieve any similar success was UNIX. UNIX however had much internal strife, and generally was slower in opening closed systems.

For the early years of UNIX, DEC’s PDP-11 systems were the only supported platforms. VAX support was added by BSD but this changed little since VAX was still a DEC machine, just more expensive and more powerful. In 1982 Sun released SunOS (the precursor to Solaris), and systems built using Motorola processors. SunOS ran only on Sun hardware, and applications required recompilation to move from other UNIX platforms, but at least it was a beginning of hardware independence for developers.

Other less closed developments happened later, like Linux and BSDI, but by that time, the IBM PC compatible market was well established and so was MS-DOS. I will not bother to cover the years since, other than to concede that yes, Microsoft has an effective monopoly on operating systems, and that dislodging them is certainly difficult due to the large amount of software designed for the Windows. A great deal of people contend that Microsoft has abused their monopoly status in many ways. I will not attempt to debate any of these contentions today however, as it is not necessary to make my point.

The point is that none of the acts Microsoft has been convicted of, settled for, nor even accused of, has been as black and white wrong as a closed hardware/software system. Preferential licensing, undocumented APIs, and bundling of extra software into operating systems would be harmful, but absolute exclusion through lack of licensing is simply and obviously inappropriate for a monopoly.

Apple users however ignore this, and advocate a system far more restrictive than the one they campaign against. In terms of OSX, Apple can legally do this as long as their market share is small, because minor vendors have that freedom if they want it, however misguided it may be. But what astounds me is the acceptance of it in the digital music arena, where Apple does have a monopoly nearly as strong as that of Microsoft in operating systems. No one has asked Apple to remove features, alter hardware or software on their iPod. The best one expects from Apple is to license the FairPlay DRM to other companies who either sell music or players so they can alter their own products. At the very least one would hope Apple would not sue companies that reverse engineered the DRM and re-implemented it, nor go to great lengths to intentionally break those implementations.

For years, Microsoft has let Wine go about its way. Recently they have designed WGA to not function on Wine. However, their official policy seems very weak on the issue. The overall effect only turns out to be that users must use a manual process for downloading updates for Office applications. If Microsoft were Apple however, they would have made WGA disable the office applications, and threatened lawsuits.

Microsoft is not perfect, but at least they are better than Apple.