Juha Palomäki - Theory of (web framework) evolution

A look at the history of web frameworks and my views on where we are heading.
1G
It all started with cgi-bin applications that really did not separate the templating from rest of the logic.

1.5G
It is little bit difficult to say if the simple scripting based approach was any better than the first cgi-bin applications. I say yes, because there you usually had a little bit more structure. The application was divided between multiple scripts, each one usually handling a specific url. The scripting framework took care of preprosessing the request and splitting for example request variables to suitable data structure.
Examples: Classic ASP, PHP, Perl, Livescript

Developers noticed that mixing logic and markup easily leads to hard-to-understand and -debug code. The obvious solution was to apply the Model View Controller that had been succesfully used with traditional GUIs. Usually this lead to a programming model where we had one controller per page.

This step did not really require new frameworks. Even the simple scripting solutions allowed one to separate the logic from the markup, all though this required some discipline, since both were usually contained in the same script file.

There was (and still is) great deal of controversy regarding the “how much logic can you have on your view” question. Some people say none, many others are more relaxed.

One big problem with the 2G solution was that they usually provided limited support for encapsulating parts of the user interface (for example a widget that needs to be on every page of the application). This problem was attacked with different kind of tiling systems, template level includes (and to some extend with JSP tags). Usually the problem was that these solutions were just on the view-level - they did not extend to the controller or model part.

Examples: Struts, ASP.net MVC, Django and many many others

Next step was to adopt more principles from the traditional GUIs to the web, this time in form of components. 3rd gen frameworks allowed users to create components, each consisting of separate markup and logic and them combining these components to larger components and finally to pages.

These new frameworks made the developers job easier for certain tasks, but they were not a suitable solution to all issues. The fundamental problem of the web development is that web is pretty much stateless. This is not just a drawback - it can also make things like building scalable web sites easy. But for a fancy component oriented web framework the statelessness is a problem - they must manage the state some how. Some decided to put the state to client (ASP.net webforms), others kept it on server. Neither solution made everybody happy, people were afraid on bandwidth use, serverside scalability or security.

Another issue that come up with the latest breed of frameworks was the lack of control. In order to work, the frameworks tended to mess up more of less with the HTML markup - leading the coder to feel out-of-control. This also caused troubles with 3rd party Javascript components. Including proved to be difficult, when the frameworks for example took control of the element ids.

Examples: ASP.Net (webforms), Tapestry, Wicket

Bigger is not better

As we have seen with other programming tools, the new generations are not always better that previous ones. Just look at the 4GL programming tools. There are certain cases where they shine, but they did not really take on for the generic application development. Same applies for web development. This is the reason why for example Microsoft is seemingly stepping back by introducing ASP.NET MVC. There are simply use cases that are easier to cover with the 2G approach than with the 3G, component oriented frameworks.

What next?

I believe this is the end for web frameworks of traditional style. Frameworks like Wicket, Tapestry and ASP.NET (WebForms) will continue to evolve and get better and better, but the fundamental problems won’t go away.

BUT meanwhile the browser has changed from what it was in the days of NCSA Mosaic. It is no longer just the dummy HTML rendering tool between the server and user that it used to be. Instead it is a full blown application execution environment. This is not exactly a new thing. It all started with Netscape adding the support for simple scripting language, later renamed to Javascript. Then came plugins, Microsoft ActiveX, Flash, Java Applets and many more. Many of them promised to bring the desktop applications to the web. But users and developers did not cheer. The applications written with these did not really feel web, they had all kinds of security and performance issues and compatibility between browsers was usually poor. Simple things like printing became difficult.

One of those technologies has done better and I argue that is the basis of the future for the Internet application development. We are already seeing examples of this future when we use products like GMail or Google Docs. The applications abandon the usual model of “click-go-to-server-serve-a-page-wait-click”.

The new way of building applications is completely different than the previous one. Revolution instead of small step in evolution. But at the same time it is nothing new. What we are seeing now is essentially the same thing we were doing on the desktops before this great new web way of doing things took over. Back then people were building so called fat client applications, with the application running on the desktop taking care of mainly user interface related tasks and the server application taking care of business logic and database access (some people also experimented with skipping the server part and going directly to database from client, but this did not prove to be a good idea, so I suggest we skip this part of evolution this time).

The tree layers in this “new” model are the same as before (I will take a closer look at these later):

Client: Modern web browser, capable of executing Javascript and preferrably supporting latest breed of web technologies (e.g. HTML5). Applications are written either with Javascript or with some other language that is compiled to Javascript. I see no future in other technologies (Flash, Java etc) on the browser - except for special use cases.
Server: Most, if not all, application servers and scripting solutions used with the 2G or 3G web application development will be usable. Instead of serving HTML pages they are going to serve XML or JSON via HTTP and pushing information via Web Sockets. The servers are likely to be as staless as possible, to make things like scaling and clustering easy.
Database: Since the database is hidden behind the server layer, there is no need to make new inventions here.