[Link]Falling Short
Falling ShortReconsider using Java when developing today's interactive Web applications
by Daniel F. Savarese
http://www.ftponline.com/javapro/2006_03/magazine/columns/proshop/
August 28, 2006
Java application servers—or, to be more precise, servlet containers—suffer from a resource utilization problem that's bothered me for years, but has finally come to a head for developers at large. Most Java application servers I'm familiar with assume HTTP request processing will be relatively short-lived and dedicate a separate thread to each request. HTTP servers in general follow this pattern, whether they use a worker pool of processes or threads. An exception is the Apache HTTPD event multiprocessing module (MPM), which assigns multiple, long-lived HTTP connections (those requesting to be kept alive) to a single thread.
Lighttpd multiplexes multiple connections through a single process, but that approach has its own problems. The first application server I've seen to tackle the problem effectively is the Jetty 6 servlet container with its continuation mechanism. Before I explain why the current request-handling approach is undesirable and why application servers need to be redesigned (or at the very least, why the Servlet API needs to be improved) and take a lesson from Jetty, I have to digress and discuss the context of the problem as I've encountered it.
This year I've been heavily involved with Web application development. I've witnessed firsthand the problems companies have with scaling Web applications to support large numbers of users. You have to make a lot of compromises when you go into a project that already has a long history. Business realities prevent you from scrapping the entire system and starting from scratch. However, what if you could start from scratch? Well, I had that opportunity and came to a number of rather controversial conclusions, the first of which was to throw Java out the window.
Been There
Since making that decision, I've been working primarily in C++ (for the heavy lifting) and Python (for the glue), something I had meant to do almost ten years ago before Java came along, and I decided to go with the mindshare. The problem I keep finding with Java-based Web applications is that they eat up memory like there's no tomorrow. In many situations, you run out of memory before you run out of processing power. This phenomenon is a direct result of the rate of growth in aggregate processor speed (not just clock speed, but total work capacity) outstripping the rate of growth in memory available to a processor (or more accurately, available to a thread of execution). With the price of rack space and electricity, underutilized processing power is wasted money.
I don't want to rekindle the age-old debate about Java performance. I know it can be five to ten times slower than C++ for the types of applications I write. If all you do is benchmark simple loops, arithmetic, and virtual function invocations, the two perform about equally (using the HotSpot server JIT). The performance you'll achieve in real applications will depend on the application and how well the programmer understands how to best exploit each environment. For example, if you don't understand how to use C++ templates effectively, you probably won't see great performance differences because you'll be dependent entirely on virtual functions and their high invocation cost for polymorphism.
Still, it's not the speed that concerned me. I'm willing to buy the argument that it's better for a company's bottom line to develop software faster than to develop faster software. After all, I have been working with Java for ten years now even though I worked primarily with C++ before Java arrived. I find that if you write C++ code that depends on dynamic polymorphism, your development times will be similar to Java development times, but you won't see much of a performance benefit these days. Once you apply static polymorphism with templates and template metaprogramming, you can fall into a quagmire of time-consuming compiler errors, portability problems, and syntactic issues (not to mention long compilation times) that increase development times, but yield much faster execution times.
Getting back on track, the problem that really concerned me was the memory consumption. My C++ programs were consuming five to ten times less memory than my Java programs. If you can write programs that are five times faster and use five times less memory you can get some serious scalability and cost improvements.
The truth of the matter is that Java programs don't have to use as much memory as they do. Multiple layers of inefficiency are thrust upon Java programs by the way Java APIs and application servers are typically implemented. They tend to be written as though there is no cost to garbage collection. Even if the cost of garbage collection were zero, you've still got garbage eating up memory before it's collected (and who knows when that will happen).
Take Out the Trash
If you're creating garbage with every method call, your program may always have heaps of uncollected garbage lying around. There's nothing that will kill Java application performance faster than executing out of swap space. I've seen it happen, and it's not pretty. Developers sometimes think they can add layer upon layer of Java support to their Web applications, forgetting about the consequences of running a Java application server, a Java distributed cache, a Java content management framework, and so on, all on top of one another. That distributed cache isn't doing you much good if it's operating out of swap space.
I've even seen people use a separate servlet container instance for each servlet, effectively using them as server processes so that if one process fails, the other won't be affected. Creating fault tolerance by relying on operating system memory protection and process separation is a good practice, but not when you're bypassing one of the primary reasons for using a servlet container (that is, to conserve resources by executing multiple servlets within a single process).
Scaling and performance problems aren't caused primarily by Java itself, but instead by how Java is used—which brings me back to request handling in Web applications. Even today's average desktop computer can handle many thousands of network requests. Except for gigabit-per-second LAN connections, network bandwidth should saturate well before processing power tops out. However, budding Java Web application developers are running out of memory before the network is saturated. The other problem they run into is a major performance bottleneck created by using databases the same way you would use main memory; but that's a story to discuss another time (and it's in no way limited to Java Web applications).
Java Web applications run out of memory before the network is saturated, not because of operating system TCP/IP stack limitations, but because of inefficient resource utilization both within the application server and the application. The changing face of Web application development is making this issue more apparent. Web applications are starting to look more like traditional desktop applications, requiring a high level of interactivity.
Every mouse movement and click in the Web browser has the potential to trigger one or more background conversations with the Web server. This behavior results in both an increase in connection volume and frequency to the Web server. Neither the browser nor the application server provides a standard API for using persistent connections to reduce the frequency of new connection establishment. Therefore, it has become common to use polling as a way of supporting the delivery of server-initiated events to the Web browser.
To handle all of these new connections, the existing thread-per-connection model has got to go. Jetty 6 addresses the problem by allowing idle connections to be suspended—freeing the active thread—and to be resumed later. You suspend the current request with a simulated continuation. I say it is simulated because continuations are implemented usually as a language-intrinsic feature, and the Jetty continuation works only within the Jetty servlet container. You can't use it in an arbitrary Java program without building your own continuation support.
To suspend a request, create a Continuation instance and call suspend(), providing a timeout value after which the continuation should be resumed. A continuation can be resumed by external events delivered by other threads (for example, the arrival of a chat message). Data associated with the event is accessible through Continuation.getObject.
Back to the Drawing Board
Within Jetty, when you call suspend, a RetryRequest exception is thrown, transferring control back to the servlet container. The container then adds the current request to a work queue and assigns another request to the thread. Suspended requests are pulled off the queue after their timeouts expire or other threads resume the requests based on external events. Jetty also provides a portable implementation of its Continuation API, based on calling Object.wait to stop executing the current thread. However, this implementation provides code portability only. In a non-Jetty servlet container, you will still be using a separate thread for each request, and each thread will block waiting for the continuation suspend timeout to expire or for another thread to deliver a resume event.
The approach used by Jetty works around the limitations of the Servlet API. However, a more efficient implementation can be achieved by building into the API standard support for handling many concurrent long-lived connections, keeping in mind the requirements of interactive Web applications. Application servers (both Java servlet containers and C-based Web servers) also must undergo redesigns to fully exploit available system resources.
About the Author
Daniel F. Savarese is the founder of Savarese Software Research Corporation, where he directs the development of distributed systems infrastructure software. He founded ORO Inc., was a senior scientist at Caltech's Center for Advanced Computing Research, and was vice president of software development at WebOS. Daniel wrote the original Jakarta ORO, Commons Net, RockSaw, and Sava Spread libraries. He also coauthored How to Build a Beowulf (MIT Press, 1999) and earned a Ph.D. in computer science from the University of Maryland College Park. Contact Daniel at www.savarese.org/contact.html.
页:
[1]