So I want to pontificate about SOA in the Enterprise for a few minutes here. I, as an “enterprise” developer obviously have an opinion on this topic, but this final rant was spawned by listening to Dalibor on Java Posse speaking of OSGi in the Enterprise. I had a similar conversation with
Brian Ehmann, the Java Posse intern, a while back. So I want to address several things.
First, I want to talk about OSGi, and why I would never allow it in “my” enterprise. In “my” enterprise I want absolute control over what every service has in its runtime environment. Indeed, as a longtime Fedora user, one of the things I loathe about its Java support is the dependency system that works just as Dalibor describes — at the OS level. The beauty of Java on the server is that applications and their API dependencies can be packaged into a single artifact (WAR/EAR) that is completely isolated from the library install or configuration of the server itself. Anyone who has dealt with install libs on Unix machines understands this. Indeed, if any two of your “enterprise apps” had to have the same version of something even trivial, like Log4J, this could become a serious problem for deployments.
Secondarily, the last thing I want is my application “updating” itself without my (or Major League Baseball’s) expressed written consent. One of the “theories” I have espoused for a long time, and indeed one of the reasons I fell in love with J(2)EE is the WAR file, and having a singular artifact from the developers. A WAR/EAR file is an artifact that can be built by development and move seamlessly from development, to QA, staging and production as a singular, atomic unit. As soon as you introduce dynamic dependency updates in your production systems, you have lost this atomicity in terms of production bits. It is true that OSGi does not mandate this loss of atomic control, but it does encourage it.
Of course, this example has a certain narrowness of focus. Of course things within the enterprise change, but one of the advantages of a Web Services based infrastructure is low-level dependencies as I have discussed to this point are maintained independently. At the service level, of course you need to control change. In my current environment we do this by endpoint management and control. This is one of the things that strikes me as a failing of “SOA” as a “solution”. Endpoint discovery is almost never a real problem for the enterprise. Low level network services like DNS are perfectly adequate for controlling this. Barring that, in the “WS” world. simple things like controlling HTTP routing through Apache or a specialized load balancing product work well. These solutions make it easier than most “SOA” products to run multiple live versions of a service without forced deprecation as well. Sure OSGi can operate at this level for service version control, but why trust it? Why allow binary update of runtimes on servers when endpoint URIs are easier to control over time?
Finally I want to address the issues of SOA as a failed technology. Bill de hOra and Joe Gregorio have noted that SOA has reached the “assign blame” state of failure. Indeed, here on the O’Reilly network shadows of this meme have kicked around.. I want to note that SOA is still a valid idea, but much like BPM, the marketing doesn’t live up to the hype. In enterprises with “architect” staff who are fundamentally unqualified– though I don’t believe the position is unnecessary in medium-large organizations — or a technical debt load that has become unmanageable, SOA proffers no real solution to any problem. Companies selling their SOA product as a silver bullet to said architects that will eradicate said debt are doing so at their own peril. Best case, their thousands of dollar projects will be quickly forgotten failures. Worst case, these enterprises will feel scorched by the failure and never want to deal with the same companies again. In reality, the former is most likely, since the people making the decision to go with the failed technology products will keep their jobs, blaming the failure not on their own lack of understanding of the enterprise they are responsible for but the vendors. Indeed, these sales relationships mean the same vendors get to come back when the next “Silver bullet” enterprise solution is available.
This betrays a real lack of understanding of what OSGi is all about. In fact it solves many of the specific problems you discuss, because it gives you (or your sysadmins) more control, not less.
OSGi applications do not go "updating themselves" automatically. That implies some kind of malevolent AI, just waiting to wrest control from its human masters. OSGi is not Skynet... it is just a module system that allows for hot-swapping of modules. What's so dangerous about that?
Here is the thing, I don't *want* hot swapping of anything within the scope of an application. I consider that to be a categorical bad idea.
There are 2 things that people always talk about when they speak of OSGi in EE apps:
1. Apps resolving dependencies at deployment time.
What a horrible idea. You have now introduced 5 new moving parts to the app deployment process. Even worse, you have lost artifact congruity between what the developer built, what QA tested, what was on the beta server and what is on production. Sure, through studious managment you can maintain this cohesion, but why would I bother.
2. Hot swapping modules in deployed applications.
Again, we are back to asking whether the bits in production are the same bits a developer intended. Given that EE apps themselves should represent your deployment module, IMHO, this is a pointless "feature."
In a discussion with Brian about OSGi he specifically mentioned that he liked the ability to deploy sub-sections of a web site without deploying a whole new app. Sure, that is clever. My question is, why is your whole "Web Site" a single deployed app?
In short, "Hot Swap" in production, is just a bad idea. Moreover, hotswapping at the module level means I have lost the ability, as I noted above, to keep multiple concurrent versions of the same stuff running in production on the same instances. It is easy to build 1.1 and run it on the server next to 1.0. Why do I need another moving part that makes that more complicated?
I really must point out that this article and Robert's reply to Neil's comment are rather grotesquely misinformed. OSGi does in fact solve most of the problems that you list *against* OSGi. When you mention resolving dependencies at deployment as being "a horrible idea", I wonder how you can be so happy with arbitrary dependency resolution at runtime while the app is in use. With OSGi you can name versions of your dependencies so that they are explicitly resolved before the bundle is started. I'd like to know before my app goes live if there is a dependency missing or if the wrong version was packaged, thanks. Another really obvious point is that OSGi bundles don't ever update themselves at runtime, unless someone goes through the effort to make it happen. When you talk about hot swapping, because of dependency versioning as I mention above, this is explicitly supported and will work much better for running concurrent versions of the same bundle using the exact same shared libraries, not different copies copied to the WAR/EAR made at packaging time (which could have changed the dependency versions, therefore creating a more inconsistent deployment). It all seems pretty straight forward and logical to me.
When you mention resolving dependencies at deployment as being "a horrible idea", I wonder how you can be so happy with arbitrary dependency resolution at runtime while the app is in use. With OSGi you can name versions of your dependencies so that they are explicitly resolved before the bundle is started.
Define *arbitrary*. This isn't about building the app. My "runtime" dependencies are those version I have packaged with my WAR/EAR file. That's it. That bundle of bits is final, and it will move, unchanged, through the test/deploy cycle. Specifying a version is less arbitrary than that, as I now have to assure that the repositories in production and dev are in sync, have the same availabilities, etc. My deployment phase now isn't "Install app" it is, "confirm the deploy environment is OK, install app, wait to see if it has issue, move forward."
When you talk about hot swapping, because of dependency versioning as I mention above, this is explicitly supported and will work much better for running concurrent versions of the same bundle using the exact same shared libraries, not different copies copied to the WAR/EAR made at packaging time (which could have changed the dependency versions, therefore creating a more inconsistent deployment).
Again, even if they changed dependency versions, those versions are packaged, final and tested before it even gets installed on the server. I *like* that each of my WARs are running in total isolation and not sharing dependencies. There isn't an "inconsistent deployment" because they are all self-contained.
I am not saying that build-time dependency resolution is a bad idea, but if that is taking place at deployment time, you are weeks too late in the cycle. Moreover, the even have that take place at deploy time, you are introducing a whole new complexity to the deployment environment beyond what I have to deal with now.
To a certain extent I can sympathize without your resistance to dynamic updates, that idea can be scary, and in a classpath based environment I could not even imagine doing it. However, what OSGi provides is strong modularity, the dynamic model is enabled by strong modularity but in no way is it mandatory. Once people start using OSGi modularity they discover that the dynamic updates can work well and are useful in many scenarios, not in the least during development. And they get (justly imho) excited about this but there is nothing in the spec that says you must use it in a dynamic way. Most JEE code can not be used dynamically anyway. The key OSGi thing is modularity ...
And that is what is a bit puzzling about your article. There seems to be a contradiction between your careful deployment process versus the completely unmanaged situation once you put your JARs on a classpath. Even if QA has done their work on a WAR it will not be fully tested. It is just impossible to test all possible paths in your code. There are many subtle and not so subtle errors in the classpath model:
split packages (classes in the same package from different jars)
multiple versions of the same packages where the first is picked
accessing classes that were supposed to be private
etc ...
Most of these errors are hard to test for and preventing and detecting these is where OSGi shines. It has very clear rules about the dependencies and verifies them before the code is run. This will not discover all errors but it finds a surprising number of them. Even if you have statically analyzed your WAR and ensured that there are no bad dependencies then your still needs to be deployed on the right application server setup with the right classpath. Again, in the current JEE situation this is not checked by the code, it all relies on the system administrator, the deployer, the developer, and others to do the right thing outside the deployment artifacts; if one of them makes an error you run into problems somewhere in the process.
For this reason, many people use the Equinox OSGi servlet bridge to package all their code in a WAR but use an embedded OSGi framework to manage their dependencies inside the WAR. This gives them all the advantages of strong modularity on a JEE application server. These people do not go for the dynamics, they go for the strong modularity because it is worth it.
It surprises me therefore that you do not like OSGi. Your organization obviously is in detailed control of the process, most of the time people that are fanatic about these issues love OSGi because it enforces a discipline through the whole development and deployment chain. As BEA said: “We thought we worked modular until we started using OSGi.”
To conclude. The dynamics are not the primary motivation for OSGi: modularity is. The strong modularity makes systems more controlled and therefore more robust. This enables the hot pluggability in the long term, as it enables many other things. But there is nothing in the spec that mandates you to do hot pluggability. We have been extremely careful to keep you in the driver seat at all times. However, it would surprise me if you wold not find use for it within 6 months of starting to use it ... :-)
Kind regards,
Peter Kriens
OSGi does not mandate this loss of atomic control, but it does encourage it
I think of OSGi as simply a mechanism that allows me to upgrade code without bringing the server down. One example where this would be useful is for deploying a hot fix (that has no dependency changes).
Obviously, you wouldn't want to deploy a new service that relies on other services without thorough regression, integration and performance testing, but I think the problem of "dependency hell" is an orthogonal concern. I've seen staged environments used effectively for these purposes - for example "dev", "qa" and "prod". Service upgrades get promoted through the environments. OSGi simply allows you to do the promotion without having to restart the application.
I would also say that "dependency hell" applies to services in general, regardless of if they are out-of-process (an endpoint) or in-process (an OSGi bundle).
As a Java and C dev, I understand where you're coming from. Static linking of binary blobs under one's own control has its advantages for specific scenarios. But so does dynamic linking of libraries outside one's control.
From the proprietary ISV POV, regardless of the language, total control over dependencies making up one's product is desirable, as it reduces the testing matrix.
From the distributions POV, who are platform providers, a strict coupling is undesirable, as it requires breaking everyone's software on security updates, for example: Let's say your app requires and checks for lib 1.0, and if the version string is not precisely 1.0, it barfs and dies, but unfortunately, lib 1.0 turns out to have had a security problem, so the platform vendor needs to upgrade it, and break your app.
Security issues are very common in software, even software written in Java. You just don't hear much about it unless you are reading the right forums, like bugtraq. Security is not nearly as much of a sexy topic as, say, enterprise SOA is, but Tomcat just had a CVE yesterday ...
So yeah, as a distributor of software, I don't want to have to dig around unversioned JAR files stuck together from arbitrary SVN checkouts on Maven's unmaintained repos using JarJar, or some other tool, in order to figure out if I'm shipping (or running) code with known security vulnerabilities. Unfortunately, the state of the art in Javaland is such, that there is often little choice about it.
Moving away from the current state of the art in the Java world of linking statical blobs of unknown provenance with each other until the green bar on JUnit passes, and into the way the C world started to work when dynamical libraries and packaging were introduced to the masses will require tools (OSGi, 277, OpenJDK in Fedora/Debian/*, etc.), education (how do I use the tools to get what I want?), and most importantly, social competence & efforts to get people to contribute to a shared web of metadata, rather than keeping all their JARs, sources and metadata in mutually incompatible silos.