It seems like a simple problem, you want to allow people to upload videos, and then you want to process those videos to Flash Video. (Admit it. Flash Video is the video format for the web at the moment.) Talk to the LAMP crowd (or the Rails crowd), and when you ask them the question: “How would you process media files on the web?” You’ll find two popular answers:
- FFMpeg for video processing
- ImageMagick for image processing
But, don’t we do everything in Java through the JMF and the JAI? How would one integrate these utilities into Java?
Video: Using FFMpeg from Java
if it relates to audio or video processing, FFmpeg can probably do it. Throw almost any format at ffmpeg and ask it to resize the video, resample the audio, spit out a thumbnail of every 10th frame, and cook your breakfast, and it will do it. If you are interested in producing Flash Video from Java, and you want to involve ffmpeg in the process you’ve got some options.
- Use a drop-in JMF replacement: Freedom for Media in Java (http://fmj.sourceforge.net/)
- Use a JMF Plugin FOBS C++ & JMF Wrapper for ffmpeg (http://fobs.sourceforge.net/index.html)
- Use Runtime.exec() to invoke ffmpeg directly.
If you are looking for video transcoding, and you are not looking at a million visitors per second, the Runtime.exec() solution gets you to “done” faster than wheeling out the JMF and fiddling with the various properties files that need to be manipulated to get a JMF plugin to run properly.
Image: Using ImageMagick from Java
ImageMagick is this powerhouse image processing utility which supports about 100 image formats, provides the ability to manipulate images, apply filters, etc. If you are interested in side-stepping the JAI, here are some options.
- Take a look at JMagick which is a “thin interface layer into the ImageMagick API”.
- Use Runtime.exec() to invoke imagemagick’s command line utilities directly.
Take a look at D’Arcy Norman’s blog entry JAI vs. ImageMagick image resizing. This is a great summary comparison of using JAI vs. JMagick vs. Runtime.exec(). It proves that even though the default java reaction is to sneer at Runtime.exec(), it is admittedly one of the best integration choices between ImageMagick and Java.
Runtime.exec() *shrug*, it’s ain’t so bad…
While Runtime.exec() might seem like a bad idea for performance of a server, you should know that the Ruby programmer down the hall is busily calling out to ffmpeg and launching something revenue generating while you are still researching the intricacies of the Java Media Framework (JMF). Because ImageMagick and FFMpeg are such powerful utilities, you might be well served to just bypass JMF altogether and invoke these utilities directly. Other alternatives are to use Java as a “stevedore” for an offline processor which runs a periodic job to translate everything in a given directory to FLV. In summary, the idea here is that there are times when Runtime.exec() is an acceptable solution, not everything has to be run through the super-generalized media API.
"you should know that the Ruby programmer down the hall is busily calling out to ffmpeg and launching something revenue generating while you are still researching the intricacies of the Java Media Framework (JMF)."
A profound statement with broader implications. (while you are still researching persistence, MVC frameworks, JMF, Hibernate, ...)
Taylor - agreed.
If you are a small agile company churning out web applications for paying customers - and you are using Java, you probably should take a long hard look at your life choices :)
I encourage anybody who is a Java only web app developer to 'step outside the box' and spend a weekend with Ruby / PHP ( even, dare I say, ASP).
I'm not saying Java is bad or the others a better, but the adage 'when all you have is a hammer everything looks like a nail' is quite true of single tech developers.
@Taylor, @S, i'm not ready to give up on Java as being something that can be very agile. The problem isn't Java in comparison with Rails, etc. I've beat that dead horse too much already. I thin the issue us that Javaists tend think too much about "the platform", this entry is an example. Just call out the Runtime.exec(), leverage the existing tool, dont try to fit the existing tool into your framework. @S, you are right, 'when all you have is a hammer everything looks like a nail'. We need to encourage every Javaist to step down to scripting, not because Java is bad, but because it's a useful perspective. I wonder how many Java programmers end up opting for the most complex option just because they didn't know of the alternative.
But, I'm not saying that the JMF and the JAI are necessarily bad things. i could see instances where it might make sense, if I were a vendor creating something that had to interoperate with the existing JMF subsystem.
I did have a client not that long ago who was trying to do the same thing as Flickr in Java, and it was a painful experience. In addition to creating a custom MVC framework, the idea was to just use JAI everywhere. Where ImageMagick+ruby could have handled the offline processor in ten lines of code, JAI required an API and hundreds of lines of repetitive code.
This topic of JAI producing inconsistent results when resizing an image is one of the urban myths floating around. People try the bicubic interpolation and is not satisfied by results, then give up. The point is that by design JAI operations are "atomic" and the best way to resize an image isn't. In other words, you just have to combine two or three operations together, but in the end you get high quality results. I wouldn't understand how the NASA guys could use JAI if it delivered inconsistent results.
BTW, a complete discussion about image resizing with JAI is here:
http://archives.java.sun.com/cgi-bin/wa?A2=ind0311&L=jai-interest&P=R9579&I=-3
Of course, one could blame JAI for keeping that information not immediately available to programmers, you have to search through the archives of the mailing list. More in general, what I see is the lack of a broad-view community for Java imaging people (not only for JAI, but also for other APIs) where this kind of information could be made more readily available.
For what concerns the verbosity, yes, JAI requires a lot of repetitive code. Unless you write a wrapper that simplifies the API. For instance take a look at mistral.tidalwave.it (work in progress). In this way you can also implement good quality resizing with a single operation ;-) Staying with Java gives a lot of advantages, including being able to work on grids using a number of technologies, such as Jini.
@Fabrizio, The NASA reference is telling, sure JAI works, but the point here isn't image processing in an organization that has enough resources to launch a reusable space vehicle. I think JAI has it's place, but it isn't necessarily an agile solution.
Thanks for the Mistral reference, the tutorial looks interesting. Mistral also appears to be in line with something that sits atop a standard API and makes it useable for the end-user - in the same way something like Hibernate allows people to benefit from JDBC without touching it.
@Tim We can discuss whether JAI is agile or not, whether such an organization as NASA is required or not to run it... but I understand that the point of the mentioned blog was about its "inconsistent" results. Well, results are consistent if the API is used properly. :-)
For what concerns JAI agility, among its success stories there is a company delivering a product for the desktop: http://java.sun.com/products/java-media/jai/success/lightcrafts.html - so looks like NASA is not required to run it ;-)
@Fabrizio, I think that's good, no problems with JAI as being a valid selection. Point of the blog post: if you are looking for media utilities and video processing specifically, ImageMagick and ffmpeg should be considered as standalone alternatives especially on the server-side. The target here isn't vendors who need to deliver cross-platform client-side products. In that case, I'd recommend JAI over bundling something like ImageMagick - that's a no brainer.
I recently started a project called CastCluster (http://castcluster.blogspot.com/) it does some video conversion... I spent a day or so looking at the sad, sad state of media manipulation in Java... then ended up with ffmpeg, Runtime.exec() and regex for figuring out what kind of media I was looking at and what bit rate it was encoded at.... it works pretty well and I'm happy with the decision.
A couple of JMF code snipets to do some basic things
http://www.coderslog.com/JMF
A neat way to get JMF to act like a VoIP server/client
http://www.coderslog.com/RTP_Server_Using_JMF
Hey Tim.
I don't think Fobs4JMF can do encoding to different formats ... it just supports playback of various media formats at its current state. I think, even if there is a performance problem as you pointed out, using the Runtime.exec() functionality is the way forward at least for the time being.