Vladimir Vukićević: Android Hacking (Part 1 of probably many)

I've been looking to understand the Android OS better, so that I can answer some questions and create plans for getting Gecko/Firefox running on Android-based devices.  One of the first questions I asked was, "How do Android apps start?"  They're clearly separate processes while they're running, but it wasn't clear how they were launched.  It turns out, there are a couple of pieces here.  I'm going to describe what I've discovered here, in case it's useful for someone else; I haven't been able to find much of this information, largely because I don't think many people need to know any of this.

At boot time, a special instance of the Java VM is launched, called the Zygote.  This process loads a bunch of the core Java classes and performs initial processing of them, making it possible to avoid this step for each app launch.  Once the initial work is done, the process listens to a socket and waits for requests.

To launch an app using the Zygote process, a command-line tool called "dvz" can be used.  It sends its arguments to the Zygote, which will fork and then start executing the main method in a given class.

So, we have these steps, dealing with the Zygote process:

• the zygote process is executed at system boot; it does initialization, and then runs a select() loop listening to requests
• the zygote process is sent a message, which includes the start args, which will include the class name for main launch
• the zygote process reads connection args in ZygoteConnection.runOnce
• the zygote process forks in ZygoteConnection.runOnce (Zygote.forkAndSpecialize, native)
• ZygoteConnection.handleChildProc is called, which does some cleanup and eventually throws a MethodAndArgsCaller (via ZygoteInit.invokeStaticMain)
• ... which takes us all the way back to ZygoteInit.main, which catches that trampoline exception, and calls run()

But, that's not fully how a new Android Activity is started.  It's a bit of a roundabout process.  To launch a new Activity, the ActivityManagerService is notified with an activity start request, including things like the name/class/etc. of the activity.  It puts that information in a list of activities to run.  Then, a new process is started with the main from ActivityThread.  This new process then contacts the ActivityManagerService and asks, essentially, "what app am I?".  The service then gives it the name of its activity class and other info, which is then loaded, and a message is enqeued on the main thread to instantiate the new activity and send it an onCreate() message.

This is interesting because it means that apps are not launched directly, but instead somewhat indirectly through specializing a generic "Activity" process for a specific activity.  A side effect of this is that I couldn't find a way to actually register an app with the ActivityManagerService if it wasn't launched by it.  So, to be a full Android app, you have to go through this normal startup process.

JNI Bridging

One of the difficulties in porting Gecko to Android is that the Android platform is built around Java, whereas Gecko is very much all native C/C++.  However, there is a fairly good native bridge layer, JNI, which is fairly heavily optimized by Dalvik.  So, the simplest way to connect these two is to write a shell app in Java, which bridges events, messages, paint requests, etc. to the native code for handling.

As a proof of concept of this, I wrote a simple test app.  It's fairly straightforward, with a few wrinkles.  Most of this stuff can be done with the stock Android SDK and NDK -- except painting.  The only API that the NDK exposes for graphics is OpenGL ES.  This is fine, but in some cases you may want to access Skia directly from native code.  This is possible, but requires version-specific code to accomplish.  You can ship multiple versions of your JNI glue layer, optimized for each Android version (or even platform), and load the right one during your app startup on the Java side.

This is, of course, not very portable, robust, or guaranteed to continue to work by Google, but it's possible.  There are some very rough hacks in the test app, but for the most part it demonstrates that this approach can work fine.

Next up, I'll probably blog about porting issues for large native apps, including library compatibility, Bionic, and integrating into a non-ant-based build system.

Burning Edge - Firefox: 2009-11-21 Trunk builds

Vladimir Vukićević: Droid Almost Does

I purchased a Droid when they came out.  It's my first Android device, and it's been an interesting experience.  I am not a fan of the iPhone, and I've been using a Blackberry for the past few years (an 8700 first, then the original Curve, then the updated 8900).   The Droid is a great looking device; I like the industrial look, with my only complaint being that the big gold-coloured area on the D-pad is way too garish; it would also have been nice had that area been a trackpad-like virtual trackball.  The keyboard leaves a lot to be desired, though.  It's a physical keyboard, which is nice, but it's no match for a Blackberry keyboard.  Typing on it is slow and cumbersome, given the very wide layout, and some keys are very oddly placed.  (I found it amusing that while the Blackberry has a dedicated unshifted key for "$", the Droid has a dedicated key for "?"...)

The feel of the OS is pretty nice, although some things are more sluggish than they really should be on an OMAP3 device.  Stuart keeps telling me that Fennec has smoother panning in the browser, and I think he's right.  It's not a deal breaker though; I find myself using the browser a lot to do all sorts of things that I never would have considered on my Blackberry (because, wow, the web browser situation there is awful), but that was a frustrating experience on my iPod Touch as well.  I've spent a while "browsing the web" on my phone, which I've never been able to say I've done before.

But, it's still a phone, and while the voice portion isn't all that important to me, the overall communication package is.  Coming from a Blackberry, the overall messaging situation on the Droid is simply horrible.  Email, whether Exchange or IMAP, is a disaster.  The email client seems designed for simple "lol r u there" type of messages, and even the message lists don't seem intended for people who get more than 5 messages a day -- turning a message list into landscape mode is worthless as you only get to see about 3-4 messages in the list (same view as in portrait mode, just along the much smaller axis of the display), no IMAP IDLE support etc. are all very strange on a top-end phone.  Exchange support works ok for Calendar sync, but for email sync it would only download the first 1000 bytes or so of a message, including headers; this meant that I often only got to see the first sentence or two of an email.  I don't know whether this is a problem with the Droid or our Zimbra Exchange connector, but switching to IMAP for work mail fixed that problem.

An recently-released version of the open-source K9 Email Client that works on the Droid resolves many of these issues, though it needs some polish.  I might write some code there, since it's close to becoming a pretty good email solution.

The Gtalk client is probably in worse shape than email.  It's almost as if Google entirely ignored Gtalk on this device (and I can't believe that would be Verizon's fault, since things like Google Voice work just fine).  First, it's in general buggy -- it's crashed on me multiple times, often freezes when returning to it from another app (after clicking a link to the browser, for example), and often shows contacts as offline with a big red message despite the contact clearly having a green dot next to their name and responding to my messages.

In the browser and in other apps, you can share a web page with someone using a "Share with" button.  The list you're presented is conspicuously missing Gtalk, despite having Facebook, Email, Messaging (SMS) and a random Twitter client I installed on there.  What gives?  All of these features are available on the Blackberry; I'm not sure if it was RIM that did the Gtalk app there, but can we get whoever it was to rewrite the Android one?

One of the best things about the Blackberry is the unified messaging; there's a single view where I can go to see all my emails, my gtalk conversations, my SMS messages, app updates, and whatever else.  No such thing exists on Android.  The closest thing is the notification bar, which requires a swipe down to use, and then only shows things that have come in since the last time you looked.  I'd prefer a more time-based list that contains both old and unread items.  Sounds like the Sony-Ericsson X10 might be doing some interesting things there, and I hope that someone figures out how to create an app like this.  What it comes down to is that anything to do with communication is faster and simpler on my Blackberry, which is really strange; you'd think Google would have spent some time working this out, as everything else about the device is far superior to my 8900.  I understand that more "enterprise oriented" customers (which apparently means those that like to use email a lot?) aren't necessarily the target market here, but they could've really attacked that market with some simple work that wouldn't have affected anything else.

The good news is that all of these are fairly straightforward software issues.  The hardware is solid, and Google has shown that they'll do frequent upgrades of the OS.  Given that the Droid is a "Google Experience" device, those updates should find their way to the device quickly.  Some fixes, combined with getting Firefox Mobile on the Droid and other Android devices, will make this a great phone.

Chris Hofmann: Open Source Education in Brasil Last Summer I got the chance to visit several Universities while traveling around Brasil. One of the stops was to meet up with Prof. Fabio Kon and students at the University of Sao Paulo and the FLOSS Competence Center. For... [Source: Planet Mozilla]

Mozilla Web Development: Mozilla Launches Facebook Security Quiz

Are you up for the challenge?

This week Mozilla launched the security quiz on Facebook. We encourage you to take the quiz and see how much you know about web security!

Similar to our plugin checker, the security quiz is a part of our larger effort to raise awareness about web security.

Help us spread the word and make the web safer for everyone. And don’t forget to check your plugins!

Blair McBride: Status update

Was stubbornly fighting the flu for part of the week, so I didn’t get as much done this week as I had hoped.

Tab matches in Awesomebar

Status

• Finished nsPlacesAutocomplete integration – works wonderfully well
• Filed bug 530209 to change the preferences UI to allow adding tab matches
• Tryserver builds at http://people.mozilla.org/~bmcbride/tabmatches/latest/

Loose ends

• Waiting on feedback

Next steps

• Unit tests
• Respond to feedback

Target for next week

• Unit tests

Binding for untrusted text in security dialogs

No change.

Miscellaneous

• Helped with some lightweight theme bugs for 3.6

Reflections

• Sometimes, there is no good solution. But there is a best solution.

Related posts:

1. Status update
2. Status update
3. Status update

Bryan Clark: Raindrop & Jetpack

The other day I did a quick hack using Raindrop & Jetpack to get new mail notifications from Raindrop.  In total it took me less than an hour.  It’s no Joe Shaw hack, so I don’t expect to get in the paper for this but I figured I’d share anyway.

This Jetpack checks Raindrop to see if there are new messages and bubbles them up as notifications if there are.  Here’s the source code:

var messages = ; 

function checkMail() {
 var api="http://localhost:5984/raindrop/_api/inflow/conversations/home?limit=10";
 jQuery.getJSON(api,
               function(data, textStatus){
                 jQuery.each(data, function(i,item){
                   if (item.unread) {
                     if (!messages[item.id] || messages[item.id] != item.messages.length) {
                       var n={title: item.subject,
                              body : item.messages[0].schemas["rd.msg.body"]["body_preview"],
                              icon : 'http://localhost:5984/raindrop/inflow/i/logo.png'};
                       jetpack.notifications.show(n);
                     }
                     messages[item.id] = item.messages.length;
                   }
               });
 });
}
setInterval(checkMail, 10000);

To try this out you’ll need Raindrop installed and running and Jetpack installed in Firefox.

Go to about:jetpack and copy the above code into the Develop tab, then click the try out this code link just below the Bespin editor.

If you don’t want to do all that you can just watch the video below (no sound, so you might want to play some music)

Mary Colvig: Ready for your close up?

Are you a Firefox fan?  Or even better, a Personas fan?  If so, we’d like you to star in a video we’re creating to showcase Personas.  Here are the details:

• Date:  Monday, November 23, 2009
• Time:  12:00 p.m
• Location:  Mozilla HQ, 650 Castro Street, Suite 300, Mountain View, CA 94041
• Sign up sheet (create an account to add your name to the wiki or comment below to sign up)

Come as yourself – no fancy costumes needed – and meet other Firefox fans.  We’ll treat you to lunch and make it worth your while!  And, we promise none of these antics…

Jump up and Dance by Gary Pauck (Firefox Flicks)

Taras Glek: Dehydra Testsuite Passes on GCC 4.5

I spent couple of days fixing the remaining test-suite failures on GCC 4.5 trunk for Dehydra. Since the last time I looked into this, GCC went from crashing all over the place to only crashing if I did something bad. It was nice to discover that as a result of switching to 4.5 Dehydra users will get saner .isExplicit behavior and more precise location info.

Treehydra will take more work due to me misunderstanding GTY annotations.

By the way, I am really grateful for all of the people who contributed GCC 4.5 fixes so far. You guys have been a big help in getting Dehydra testsuite to 100% on 4.5. Looks like I will meet my goals to finish De+Treehydra by the end of the year in time for GCC 4.5 release and my “Introducing Dehydra to the Developer World”-type talk at LinuxConf.au.nz 2010.

Startup
I reduced my focus on startup speed at the moment to catch up on Dehydra. I plan to work on reducing xpconnect overhead during startup next, ie more of this bug.

Dietrich Ayala: Firefox Startup Performance Weekly Summary

Current numbers are available on the Performance Snapshot page.

Summary, relative to Firefox 3.5:

• Warm startup: For Mac, 36% better on 3.6 and 35% better on 3.7. For Windows, 5% and 5%. Flat on Linux. Also, Warm startup for Mac on 3.6 is a whopping 13% better than last week, due to the landing of bug 517804.
• Cold startup:  For Mac, 20% better on both 3.6 and 3.7. For Windows, not measuring yet. For Linux, we’re seeing a regression of ~9% across branch and trunk in the snapshot but not on the graphs, so I need to figure out where the discrepancy is.

This week’s activity:

• Dirty-cold-Ts went live this week, thanks to Alice and Lukas. Example: cold startup with a large places.sqlite on Mac.
• Joel is making progress on making a super-static Firefox in bug 525013.
• Ben is making progress on the fastload replacement in bug 520309.
• No updates on Windows cold-startup testing for Talos on bug 522807. I need to test on Vista, and turn off Pre/Superfetch.
• Taras has patches up for service caching (bug 516085) and super-fast-path-ing of Components.* (bug 512584), however the latter he’s hit a wall, passing on to Blake or someone else who knows that code.
• Ted landed rebasing on Windows in bug 484799.
• Jonathan Kew has a new patch in bug 519445 for further reductions in Mac startup time spent in font system initialization, just about there…

Projects in a holding pattern:

• JARification: David abandoned moving JS modules into a JAR file, since those files are fastloaded. However, since we want things like post-extension-install restarts to be fast, and those cause fastload cache invalidation, we might want to do things like this anyways. I filed a bug for the same treatment for components. These are lower priority, since they’re not the normal startup case. Follow along with all JAR-ification via the tracker bug.
• Startup Timeline: No updates, still not landed. Add [ft] in the whiteboard of your bug w/ the function names you want timed and David will generate it and update the bug.
• Static Analysis: No progress on bug 506128. David needs to file a bug with the final log of named-yet-uncalled functions.
• Dirty Profile Testing: No progress. Need to list scenarios, file bugs for each, generate Talos config patches and profile data, and then move it into Rel-Eng territory. Also, need to get a separate Tinderbox tree, since it’s going to cause a bazillion new columns.
• Joel Reymont noted in bug 513076 that there are serious drawbacks to getting our libraries in the dyld shared cache on Mac, so has deprioritized that work.
• No updates on Zack’s CSS parser changes in bug 513149.

As usual, more details and links are available on the project wiki, and we’re available to answer questions in #startup on irc.mozilla.org.

Caitlin Looney: Product Shots of Firefox on N900

CNET UK recently reviewed Firefox running on the Nokia N900 (see Firefox Mobile on Nokia N900 hands-on photos: Fire in your trousers).  I liked their  product shots of Firefox in action (who wouldn’t?) and wanted to share some of those shots with you all today.Nice pic of the Firefox start page.  We’ve incorporated a mini animation to help guide first-time users to navigate around the UI.

…A quick slide to the right reveals open tabs in thumbnail view so you can easily see what website you want to select. Tap on the corner of the thumbnail to delete the open tab, or tap on the button below to open a new one.

…A quick slide to the left shows the stowed away controls: bookmarking, back and forward, as well as preferences. Bookmark a page you like with one touch and edit the tag if you’d like.

Next to the new tab button is the WeaveSync button. Tap on that button and WeaveSync synchronizes and delivers your open tabs from your PC.  This is a great example of how you can work away at your desktop, get up and go, pull out your mobile, and have everything waiting for you (browsing history, saved passwords, bookmarks, as well as open tabs) just as you had left it.

By tapping on the Tools button and going to your preferences in Firefox, you can select the add-on button to search and install your favorite add-ons from your mobile device.  You can also manage your search engines here that appear at the bottom of the screen when you’re conducting a search with the Awesome Bar.

Ah yes, the glorious Awesome Bar in action. We know typing is hard so the Awesome Bar helps you get where you’re going in only a few keystrokes. With WeaveSync, the Awesome Bar gets that much more powerful as it recalls your browsing history from both your PC and mobile. See the search engines below so you can narrow your search further. Quick access to Wikipedia gets me one step closer to winning Bar Trivia Night.

Hope you enjoyed the photos…I know I did. I’ll continue to post the latest and greatest screenshots on my Flickr stream: http://www.flickr.com/photos/missylooney/

 

Sid Stamm: update on HTTPS security Version 2.0 of my Force-TLS add-on for Firefox was released by the AMO editors on Tuesday, and in incorporates a few important changes: It supports the Strict-Transport-Security header introduced by PayPal, and also has an improved UI that lets you add/remove sites from the forced list. For more information see my Force-TLS web site.

On a similar topic, I've been working to actually implement Strict-Transport-Security in Firefox. The core functionality is in there, and if you want to play with some demo builds, grab a custom built Firefox and play. These builds don't yet enforce certificate integrity as the spec requires, but aside from that, they implement STS properly.

Unlike any add-ons that implement this, the built-in version performs an internal redirect to upgrade channels -- before any request hits the wire. This is an improvement over the way the HTTP protocol handler was hacked up by version 1 of Force-TLS, and doesn't suffer from any subtle bugs that may pop up due to mutating a channel's URI through an nsIContentPolicy. It's not that add-ons that do STS are poorly written, but rather there is no way to trigger the proper internal redirect from an add-on, so the only way to 100% correctly implement STS in Firefox without obscure side-effects is through a patch.

Giorgio Maone: IE’s XSS Filter Creates XSS Vulnerabilities

Internet Explorer 8’s famous XSS filter can be exploited to perform successful XSS attacks against web sites which would be otherwise safe. In other words, XSS “protection” is helping XSS attackers, oh the irony.

Well, this is not exactly news among security researchers, but those aware of the details (including Microsoft of course, Eduardo “Sirdarckcat” Vela and myself) have kept a low profile so far. Check, for instance, slide #17 in my OWASP presentation, given two weeks ago.

However, after Microsoft left it unfixed for many months, someone apparently decided to whisper this dirty little secret in Dan Goodin (The Register)’s ear.

To Microsoft’s credit, this problem has no quick fix: in fact, it’s way worse than a simple implementation bug. Its root is a flawed design choice: when a potential XSS attack is detected, IE 8 modifies the response (the content of the target page) in order to neuter the malicious code. This is, incidentally, the only significant departure from NoScript’s approach, which modifies the request (the data sent by the client) instead, and is therefore immune.

Anyway, here’s the juice: IE 8’s response-changing mechanism can be easily exploited to turn a normally innocuous fragment of the victim page into a XSS injection. The attacker just needs a certain degree of control on the content of the web site to be injected: social networks, forums, wikis and even Google Apps are good prey. To be fair, Google Apps are not vulnerable anymore, since Google’s properties wisely choose to deploy the X-XSS-Protection: 0 header, which is the “safety switch” disabling IE 8’s XSS protection.

So, web site owners’ dilemma is, opt out or not opt out?
For browser users, there should be no dilemma at all ;-)

Robert Strong: App update status – week of 11/20

It has been a good couple of weeks. There are several bugs I am relieved that are now fixed for Firefox 3.6… especially that we now check if Firefox is in use prior to updating and prevent launching Firefox during an update. Also, checking for updates for users that aren’t able to apply updates. Beltzner did his usual beltzner thing by catching what I see as a major usability flaw in that the original patch notified users repeatedly for the same release until Firefox was upgraded which I was able to fix. I’m still kicking myself for not catching that myself.

Progress:

• WOOT! Landed on trunk and 1.9.2 branch – Bug 407875 [Toolkit] – “Unprivileged users are not notified of security updates [All]“. The next bugs to fix that are similar are the dependent bugs of Bug 318855 [Toolkit] – “App update should provide method to update when the user doesn’t have privileges [All]“.
• Landed on trunk and 1.9.2 branch – Bug 510501 [Toolkit] – “not granting UAC permission to updater.exe causes full update to be downloaded [Windows]“. The next bug to fix that is similar is Bug 336267 [Toolkit] – “If software update is disabled or “ask” after an update has been downloaded, the update should be disabled or asked [All]“.
• Created a wiki page for the work on Bug 410639 [Toolkit] – “Provide ability to change update channel within the application [All]” and emailed dev-apps-firefox / dev-platform (followups to dev-apps-firefox) for this proposal.

Future targets (short work week so no way this will all get done):

• Bug 336267 [Toolkit] – “If software update is disabled or “ask” after an update has been downloaded, the update should be disabled or asked [All]“
• Investigate Bug 526441 [Toolkit] – “Unable to use FileUtils.jsm in nsExtensionManager.js.in on 1.9.2 due to reftest failures”.
• Yes, I still need to blog about the lessons I’ve learned while trying to improve startup time for app update but the Firefox 3.6 took precedence.
• Investigate Bug 529948 [Toolkit] – “Cannot check for updates on trunk when the download server is down” along with its friends

I’m taking Wednesday off so next week is a two day work week for me since Thursday and Friday are holidays.

Armen Zambrano Gasparnian: hy-AM (Armenian) moving forward Robert Sargsyan has been localizing Firefox into Armenian for a really long time through Narro.
He recently has contacted me to get things rolling since he has translated 98-99% (94% according to compare-locales) of the strings.

It is now my turn to get into the technical details and move it to mercurial. These are the steps that we have taken:

• Robert ported the strings from 3.5 to 3.6 (Narro allows you to do this)
• Through Narro's interface I exported the project and downloaded the zip file that it generates
• I checked out my clone of the Armenian 1.9.2 tree
  
• I overwrote my tree with the contents of the zip file
• I run compare-locales like this:
  compare-locales /Users/armenzg/moz/repos/mozilla-1.9.2/browser/locales/l10n.ini .. hy-AM-1.9.2
• I removed the files that were indicated to be removed
• I pushed my changes to my repository

• generate a langpack
• submit it to AMO (submit page)
  
• promote the add-on
• get people's review
• convince drivers to give us commit access
• push the changes to the official Mozilla hy-AM repositories
  

More News