O'Reilly Network: Mozilla DevCenter

Here is what the internals of the GeeksPhone Keon powering Firefox OS looks like:

Here are some photos of my unboxing of the GeeksPhone Keon running Firefox OS that I got today:

One thing I immediately liked was the packaging for the Keon is pretty darn unique.

�� I like the images on the side of the box which represent some of the things it can do.

The inside of the box is neat and lets you use it to keep your phone organized and also holds a pair of earbuds, european USB charger and a regular USB cable.

The vinyl screen protector it shipped with was a bit messy but it didn’t matter much to me.

For those who have not been using Firefox OS for while that sums up much of the platform. There really is so much more including the Marketplace and speed that make this open mobile platform the wave of the future.

I think were moving towards a time where mobile platforms like Firefox OS, Ubuntu Touch and maybe even Tizen can compete with platforms like Android and iOS. I think Mozilla is doing a great job and has a super strategy for evangelizing the platform and getting partners and developers on board.

Our monthly newsletter, Firefox & You, has seen quite a few changes since its inception in October of 2010.

October 2010

April 2011

April 2013

May 2013

And this month, it got another face-lift and its code became mobile friendly (click on the image to see the full email):

Desktop

Mobile

Thank you Team!

After countless hours of design, coding, testing across 30 different email clients, browsers and mobile devices, localizing, re-testing, and troubleshooting a persnickety Android Tablet rendering issue (that made me and nigelb feel a little like this… who knew Android tablets would be so picky about any extra space in a style attribute?) — the May newsletter has finally launched to all 7 languages. (And Polish is on deck to launch next month!)

I think I can finally go into this 3-day holiday weekend knowing a great feat has been accomplished… just in time to start the June newsletter next week :)

PS – Want to know what a checkbox list looks like for putting together a new newsletter in 7 different languages? Take a look…

Second, there are plenty of extensions available for reading web feeds. I used to use Sage. Other popular ones:

Joey Armstrong: Transitioning to moz.build If you have been wondering about status for the transition to mozbuild as a replacement for makefile infrastructure, a source is becoming available. People involved with the conversion have been meeting bi-weekly on the topic – details and content can … Continue reading → [Source: Planet Mozilla]

This post was originally posted on Medium, a new blogging platform made up mostly of people who aren’t necessarily subscribed to Planet. So, please forgive the obvious statements, as the target audience are people who don’t know very much about Postgres. Tom Lane, taken by Oleg Bartunov

For 15 years, Tom has contributed code to Postgres, an advanced open source relational database that started development around the same time as MySQL but has lagged behind it in adoption amongst web developers. Tom’s move is part of a significant pattern of investment by large corporations in the future of Postgres.

For the past few years, Postgres development has accelerated. Built with developer addons in mind, things like PLV8 and an extensible replication system have held the interest of companies like NTT and captured the imagination of Heroku.

Tom has acted as a tireless sentry for this community. His role for many years, in addition to hacking on the most important core bits, was to defend quality and a “policy of least surprise” when implementing new features.

Development for this community is done primarily on a mailing list. Tom responds to so many contributor discussions that he’s been the top overall poster on those mailing lists since 2000, with over 85k messages.

Really, he’s a cultural touchstone for a community of developers that loves beautiful, correct code.

When I read that last October, I was filled with glee, quickly followed by terror. I love my small database community, my friends and my job. What if Oracle shifted it’s attention to our community and attacked it, directly? So far, that hasn’t happened.

Tom’s move probably won’t change much for the day-to-day operation of Postgres itself. Hopefully, things are about to get real at Salesforce.

I’m a major contributor to Postgres. I started in 2006, learning about relational databases through work at a small bike parts manufacturer and ERP. My contributions include code, starting conferences, encouraging user group leaders and introducing Postgres to communities that otherwise would never hear from us. I’m a data architect at Mozilla.

Our heroes continued on their quest to move add-on- and front-end-accessible APIs off the main thread. Arrayed against them, work weeks, conferences, holidays and terrible weather. Still, they pressed on.

Tooling and new APIs

Refactorings

Testing

Yes, even I’ve got my Geeksphone Peak, and yes I like it. I am sure, that I have collectible item in my hands … it will be either that historical model which changed the world to the open standards on mobile, or maybe it will be in one of those “10 biggest flops” articles (yes, I would love to have Lisa!). I am not sure about this mass success, but I feel like I may eventually like it for myself (which is actually all I care for; after all, I am a happy user of Linux). The first news is very good news: it works perfectly as a phone, which means I use it as my day to day phone (well, music player doesn't switch off on picking up the phone). Also, camera Just Works™, it is possible even lightly edit the image on the phone (a feat which the standard Android camera was not able to accomplish, BTW). Now the problematic parts.

Calendar just doesn't work for me. It works with CalDAV on Firefox OS Simulator 3.0, but not with my phone. Sad but expected (I don’t know why Calendars get so bad treatment by most software providers, not mentioning missing Tasks). Also, originally, Mail didn’t work for me, but I was able to get it working more or less with some later images from Geeksphone (still from time to time the app doesn't load the body of the mail ever). Which leads to another problem: there are no OTA updates yet. I am willing to use Debian/unstable or Fedora/Rawhide, but I really need to be sure, that all those bugs which I hit will be soon fixed and delivered to me. Getting developing-quality with no updates, is really a bad experience for the user, I would say. In the end I at least found the latest images from Geeksphone, but that still has only 1.0.1 version, although there are rumours of 1.1 already existing on the Mozilla servers. Other bad experience is that fonts are tiny, almost unreadable, and I haven’t a way (even a one which would include adb or something similarly drastic) to work around it.

The worst and the best part of the phone is the situation with Contacts. Out of the box, FF OS imports just from SIM card (who still uses contacts there?) and from Facebook (I don’t have an account, and even if I had I would never ever store my main address book there). That was pretty bad, as I really didn't look forward typing of couple hundred contacts I had on pretty crappy keyboard (with unreadably small characters). So, my programmer’s pride raised its head, and I have wrote Javascript LDIF parser and with that I was able to write a simple (and ugly) single-webapp for importing a LDIF file to the system address book. While doing that I have found that there is already Importer from GMail Contacts. I don’t use GMail, so it wasn’t useful for me as such, but eventually I hope to merge my code into this app, because it has much better user interface.

Generally I am enjoying the ride so far, and I am looking to the time where I could get my fix of nightlies everyday.

Warning: This post is rather heavy with the use of Flash to show videos. Sorry!

I've seen a bit of weirdness involving records, of all things, lately. Records, specifically Vinyl Records, are a pretty obsolete technology at this point. Yes, I'm well aware of the undying love for the "superior" sound quality of vinyl from a certain subset of bibliophiles. These people adore their record collections until the end of time. The rest of us, like my wife, have long given up on vinyl and moved, first, to compact discts (if not tapes) and then to mp3 files and our iPod-like devices. This moves record technology for most people, especially the young, into the same realm as wind up gramaphones or rotary telephones and HAM radio: old technology loved by a view but looked upon oddly by the rest.

Now, in spite of this (or maybe because of this), people are doing some weird things with record technology right now as artists. I wanted to share a few instances that I have come across, two from the same person. I came upon all of these within the last two weeks so they kind of stuck in my mind.

A band, the Shout Out Louds, sent out ten kits to people that have a silicone negative image of a single from their recent (unreleased at the time, I believe) album. This came with instructions to pour enclosed distilled water on it, freeze it for six hours, and pop and ice record from the mold to then play. You can view the results above.

Next is Amanda Ghassaei's attempts to play with record technology as an artist. First, she used some high end printers to 3D print a record. She details this (at great length) on a post on instructables on this. She figured out a method to convert an mp3 music file into a grooved shape thaat would play on a traditional record player. This was then printed on a high end printer, though I suspect that a well tuned hobbyist printer might work well enough. She mentions that it has a sampling rate of only 11kHz and 5-6bit resolution, which makes it sound a bit like an ancient Edison era recording. The video below gives an example of the result:

Amanda has more recently gone on to try the same trick using a laser cutter, resulting in a wood record etched by the laser. She wrote another instructable describing this process. You can also watch her video about it below to hear the wood one:

She's actually done this with acrylic using the laser as well and I think that it sounds a bit better than the wood one:

All of that said, the audio on all of these is horrible. I actually think the ice record sounds the best. What I've found the most interesting about this is the intersection of an obsolete and very analog technology, the record player and the records, with interesting art and even cutting edge tools like a laser cutter or a 3D printer. Technically, the ice record could have been made 50 years ago (I don't think making a silcone negative is likely a difficult technical challenge) but this marriage of art and technology is fascinating.

People often ask me, "What do you make with a 3D printer?" The joke, of course, is the answer, "Parts for more 3D printers" (thought it is often true). That said, the ability to quickly prototype, especially to prototype from an artistic vision created within a computer, is really cool for art. Amanda wrote a script in Processing, an open source language used by artists a lot and often used with arduinos, to do the conversions for her to turn the music files into a three dimensional construct that she could then print or cut. What other art do people have locked inside of them that they can use these tools to express?

Ever so often, friends send me emails with pictures they took on their phones. When looking at the photo, it frequently appears turned on its side, like this:

A little research revealed that cameras take both sideways ("landscape") and upright ("portrait") photos the same way. Then, instead of arranging the pixels differently (an expensive process), many cameras, especially phones, just add a little bit of EXIF metadata defining the way the device was oriented when the picture was taken.

It is then up to the image viewing software to read this metadata and display the image correctly.

An old bug

Some image viewers respect this and show the image right-side-up. Firefox, it turns out, does not.

The bug report, almost 8 years old, lists many good reasons why you wouldn't want to auto-rotate images inside web pages based on their metadata. For example, some image editors will turn your image, but not clear the orientation metadata, so the browser would overcompensate if it applied additional rotation. Also, silently changing a major part of the web like image rendering is likely to break many existing sites that have come to rely on the previous behavior.

On single-image pages ("view image"), however, I and many people I talked to believe the image metadata should be honored. Not doing so, for fear of the off chance the image has broken metadata, is breaking the 99% case (photos from digital cameras) in favor of the 1% false positive (broken image editing software).

Time for a fix

So I went ahead and wrote a Firefox add-on that fixes this issue. Meet imagetwist. It adds two features to your browser.

First, it fixes the problem described above: If you look at an individual image, and the metadata says it should be turned, the add-on will turn it automatically.

Second, it provides a right-click menu on any image, anywhere, allowing you to rotate it as needed (to the right, left, or 180 degrees). This is useful for the occasional case where a wrongly oriented image is visible on a website. For this feature, image metadata is not considered, you can rotate images any way you want.

How does it work?

To turn an image, I spin up a canvas element and project the image on it, rotated. Then I export the canvas data and replace the image source with it.

Granted, I would have preferred a pure CSS solution. I explored CSS transforms, but that will only rotate the image itself, not its bounding box. So, frequently, the image will rotate underneath some other content on a page and become partially obscured.

A better CSS option would be the image-orientation property, but alas, it's not implemented quite yet. When it lands, I will happily adapt imagetwist's code accordingly.

Try it out! Contribute!

Centre for Internet and Society celebrated their 5-year anniversary with an exhibition at their Bangalore and Delhi offices and a series of talks in Bangalore. I was there on Tuesday and managed to spend some time at the exhibition and attend the talks.

The exhibition showed off some of the work that CIS has been doing and the work of several independent artists. The bits that are particularly in my memory is Tara Kelton’s work as well as Sharath’s work.

Later in the day, Lawrence Liang talked about the Encyclopedia of Indian cinema. It was a very interesting talk, especially for me since it encompasses open data, open source software, and copyright issues! A convergence of a lot of my interests :) Lawrence talked about what they’ve built and the problems they’ve faced and how internet as a medium for a film encyclopedia is very powerful, but is limited by the legal issues surrounding copyright laws.

I know Disney is great, but I’m not sure I like them as much after this video.

Using hg & git for the same codebase

Speaker Note

Following are the slides I presented at the RELENG 2013 workshop on May 20th, 2013. Paragraphs like this were not part of the presented slides - rather speaker notes.

If you prefer, here is a PDF version.

Hal Wine hwine@mozilla.com
Release Engineering
Mozilla Corporation

Issues and solutions encountered in maintaining a single code base under active development in both hg & git formats.

Background

Mozilla Corporation operates an extensive build farm that is mostly used to build binary products installed by the end user. Mozilla has been using Mercurial repositories for this since converting from CVS in 2007. We currently use a 6 week “Rapid Release” cycle for most products.

Speaker Note

We currently have upwards of 4,000 hosts involved in the continuous integration and testing of Mozilla products. These hosts do approximately 140 hours of work on each commit.
Firefox Operating System is a new product that ships source to be incoporated by various partners in the mobile phone industry. These partners, experienced with the Android build process, require source be delivered via git repositories. This is close to a “Continuous Release” process.

Speaker Note

A large part of the FxOS product is code used in the browser products. That is in Mercurial and needs to be converted to git. Most new code modules for FxOS are developed on github, and need to be converted to Mercurial for use in our CI & build systems.

Summary

What we initially set out to do:
- Make it purely a developer choice which dvcs to use.
  
  Speaker Note
  
  Ideal was to allow developers to make dvcs as personal a choice as editor.
- Support multiple social coding sites.
  
  Speaker Note
  
  These social coding sites, such as github and bitbucket, make it much easier for new community members to contribute.
That was much tougher than anticipated.

In theory, git & hg are very close...

... In practice, “the devil is in the details”.
Where we are:
- Changed direction to support FFOS release to partners.
- Quickly mirror Repository of Record (RoR) between git & hg.
- CI/build system remains Mercurial centric.

Challenge Areas

Changesets have different hashes in Mercurial and git.
- We added tooling to support both in static documents such as manifest files.
- All tools continue to use hg hash as primary value for indexing and linking.
Propagation delays of changesets to the “other” system.
Speaker Note

For most use cases, the approximately 20 minute average we’re achieving is acceptable.
- Compounded by hash differences between two systems.
  
  Speaker Note
  
  A common use case here is a developer wanting to start a self serve build. If the commit was to git, the self serve build won’t be successful until that commit is converted to hg.
  
  We are continuing work on this. It is closely tied to determining which commit broke the build, when multiple repositories are involved.
Build details
- Movable tags are not popular in git based workflows, but have been a common technique at Mozilla to mark “latest”.

Challenge Areas (Con’t)

Mixed philosophies are often linked with mixed repositories.
- Android never wants history to appear to change. Downstream servers allow only fast forward changesets and deny deletions.
- Mozilla uses “RoR is authoritative”.
  
  Speaker Note
  
  Either approach is self consistent. It is when the two need to interact that challenges arrise.

Conversion failures
- Occasional hg-git conversion failures, due to implementation details of hg & git.
  
  Speaker Note
  
  Dates in export patches (e.g. hg uses seconds, git uses minutes, in time)
  
  Email validation (git stricter than hg)
- Since commit already accepted by hg, hg-git must be modified
  
  Speaker Note
  
  This requires inhouse resources to respond urgently to patch the conversion machinery. Without conversion, there are no builds.

Alternate Approaches

To support your own “use the DVCS you want” infrastructure requires:
- production quality hg server
- production quality git server
- in house ability to address conversion issues (as already mentioned)
I’m aware of two commercial alternatives. Both of these use a centralized RoR which supports git and/or hg interfaces for developer interaction.

Speaker Note

And at least one explicitly does not have a git back end.
You can leave it to developers to scratch their own itch independently. Given diversity of workflows, this may be more cost effective than obtaining consensus.

Future Research

Areas of particular interest for further study include:

What is the set of enforceable assertions which would ensure the tooling can maintain lossless conversion between DVCS?
What minimum conditions must be maintained in conversions to preclude downstream conflicts?
What workflows can be supported to minimize issues?
Are there best practice incident management protocols for addressing problem commits.

Speaker Note

The common example is a commit contains sensitive material it should not. There are cases were limiting the scope of distribution can have significant business value.

Thank You

References:

Blog:

http://dtor.com/halfire/

One of the best ways to help people understand something they’ve not come across before is through metaphor and analogy.

A year or so ago, for example, my son had a cold and said “my nose is deaf” – and I knew exactly what he meant. It contained just the right balance of ambiguity.* When explaining Open Badges to people I’ve found “X is kind of like Y because of Z” helpful in getting them to grasp what I mean. The more useful metaphors, similes and analogies I can find, therefore, the better.

Below are some I’ve used recently to explain Open Badges. They may or may not help you or the people you’re talking to about badges. But give them a read and tell me what you think. Oh, and the animated GIFs are just for fun!

The difference between ‘a badge’ and ‘a badge system’

Mozilla is developing the OBI – the Open Badges Infrastructure. People are free to use it to create their own badges for whatever purpose they like.

It’s a bit like a water company providing the infrastructure so that instead of having to go to a well, you can get water coming out of a tap. What you use that water for, what you mix it with, and how you share it is entirely up to you.

A different analogy might be that a badge is akin to an ‘app’ in an app store. Mozilla may produce some badges of its own, but it’s looking after the entire ‘app store’ in terms of the OBI. This metaphor breaks down for two reasons, however: there’s no one place to see all of the badges (at present) and it’s not a walled garden as many app stores are. Anyone can use badges for any purpose without reference to Mozilla. It’s an open, decentralised system and standard.

Metadata in badges

Metadata is data about data. It’s like when you tag someone in a photo on Facebook – you’re adding data about the data already in the system. In this case the data is the photograph and the metadata you’re adding is the name of an individual in the photo. The index at the back of a book is metadata as well – data about the data in the book.

One way to think about Open Badges is that they’re a bit like barcodes that can be understood by humans. Just as when you scan a barcode you get extra data such as the price of a product, so when clicking a badge you get details of what the earner had to do to get the badge, the evidence for it, etc.

The metadata is hard-coded into the Open Badge. So, just like when you make a cake, it’s made up of lots of different ingredients (the name of the badge, the identity of the badge earner, the Criteria URL, etc.). Once you’ve baked the cake or the badge, you can’t change those ingredients or get them out. That badge is unique to the individual. If you’ve baked a chocolate cake and now you want a Victoria sponge, then you’re going to have to bake another cake. Similarly, you can’t change a badge once it’s been issued.**

Badge pathways

In life, some pathways and routes definitely lead somewhere. That could be a route into employment, a journey to a holiday destination, or some other ‘place’ that you want to get to. There are almost-guaranteed ways to get to that destination, such as going to a travel agent and getting them to take care of your flight, transfers and accommodation. Likewise, completing a recognised project management qualification greatly increases the chances of being employed as a project manager.

There are other ways of getting to your holiday destination and becoming a project manager, however. You could book all of the different parts of the trip yourself. You could hitch-hike. You could use websites like Couchsurfing or Airbnb. Likewise, with the project manager position you could have learned how to manage projects on the job and have lots of experience of delivering successfully. Or, indeed, you may have transferable skills.

But there are some people for whom the journey is the destination. They don’t have a particular path in mind – or, perhaps, they’re blazing a new trail unsure of where it will lead. Being able to capture the knowledge, experience and skills they gain along the way would seem to be a useful thing to do. It surfaces the slightly meandering journey that I think we’ve all experienced during our careers. Badges can help validate these non-linear pathways.

Badge quality

Think of the last time you stayed in a hotel. Unless that was booked on your behalf, how did you end up staying where you did? Some of it may have been down to money, but what other factors were involved? There would have been the 5-star rating system which, until recently, would have been one of the only ways to ascertain the quality of a hotel. But is that the only thing you used? I bet, nine times out of ten, it was either TripAdvisor or some other social ratings/recommendation system.

The value of an Open Badge comes from at least from three different places. First, there’s the reputation of the individual or organisation that issues (or, in future, endorses) the badge. Second, there’s the (essential) Criteria URL in the badge that tells the consumer what the earner of the badge had to do to get it. And, finally, there’s the (optional) Evidence URL that shows just what the earner did with that criteria. It’s a triangulation very much akin to deciding which hotel to stay at: the star rating, a description of the facilities/amenities, and reviews from sites like TripAdvisor.

The point here is that top-down ‘quality’ systems can work, but they’re even more powerful (and can sometimes be replaced) by horizontal, peer-to-peer recommendation engines. It’s the difference between how a system should work and how it actually works.

Badge equivalency

Deciding that one thing is equivalent to another is not something that Mozilla is (at the moment, at least) concerned with.

I think of badge equivalency as being a bit like mobile phone tariffs. There’s many different plans and tariffs that it’s possible to use/sign up for as a mobile phone user. Most of them offer fairly similar combinations of talktime minutes, SMS messages and 3G data. Some, however, may offer 4G data. Even if there are differences between providers, it’s still possible to weigh up what’s best for you. What you decide to be ‘equivalent’ might not be the same as what someone else believes to be so. It depends upon context.

There will potentially be many different providers of similar badges. The value of the badge will be ascertained by employers and other people providing opportunities by comparing those badges against the others available. Credentials are always used for a purpose, after all. Eventually, some kind of ‘BadgeRank’ algorithm (similar to Google’s ‘PageRank’) may help both earners and employers find the most relevant badges in their industry.

Badge backpacks

Badges are hosted in a badge backpack and then displayed across the Web. It’s similar to videos being stored on YouTube or Vimeo and being embedded on many different websites. Likewise, you can make them private or public. The difference here is that, once we’ve got federated badge backpacks, you will get to choose where your ‘videos’ (badges) are hosted as well as where they’re embedded.

Some of these work better than others. I’d very much appreciate feedback as well as any analogies you’ve used successfully!

** So, technically, we are thinking very carefully about badge revocation but we don’t want people reaching into people’s badge backpacks willy-nilly and changing them. There possibly will be a ‘nuclear’ option for revocation, however – such as when you’ve accidentally awarded a PhD-level badge to a six year-old…