Jonathan Wilde View RSS

Want more context? See the introduction to Gossamer and previous update.

Another Demo

Let’s say you make a code change to your browser and you want it today. After making your change, you need to restart the app, or in the case of browser.html clear caches and refresh the page.

With our experimental fork of browser.html, we can now apply a lot of different types of changes without a refresh.

Let’s say we want to change the experiments icon in the upper right of our browser and make it red and larger. You just make the change and hit save. The changes appear in your running browser, without any loss of state.

We’re doing this with Webpack Hot Module Replacement and React Hot Loader.

In the demo, I’m running browser.html from Webpack’s development server. It watches and serves the browser.html files from my working copy, performs incremental module builds, and has an open socket.io connection to the browser notifying it of build status.

When the working copy changes, it performs an incremental build and notifies the browser of new code. The browser can apply the changes without a restart.

What I Did on Saturday

• Restructured browser.html so that there is one component exported per file, which plays better with React Hot Loader at the moment.
• Moved browser.html to CommonJS, which plays better with the early builds of Webpack 2 at the moment.
• Shifted the npm start script from ecstatic to Webpack’s development server.

Next Steps

• Graft webpack onto the Gossamer build server.

I’m currently working with Lyre Calliope on a project to improve tooling for developing and sharing web browser features.

I’ll be documenting my progress on this Mediapublic-style.

First, a Little Demo

In order to tinker with your web browser’s source today, you need to download a working copy of the source, set up a build environment, and have your text editor selected and configured. It can take hours, even for people who’ve done it before.

Why can’t we just edit and share web browser UI changes from a web application, like we can with documents and other things?

In our experimental fork of browser.html, we can up the GitHub web interface (even from the browser you’re trying to edit), make edit the color, and when the update popup appears in the web browser, click “Apply”.

We don’t have to configure Gossamer to continuously build and ship our branches, and other people testing the same Gossamer branch receive that update, too.

In case you’re curious, here’s the commit I made in the demo.

What I Did Thursday and Friday

• Removed analytics and news feed for now. Remove as much UI as possible.
• Restructured the build process. Our one cheap build transform is performed as late as possible, and not muddled in with our cache of repo data.
• Remove the need to explicitly register branches as experiments.
• Add “base” branch that is used when you don’t have a branch explicitly selected. Drop requirement to be logged in to access builds.
• Add Webhook support for receiving push notifications.

Next

• Standing up Webpack and react-hot-loader locally with minimal changes to browser.html.

Stay tuned for more!

A few summers back, I worked on the Firefox Metro project. The big challenge I ran into the first summer–back when the project was at a very early stage–was figuring out how to distribute early builds.

I wanted to quickly test work-in-progress builds across different devices on my desk without having to maintain a working copy and rebuild on every device. Later on, I also wanted to quickly distribute builds to other folks, too.

I had a short-term hack based on Dropbox, batch scripts, and hope. It was successful at getting rapid builds out, but janky and unscalable.

The underlying problem space–how you build, distribute, and test experimental prototypes rapidly?–is one that I’ve been wanting to revisit for a while.

So, Gossamer

This summer, Lyre Calliope and I have had some spare time to tinker on this for fun.

We call this project Gossamer, in honor of the Gossamer Albatross, a success story in applying rapid prototyping methodology to building a human-powered airplane.

We’re working to enable the following development cycle:

1. Build a prototype in a few hours or maybe a couple days, and at the maximum fidelity possible–featuring real user data, instead of placeholders.
2. Share the prototype with testers as easily as sharing a web page.
3. Understand how the prototype is performing in user testing relative to the status quo, qualitatively and quantitatively.
4. Polish and move ideas that work into the real world in days or possibly weeks, instead of months or years.

A First Proof-of-Concept

We started by working to build a simple end-to-end demonstration of a lightweight prototyping workflow:

Starting a two-week Mozilla heartbeat-style project sprint with @hellojwilde tomorrow involving Mozilla's browser.html experiment. Excited!

— Lyre Calliope (@CaptainCalliope) May 25, 2015

(Yeah, it took longer than two weeks due to personal emergencies on my end.)

We tinkered around with a few different ways to do this.

Our proof-of-concept is a simple distribution service that wraps Mozilla’s browser.html project. It’s a little bit like TestFlight or HockeyApp, but for web browsers.

To try an experimental build, you log in via GitHub, and pick the build that you want to test…and presto!

Sequence shortened.

About the login step: When you pick an experiment, you’re picking it for all of your devices logged in via that account.

This makes cross-device feature testing a bit easier. Suppose you have a feature you want to test on different form factors because the feature is responsive to screen dimensions or input methods. Or suppose you’re building a task continuity feature that you need to test on multiple devices. Having the same experiment running on all the devices of your account makes this testing much easier.

It also enables us to have a remote one-click escape hatch in case something breaks in the experiment you’re running. (It happens to the best developers!)

To ensure that you can trust experiments on Gossamer, we integrated the login system with Mozillians. Only vouched Mozillians can ship experimental code via Gossamer.

To ship an experimental build…you click the “Ship” button. Boom. The user gets a message asking them if they want to apply the update.

And the cool thing about browser.html being a web application…is that when the user clicks the “Apply” button to accept the update…all we have to do is refresh the window.

We did some lightweight user testing by having Lyre (who hadn’t seen any of the implementation yet) step through the full install process and receive a new updated build from me remotely.

We learned a few things from this.

What We’re Working On Next

There’s three big points we want to focus on in the next milestone:

1. Streamline every step. The build service web app should fade away and just be hidden glue around a web browser- and GitHub-centric workflow.
2. Remove the refresh during updates. Tooling for preserving application state while making hot code changes to web applications based on React (such as browser.html!) is widely available.
3. Make the build pipeline as fast as possible. Let’s see how short we can make the delay from pushing new code to GitHub (or editing through GitHub’s web interface) to updates appearing on your machine.

We also want to shift our mode of demo from screencasts to working prototypes.

Get Involved

This project is still at a very early stage, but if you’d like to browse the code, it’s in three GitHub repositories:

• gossamer - Our fork of browser.html.
• gossamer-server - The build and distribution server.
• gossamer-larch-patches - Tweaks to Mozilla’s larch project branch containing the graphene runtime. We fixed a bug and made a configuration tweak.

Most importantly, we’d love your feedback:

• Tweet at us! We’re @CaptainCalliope and @hellojwilde.
• Come say hi in #gossamer on Mozilla IRC.
• File issues on GitHub. We’re centralizing issues in the gossamer repo for now.

There’s a lot of awesome in the pipeline. Stay tuned!

Originally posted on Wordpress.

“Let’s give it up for ‘worse’!”

Tom Lehman of Rap Genius emceed this year’s HackMIT–a competition where teams try to pull off the most impressive stunt with technology in 24 hours.

The day before, he discussed the glory of poorly-written software, preaching that “worse is better” for hackathon projects. Teams only have a bit of time to build something amazing, so they better focus on the amazing bit, not ensuring there are elegant or proper solutions for all of the underlying coding problems for the project that judges and users will never see.

As expected, the crowd gave it up for “worse”. But when Lehman asked the crowd to “give it up for ‘better’,” people cheered for “better” just as loud as they did for “worse”.

Lehman wondered aloud whether anybody had learned anything in the past day.

Steve Jobs continually talked at Apple about how “real artists ship”, meaning that getting something out into the real world to inspire people about a vision was more important than getting a product out onto the market that fulfilled the maximum extent of the vision.

And as Fred Vogelstein discussed recently in NYT Magazine, Jobs applied this mentality to the iPhone project.

As early as 2003, a handful of Apple engineers had figured out how to put multitouch technology in a tablet.

The fundamental capabilities of the technology were amazing, but with the technology and resources available at Apple at the time, it was going to be impossible to launch a tablet in any sort of reasonable timescale.

They simplified the product somewhat, creating phone rather than a tablet. But buying a phone is complicated. It takes time for people to work out contracts with their carrier. They have to budget money. They need to tell their friends about how excited they are about the new product.

Apple needed to run a demo far before the product was ready.

Not only was he introducing a new kind of phone — something Apple had never made before — he was doing so with a prototype that barely worked. Even though the iPhone wouldn’t go on sale for another six months, he wanted the world to want one right then.

The difficulty was figuring out how to inspire the crowd without hitting any of the kinks that would shatter impressiveness of the vision. So, they constrained the scope of the demonstration to avoid those problems.

Hours of trial and error had helped the iPhone team develop what engineers called “the golden path,” a specific set of tasks, performed in a specific way and order, that made the phone look as if it worked.

And after the successful demo was over, they put an iPhone on display, but on a stand inside a transparent cylinder. People couldn’t touch and potentially find the problems with it. And, if anything, it visually elevated the iPhone from something normal to something special you might find in an art museum.

A year ago, I balked at the concept of going to a hackathon. They seemed like the clearest expression of the “brogrammer” concept, who skip sleep and binge guzzling Red Bull and beer while coding in an attempt to demonstrate how amazing they are.

And on a deeper level, I was frustrated with how British Airways and other organizations presented their hackathons as ways to “change the world” with concepts ready to present to UN delegates and other officials in a short period of time.

Initial hackathon concepts are just a start. They have to be prototyped and those prototypes need to be tested with users. A final concept to change the world–one that’s going to be worthwhile to present to lawmakers–is going to take persistence. It’s probably going change in response to new data about the world.

That’s why massive-growth startup accelerator Y Combinator has always been about finding smart people, and not obsessing over their current concept:

I care more about the founders than the idea, because most of the startups we fund will change their idea significantly. If a group of founders seemed impressive enough, I'd fund them with no idea. But a really good idea will also get our attention—not because of the idea per se, but because it's evidence the founders are smart.

Granted, hackathons like the Ungrounded are a great way to get pre-vetted groups of likely unconnected smart people to talk, enabling them to create amazing concepts down the road that they might not otherwise, but presenting the ideas developed at such an event to dignitaries always seemed like a bit of a stretch.

Over the summer, I decided to sign up for Greylock Hackfest with two friends and a friend of a friend I hadn’t met yet. The contest had a reputation for being a place where teams produced some pretty awesome bits of code.

During the opening speeches of the event, Julie Deroche and other members of Greylock Partners dealt with this issue head-on. Hackathons aren’t really the ideal place to find the next idea worth funding. But, they’re a way for Greylock to find new, interesting people and help interesting people find each other. Greylock can see how students get creative with product ideas and implementations when there are tightly limited resources–the core types of skills you’d need in an early-stage venture, in a startup or otherwise.

Basically, hackathons are a practice-run that Greylock can use to find who’s going to be able to pull off a demo like that very first iPhone presentation, rather than a way to actually find the next iPhone.

The crowd’s cheering for “better” during Lehman’s speech was probably mostly due to sleep deprivation at the time, but I’ve found a deeper underlying truth to the response.

After working at three major hackathons (Greylock Hackfest, HackNY, and HackMIT) and numerous other side projects (more on these in future posts), I’ve learned that getting started and getting something built that has laser-like focus on the bare minimum and doesn’t get lost in details is incredibly difficult. It’s something that I haven’t practiced sufficiently in my pet projects. And there isn’t a huge emphasis on it in school.

In a typical project for a university computer science course, you’re given a pretty tightly fixed input and told to give a pretty tightly fixed type of output. You have a fair amount of leeway to decide how much time to dump on building the problem. This is restricted by your other commitments as a student–but you probably have a week or two to get it done.

But more importantly, a project is graded on how “good” your implementation is according to various metrics about code quality, accuracy of transforming inputs to outputs, etc.

It’s a really fantastic system for teaching you the craft of computer science, which is foundational to building a software product. However, there’s skills beyond these–namely understanding how to pivot and how to understand what problem to solve in the first place–that are key to understanding how to launch a product that this model is not spectacular at conveying.

Hackathons are essentially the inverse of this model. The underlying goal of a hackathon is seeing how competitors figure out what vision is important to solve and what inputs and outputs to support that still demonstrate this vision.

Time is incredibly short in a hackathon. If a team tries to build an app that covered all of the real-world inputs and outputs for a vision, they won’t be able to be very ambitious with their vision. Or just fall flat and miss implementing the vision entirely.

The key to winning one is to figure out an incredible product vision and how to traverse a “golden path” like that of the iPhone demo to demonstrate that vision to the judges.

The judges–and for that matter, audiences seeing a demo of a first-generation product–will never see or care about the code used to accomplish that. It doesn’t even matter if you build something that’s not robust. You don’t have to give them the opportunity to touch the product.

Besides, writing something elegant, proper, and robust takes extra time that just doesn’t exist at a hackathon–much like the situations you’ll find in the early stages of a startup.

So, as Lehman says, here, worse is better.

Windows 8 is ambitious. Most operating systems support an imprecise input method (touch) or a precise input method (mouse or pen). Windows 8 supports both, across tablets, laptops, and desktops.

Firefox Metro supports the same spread as Windows 8 itself. So, as a Mozilla summer intern developing its front-end code, I needed to regularly test my work on a tablet in addition to my development desktop.

I wanted Metro Firefox on the tablet to run untethered from the desktop. That way I’d be able to bring it up and demo it easily for people.

This meant maintaining separate Mercurial working copies and running builds on each device. Other team members depend on Elm, Metro Firefox’s project repository, not breaking. My code changes weren’t polished or stable initially. Pushing them directly to Elm wasn’t an option.

Mercurial Queues is a popular way to deal with this. It encapsulates in-progress changes as portable patch files, which you can synchronize between working copies with another Mercurial repository.

But, it’s not a simple solution. Pushing a patch from desktop to tablet involves updating the patch file, committing and pushing it to hg.mozilla.org, pulling Elm changes to the tablet, pulling my patch changes, reapplying the patch, building on the tablet, and restarting Firefox.

This is cumbersome. What if I all I had to do was build on the desktop, wait a bit, and restart Firefox on the tablet?

Enter, Dropbox

I figured I could accomplish this by using Dropbox to synchronize my working copy.

However, a typical Firefox working copy weighs in at several gigabytes, most of which is unnecessary for testing out a pre-built copy of Firefox. To save time and space in my Dropbox, it made more sense to only sync obj/dist, the core files needed to run Firefox.

Windows doesn’t seem to provide file change events about what’s happening behind a symlink, so I couldn’t just symlink obj/dist from Dropbox to my working copy. Instead, I had to copy obj/dist into Dropbox, and then link obj/dist symlink from my working copy to obj/dist in Dropbox.

After getting sync working, I needed to sort out some remaining Windows Registry and permissions issues.

To be a Metro style enabled desktop browser and have a proper tile on the Start Screen, there need to be Windows Registry entries pointing to the Firefox executable. There already was a makefile in Elm to generate registry entries pointing to a Firefox executable in obj/dist for the current working copy. Since the path of my Dropbox folder on the tablet differed from the location of my working copy, I had to tweak the paths in the generated registration scripts.

But, event after getting registered, Metro Firefox wouldn’t launch. The Windows Event Log contained to permissions errors.

To keep file paths short on the command line, I put Dropbox folder in the root of the tablet’s C:\ drive. Since Windows doesn’t automatically give files in the drive root the required execution permissions, Metro Firefox wouldn’t launch.

Firefox started working after I moved the Dropbox folder to my user directory on the tablet.

Sharing

A few weeks later, Yuan from UX asked:

15:23:55 - yuan: is there any other way for you to share your front-end work? jwilde

We couldn’t yet generate an installer for Metro Firefox because some key build system changes hadn’t landed.

So, I decided turn my Dropbox builds into a shared folder for Yuan to use. I automated the setup process with some Python scripts that generated the appropriate Windows Registry files based the user’s Dropbox folder location and documented the installation process publicly on Etherpad. It was janky, but usable as a temporary solution for bleeding-edge testers.

A bit later, she had a working copy of Metro Firefox that would receive updates in real-time.

Yuan posted a link to the Etherpad on the Mozilla Wiki so that other people could try out the build. Given the roughness and instability of the builds in the Dropbox, I chose to keep the Dropbox private but give out access to anybody who asked. By the end of the summer, 13 people were actively testing the Dropbox build and providing incredibly useful feedback.

As more people joined the folder, I felt a responsibility to do a better job at keeping builds working than I did when it was just me and my tablet.

Directly real-time syncing obj/dist meant that shared folder members could see all changes, both good and ugly. To reduce the chance users would see ugly changes, I got rid of the symlinks to the Dropbox folder. I started compiling into a staging folder and using a Bash script to copy obj/dist into Dropbox once I’d checked that everything looked good.

This cleaned things up some, but with ~13 people in the folder, there was a decent chance that at least one person would have Firefox open, putting file locks across their copy of Firefox. When Dropbox receives changes a locked file, it tags the file “in conflict” and uploads the alternate version. Because all members of private shared folders have write access, there were duplicate files strewn across the Dropbox folder, leading to clutter.

Despite the issues that I ran into, I think that Dropbox is a workable development deployment solution for small number of devices when products like Adobe Shadow or Visual Studio Remote Debugging aren’t workable. A public, read-only shared folder might have handled the scaling issues better, but I’d stick to proper updater systems for when apps grow beyond one or two testers.

I recently had the chance to make a very special trip to the Googleplex. It is an event I definitely won’t forget for a really long time.

I met and socialized with a students and mentors from all over the world—including from Thailand, Russia, South Africa, the UK, Germany, Poland, Switzerland, Belgium and the USA—to celebrate the conclusion of the first-ever Google Highly Open Participation Contest. In addition to all of this social interaction, I had the chance to ride roller coasters at Great America, tour the Googleplex, talk to other GHOP students and mentors, eat at two of Google’s famous cafés, and listen to a set of amazing presentations by various tech luminaries who work for Google.

Today, I decided to put together a recap entry about the activities of the two days and what we did during them:

Thursday

I started the day by driving down to the hotel with my mom where the other GHOP students were staying for the four days of their trip. We picked up our Google/GHOP swag kits, which each included various items such as: a hat, a backpack, a sweatshirt, a t-shirt, etc. Afterwards, we proceeded to talk to other students about the projects that we worked on.

A few minutes later, our group’s bus arrived and then we drove off to the secret location of where breakfast was supposed to be. As the bus stopped, people began to chuckle. We were going to have our celebratory GHOP breakfast at an IHOP restaurant!

Many fun discussions, a set of pancakes cut into the shapes of specific letters and arranged so they said GHOP, and several group photos later, we all boarded our bus and headed off to our next destination: Great America. We spent the rest of the day (until 7pm) riding roller coasters, water rides, and other various attractions. At 7pm, we boarded our bus and went back to the hotel. While many decided to go to the Bay Area Python Interest Group’s meeting, I instead decided to go home (I was much too tired).

Friday

On Friday, we all started the day by meeting in the lobby of the hotel. After thirty minutes of fun, interesting discussions, our bus stopped in front of the hotel. We all hopped on board and began the short journey off to the Googleplex.

When bright, multi-colored umbrellas appeared next to the bus, we all knew that we were there. We all jumped off the bus, signed in, and headed off to Café Moma for breakfast. We spent breakfast discussing our various open-source projects. Roughly an hour later, we went on a tour of the Googleplex.

Around 11 AM, we began our awards ceremony. Chris DiBona, Alan Eustace, and Leslie Hawthorn led the awards ceremony, gave speeches, and handed out the awards to students. Mentors, students, and parents posed for photos.

Afterwards, we all walked over to Charlie’s Café and had lunch. I had the opportunity to have lunch with Alexander Limi, co-founder of the Plone project. During lunch, various student/mentor pairs were taken to a rather shiny Google video studio and interviewed for use on YouTube. Alexander Limi and I were interviewed in one of these videos.

After another group photo session in front of Google signs, we went back to the room where the awards ceremony occurred and finished the day by listening to a set of talks given by tech-industry stars such as Guido van Rossum, Romain Guy, Jeff Dean, Bharat Mediratta, and Mike Bland. Halfway though the talks, we took a short break to devour a beautifully-made cake with the GHOP logo on it.

Photos

You can find photos of the event in the ghop07 tag on Flickr and in the ghop07 tag on Picasa Web Albums.