Allen Pike 2015-05-01T01:35:34-07:00 Allen Pike User agents of change 2015-04-30T18:00:00-07:00 Allen Pike <p><img style='max-width: 100%' src="/images/2015/ie-edge.jpg" width="250" /></p> <p>Yesterday, Microsoft released a preview of Edge, their next-generation web browser. Edge’s new rendering engine brings it more in line with modern layout engines like WebKit, and finally introduces a modern replacement for Internet Explorer. IE’s dark past means that millions of existing websites serve it old and busted markup and JavaScript, which should thankfully no longer be necessary with Edge’s modern engine. As such, it was time for Microsoft to revisit the browser’s user-agent.</p> <p>For Edge, they worked to remove the <a href="">gross middleware junk that cluttered IE’s user-agent</a> and simply advertise Edge as a modern browser that can handle modern web apps. With this in mind, Edge identifies itself with the following <a href="">new, streamlined user-agent</a>:</p> <blockquote> <p>Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36 Edge/12.0</p> </blockquote> <p>Short and sweet. The user-agent is more thorough on mobile:</p> <blockquote> <p>Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; <em>DEVICE INFO</em>) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Mobile Safari/537.36 Edge/12.0</p> </blockquote> <p>That is to say, Microsoft Edge claims to be every computing platform ever conceived - except for Internet Explorer. On its surface, this bold claim is surprising.</p> <h2 id="i-am-everyone-and-no-one">I am everyone and no one</h2> <p>The user-agent HTTP field was conceived in 1992 with a clear and simple purpose: let browsers identify themselves to websites. It let web developers collect stats about how many luddites were using “NCSA_Mosaic/2.0” and how many hotshots were using “Mozilla/1.0”, the Mosaic killer officially known as Netscape.</p> <p>Netscape did in fact kill Mosaic, and it did so by adding more features. By the mid 90s, savvy web developers were checking the user-agent for “Mozilla” so they could send Netscape fancy new markup but still support older browsers with plainer content. With this user-agent detection technique, developers could safely use Netscape’s JavaScript to pop up insightful alert dialogs, or serve fancy frame-based layouts that leveraged expansive 800x600 “<a href="">Super VGA</a>” displays. It was a crazy time, full of naive optimism and developers drunk on blink tags.</p> <p>In the meantime, Microsoft was busy developing Internet Explorer. As expected, they specified their user-agent string as “Microsoft Internet Explorer/1.0 (Windows 3.1)”. Unfortunately for Microsoft, and anybody who would ever need to make sense of a user-agent again, this meant IE 1.0 was served pages without the fancy Netscape functionality. These Mozilla-detecting sites made IE seem crappy - a designation it had yet to earn. The next version of IE instead shipped as “Mozilla/1.22 (compatible; MSIE 2.0; Windows 3.1)” and fixed the problem <a href="">once and for all</a>. IE users got JavaScript and frames, and web developers got <a href="">an endless cycle of pain</a>.</p> <p>In the 20 years since, every new web browser has been stuck with the same unpleasant choice. They can either fight the long, hard fight of evangelizing <a href="">feature detection</a> and battling one by one to get sites to update fragile and out of date browser detection code across the entire web… or they can just tack a new token onto the existing trainwreck and be done with it. Chrome could have simply launched as Chrome/1.0, but instead it made its debut as</p> <blockquote> <p>Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/ Safari/525.19</p> </blockquote> <p>The HTTP 1.1 spec <a href="">specifically discourages this</a>, since it further entrenches browser detection. Unfortunately, appending yet more junk to the user-agent is the least bad way for a new browser to get modern behaviour from existing websites, but still allow new code and analytics packages to identify it.</p> <p><img style='max-width: 100%' src="/images/2015/katamari.jpg" alt="Katamari Damacy" /></p> <p>And so the user-agent string has become a never-ending katamari that appends the string of every browser that was ever popular. After 20 years of rolling in more and more tokens, every HTTP request Edge makes has to include more than 150 bytes of text to simply convey that it is in fact Edge - a fact that only contains perhaps two bytes of entropy. As things stand, the string will continue to roll on and on indefinitely until it is large enough to pick up buildings and oil tankers.</p> <p>Thankfully, an end to this madness is in sight. Analysis of major browser releases over the last 20 years shows that user-agents have grown in length roughly linearly at a rate of about 5 characters a year. This pace will eventually become unsustainable, since the popular Apache web server <a href="">limits header size to only 8190 bytes</a>. With this limit in place, user-agents can only grow at their current rate for another 1608 years. The clock is ticking for browser vendors and web developers alike to work together to forge a new solution to this problem - before it’s too late.</p> Moving mountains 2015-03-31T18:00:00-07:00 Allen Pike <p>Let’s say you’re an independent developer, and you want to convince the most profitable and successful tech company in history to make a change that benefits you. How might you go about that?</p> <p>Naturally you start by <a href="">filing a Radar or Getting the FO</a>, but beyond that, it is infamously hard to determine who at Apple is actually responsible for your issue. It would be nice if every developer had a knowledgeable and responsive Developer Relations rep they could contact, but given that there are hundreds of thousands of iOS developers, that’s hardly practical. The same is true on Google’s and Microsoft’s platforms — at a certain scale, they can’t practically listen to every voice. As such, we reach to the most classic of persuasive methods: the critical article.</p> <p>A critical article about some Apple technology or policy is like a kind of thought virus. If you make a compelling argument, you can seed it on the open internet, and by its nature the article will spread among people who care about Apple and its success or failure. Naturally, this includes Apple employees. While it may be impossible from the outside to discern who is responsible for a particular iOS 8 usability issue, a thoughtful critique of the problem has a decent chance of making its way to that team and driving change for the better.</p> <h2 id="running-to-the-press">Running to the press</h2> <p>The introduction to the App Store Review Guidelines give this advice:</p> <blockquote> <p>If your App is rejected, we have a Review Board that you can appeal to. If you run to the press and trash us, it never helps.</p> </blockquote> <p>A lot of folks have argued that this is hypocritical to say, since negative press around a ridiculous rejection does seem to expedite resolution. At its root though, there is some truth: if you do a press interview feeding the latest “Apple is doomed because they rejected my fart app” article on Valleywag or Forbes, then Apple PR and the App Store team aren’t going to be enthusiastic about helping you, even if they may have their hand forced. On the other hand, if you write some thoughtful criticism on some actual problems with one of their policies or APIs that happens to be circulated in the press, there will be people at Apple who want to solve the problems. There’s a difference between trashing a company and criticizing their policies.</p> <p><a href="" title="Photo: Rubin Starset"><img style='max-width: 100%' src="/images/2015/signs.jpg" alt="Photo: Rubin Starset" /></a></p> <p>In his recent short novel “<a href="">Fear of Apple</a>,” Eli Schiff argued that independent iOS developers aren’t critical enough of the fact that it’s increasingly difficult to make a living on the App Store. Now, I don’t know what iOS developers Eli has been talking to, since I’ve seen more articles, talks, and rants on this topic than any other in the community. What was more interesting to me, though, was that Eli attributed the supposed lack of criticism to a fear of Apple.</p> <p>While there are enough “vengeful Apple” stories from the Jobs era to give some long-time Apple developers pause, it’s unclear to me what potential critics are afraid that a modern Apple might do to them. Pulling some indie’s app from the store because they wrote a critical blog post is hardly Apple’s MO. Hell, I <a href="">called the iOS shift key the “the worst thing to happen in the history of software”</a> and I privately got positive feedback from folks at Apple. As <a href="">Marco put it</a>:</p> <blockquote> <p>No sensible developer should be worried about angering “Apple” by fairly expressing legitimate criticism.</p> <p>There is no single “Apple” to anger, as the company comprises thousands of people across many different departments, all of whom can think for themselves. I’m sure some of them can’t take criticism well and may be vindictive — any large group of people will contain almost every personality type — but that’s not the attitude of any of the Apple people I’ve interacted with.</p> </blockquote> <p>Admittedly, if for some reason Apple PR noticed your dissection of serious bugs in some new API, they wouldn’t be pleased — but how often is your indie app business dependent on Apple’s PR department? Contrast that to how often you benefit from the work Apple’s API teams are doing on the many technologies we depend on to build our businesses. Consider how much you benefit when they improve the platforms you and your customers live on. Every fix you might motivate could affect hundreds of millions of users around the globe. For me, the math is clear: if there’s a chance you can help Apple build better software, it’s worth writing something critical.</p> <p>That said, the nature of developers’ critical pieces is going to be different than the dramapress’. We understand how hard software is, we know that Apple’s designers and engineers made the tradeoffs they did for a reason, and we actually want to effect positive change, not just get pageviews because we’ve riled up a frothing mass of anti-Apple noise. Apple makes various products and decisions that are interesting to talk about. Many of them are good, some of them are bad. In the end though, they’re a <a href="">company made of people</a>, and if we want to actually convince them to do better, we need to do just that: convince them.</p> <p>As a group, indies are saddened and frustrated by all the business challenges that unsustainable pricing and oversupply have created. We have been and will continue to be critical of specific App Store decisions, like lack of upgrades, that make it harder to keep writing good software for iOS and the Mac. We should continue to push issues like App Store search that hurt users and developers alike every day. Still, yelling at Apple for the fact there are 500,000 people flooding the store with mediocre apps isn’t going to effect change.</p> <p>Independent app developers live in the shadow of a mostly benevolent giant that can at a whim create and destroy entire markets. We’re usually too small to be intentionally slighted, so we have the freedom to speak up that folks at larger companies might not have. We may fear Apple the same way we fear nature: it’s a powerful force we don’t control. Unlike nature, though, it’s a force that’s trying to do good, and it’s one we can influence - if we make a compelling argument.</p> A JS framework on every table 2015-02-28T18:00:00-08:00 Allen Pike <p>Most programming languages support a small number of popular, stable application frameworks. Objective-C and Swift apps use Apple’s excellent Cocoa framework. Ruby apps more often than not use Rails. Java has a handful of established web app frameworks, and they come and go relatively slowly.</p> <p>In the meantime, the latest and greatest JavaScript framework comes around every sixteen minutes.</p> <p>Studies show that a todo list is the most complex JavaScript app you can build before a newer, better framework is invented. Luckily, there’s an excellent site called <a href="">TodoMVC</a> dedicated to comparing JavaScript frameworks by way of todo sample projects. There, you can see how 63 JavaScript app frameworks’ todo examples compare to just jQuery or vanilla JavaScript. If you think 63 frameworks are a lot, you ain’t seen nothing yet: the TodoMVC team <a href="">gets multiple pull requests weekly</a> from people flogging new JavaScript frameworks. </p> <p>For example, <a href="">the most recent TodoMVC pull request</a>, as of this writing, wants to add Riot.js 2.0. What is Riot.js 2.0, you ask? Apparently, it’s the second version of something called Riot.js, an app framework that’s like React.js, but better in some ways that are definitely important. “But wait,” you may ask, “didn’t React like just come out and isn’t even 1.0 yet? How can a thing based on it be 2.0 already?!” The answer, my friend, is JavaScript.</p> <p><img style='max-width: 100%' src="/images/2015/javascript-guy.jpg" /></p> <p>Why is the JavaScript framework environment so unstable? What is driving this insanity? Why are Ruby developers using Rails 4.2 but client-side developers are hyping boron.js 0.2.1?</p> <p>The problem is, as always, the browser. The modern web browser is an incredible feat of engineering, especially considering its humble roots as a way to display simple documents. There is definitely a substantial amount of framework turnover in the node/io.js community, but it’s nothing compared to the zoo that is the browser app framework world. The wild instability of the client-side JavaScript landscape is a product of the browser: both its historical limitations and its incredible rate of improvement.</p> <p>The browser is the most ubiquitous app runtime in history by a wide margin, and to write applications for it you need to write JavaScript. As a consequence, JavaScript is the least elective programming language in the world – most people who are writing client-side JavaScript applications are not doing so because they chose JavaScript, but because they chose client-side web development. These people come from a wide variety of backgrounds, and have a wide variety of goals. This constant inflow of new ideas and interests has made the JavaScript community unusually diverse.</p> <p>Meanwhile, the browser itself changes far more rapidly and unpredictably than the underlying languages on which other frameworks are built. The browser has changed more since Backbone.js debuted in 2010 than Objective-C has changed since Cocoa debuted all the way back in 1988. Browser improvements that inspire new generations of frameworks happen constantly.</p> <p>These inputs fuel an incredible amount of experimentation, excitement, and adaptation. Every day new frameworks, new approaches, and new tools <a href="">erupt from the geyser we call Github</a>. Every year there are more instant-classic JSConf talks about <a href="">crazy new ideas</a>, <a href="">impressive new browser features</a>, and <a href="">rethinking best practices</a>. This beautiful chaos is exciting in a way that I’ve never seen in another developer community.</p> <p><a href="" title="Photo: Adrián Pérez"><img style='max-width: 100%' src="/images/2015/jsconf-eu-banner.jpg" alt="Photo: Adrián Pérez" /></a></p> <p>Unfortunately, while the community is busy driving new features and new frameworks, the browser is is busy killing them.</p> <p>Unlike the frameworks we use in C++ or Swift, the entirety of your app framework must be downloaded, parsed, and turned into machine code before users even see your app. Caching and offline access continue to improve, but the very real constraints on bandwidth, latency, memory, and loading time are a huge challenge for the kind of huge app frameworks that we take for granted in native app development.</p> <p>This is especially true on mobile, where the already problematic constraints of memory, CPU, and network bandwidth can be crippling. While JavaScript is now a remarkably fast language, the ubiquity of the web means that your code and framework can be downloaded and run in some incredibly hostile environments. Frameworks that can take advantage of Chrome Canary’s supernatural powers on a Mac Pro aren’t going to enjoy themselves on “Browser” on an old Acer EvoAMAZE Plus 3G E.</p> <p>The churning seas on which client-side frameworks sail are especially treacherous for large, ambitious frameworks that try to create a rich environment and UI library for building apps the way we do natively.</p> <p>When SproutCore and Cappuccino appeared in 2007, I was totally sold by <a href="">the amazing demos</a>. I’d thought Moore’s law had finally delivered stable, featureful JavaScript frameworks for building beautiful desktop-like apps in the browser. Then I spent two years <a href="">shipping stuff</a> with SproutCore, dealing with megabytes’ worth of somebody else’s JavaScript, wrestling with weird technical choices necessitated by horrible DOM performance on IE7, and somehow cramming the whole thing into the iPad’s 256MB of RAM. Meanwhile, the documentation never caught up to the pace of breaking changes, and performance was a constant battle. Before long, the dream felt more like an evolutionary dead end.</p> <p><a href="" title="&quot;Sharovipteryx BW&quot; by Nobu Tamura ( Yes, this was a thing."><img style='max-width: 100%' src="/images/2015/flying-dino.jpg" alt="&quot;Sharovipteryx BW&quot; by Nobu Tamura ( Yes, this was a thing." /></a></p> <p>While in many ways I still dream of a framework that could deliver on the promises that SproutCore and Cappuccino made, I now have much more respect for how hard such a thing is to build, especially in JavaScript. The same flexible, untyped nature that makes JavaScript fun to write a microframework in makes it kind of unpleasant to learn or maintain a large framework in. While the strong typing of native app frameworks can feel like a drag on tiny projects, it makes their often-massive APIs easy to explore and use effectively: just tab-complete your way to victory.</p> <p>A deprecated API in the Cocoa framework just means that a warning comes up when I open my project, and sometimes I can even right-click to replace the old way of doing things with the new way. Meanwhile in JavaScript, poking around in the Webkit REPL to find APIs you might make use of is a good way to find unsupported accidentally public APIs that will blow up your app at runtime after some point update of your framework.</p> <p>All these forces – the rapid changes, the diverse needs, the hostile environment, and the loose language features – make large, feature-rich JavaScript frameworks slow, lumbering prey. A horde of young, nimble <a href="">microframeworks</a> swarm them, take them down, and fight over the meal. Modularity and componentization reigns.</p> <p><a href=""><img style='max-width: 100%' src="/images/2015/tar-pit.jpg" alt="Evolution's a bitch." /></a></p> <p>A world dominated by endless tiny frameworks actually works okay for JavaScript experts and consultants. We become skilled at mixing and matching components, are able to spend time keeping up to date on the latest options and their relative strengths, and gain a lot of fodder for our conference talks. Yet for those who are new to JavaScript or who aren’t full-time client-side developers, any given tiny framework can’t possibly solve the wide range of problems that JavaScript developers struggle with. Product companies building large, long-lived client-side apps need to settle on some framework, and live with the consequences for years as they discover its limitations.</p> <p>The well-regarded <a href="">Mithril.js framework is only 5kb</a>. That’s great for load and parse time, but it necessarily can’t solve a rich array of problems. Rails, by contrast, has more than 100x as much code. <em>Just ActiveRecord is 1274kb!</em> In the browser, you can’t sanely use a 1.2MB framework, and so the proliferation continues.</p> <p>Neck deep in frameworks, <a href="">choosing one we’re actually happy with</a> becomes virtually impossible. The Paradox of Choice means that knowing you’re probably not using the right framework causes endless cognitive dissonance. Ironically, this dissatisfaction drives even more people to create their own frameworks.</p> <p><a href="" title="Cube Drone by Curtis Lassam."><img style='max-width: 100%' src="/images/2015/cube-drone-angular.jpg" alt="Cube Drone by Curtis Lassam." /></a></p> <p><a href="">Framework</a> <a href="">fatigue</a> has driven down the number of frameworks that JavaScript developers actually try. Who is going to actually build a non-trivial application with Dojo, YUI, ExtJS, jQuery UI, Backbone, Ember, Cappuccino, SproutCore, GWT – oh man remember GWT? – Angular, Sencha, jQuery Mobile, Knockout, Meteor, Ampersand, Flight, Mithril, Polymer, React and Flux? Seriously?</p> <p>Nobody can try all the popular options, but everybody wishes for consolidation and more wood behind fewer frameworks. As such, a lot of developers have fallen into a pattern: avoid learning a new framework until it seems like one is finally ascending to the throne as the widely popular, exceptionally capable, “won’t get you fired” choice for building JavaScript frameworks. While Google Trends is no double-blind study, it seems clear that interest in Angular has far surpassed that of any historical web app framework. After a decade of chaos, it finally looks like one we’ve found the chosen one!</p> <p><img style='max-width: 100%' src="/images/2015/rise-of-angular.jpg" alt="Composition of Google Trends searches of &quot;%s tutorial&quot; for 9 frameworks." /></p> <p>Yes, Angular,js, the framework that I initially described in 2009 as “unreasonably weird” seems to have achieved a level of popularity that no JS app framework ever has. Between serious support from Google, an easy ramp-up path, and a rich community, Angular looks poised to finally settle the…</p> <p>Wait, what’s that? The Angular team has decided that <a href="">Angular 2 will not be compatible with Angular 1.x</a>, and there will be no easy upgrade path?</p> <blockquote> <p>Developers familiar with the Angular 1.X will encounter a drastically different looking framework and will need to learn a new architecture.</p> </blockquote> <p>Oof.</p> <h2 id="is-this-forever">Is this forever?</h2> <p>Perhaps we should just resign ourselves to it always being this way. While I’ve long argued that creating your own JavaScript framework out of a microframework and a DOM library is madness, maybe it’s the least bad option. Maybe the quirks of the language and the constraints of the browser make a sophisticated but bulletproof framework like Cocoa or Rails just kind of impossible.</p> <p>Maybe I should just call off the hunt. Though, people do seem pretty excited about React.js. They have some smart developers and exciting ideas, at least from what I’ve seen.</p> <p>You know, maybe I should build something new with it and see what it’s like. Maybe… maybe it’s the one.</p> <hr /> <p><em>Special thanks to Henrik Joreteg, Angelina Fabbro, and Nigel Brooke for their feedback on the ideas behind this piece.</em></p> <p><em>This article has been <a href="">translated to Japanese</a>.</em></p>