Allen Pike 2016-06-30T22:26:06-07:00 Allen Pike Organizational Size Classes 2016-06-30T18:00:00-07:00 Allen Pike <p>The best thing about organizations is that they are made of people. However, just as more money causes more problems, so it is with people. In fact, the number of people in an organization may be the most important signal in understanding how it thinks and acts.</p> <p>For example, there’s a huge transition when a company grows beyond <a href="">Dunbar’s number</a>, since when a company grows beyond 150 people, it effectively splits into multiple organizations. The other big discontinuity in how a company works – the one that’s more interesting to me due to it currently staring me in the face – happens around 15 people.</p> <p><img style='max-width: 100%' src="/images/2016/steamclock-office-banner.jpg" /></p> <p>You see, a small company is a thing of beauty. Everybody is on the same page. You know what each other are up to, you know how each other think, and everyone being in the same Slack channel doesn’t drive you bananas. You can have lunch together. You don’t have teams, managers, direct reports, and you certainly don’t have departments. You just have folks doing stuff, with maybe one or two people coordinating, and it’s great.</p> <p>I have a lot of love for small, flat teams, and when we started Steamclock six years ago the idea was to build the best small team in Vancouver. My love for indie software companies has always been intertwined with my love for small teams. The idea of every employee actually designing or developing software is delightful.</p> <p>Unfortunately though, a leader can only directly lead a certain number of people. Much beyond 10, and things start to fall through the cracks. The team gets out of sync and people don’t get the support they need. In a one-level team you can often stretch it to 15, but at that point there are two main solutions: stop growing, or start adopting some sort of management structure.</p> <p>Growth wasn’t a goal when we started Steamclock. The fact that we’re at 12 people still seems odd to me some days. One thing I’ve learned over the years, though, is that while overly aggressive growth is dangerous, stagnancy can be too. Specifically, growth opens opportunities for your people to grow with your company rather than outgrow it. While the most important aspect of this is retaining employees, the same goes for founders. In retrospect, I think that the slow ramp in challenge in my job as we’ve grown is what’s kept me engaged and enjoying this job longer than any I’ve ever had.</p> <p>Yet now that <a href="">Pikelet</a> is fast approaching, I’ve been spending a lot of time thinking about how the company is going to roll while I’m on parental leave. It turns out, too often I’ve been the bottleneck to a given project or process, often for no good reason. Where at 5 people it felt like the team was accelerating me, at 12 people it’s sometimes felt like I’m slowing them down.</p> <p>So, more and more, I’ve been working consciously to push information, trust, and authority to the team. When somebody asks me a question where the answer is clear, I try not to just answer it, but <a href="">spread the philosophy behind the answer</a>. When somebody asks for information or access that only I have, I think about how to open that up. I’m constantly looking for ways to not be a bottleneck or switchboard. Years ago if I saw two people collaborating on something that I didn’t know about, it would make me uncomfortable – now it makes me proud.</p> <p>Now, despite these efforts, if we grow larger than 15 or 20, then we’re probably going to need some sort of management structure. This still weirds me out, but increasingly, I can see it working really well. I’ve had some great managers in my career, and I’ve seen how they can provide support, give feedback, help define the mission, and bring attention to problems before shit gets real. Ideally, then, I can work increasingly <a href="">on the business, rather than just in it</a>.</p> <p>And, you know, raise a baby.</p> Too Many Slacks 2016-05-11T18:00:00-07:00 Allen Pike <p><img style='max-width: 100%' src="/images/2016/slack-signin.png" style="width:310px" /></p> <p>We all know the frustration. You’re invited to collaborate with a new team or group. You’re excited to get going, your mind filling with possibilities, when <strong>thwack</strong>. You’re hit with a signup form. Not only are you forced to create a new account, you now need to keep track of one more set of credentials, forcing you to fight to stay logged in to one more thing across your array of devices.</p> <p>Thankfully, Slack just introduced a new feature called “<a href="">Sign in with Slack</a>”. Similarly to Facebook or Twitter authentication, this new feature lets you simply log in to the apps you use for work using your Slack login. You know, your one Slack login. The single one you have, since having more than one Slack login would make this really confusing.</p> <h2 id="welcome-to-slack">Welcome to Slack!</h2> <p>Since October 2014, I have been invited to 19 Slack teams. In addition to the main Steamclock Slack account I use day to day, I have Slack accounts for 5 conferences, 5 developer groups, and 9 clients. Due to the nature of my work and the ever-growing popularity of Slack, I’m now invited to a new Slack on a monthly basis.</p> <p>For each account, I have a username, a password, an avatar, an unread badge, and a helpful red dot telling me what features are new in Slack. Each team takes up time, attention, and a couple hundred megabytes of memory.</p> <p>As such, every time I set up an iPhone, iPad, or Mac, I need to evaluate how many Slacks it’s worth logging in to. 5? 10? Certainly not 19. Over time I’ve gone inactive in more and more social Slacks, which is a shame because some of them I really enjoy, but it’s just too much to juggle.</p> <p>So Slack is a really powerful tool for those of us who collaborate across a lot of teams, and it’s well known that they <a href="">have a problem with account management</a>. More interesting to me is why they let it get this bad, and whether or not they’re going to fix it.</p> <h2 id="july7yardsaleslackcom"></h2> <p>Architecturally, every Slack team is separate. It is on an entirely separate subdomain, and it shares no data with any other Slack team. This is wonderful – for Slack. This separation means each team is its own scalability island, and it can be put on any server, running any version of the software, in any A/B test or custom group, and can completely ignore the now-staggering number of other Slack teams. Siloed clusters of accounts are much easier to scale than a giant interconnected graph, and when you’re growing as crazy fast as Slack has been, scalability is good.</p> <p>An even bigger of advantage of Slack teams being siloed is that enterprise customers have very specific expectations around their data. These customers are willing to pay a lot to control very specific things about how authentication and data storage behaves for their users. If a Very Important Enterprise Customer needs some crazy security feature, or even (god forbid) wants an on-premise install (gasps of terror from the audience, foreboding sound effect plays) then their team can easily be sliced off and put onto a separate server. Meanwhile, a different enterprise customer may require that their users use two-factor auth, or that their usernames must fit a certain format, or that users are immediately logged out if they blaspheme <a href="">Ha’atu the All-Seeing</a>.</p> <p>In this vein, the folks at Slack have been hard at work building out an array of authentication and account features for enterprises. In fact, Slack already supports single sign-on – just not the kind that we want. “Plus” accounts can now <a href="">use SAML</a> to enable signing into Slack at the same time as an enterprise’s other web apps using a set of credentials that the company controls.</p> <p>Slack is also getting ready to launch a set of <a href="">long-anticipated enterprise features</a> centered around security and administration. While you and I may not get out of bed in the morning craving some hot fresh data retention and compliance policies, Slack’s pricing page has pencilled in $32/user/mo as the price for these highly-desired features. Considering that companies need to get quite large before they even know these buzzwords let alone want them, these new customers should be incredibly profitable for Slack.</p> <p>Meanwhile, we freeloaders are whinging about having to juggle our 27 free Slack teams.</p> <h2 id="thanks-for-your-feedback">Thanks for your feedback</h2> <p>So, we know why the problem exists, and why it hasn’t been fixed yet, but the key question remains: are they going to fix it anytime soon? Do we just wait out the chaos, or do we start evaluating alternatives, even just for the conferences-and-networking use cases that are exacerbating this Slacksplosion?</p> <p>Clearly, fixing something like authentication retroactively is a ton of work. A change like this requires rearchitecture, serious scaling preparation, and a heavy dose of migration. It took years from Basecamp’s launch until they <a href="">finally launched a system-wide account system</a>, and at that point they still hadn’t reached the scale that Slack already enjoys after less than three years.</p> <p>That being said, the folks who work at Slack give a shit about this kind of UX problem, even if those of us on Free and Standard teams don’t generate as much growth juice as a the Enterprise whales do. They’re aware of the pain, as we can see in the toil of the SlackHQ Twitter account.</p> <blockquote> <p>Oh and setting up 2-factor auth EIGHT TIMES as well. Seriously @SlackHQ how about some consolidation here?</p> </blockquote> <blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr"><a href="">@kronda</a> Sorry! 😞 Accounts are all kept separate for now, but we hope to improve the experience in future (and hoping never to do it again)</p>&mdash; Slack (@SlackHQ) <a href="">March 27, 2015</a></blockquote> <script async="" src="//" charset="utf-8"></script> <p>While feeling bad about it was a good start, by this March their tone <a href="">had become more guarded</a>:</p> <blockquote> <p>Hey @SlackHQ, how’s it coming with single-sign-on for many accounts?</p> </blockquote> <blockquote class="twitter-tweet" data-lang="en"><p lang="en" dir="ltr"><a href="">@designatednerd</a> <a href="">@corvino</a> We&#39;ve no near-term plans yet but we understand the usefulness! It&#39;s something to think about <span style="font-size: 28px; position: relative; top: 3px">🍮</span></p>&mdash; Slack (@SlackHQ) <a href="">March 19, 2016</a></blockquote> <script async="" src="//" charset="utf-8"></script> <p>While we can all agree on the creamy appeal of CUSTARD (U+1F36E), the phrase “no near-term plans” sounds pretty clear. The juggling will continue until morale improves.</p> <h2 id="and-yet">And yet…</h2> <p>Even if unified sign-in is a dream for now, there are a few intermediate things Slack could do to make working across teams less painful. At <a href="">Slack’s roadmap event in March</a>, they announced that they’re working on a feature called Shared Channels. The idea here is that teams will be able to share a channel between them, a more native and capable version of <a href="">Slackline</a>. This would be an A+ slam dunk for me, since this would eliminate the need for me to be in 8 of the client Slack teams I’m part of.</p> <p>Although it’s likely not part of the initial vision for their Shared Channels feature, an evolution of it could further reduce the unnecessary proliferation of social and conference Slack teams, since most of them fundamentally only need one channel. Occasionally a social group or conference can justify a variety of channels (the XOXO Slack did a surprisingly good job of this) but most of the time, what people creating a social Slack team need is a single channel to chat with an ad-hoc group. In essence, an IRC channel for the 21st century.</p> <p>I for one would be happy to pay Slack on top of what we pay for Steamclock’s team to create a personal team if it would let me consolidate a number of Shared Channels: XOXO, Úll, VanCocoa, &amp;you, Rands Leadership, WWDC, and so on. Since this is a perversion of Slack’s concept of a team it would need further product design work, but it could be simpler to implement than full single sign-on and would reduce the overbearing feeling you get when you context-switch into a social slack. More often than not, to justify the existence of an entire team, an optimistic admin has created two dozen channels, only one of which contains the real conversation, while the other 11 are perpetually marked unread by an ill-advised zombie bot that posts in the channel every six seconds when somebody mentions the word “Swift” on Twitter.</p> <p><img style='max-width: 100%' src="/images/2016/slack-accounts.png" style="width:300px" /></p> <p>The other ray of light comes from our new friend, “Sign in with Slack.” While testing out the feature on the shared channel tool Slackline, I was pleasantly surprised to see a team chooser. This is, as far as I know, the first product feature Slack has launched that shows centralized access to your teams. While it’s disconcerting that I may one day may need this to figure out which of my many Slack teams I associated with some web service, it’s great to see acknowledgement from Slack, not just in their support tweets but in their product, that this is a real problem they need to solve.</p> <p>While I feel strongly about this UX problem, I know that there are a lot of pieces in play and fixing it after the fact is fraught with tradeoffs and costs. It’s a tough issue, and there’s a lot to it. As such, if you work at Slack, I’d be willing to chat in more detail about this issue. Just get in touch, and I’d be happy to invite you to the brand new Slack team I’ve created for it.</p> Diagrammatically Correct 2016-03-31T18:00:00-07:00 Allen Pike <p>Once upon a time, back when Netscape roamed the earth, software development was hard. Desktop apps shipped in cardboard boxes, web apps ran on Solaris boxes, and video games came on little plastic boxes. In this box-crazed world, making sure software was right the first time was Super Important™.</p> <p>Back then, writing an app – or “application” as it was referred to in the archaic English of the time – was an arduous process. Good design patterns were still evolving, and system libraries were of little help. Much of the code in any given app was, effectively, boilerplate. Collection classes, sync frameworks, windowing systems, memory management schemes – a wretched hive of tricky computing science problems.</p> <p>Meanwhile, it had become well understood that software projects, especially large ones, had a disturbing tendency to fail spectacularly. While every field has its blunders, research found that software developers as a discipline were hilariously bad at actually achieving the goals they set out to achieve.</p> <p>In response, software developers armed themselves. They increasingly adopted tools and systems for planning, communicating, and documenting their systems. They formalized requirements before designing, diagrammed objects before coding, and debated architectures before ordering two million dollars of servers. “If you fail to plan, plan to fail,” they would say, and then resume architecting their system’s UML to achieve optimal domain synergy. It was a strange time.</p> <h2 id="years-later">Years later</h2> <p>Recently a <a href="">good friend</a>, a survivor of this gilded era of software engineering, asked me about a hiring problem. You see, he was interviewing to hire a senior software engineer, but many of his candidates were struggling with the systems design portion of the interview. These are supposed to be senior engineers, yet their diagramming skills seemed totally absent. How much systems diagramming do you do at Steamclock?</p> <p>“Uh, well… systems diagramming hey? Well, we don’t exactly… I wouldn’t say we formally… I mean… hm.”</p> <p>After I quelled a wave of guilt, I admitted that we don’t do very much. Of course we’ll diagram out proposed AWS setups, graph gnarly retain cycle issues, and whiteboard the hell out of UI flows, but we haven’t produced a UML document in the history of the company. While we tirelessly design and refine our user interfaces, our technical designs are rarely formalized before we start prototyping.</p> <p>While this is not what we were taught in school, this kind of modern ad-hoc process is now incredibly common in the industry. For every blog post or podcast about diagramming and systems architecture, there are ten thousand on choosing frameworks or hiring. Why design a system up front when it’s just going to change? Why make a document when it’s just going to get out of date?</p> <p>Well, because sometimes that’s what’s necessary to scale software, that’s why. We know how valuable diagramming can be as a tool for thinking through hard problems. We also know that thoughtful systems design is important for building complex, novel systems. Up-front systems design put us on the moon. So, why don’t we do any?</p> <h2 id="meet-the-prototyper">Meet the Prototyper</h2> <p>From a technical perspective, most software projects today just aren’t that special. If you want to explore just how not-special most software projects are, subject yourself to a project review for the Canadian <a href="">SR&amp;ED Tax Credit</a>. This program provides tax credits for technically novel software development, which when you dig into it is very little of what happens in a typical software project. Review a year’s worth of time entries on a challenging project, and you’ll often find that nine hours out of ten, you were just picking libraries, adapting to new requirements, and improving the UI.</p> <p>Modern software projects are often just the cherry on top of a wheelbarrow-sized sundae of opaque 3rd party code. Why write a blogging engine from scratch when you can make one that fits your needs perfectly in a few hundred lines, by just importing <a href="">200,000 lines of dependencies</a>? Why reinvent the wheel when you can <code>gem install wheel</code> or add <code>wheel 0.1.x</code> to your package.json?</p> <p>In this environment, more and more software developers now spend their careers as prototypers. Startups, product consulting shops, and indies typically spend a lot of time starting stuff, and relatively little time scaling it. In these kinds of businesses, prototypers’ shipping skills are invaluable. Any given project may be relatively small, and may or may not turn into something big, but if you can ship it, you’ve overcome the biggest threat to most new software: never shipping in the first place.</p> <p><a href=""><img style='max-width: 100%' src="/images/2016/cube-priority.jpg" /></a></p> <p>Our culture has never been better focused on shipping. Agile – <a href="">the manifesto</a>, not <a href="">the catchphrase</a> – makes no bones about the need to deliver sooner rather than later:</p> <blockquote> <p>Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. … Working software is the primary measure of progress.</p> </blockquote> <p>Say what you want about agile the catchphrase, but the increased focus on iteration and delivery has been a huge positive force in the industry. Fifteen years later, we have a whole new generation of developers for whom this mentality has always been the norm.</p> <p>While this has been a net win, there are some interesting consequences to this mindset. Folks who spend their careers working on a series of small iterative projects tend to think less in terms of traditional software engineering, and more in terms of immediate user goals and short term improvements. Besides leading to a lower prioritization of systems design and planning up front, this mentality can also lead to laissez-faire attitudes about documentation, unit testing, and refactoring. The OG software engineers would not approve.</p> <p>For better or worse though, the iterate-now-plan-later philosophy has come to dominate the popular discussion around software development. This outlook is an easy sell at lean startups, UX-focused development shops, and any studio more focused on solving user problems than technical ones: who enjoys writing specifications anyway? As a result, we now have an entire generation of software developers raised on the “move fast and break things” approach to solving problems.</p> <h2 id="dude-wheres-my-composite-structure-diagram">Dude, where’s my Composite Structure Diagram?</h2> <p>Unfortunately, at a certain scale breaking things is not an option. Or, more accurately, breaking things becomes staggeringly costly. Moving fast is great, but if you’re serving <a href="">billions of push notifications per day</a>, it becomes more important that everything doesn’t go to shit.</p> <p>Projects at scale have an outsized impact, and early mistakes can become hard to undo. The larger and more technically novel a project gets, the more you suffer the pain of “not maintainable here”, and the less bananas it is to design and build your own custom frameworks and other components.</p> <p>Naturally, folks that spend their careers building and maintaining large systems at scale typically spend a lot more time thinking about systems design and software engineering. At a certain point, generalized “best practices” that work for the typical system stop working, and speculatively iterating towards your servers not bursting into flames can be untenable.</p> <p>All this necessitates serious-business systems planning, diagramming, and projections. Triply so for backend systems, which get more complicated and have longer lives than our beloved clients.</p> <p>So, where are the senior systems designers? Well, they’re doing what they do best: building massively scaleable backends at Amazon, Apple, and Google. They’re launching spacecraft to Mars and running nuclear power plants. They’re grappling with the face-meltingly complex and fragile enterprise systems that keep airlines, banks, and telecoms from collapsing under decades of technical debt. Basically, they’re saving our bacon.</p> <p>Still, these experienced systems designers make up a smaller and smaller proportion of software developers over time. Unfortunately, all too often their respective corporate bureaucracies prevent them from blogging, podcasting, or even tweeting about the work they do. This reinforces the echo chamber of startups and indie product developers, who like to celebrate a lack of process, and think of rapid iteration and thoughtful software engineering as mutually exclusive. The pendulum has swung so far that the very idea of up-front design is thought of as old and busted.</p> <h2 id="one-drop-does-it">One drop does it</h2> <p>I’ll readily admit that I’m most comfortable wearing my “move fast” cap. We work on a lot of new apps, put a big focus on UX, and do a lot of work for startups. Like many studios, getting bogged down on exhaustive up-front planning isn’t a great tradeoff for most of our projects.</p> <p>That said, it’s crucial that we keep in mind the benefits of the tools in our software engineering toolbox, and make thoughtful choices of when to use them and when not to. Rather than debating whether all projects should adopt this process or that process, we should debate how to tell when a certain project justifies more formal engineering.</p> <p>For example, projects for larger companies typically have a higher likelihood of hitting scale, so that’s a flag that their projects justify spending more resources on meticulous planning, testing, and preparation. In contrast, an experimental product from a startup would more likely benefit from moving as quickly and efficiently as possible.</p> <p>So, next time you’re starting a new project, keep an eye out. A diagram here, a short planning document there – where there are technical risks, there are opportunities to use system design. It might not be sexy or futuristic, and it sure as hell doesn’t need to be UML, but sometimes, a little planning can go a long way.</p>