Saturday, June 28, 2014

What is DevOps Days Brisbane 2014?


I'm one of the organisers for an awesome conference coming up next month July 25-26.  It's called DevOps Days Brisbane 2014.  If you've never heard of "DevOps Days", it's an international grassroots conference run for the benefit of the IT community in cities all over the world.  For the Australian branch, formerly known as "DevOps Downunder", this is the first year it has been held in Brisbane.

We're really keen to get as many people along as we can and help put Brisbane on the international map.  We're not-for-profit - run solely for the benefit of the community so tickets are really affordable at $185 (+fees) each for a 2-day conference in the Brisbane Convention & Exhibition Centre with full food & beverage + reception drinks - crazy!  It's held over a Friday & Saturday so you or your employees need take only one day off work.

The program has just been announced with great local, national, international speakers from companies such as Chef, ThoughtWorks, RedHat, IOOF, REA & Wotif Group (including me - my first devops days talk!).  And the keynote address will be delivered by none other than world-renowned human factors and safety expert Sidney Dekker.

The format for both days is a mix of great presentations in the mornings and collaborative tech talk and problem solving in the afternoons:

  • In the mornings we have high quality single-track presentations and rapid-fire ignite talks with well respected international speakers as well as national and local thought leaders.
  • In the afternoons we have multi-tracked "openspaces" - where the content is driven by attendees for attendees - we provide advice for each other's technical and cultural problems, talk about the latest technology, and help each other think outside the square to where we want to be as an industry. It's a great format where attendees can not only receive advice from thought-leaders and peers but also bring the benefits of their perspectives to the wider community in an interactive manner.

DevOps Days conferences are inspiring, challenging, thought-provoking, and lots of fun.  Attendees walk away with new friends and a sense of camaraderie.  As well as new perspectives, not just on the latest technology, but importantly also on culture and how to collaborate, communicate and empathise with other disciplines within their own company.   DevOps is essentially an extension of Agile - bringing all the goodness of fast feedback loops (with added emphasis on collaboration and automation) to the "last mile" of deployment to production. It's not only open to developers and operations, we find that managers, architects, testers and BAs all get value out of it and contribute too.

Website with registration and full program details is here: http://www.devopsdays.org/events/2014-brisbane/

And there's still a chance to sponsor the event and get your logo alongside great international companies like PuppetLabs and AppDynamics.  Gold sponsors get 4 free tickets and Silver sponsors get 2 free tickets to send their employees or mates along.

I'm happy to come and talk in person with teams, groups, individuals around Brisbane and answer questions anyone may have.  Let me know if there's anything I can do to help you help us put Brisbane on the map!

Twitter: @mcallana
Google+: matthew.callanan
Email: matt at mattcallanan.net



Tuesday, May 6, 2014

Is TDD Dead?

An interesting read: http://david.heinemeierhansson.com/2014/tdd-is-dead-long-live-testing.html
A purposely inflammatory post that had the desired effect of bringing out some TDD heavyweight advocates on Twitter including Bob Martin and Martin Fowler....

There are some interesting points to draw from David's Twitter stream:
  • He learnt the ins-and-outs of TDD first before moving on (and he learnt a lot from it).
  • He says his view of the testing world only applies to Rails apps where tight Web + DB integration/coupling is a design choice.
  • He has very opinionated beliefs about how Rails apps should be built including
    • Rails apps should remain "small-scale"
    • ActiveRecord usage should be carefully maintained
    • ... both of which are often not the case in reality.
  • Patterns of building/testing Rails apps like Hexagon are overkill if you're building a small Rails app with business logic intended only for web audience - but can be useful if that Rails app is a peer among other interfaces.

My summary:
  • Learn TDD inside-out, learn where it fits rather than getting caught up in dogma. 
  • Know what type of app you're building
  • Then carefully design and architect your application to be testable through careful and appropriate use of unit/integration/system testing
  • Ensure your architecture is carefully and purposefully maintained.
I personally use a "Test Oriented Development" approach:

  • Exploratory coding should be done as part of a spike - normally on a "spike branch"
    • There's often not much point writing up-front tests when you need to try something you've never done before - this can be a form of waste
    • Kent Beck (the inventor of TDD) himself doesn't write tests before exploratory code - podcast
  • When you've solved the problem, jump back onto master and rewrite the solution this time with testing in mind - carefully plan Unit Tests, Component/Integration Tests, & Acceptance Tests
  • If you can't pair commit to master, push to a temporary review branch and submit a pull/merge request to a reviewer



Interesting response: https://www.destroyallsoftware.com/blog/2014/tdd-straw-men-and-rhetoric

And another one from Martin Fowler about self-testing code:  http://martinfowler.com/bliki/SelfTestingCode.html
These kinds of benefits are often talked about with respect toTestDrivenDevelopment (TDD), but it's useful to separate the concepts of TDD and self-testing code. I think of TDD as a particular practice whose benefits include producing self-testing code. It's a great way to do it, and TDD is a technique I'm a big fan of. But you can also produce self-testing code by writing tests after writing code - although you can't consider your work to be done until you have the tests (and they pass). The important point of self-testing code is that you have the tests, not how you got to them.

Monday, April 14, 2014

Ideas for Running Cucumber-JVM Selenium Tests in Parallel

Ideas on how to run Selenium-backed Cucumber tests in parallel


Use Selenium Grid 2 to coordinate the Selenium tests across a server farm of Selenium RC servers



Example Parallel Cucumber JVM execution with Maven surefire plugin forkCount

Alternatively,

Test Load Balancer






Thursday, February 27, 2014

How Etsy do Code Reviews with Continuous Delivery (incl. branching, culture)

Continuous Delivery strongly advocates developing on mainline/master/trunk.  It also advocates releasing as frequently as possible, pointing out that Continuous Deployment, where all commits that pass all the stages of the build pipeline are automatically released, is the ultimate logical extension.  But how do teams have a chance to perform code reviews if all commits to master are potential release candidates?  This question was posed on the Continuous Delivery Google Group recently, with a response from Etsy's Mike Brittain about how they do it.  TL;DR: It involves temporary short-lived branches for reviews and a culture: of keeping changes small, not deploying without review & being good at assessing risk of releasing untested code.


  • Are code reviews mandatory?  Any formal process or tool that "gates" check-ins to trunk, based on peer review?
    • Code review is not mandatory, nor does it gate check-ins to trunk or deployments to production. 
    • Some of the hangups we have about gating deploys in this fashion is that gating artificially slows us down emergencies/outages, and as Jez put it in the topic you linked to, gating assumes stupidity that has to be corrected for. 
    • We have built the culture in our Engineering team to assume everyone is going to do the right thing, and we should trust them. When that trust falls apart, I think of it as a bug in our hiring and on boarding processes.
    • It is socially unacceptable at Etsy to deploy without some form of review, and that's true for nearly every sub-team. 
    • Many of our reviews happen in GitHub (Enterprise), and so have an audit trail attached to them of who reviewed and what was discussed. 
    • Other code reviews might be a simple copy-and-paste of a patch file, or an over the shoulder review by a peer. 
    • Most changes are relatively small and can be easily grokked
  • With respect to config flags (feature switches) and other *new* code:
    • It's possible that we'll deploy code to our production servers that is not live (e.g. "dark code"), but that a more formal code review will happen later when all of the changes for some new feature or product are complete. 
    • Maybe this is just a side effect that we don't use feature branches, and that we think of committing to git as deploying to web servers (they happen nearly in sync). 
    • That doesn't mean that the code will be live to a public user. (Think: new class files, or new and unlinked JS or CSS, features that are gated off so that they are only available to staff members).
  • Likely that any change which would negatively impact customers in the short-term will be caught by automated tests?
    • I'd argue that there are a number of changes that could negatively impact consumers or, more to the point, their behavior, that would never be caught by automated tests
    • These are the types of things you might think of for classic A/B tests—size, shape, position, color of UI elements, feature discoverability, fall-off in multi-page flows. 
    • And that is why we move quickly to get these changes to the (A/B) testing phase.
    • This statement seems to imply that automated tests are the only way to catch bugs. We find a lot of bugs through production monitoring that we would have never conceived of writing tests for.
  • We have the benefit of operating our own products - not dealing with contractual obligations to deliver "bug-free" code to clients. 
    • We're deeply immersed in the trade-offs of testing vs. speed, and I think that's well understood by many engineers on the team who are making decisions around those trade-offs every day
    • We regularly make mistakes at this, identify the mistakes through open discussion, and those engineers who were involved come away more experienced and wiser for it.
  • Assumption of "under-reviewed code"?
    • Many cases where we deploy code, even publicly, that you would consider to be "under-reviewed"
    • The relative degree of testing that we apply is proportional to the amount of risk involved. 
    • We are decent at assessing and socializing risk :) 
    • Our Payments team, for example, is much more rigid about their review process than teams that, say, are running experiments at the UI level. 
    • It's not one-size-fits-all.
  • You mentioned that you don't use feature branches, but also mentioned that most of your reviews are done in Github Enterprise?
    • What you're pointing out is some subtlety around how we refer to branches and reviews.
    • In terms of development process, we don't use feature branches in the traditional sense—especially if you think of branches existing for more than a day. 
    • In practice, there are certainly cases where individuals will use a short-lived branch to isolate a couple of things they're working on at a time, much like you might use git stash—such as to switch context to work on an immediate bug fix that needs to go to production.
    • Our review process does utilize branches but only to support the review process
    • As an engineer prepares to commit a change set, they create a (short-lived) review branch in git. 
    • This allows us to look at the review in GitHub Enterprise and use their commenting and visual diff'ing
    • It also allows other engineers to checkout the branch and put it through its paces, when necessary.
    • Once the code review is complete, the branch is typically auto-merged in GitHub when the engineer is prepared to deploy it.
    • Yes, it's a subtle (and perhaps stubborn) distinction. :)
    • It seems that "time to review" would be a fairly key metric in that case
      • This is true, but it's not really a problem. 
      • Daily stand-ups generally work for socializing if you're blocked on someone's feedback on a code review
  • For your review process, would you say you're using a variant of what GitHub calls "GitHub Flow"?  
    • http://scottchacon.com/2011/08/31/github-flow.html - (as distinct from what Scott calls the overly "complicated" gitflow model).  
    • Perhaps tweaking step #2 to happen after making local changes to 'master' on an as-needed basis
      • That's pretty much right. We only go onto a branch for the review stage
  • How do you ensure branches (whether for bugfixes or reviews) remain short-lived
    • It's a cultural thing. 
    • You learn that practice in the first days of work on the engineering team. 
    • If you get into a review where you've got a boatload of code that's difficult to understand or to assess the risk, your peers will point that out.
    • This is also the reason we don't advocate new work to start immediately on a feature branch - those have a tendency to allow code to build up, especially in local branches.
    • Small, frequent changes to master by our engineers probably cause a greater frequency of bumping into each other, but at the same time help avoid large-scale merge conflicts.
    • Scott lays out that remote branch tracking (in GitHub and "git pull") is excellent for monitoring what people are working on. 
      • We get the same through commit logs to master and high visibility into the state of master (i.e. what's in production). 
      • I'm not going to suggest that one is inherently better than the other. 
      • Both of us have models that have worked well within our own teams.