Matt Callanan's Blog: September 2011

Tuesday, September 27, 2011

Monday, September 26, 2011

Groovy Regex

http://groovy.codehaus.org/Regular+Expressions

// simple group demo
// You can also match a pattern that includes groups.  First create a matcher object,
// either using the Java API, or more simply with the =~ operator.  Then, you can index
// the matcher object to find the matches.  matcher[0] returns a List representing the
// first match of the regular expression in the string.  The first element is the string
// that matches the entire regular expression, and the remaining elements are the strings
// that match each group.
// Here's how it works:
def m = "foobarfoo" =~ /o(b.*r)f/
assert m[0] == ["obarf", "bar"]
assert m[0][1] == "bar"

XML & HTML Slurping with Groovy

XML Slurping String Content:
GPathResult feed = new XmlSlurper().parseText(source)
def rows = feed.FORM[0].TABLE[0].TR[1..-1]
// ...

Note: element names are case-sensitive

HTML Slurping with TagSoup:
slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser())
url = new URL("http://...")
url.withReader { reader ->
    def html = slurper.parse(reader)
    // ...
}

HTML Slurping with NekoHtml:
slurper = new XmlSlurper(new org.cyberneko.html.parsers.SAXParser())
new URL(url).withReader { reader ->
    def html = slurper.parse(reader)
    // ...
}

Friday, September 23, 2011

Cloning GitHub From Behind Proxy – Building Gradle From Source

How to clone a GitHub fork (Gradle in this case) from behind a proxy on a Windows machine without admin privileges…

[Windows] Download Portable Git from: http://code.google.com/p/msysgit/downloads/list?can=3
Follow instructions here: http://help.github.com/win-set-up-git/
Using git-bash: export http_proxy=http://proxyuser:passwd@proxy:port
Setup /h/.ssh/config as per: http://stackoverflow.com/questions/5103083/problem-of-testing-ssh-in-git-behind-proxy-on-window-7
```
ProxyCommand /c/dev/opt/git/bin/connect.exe -H proxy:port %h %p

Host github.com
User git
Port 22
Hostname github.com
IdentityFile "/<userhome>/.ssh/id_rsa"
TCPKeepAlive yes
PreferredAuthentications publickey
IdentitiesOnly yes

Host ssh.github.com
User git
Port 443
Hostname ssh.github.com
IdentityFile "/<userhome>/.ssh/id_rsa"
TCPKeepAlive yes
PreferredAuthentications publickey
IdentitiesOnly yes 
```
git clone https://mattcallanan@github.com/mattcallanan/gradle.git gradle
1. Provide proxy password
Add the following to <project-dir>/gradle.properties:
    systemProp.http.proxyHost=proxy
    systemProp.http.proxyPort=port
    systemProp.http.proxyUser=username
    systemProp.http.proxyPassword=password
    systemProp.http.nonProxyHosts=myrepo|localhost
If you have a hosted Maven repo, change gradle/build.gradle:
```
ivy {
    // artifactPattern('http://repo.jfrog.org/artifactory/gradle-plugins-snapshots/[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]')
    artifactPattern('http://myrepo/content/repositories/public/[organisation]/[module]/[revision]/[artifact]-[revision](-[classifier]).[ext]')
    ...
} 
```
[OPTIONAL] Proxy settings for For Ivy (not sure if this is necessary):

set ANT_OPTS=-Dhttp.proxyHost=proxy -Dhttp.proxyPort=port -Dhttp.proxyUserName=username -Dhttp.proxyPassword=password -Dhttps.proxyHost=proxy -Dhttps.proxyPort=port

It seems minlog is in between com.esotericsoftware and com.googlecode, i.e. it’s moved but there’s still some old dependencies on esotericsoftware. Download com.esotericsoftware:minlog:1.2 jar and pom, upload to your Maven repo, add repositories
```
repositories {
    ...
    mavenCentral()
    mavenRepo(url: 'http://myrepo/content/repositories/public')
    ...
}
```

Still working through some dependency issues attempting to build Gradle…

Ruby

http://mislav.uniqpath.com/poignant-guide/book/chapter-3.html

Constants are capitalised
Symbols are prefixed with a colon and are lightweight strings

Question marks and exclamation marks can be used in method names

Class methods are invoked with double colon '::' instead of a '.'

Global variables begin with '$'

Instance variables begin with '@' ("attribute")
Class variables begin with '@@' ("attribute all")

Code blocks can be surrounded with either {} or do-end
Block variables are surrounded by | and separated by commas. E.g.

{ |x,y| x + y }

Range. E.g. (1..3) ('a'..'z')

Hashes: {'a' => 'aardvark', 'b' => 'badger'}

Regexp: /ruby/, /[0-9]+/ and /^\d{3}-\d{3}-\d{4}/

Thursday, September 22, 2011

Reverse AJAX

http://www.ibm.com/developerworks/web/library/wa-reverseajax1/

Tuesday, September 20, 2011

Windows: Kill all Processes and Pipe to Clipboard

Kill all processes named "cmd.exe"
wmic process where name=”cmd.exe” delete

Pipe output to Clipboard
dir | clip

Monday, September 19, 2011

Programming in Scala learnings

"=" = "update"

val greetStrings = new Array[String](3)
greetStrings(0) = "Hello"

compiler transforms to:

greetStrings.update(0, "Hello")

Arrays are mutable, lists are immutable.

Operator Associativity
Operators are left associative unless they end with a colon. Therefore, in 1 :: twoThree, the :: method is invoked on twoThree, passing in 1, like this: twoThree.::(1).

List Append
Class List does offer an “append” operation —it’s written :+ and is explained in Chapter 24— but this operation is rarely used, because the time it takes to append to a list grows linearly with the size of the list, whereas prepending with :: takes constant time. Your options if you want to build a list efficiently by appending elements is to prepend them, then when you’re done call reverse; or use a ListBuffer, a mutable list that does offer an append operation, and when you’re done call toList.

Tuples
Access tuple fields with ._1, ._2, etc.
val pair = (99, "Luftballons")
Scala infers the type of the tuple to be Tuple2[Int, String] the Scala library only defines them up to Tuple22. You can’t access the elements of a tuple like the elements of a list, for example, with “pair(0)” because a list’s apply method always returns the same type, but each element of a tuple may be a different type: _1 can have one result type, _2 another, and so on. These _N numbers are one-based, instead of zero-based, because starting with 1 is a tradition set by other languages with statically typed tuples, such as Haskell and ML.

reduceLeft()
val longestLine = lines.reduceLeft((a, b) => if (a.length > b.length) a else b)
The reduceLeft method applies the passed function to the first two elements in lines, then applies it to the result of the first application and the next element in lines, and so on, all the way through the list. On each such application, the result will be the longest line encountered so far, because the passed function, (a, b) => if (a.length > b.length) a else b, returns the longest of the two passed strings

Unit
The following are all the same:

def add(b: Byte): Unit = {
    sum += b
}

def add(b: Byte): Unit = sum += b

def add(b: Byte) { sum += b }

def f(): Unit = "this String gets lost"

def g() { "this String gets lost too" }

def h() = { "this String gets returned!" } // h: ()String

Class vs Object
A class is a blueprint for objects. Once you define a class, you can create objects from the class blueprint with the keyword new.

Classes in Scala cannot have static members - instead, Scala
has singleton objects. A singleton object definition looks like a class definition,
except instead of the keyword class you use the keyword object.
When a singleton object shares the same name with a class, it is called that class’s companion object. You must define both the class and its companion object in the same source file. The class is called the companion class of the singleton object. A class and its companion object can access each other’s private members.

If you are a Java programmer, one way to think of singleton objects is as the home for any static methods you might have written in Java. You can invoke methods on singleton objects using a similar syntax. A singleton object is more than a holder of static methods, however. It is a first-class object. You can think of a singleton object’s name, therefore, as a “name tag” attached to the object.
The type is defined by the class not the object. However, singleton objects extend a superclass and can mix in traits. Given each singleton object is an instance of its superclasses and mixed-in traits, you can invoke its methods via these types, refer to it from variables of these types, and pass it to methods expecting these types.

One difference between classes and singleton objects is that singleton objects cannot take parameters, whereas classes can. Because you can’t instantiate a singleton object with the new keyword, you have no way to pass parameters to it. Each singleton object is implemented as an instance of a synthetic class referenced from a static variable, so they have the same initialization semantics as Java statics. (The name of the synthetic class is the object name plus a dollar sign. Thus the synthetic class for the singleton object named ChecksumAccumulator is ChecksumAccumulator$.) In particular, a singleton object is initialized the first time some code accesses it.

Standalone Object
A singleton object that does not share the same name with a companion class is called a standalone object. You can use standalone objects for many purposes, including collecting related utility methods together, or defining an entry point to a Scala application

Scala Application
Any standalone object with a main method of the proper signature (takes one parameter - an Array[String] - and has a result type of Unit) can be used as the entry point into an application.

object Blah {
def main(args:Array[String]) { println "hello" }
}

Implicit Imports
Scala implicitly imports members of packages:

java.lang
scala
scala.Predef (println, assert, etc)

You can name .scala files anything you want, no matter what Scala classes or code you put in them - recommend naming non-script .scala files after the classes they contain.

Can import methods from any object (not just singleton objects) with "import Object.method" (no need for "static" keyword).

A script must end in a result expression (not a definition).

fsc - Fast Scala Compiler
Starts up a daemon to eliminate JVM load time on every compile.
use fsc -shutdown to kill.

scala
The actual mechanism that the "scala" program uses to “interpret” a Scala source file is that it compiles the Scala source code to Java bytecodes, loads them immediately via a class loader, and executes them.

Application Trait
object Blah extends Application {
println "hello"
}

Application trait declares a main method of the appropriate signature, which your singleton object inherits, making it usable as a Scala application. The code between the curly braces is collected into a primary constructor of the singleton object, and is executed when the class is initialized.

Application Trait Shortcomings

you can’t use this trait if you need to access command-line arguments, because the args array isn’t available
because of some restrictions in the JVM threading model, you need an explicit main method if your program is multi-threaded
some implementations of the JVM do not optimize the initialization code of an object which is executed by the Application trait

Types
scala.Byte, scala.Short, scala.Int, scala.Long, and scala.Char are called integral types. The integral types plus Float scala.and scala.Double are called numeric types.

Operators
Can use any method in infix operator notation.
E.g. "abc" indexOf 'b' // Scala invokes "abc".indexOf('b')

Can even use it for methods that take more than one argument.
E.g. s indexOf ('o', 5) // Scala invokes s.indexOf(’o’, 5)

Prefix Operators
In prefix notation, the operand is to the right of the operator. Some examples of prefix operators are 2.0, !found, and ~0xFF. These prefix operators are a shorthand way of invoking methods. In this case, however, the name of the method has “unary_” prepended to the operator character. For instance,
Scala will transform the expression 2.0 into the method invocation “(2.0).unary_”.

The only identifiers that can be used as prefix operators are +, -,!, and ~. Thus, if you define a method named unary_!, you could invoke that method on a value or variable of the appropriate type using prefix operator notation, such as !p. But if you define a method named unary_*, you wouldn’t be able to use prefix operator notation, because * isn’t one of the four identifiers that can be used as prefix operators. You could invoke the method normally, as in p.unary_*, but if you attempted to invoke it via *p, Scala will parse it as if you’d written *.p, which is probably not what you had in mind!

Postfix Operators
Postfix operators are methods that take no arguments, when they are invoked without a dot or parentheses.

The convention is that you include parentheses if the method has side effects, such as println(), but you can leave them off if the method has no side effects, such as toLowerCase invoked on a String.

These are all the same:
s.toLowerCase()
s.toLowerCase
s toLowerCase

Conditional Initialisation
val filename =
if (!args.isEmpty) args(0)
else "default.txt"

Using a val instead of a var better supports equational reasoning. The introduced variable is equal to the expression that computes it, assuming that expression has no side effects

Using vals helps you safely make this kind of refactoring as your code evolves over time.
Look for opportunities to use vals. They can make your code both easier to read and easier to refactor.

It turns out that a value (and in fact, only one value) exists whose type is Unit. It is called the unit value and is written (). The existence of () is how Scala’s Unit differs from Java’s void.

Comparing values of type Unit and String using != will always yield true. Whereas in Java, assignment results in the value assigned, in this case a line from
the standard input, in Scala assignment always results in the unit value, (). Thus, the value of the assignment "line = readLine()" will always be () and never be "".

Challenge while loops in your code in the same way you challenge vars.

For Loops
The expression to the right of the <- symbol in a for expression can be any type that has certain methods, in this case foreach, with appropriate signatures.

Can iterate through Arrays, Ranges

Filters
filter: an if clause inside the for’s parentheses
for (
    file     if file.isFile
    if file.getName.endsWith(".scala")
) println(file)

If you add multiple <- clauses, you will get nested "loops."

If you prefer, you can use curly braces instead of parentheses to surround the generators and filters. One advantage to using curly braces is that you can leave off some of the semicolons that are needed when you use parentheses, because as explained in Section 4.2, the Scala compiler will not infer semicolons while inside parentheses.

Mid-stream variable bindings
for {
    file     if file.getName.endsWith(".scala")
    line     trimmed = line.trim
    if trimmed.matches(pattern)
} println(file +": "+ trimmed)

Producing a new collection
To generate a value to remember for each iteration you prefix the body of the for expression by the keyword yield. Each time the body of the for expression executes it produces one value. When the for expression completes, the result will include all of the yielded values contained in a single collection. The type of the resulting collection is based on the kind of collections processed in the iteration clauses.
For example, here is a function that identifies the .scala files and stores them in an array:
    def scalaFiles =
        for {
            file             if file.getName.endsWith(".scala")
        } yield file

Exceptions
"throw" is an expression that has a result type.
it is safe to treat a thrown exception as any kind of value whatsoever. Any context that tries to use the return from a throw will never get to do so, and thus no harm will come.
an exception throw has type Nothing
You can use a throw as an expression even though it will never actually evaluate to anything
E.g. One branch of an if computes a value, while the other throws an exception and computes Nothing. The type of the whole if expression is then the type of that branch which does compute something.

Catch pattern matching
    ...
} catch {
    case ex: FileNotFoundException => // Handle missing file
    case ex: IOException => // Handle other I/O error
}

Scala does not require you to catch checked exceptions, or declare them in a throws clause. You can declare a throws clause if you wish with the @throws annotation, but it is not required

try-catch-finally results in a value

Sunday, September 18, 2011

Add syntax highlighting to Blogger

http://www.cyberack.com/2007/07/adding-syntax-highlighter-to-blogger.html
http://geektalkin.blogspot.com/2009/11/embed-code-syntax-highlighting-in-blog.html

Example:


Your 'HTML' code goes here

Ruby/Rails Setup

Setting up Ruby on Windows

http://www.richardlawrence.info/2011/08/20/getting-started-with-ruby-cucumber-and-capybara-on-windows/

Download Ruby*.exe from: http://rubyinstaller.org/downloads/
Download devkit*.exe from: http://rubyinstaller.org/downloads/
Follow instructions here: https://github.com/oneclick/rubyinstaller/wiki/Development-Kit
ruby dk.rb init
ruby dk.rb install
gem install rdiscount --platform=ruby
ruby -rubygems -e "require 'rdiscount'; puts RDiscount.new('**Hello RubyInstaller**').to_html"

gem install bson_ext --platform=ruby

Cucumber
Download AnsiCon: http://adoxa.110mb.com/ansicon/index.html
Run c:\opt\ansi140\x64\ansicon -i

gem install cucumber
gem install capybara
gem install rspec
Install FireBug: http://www.getfirebug.com/

JRuby
http://jruby.org/download
http://stackoverflow.com/questions/2762086/getting-jruby-to-work-in-rubymine

Setting up Rails on Windows
gem install rails http://rubyonrails.org/download
To get around "file 'lib' not found" error: http://help.rubygems.org/discussions/problems/304-file-not-found-lib

Didn't work: gem install rdoc
Worked: gem install rails --no-ri --no-rdoc

Test: 

> rails new path/to/your/new/application 

> cd path/to/your/new/application 

> rails server

gem update --system

RubyMine
http://www.jetbrains.com/ruby/
http://stackoverflow.com/questions/2762086/getting-jruby-to-work-in-rubymine
RubyMine does not have any support for Java and does not recognize Java classes - JRuby is only supported as a target platform for running regular Ruby code. Full Java/Ruby interoperability is supported in IntelliJ IDEA with the Ruby plugin.

Demo: http://www.jetbrains.com/ruby/demos/rubymine_7_minutes.zip
Tools -> Load Rake Tasks
ERROR: uninitialized constant Rake::DSL
http://stackoverflow.com/questions/6085610/ruby-on-rails-and-rake-problems-uninitialized-constant-rakedsl

rake --version

    rake, version 0.8.7

bundle update rake

bundle show rake
    C:/opt/Ruby192/lib/ruby/gems/1.9.1/gems/rake-0.9.2
rake --version
    rake, version 0.8.7

gem uninstall rake -v=0.8.7

gem install rake -v=0.9.2 

rake --version

    rake, version 0.9.2

RubyMine Database Integration Rails data sources detected, but some additional drivers are required for RubyMine to connect to the database.

    Zentus SQLiteJDBC (v056)

http://localhost:3000/ideas

ExecJS::RuntimeError in Ideas#index 

Showing C:/dev/RubymineProjects/Notebook/app/views/layouts/application.html.erb where line #6 raised:

䌧尺坜湩潤獷屜祓瑳浥㈳屜獣牣灩⹴硥履‧獩渠瑯爠捥杯楮敺⁤獡愠⁮湩整湲污漠⁲硥整湲污挠浯慭摮ബ漊数慲汢⁥牰杯慲⁭牯戠瑡档映汩⹥਍

(in C:/dev/RubymineProjects/Notebook/app/assets/javascripts/ideas.js.coffee)

Server stack trace in RubyMine:

ActionView::Template::Error (䌧尺坜湩潤獷屜祓瑳浥㈳屜獣牣灩⹴硥履‧獩渠瑯爠捥杯楮敺⁤獡愠⁮湩整湲污漠⁲硥整湲污挠浯慭摮ബ漊数慲汢⁥牰杯慲⁭牯戠瑡档映汩⹥਍

  (in C:/dev/RubymineProjects/Notebook/app/assets/javascripts/ideas.js.coffee)):

    3: 

    4:   Notebook

    5:   <%= stylesheet_link_tag    "application" %>

    6:   <%= javascript_include_tag "application" %>

    7:   <%= csrf_meta_tags %>

    8: 

    9: 

  app/views/layouts/application.html.erb:6:in `_app_views_layouts_application_html_erb___633557058_33725952'

  app/controllers/ideas_controller.rb:7:in `index'

Can't read Chinese... giving up.

turning to...

Getting Started with Rails

http://guides.rubyonrails.org/getting_started.html

rails new blog

cd blog

rake db:create

rake -T

rails server

http://localhost:3000/

rails generate controller home index

del public\index.html

edit config\routes.rb

    root :to => "home#index"

Same Chinese error message... "Learn Mandarin" now on my todo list...

Turning to Linux Ubuntu

Setting up Ruby and Rails on Ubuntu

https://help.ubuntu.com/community/RubyOnRails

sudo apt-get install ruby-full build-essential

sudo apt-get aptitude

sudo aptitude install ruby build-essential libopenssl-ruby ruby1.8-dev

sudo apt-get install rubygems

sudo gem install rails --no-ri --no-rdoc  (Using same workaround for 'lib' problem)

echo "export PATH=/var/lib/gems/1.8/bin:$PATH" >> .bashrc

mkdir www; cd www

rails new blog

cd blog

sudo apt-get install libsqlite3-dev
sudo gem install sqlite3-ruby

sudo gem install uglifier

rails server

      /var/lib/gems/1.8/gems/execjs-1.2.7/lib/execjs/runtimes.rb:47:in `autodetect': Could not find a JavaScript runtime. See https://github.com/sstephenson/execjs for a list of available runtimes. (ExecJS::RuntimeUnavailable)

sudo gem install execjs

sudo gem install therubyracer

      same problem

http://stackoverflow.com/questions/6282307/rails-3-1-execjs-and-could-not-find-a-javascript-runtime

https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager

sudo apt-get install python-software-properties
sudo add-apt-repository ppa:chris-lea/node.js
sudo apt-get update
sudo apt-get install nodejs

rails server

    Success!

http://localhost:3000/

rails generate controller home index

del public\index.html

edit config\routes.rb

    root :to => "home#index"
rails generate scaffold Post name:string title:string content:text
rake db:migrate
vi app/views/home/index.html.erb

    <%h1>%Hello, Rails!

    <%/h1>%
<%= link_to "My Blog",  posts_path %> 

TextMate for GEdit: https://github.com/gmate/gmate

sudo apt-add-repository ppa:ubuntu-on-rails/ppa
sudo apt-get update
sudo apt-get install gedit-gmate

Monday, September 12, 2011

Logback

Reasons to prefer Logback over Log4j:http://logback.qos.ch/reasonsToSwitch.html

· Gradle dependency:

compile("ch.qos.logback:logback-classic:0.9.29") // transitively includes"ch.qos.logback:logback-core:0.9.29" and"org.slf4j:slf4j-api:1.6.2"

· Add logback.groovy to classpath (e.g. src/main/resources)

import ch.qos.logback.classic.encoder.PatternLayoutEncoder

import ch.qos.logback.core.status.OnConsoleStatusListener

import clover.org.apache.log4j.FileAppender

import static ch.qos.logback.classic.Level.DEBUG

// We highly recommended that you always add a status listener just after the last import statement and before all other statements

statusListener(OnConsoleStatusListener)

def bySecond = timestamp("yyyyMMdd'T'HHmmss")

appender("FILE", FileAppender) {

file = "log-${bySecond}.txt"

encoder(PatternLayoutEncoder) {

pattern = "%logger{35} - %msg%n"

}

root(DEBUG, ["FILE"])

· If Groovy is not available in the current classpath, logback will have this error:ERROR in ch.qos.logback.classic.LoggerContext[default] - Groovy classes are not available on the class path. ABORTING INITIALIZATION.

o (found by doing: logger.getLoggerContext().getStatusManager().getCopyOfStatusList())

· Or, use logback.xml

</rollingPolicy>

<!-- Prudent mode provides safety when multiple JVMs are writing to the log file at the same time.

See http://logback.qos.ch/manual/appenders.html#prudentWithRolling -->

<pattern>%level %date [%thread] %logger{40} [%file:%line] %msg%n</pattern>

</encoder>

</appender>

<pattern>%level %date [%thread] %logger{40} %msg%n</pattern>

</encoder>

</appender>

<appender-ref ref="FILE"/>

<appender-ref ref="STDOUT"/>

</root>

</configuration>

· If logback-test.xml exists it will be used instead of logback.xml (logback.groovy takes precedence over all xml)

Continuous Delivery vs Continuous Deployment

Continuous Deployment -http://timothyfitz.wordpress.com/2009/02/08/continuous-deployment/

· Every commit should be instantly deployed to production.

o Alex commits. Minutes later warnings go off that the cluster is no longer healthy.

o The failure is easily correlated to Alex's change and her change is reverted.

o Alex spends minimal time debugging, finding the now obvious typo with ease.

o Her changes still caused a failure cascade, but the downtime was minimal.

· Fail Fast – the closer a failure is to the point where it was introduced, the more data you have to correct for that failure.

o In code, Fail Fast means raising an exception on invalid input, instead of waiting for it to break somewhere later.

o In a software release process, Fail Fast means releasing undeployed code as fast as possible, instead of waiting for a weekly release to break.

· Continuous Deployment is simple: just ship your code to customers as often as possible. Maybe today that's weekly instead of monthly, but over time you'll approach the ideal and you'll see the incremental benefits along the way

Continuous Delivery -http://continuousdelivery.com/2010/08/continuous-delivery-vs-continuous-deployment/

Continuous delivery is about putting the release schedule in the hands of the business, not in the hands of IT

o Implementing continuous delivery means making sure your software is always production ready throughout its entire lifecycle – that any build could potentially be released to users at the touch of a button using a fully automated process in a matter of seconds or minutes

o This in turn relies on comprehensive automation of the build, test and deployment process, and excellent collaboration between everyone involved in delivery – developers, testers, DBAs, systems administrators, users, and the business

· So when can you say you're doing continuous delivery? I'd say it's when you could flip a switch to go to continuous deployment if you decided that was the best way to deliver value to your customers. In particular, if you can't release every good build to users, what does it mean to be "done" with a story? I think at least the following conditions must apply:

o You have run your entire test suite against the build containing the story. This validates that the story is delivering the expected business value, and that no regressions have been introduced in the process of developing it. In order to be efficient, that means having comprehensive automated tests at the unit, component and acceptance level.

o The story has beendemonstrated to customersfrom a production-like environment. Production-like means identical with production, within the bounds of reason. Even if you're deploying to an enormous cluster, you can use a technique like blue-green deployments to run a different version of your app in parallel on the production environment without affecting users.

o There are no obstacles to deploying to production. In other words, you could deploy the build to users using a fully automated process at the push of a button if you decided to. In particular, that means you've also tested it fulfills its cross-functional characteristics such as capacity, availability and security. If you're using an SOA or you have dependencies between your application and other systems, it means ensuring there are no integration problems.

Continuous Deployment Example -http://timothyfitz.wordpress.com/2009/02/10/continuous-deployment-at-imvu-doing-the-impossible-fifty-times-a-day/

· Continuously integrate (commit early and often). On commit automatically run all tests. If the tests pass deploy to the cluster. If the deploy succeeds, repeat.

· Our tests suite takes nine minutes to run (distributed across 30-40 machines). Our code pushes take another six minutes. Since these two steps are pipelined that means at peak we're pushing a new revision of the code to the website every nine minutes. That's 6 deploys an hour. Even at that pace we're often batching multiple commits into a single test/push cycle. On average we deploy new code fifty times a day

· Continuous Deployment means running all your tests, all the time

· The magic is in the scope, scale and thoroughness. It's a thousand test files and counting. 4.4 machine hours of automated tests to be exact. Over an hour of these tests are instances of Internet Explorer automatically clicking through use cases and asserting on behaviour, thanks to Selenium

· schema changes are done out of band. Just deploying them can be a huge pain. Doing an expensive alter on the master requires one-by-one applying it to our dozen read slaves (pulling them in and out of production traffic as you go), then applying it to the master's standby and failing over. It's a two day affair, not something you roll back from lightly. In the end we have relatively standard practices for schemas (a pseudo DBA who reviews all schema changes extensively) and sometimes that's a bottleneck to agility. If I started this process today, I'd probably invest some time in testing the limits of distributed key value stores which in theory don't have any expensive manual processes.

o My guess is that we could build a decent automated system by having a replicated stand by machine just for schema updates. You'd apply new schemas to the stand by and then automatically fail over to it. If you have to roll back, you just swap back the original machine, after which you have to rebuild the stand by from backups or some other expensive recovery option. You'd still need yet another safety net, in case of mysql crashes or other shenanigans, so you'd end up with a 2nd stand by machine (3 hosts per database role).

· A symlink is switched on a small subset of the machines throwing the code live to its first few customers. We have a fixed queue of 5 copies of the website on each frontend. We rsync with the "next" one and then when every frontend is rsync'd we go back through them all and flip a symlink over. i.e. rsync the code into a separate folder, then flip a symlink to make it live. e.g. the document root is /var/www/current, which is a symlink to /var/www/1. The next deployment rsyncs to /var/www/2 and flips the /var/www/current symlink to point to that when ready

Thursday, September 8, 2011

Git is simpler than you think

http://nfarina.com/post/9868516270/git-is-simpler

Git - Documentation Links: http://git-scm.com/documentation
Git - SVN Crash Course: http://git-scm.com/course/svn.html
Cheat Sheet: http://appletree.or.kr/quick_reference_cards/cvs-subversion-git/git-cheat-sheet.pdf
RefCard: http://refcardz.dzone.com/refcardz/getting-started-git

Thursday, September 1, 2011

97 Things Every Software Architect Should Know

http://97things.oreilly.com/wiki/index.php/97_Things_Every_Software_Architect_Should_Know_-_The_Book