- Automate Everything.
- Test Everything.
- Version Everything.
- Track and Plan Everything.
- Monitor and Audit Everything.
- Dashboard Everything.
Snippets and thoughts from a passionate software developer interested in all things DevOps, Continuous Delivery, Functional Programming, Distributed Systems. Also on Twitter as @mcallana.
Monday, April 16, 2012
IBM: 6 Ways for Enterprises to Adopt DevOps
Sunday, April 15, 2012
CouchDB on Amazon EC2
https://github.com/iriscouch/build-couchdb (from http://wiki.apache.org/couchdb/Installing_on_Ubuntu)
- Setup new instance: Ubuntu 11.10
- ssh to new box
- run the following to build couchdb from source (packages available via "sudo aptitude install couchdb" are too old):
sudo apt-get install git
sudo apt-get install make gcc zlib1g-dev libssl-dev rake
git clone git://github.com/iriscouch/build-couchdb
cd build-couchdb
git submodule init
git submodule update
rake
couchdb
ubuntu@ip-10-128-141-191:~/build-couchdb$ couchdb
Apache CouchDB 1.0.1 (LogLevel=info) is starting.
=CRASH REPORT==== 19-Apr-2012::11:28:09 ===
crasher:
initial call: couch_event_sup:init/1
pid: <0.80.0>
registered_name: []
exception exit: {{badmatch,
{'EXIT',
{{badmatch,{error,eacces}},
[{couch_log,init,1},
{gen_event,server_add_handler,4},
{gen_event,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}}},
[{couch_event_sup,init,1},
{gen_server,init_it,6},
{proc_lib,init_p_do_apply,3}]}
in function gen_server:init_it/6
ancestors: [couch_primary_services,couch_server_sup,<0.32.0>]
messages: []
links: [<0.79.0>,<0.6.0>]
dictionary: []
trap_exit: false
status: running
heap_size: 377
stack_size: 24
reductions: 116
neighbours:
=SUPERVISOR REPORT==== 19-Apr-2012::11:28:09 ===
Supervisor: {local,couch_primary_services}
Context: start_error
Reason: {{badmatch,{'EXIT',{{badmatch,{error,eacces}},
[{couch_log,init,1},
{gen_event,server_add_handler,4},
{gen_event,handle_msg,5},
{proc_lib,init_p_do_apply,3}]}}},
[{couch_event_sup,init,1},
{gen_server,init_it,6},
{proc_lib,init_p_do_apply,3}]}
Offender: [{pid,undefined},
{name,couch_log},
{mfargs,{couch_log,start_link,[]}},
{restart_type,permanent},
{shutdown,brutal_kill},
{child_type,worker}]
=SUPERVISOR REPORT==== 19-Apr-2012::11:28:09 ===
Supervisor: {local,couch_server_sup}
Context: start_error
Reason: shutdown
Offender: [{pid,undefined},
{name,couch_primary_services},
{mfargs,{couch_server_sup,start_primary_services,[]}},
{restart_type,permanent},
{shutdown,infinity},
{child_type,supervisor}]
=CRASH REPORT==== 19-Apr-2012::11:28:09 ===
crasher:
initial call: application_master:init/4
pid: <0.31.0>
registered_name: []
exception exit: {bad_return,
{{couch_app,start,
[normal,
["/etc/couchdb/default.ini",
"/etc/couchdb/local.ini"]]},
{'EXIT',
{{badmatch,{error,shutdown}},
[{couch_server_sup,start_server,1},
{application_master,start_it_old,4}]}}}}
in function application_master:init/4
ancestors: [<0.30.0>]
messages: [{'EXIT',<0.32.0>,normal}]
links: [<0.30.0>,<0.7.0>]
dictionary: []
trap_exit: true
status: running
heap_size: 987
stack_size: 24
reductions: 115
neighbours:
=INFO REPORT==== 19-Apr-2012::11:28:09 ===
application: couch
exited: {bad_return,{{couch_app,start,
[normal,
["/etc/couchdb/default.ini",
"/etc/couchdb/local.ini"]]},
{'EXIT',{{badmatch,{error,shutdown}},
[{couch_server_sup,start_server,1},
{application_master,start_it_old,4}]}}}}
type: temporary
- sudo apt-get update && sudo apt-get -y upgrade
- sudo apt-get install erlang libmozjs-dev libicu-dev libcurl4-gnutls-dev make subversion automake autoconf libtool help2man
- Fetch, build, and run CouchDB:
- svn checkout http://svn.apache.org/repos/asf/couchdb/trunk couchdb
- cd couchdb
- ./bootstrap
- ./configure
- Is the Mozilla SpiderMonkey library installed?
- http://tommy.chheng.com/2009/07/10/installing-couchdb-on-ubuntu-problems-and-fixes/
- http://wiki.apache.org/couchdb/Installing_SpiderMonkey
sudo add-apt-repository ppa:launchpad/ppa sudo apt-get install libmozjs-dev
- ./configure
- make && sudo make install
- sudo adduser --system --home /usr/local/var/lib/couchdb --no-create-home --shell /bin/bash --group --gecos 'CouchDB account' couchdb
- sudo chown -R couchdb.couchdb /usr/local/var/{lib,log}/couchdb
- (Optional) Enable direct web access. NOTE: This step makes your CouchDB instance available for everyone. See the FaQ for some (but not all) security options.
- sudo vim /usr/local/etc/couchdb/local.ini
- Search for "bind_address", uncomment it, and change it to 0.0.0.0
- Save and exit
- (Alternative option) Set up port forwarding to access your DB, for example with SSH.
- ssh -L 5984:localhost:5984 -l ubuntu <public DNS name>
- Leave that session open as long as you need the proxy to work
- Your new DB URL will be http://127.0.0.1:5984/ instead.
- sudo -i -u couchdb couchdb
- Test it by going to http://<public dns name>:5984/_utils/
Saturday, April 14, 2012
BusinessInsider: 8 Mind Games That Recruiters Play During Interviews
8 Mind Games That Recruiters Play During Interviews
BusinessInsider: 17 Tips On Becoming A Charismatic Leader
Friday, April 13, 2012
Deploying Grails app to CloudFoundry
http://blog.springsource.com/2011/04/12/one-step-deployment-with-grails-and-cloud-foundry/
http://docs.cloudfoundry.com/tools/vmc/installing-vmc.html
http://grails-plugins.github.com/grails-cloud-foundry/docs/manual/index.html
- grails install-app cloud-foundry
- Create /<homedir>/.grails/settings.groovy:
- grails.plugin.cloudfoundry.username = "<email>"
- grails.plugin.cloudfoundry.password = "<passwd>"
- grails cf-info
- VMware's Cloud Application Platform
For support visit http://support.cloudfoundry.com
Target: http://api.cloudfoundry.com (v0.999)
User: <email>
Usage: Memory (1.0G of 2.0G total)
Services (1 of 16 total)
Apps (2 of 20 total) - grails prod cf-push
- choose app name
- say yes to MySQL
- say no to postresql
- grails cf-services
- ============== System Services ==============
+------------+---------+---------------------------------------+
| Service | Version | Description |
+------------+---------+---------------------------------------+
| mysql | 5.1 | MySQL database service |
| postgresql | 9.0 | PostgreSQL database service (vFabric) |
| mongodb | 1.8 | MongoDB NoSQL store |
| redis | 2.2 | Redis key-value store service |
| rabbitmq | 2.4 | RabbitMQ message queue |
+------------+---------+---------------------------------------+
=========== Provisioned Services ============
+---------------+---------+
| Name | Service |
+---------------+---------+
| mysql-f9bee59 | mysql |
+---------------+---------+ - grails cf-apps
- +-------------------------+----+---------+------------------------------------------+---------------+
| Application | # | Health | URLs | Services |
+-------------------------+----+---------+------------------------------------------+---------------+
| blog | 1 | RUNNING | gailsblog.cloudfoundry.com | mysql-f9bee59 |
+-------------------------+----+---------+------------------------------------------+---------------+
| cloudfoundry_grailsblog | 1 | RUNNING | cloudfoundry_grailsblog.cloudfoundry.com | |
+-------------------------+----+---------+------------------------------------------+---------------+ - grails cf-map grailsblog.cloudfoundry.com
- cf-apps
- +-------------------------+----+---------+---------------------------------------------------------+---------------+
| Application | # | Health | URLs | Services |
+-------------------------+----+---------+---------------------------------------------------------+---------------+
| blog | 1 | RUNNING | gailsblog.cloudfoundry.com, grailsblog.cloudfoundry.com | mysql-f9bee59 |
+-------------------------+----+---------+---------------------------------------------------------+---------------+
| cloudfoundry_grailsblog | 1 | RUNNING | cloudfoundry_grailsblog.cloudfoundry.com | |
+-------------------------+----+---------+---------------------------------------------------------+---------------+ - grails cf-unmap gailsapp.cloudfoundry.com
- grails cf-update-instances 3
- Scaled 'blog' up to 3 instances.
- grails prod cf-update
- Updating Application: Checking for available resources: OK Processing resources: OK Packing application: OK Uploading (28K): OK
Application 'blog' stopped.
Trying to start Application: 'blog'....Application 'blog' started at http://grailsblog.cloudfoundry.com - grails cf-delete-app
Wednesday, April 11, 2012
Grails: Accessing paths, properties and URLs
In Service classes:
· ApplicationHolder.application.metadata.'relative.dir' --> returns "relative.dir" property value from application.properties
· ApplicationHolder.application.getMainContext().getResource('/').getFile().getAbsolutePath() --> returns the local root directory of the webapp on the server
· [inject] LinkGenerator grailsLinkGenerator (literally)
o grailsLinkGenerator.serverBaseURL + "/path" --> http://localhost:9999/context/path
o grailsLinkGenerator.link(uri: "/path", absolute: true) --> same as above (see javadoc on link())
In gsp:
· grailsApplication.metadata.'relative.reports.dir' --> returns "relative.dir: property value from application.properties
· request.getSession().getServletContext().getRealPath("/") --> returns the local root directory of the webapp on the server
· request.getSession().getServletContext().getRealPath("/path") --> returns /path relative to the local root directory of the webapp on the server
Tuesday, April 10, 2012
Monday, April 9, 2012
Kanban vs Scrum, Shock Therapy
Difference between Kanban and Scrum: http://www.valueflowquality.com/the-difference-between-kanban-and-scrum-in-one-word/
Jeff Sutherland Shock Therapy: http://scrum.jeffsutherland.com/2012/01/scrum-shock-therapy-how-to-change-teams.html
· http://danrough.net/2009/06/01/one-piece-flow-one-piece-at-a-time/
Sunday, April 8, 2012
Saturday, April 7, 2012
Create Grails app
- create-app blog
- create-domain-class post
- Edit Post.groovy:
- package blog
class Post {
String title
String content
Date date
static constraints = { }
} - generate-all blog.Post
- create-domain-class comment
- Add comments to Post.groovy:
- class Post {
String subject
String content
Date date
static hasMany = [comments: Comment]
static constraints = {
// Ordering constraint
subject()
content()
date()
comments() }
} - generate-all *
- add to views/post/show.gsp
- <g:link class="comment" action="comment" id="${postInstance?.id}"><g:message code="default.button.comment.label" default="Comment" /></g:link>
Wednesday, April 4, 2012
OWASP Agile web security: Evil Stories & Top 10 Web Security tips
- Example #1. "As a hacker, I can send bad data in URLs, so I can access data and functions for which I'm not authorized."
- Example #2. "As a hacker, I can send bad data in the content of requests, so I can access data and functions for which I'm not authorized."
- Example #3. "As a hacker, I can send bad data in HTTP headers, so I can access data and functions for which I'm not authorized."
- Example #4. "As a hacker, I can read and even modify all data that is input and output by your application."
1. Injection
2. Cross-Site Scripting (XSS)
3. Broken Authentication and Session Management
4. Insecure Direct Object References.5. Cross-Site Request Forgery (CSRF)
6. Security Misconfiguration
7. Insecure Cryptographic Storage
8. Failure to Restrict URL Access
9. Insufficient Transport Layer Protection
10. Unvalidated Redirects and Forwards
Dwight Merriman: MongoDB, CAP theorem
Dwight Merriman talks with Robert about the emerging NoSQL movement, the three types of non-relational data stores, Brewer’s CAP theorem, the weaker consistency guarantees that can be made in a distributed database, document-oriented data stores, the data storage needs of modern web applications, and the open source MongoDB.
- 10-gen
- NoSQL – the Shift to a Non-Relational World by Dwight Merriman
- A series of blog posts on consistency in a distributed database (March 26 to April 15, 2010)
- Part 1 - Introduction and CAP
- Part 2 - Eventual Consistency
- Part 3 - Network Partitions
- Part 4 - Multi Data Center
- Part 5 - Multi Writer Eventual Consistency
- Part 6 - Consistency Chart
- Slides and video from MongoSF 2010
- Brewer’s presentation on the CAP theorem
Sunday, April 1, 2012
Silos
http://www.resorgs.org.nz/pubs/Silos.pdf
- "It is helpful to think of silos existing in degrees along a continuum,
- i.e. silo existence isn’t a binary, black and white matter.
- Most organisations and organisational units will have some silo characteristics at any given time.
- But silos become problematic when they are develop to a point that good performance suffers.
- The challenge is to indentify silos that are problematic or that threaten to become so, and to identify and take remedial steps."
Linked from: http://www.jedi.be/blog/2012/01/08/devops-a-wicked-problem/
