UberConf 2012 notes

Complexity of Complexity

by Ken Sipe - Wednesday 3:15 PM

 

Essential complexity = plane 
Accidental complexity = Indian power lines 
V1 user does not like 
V2 overly complex 

Everything is diffing mental models 
Size of team matters adding ppl is counterproductive 

Application of knowledge, level of experience 

Dreyfus model 

Experts create recipes for novices and advanced beginners 

Brian Goetz, extending Java with backward comp 

1952 
Make coding comprehensible 

Test reviews more important than code reviews 

2.0 - 0.9 = 0.89999999 
Have to have humans working on code that understand this 

Possible to create obj hierarchy where legal equals is not possible 

Must use most powerful language at disposal 

Increased flexibility = increased complexity 

ESB - complexity does not go away, need to be smart 
Internals are not standardized 

CAP Availability = caching 
"if u can't split it u can't scale it" shoelz eBay 

Level 3 = competent

Super observer sees all consistency - not necessary? 

What is core business? Can solve with fault tolerant hardware instead of sware complexity? 

Context... Hard problem 
Google talk hardest problem is social graph (friend status) 
IO throughput, CPU throughput, two solutions 

Request, asynchronous mq, response 
Don't cache preemptively

MongoDB: Scaling Web Applications

by Ken Sipe - Wednesday 5:00 PM

 

Sharing 3x3 is magic 
3 shards, replicate sets

Mobile Performance Tips n' Tricks

by Pratik Patel - Wednesday 5:00 PM

 

Blaze.io/testing

Visualizing Data on the Web

by Brian Sletten - Wednesday 8:30 PM

 

Preattentive 200ms 

Stevens power law ppl find length easier to perceive than area - pie charts are harder to understand 

Show the data - don't accentuate, "lie factor" 

Max ideas presented min time, min ink, min space 
"Chart junk" 

Spark lines, word-size graphics, eg glucose level inline with text 

Jquery sparkling 
Rgraph 
Highcharts commercial eg Gatling 
Raphael 
D3 data drives with little code, based on protovis 
- multivariate relationships 
- parallel coordinates 100g 
Tangle worrydream, Bret Victor 
- interactive document 
exhibit, simile-widgets 

Flowingdata.com "visualize this" book 

Many eyes IBM 
Selection.datavisualization.ch 
inmaps.linkedinlabs.com

Clojure Workshop

by Neal Ford - Thursday 9:00 AM

Watch Rich Hickey videos 

Dynamic 
Strongly typed 
Functional 
High-performance 
Lisp 

Homoiconic 
Only references are mutable 

Regex #"" 
Symbols 
Keywords special symbols :foo ::foo, constant intern strings 
Data literals 
-List. Singly linked insert at front () 
-Vector. Indexed, insert at rear [] 
-Map key/value {:} 
-Set #{} 

Few data structures many functions 

Seq interface

Convert Groovy NodeChild xml element to XML string

new groovy.xml.StreamingMarkupBuilder().bindNode(nodeChild) as String

ElasticSearch terminology

Relational DB ⇒ elasticsearch

database ⇒ index
table ⇒ type
row ⇒ document
column ⇒ field
schema ⇒ mapping
index ⇒ everything is indexed
SQL ⇒ query DSL

Kibana: A brower based log analysis tool for Logstash and ElasticSearch

http://rashidkpc.github.com/Kibana/

You have logs. Billions of lines of data. You shipped, dated it, parsed it and stored it. Now what do you do with it? Now you make sense of it. Kibana helps you do that. Kibana is an alternative browser based interface for Logstash and ElasticSearch that allows you to efficiently search, graph, analyze and otherwise make sense of a mountain of logs. 

Installing Clojure

• Install Clojure
• Download clojure-1.4.0.zip from http://clojure.org/downloads
• Extract
• Add c:\opt\clojure-1.4.0 to PATH
• Create clojure.bat file in c:\opt\clojure-1.4.0 with the following:
• java -cp C:\opt\clojure-1.4.0\clojure-1.4.0.jar clojure.main
• Install Counterclockwise in Eclipse:
• Add following update site to Eclipse: http://ccw.cgrand.net/updatesite/
• Install Leiningen (Clojure build tool)
• Download Leiningen: git clone https://github.com/technomancy/leiningen/
• Run leiningen\bin\lein.bat self-install 
• Add leiningen to PATH
• Download Clojure Koans: https://github.com/functional-koans/clojure-koans/downloads
• Extract
• script\run

Neo4j

Download from: http://neo4j.org/download_a/
Start server
Connect: http://localhost:7474/webadmin
or
http://localhost:7474/db/data/

bhenerey: Ideal Ops List

https://gist.github.com/2777306

Ruby: Printing the current stacktrace

begin

  Math.sqrt(-1)  # any error will do
rescue => e
  puts e.inspect
  puts e.backtrace
end


http://www.ruby-forum.com/topic/125107

Geppetto: Puppet IDE

http://puppetlabs.com/blog/geppetto-a-puppet-ide/ (old but good example screenshots)
https://github.com/cloudsmith/geppetto
http://cloudsmith.github.com/geppetto/


"As usual, the team ignored Management and did whatever they damn well pleased, working on Geppetto in secret through 15 long months of nights, weekends and vacations"
http://blog.cloudsmith.com/?p=803

Lifehacker: Focussing goals

http://lifehacker.com/5912971/focus-your-ambitions-with-the-lifehacker-hierarchy-of-goals

THORIN KLOSOWSKI

Level 1: The Primary Goals

• Base of all other goals - one or two things you aspire to do before you die - that will truly matter to you in 20 years. 
• Everything above this bottom level should help you to one day complete these goals. 
• Should only have two or three high stakes goals

Level 2: Long Term Goals

• Major goals that are required to get to the primary goals. 
• E.g. Sustainable habits you need to form over the years 
• E.g. Achievements you want to reach

Level 3: Short Term Goals

• Weeks or months out. 
• Short term, not short-sighted. 
• What you can do now to make primary goal happen?

Level 4: Recurring Goals

• What you want to do daily/weekly/monthly regardless of what else is going on. 
• These aren't quite the same as short-term goals because they're to form a habit. 
• Ask yourself what you need to do on a daily basis to make that happen in the long run.

Level 5: Immediate Goals

• Goals and to-dos that you can and want to accomplish right this second. 
• Ever-changing but necessary part of pyramid - allows you to measure your daily duties to see how they have an affect on overall life goals. 
• Could be as simple as making a phone call. 
• How do they affect other aspects of the pyramid both positively and negatively?

Weed Out Junk and Accomplish Your Goals

• "Life consists of what you pay attention to" - you can structure your goals the same way.
• When you have too many goals conflicting with each other your attention is shifted too often. 
• Trim away junk goals to get things done and find an actionable path.
• The goal of the pyramid is to ensure that every aspect of your goals work together:
• Can see where your ideas fail and aren't coalescing.

1. Start at the bottom of your pyramid and draw lines up through goals that match each other. The line should move through each level and hit one or two different goals along the way.
2. Do this with all your goals moving upwards through the pyramid.
3. When you're done you'll probably have a few outliers scattered about. 
1. Ask yourself: Why do I want this? 
2. Does this relate to anything else I want? 
3. If you don't have a good answer, cut them from the list. 
4. If you want to keep goals then focus them to help you with another goal.
4. Go back through your levels and see what goals you can outsource to other people.

Focus only on the goals that matter, break them into smaller steps, and start work immediately

HTTP Status Decision Graph

Just saw a demo of webmachine's decision graph by @TheColonial:
http://blog.beerriot.com/2009/03/20/http-decision-graph-comes-to-life/

Inspired by:
http://http-headers-status.googlecode.com/files/http-headers-status%20v3%20draft.png

Creating AWS CloudFormation Puppet cluster

http://puppetlabs.com/blog/using-cloudformation-to-build-out-fully-functional-stacks-of-puppet-enterprise/

https://github.com/puppetlabs/puppetlabs-cloudformation/blob/master/README.markdown

http://forge.puppetlabs.com/puppetlabs/cloudformation

Puppet Learning VM

http://docs.puppetlabs.com/learning/

Download latest VMX from http://downloads.puppetlabs.com/learning/

Follow instructions:
http://docs.puppetlabs.com/learning/#get-equipped

Log in as root, with the password puppet

Success

To view the Puppet Enterprise web console, navigate to https://(your VM’s IP address).
(Note the https)
Log in as puppet@example.com, with the password learningpuppet.

Doesn't work with LearningVM v2.0.1

Downloading LearningVM v2.5.1...

It works!

The Shirky Principle

"Institutions will try to preserve the problem to which they are the solution"

Hadoop & Amazon

http://wiki.apache.org/hadoop/AmazonEC2

http://www.daltonclark.com/blog/2011/03/30/cloudera-hadoop-amazon-howto/