Thursday, April 19, 2012

MongoDB on AWS (Marketplace)

With Amazon's AWS Marketplace it's very easy to have an EC2 instance up and running with MongoDB already installed.

Go to AWS MongoDB in the AWS Marketplace: https://aws.amazon.com/marketplace/pp/B007IBMJPI/ref=gtw_msl_image?ie=UTF8&pf_rd_r=017SRM6GKB2SRH73AWGW&pf_rd_m=A33KC2ESLMUT5Y&pf_rd_t=101&pf_rd_i=awsmp-gateway-1&pf_rd_p=1355801782&pf_rd_s=center-3

Choose Region: e.g. Singapore
Continue

Version: 2.0.4 release 04/03/2012
Region: Asian Pac (Singapore)

EC2 Instance Type: Standard Micro (t1.micro), 613MB EBS storage, 64-bit
Key Pair: Create a new key pair in AWS Mgmt Console in the same region

Agree and proceed...
1 minute later:

An instance of this software is now deploying on EC2.

  • You'll receive an email shortly to confirm your subscription.
  • If you would like to check the progress of this deployment, go to the AWS Management Console 
  • Once the software is ready (at least 2-3 minutes), you can access the software via Your Software.
  • Now that you are subscribed, you can launch the AMIs containing this software with 1-click Launch, or via the EC2 console, EC2 APIs, or with other AWS management tools.
Right-click connect from AWS mgmt console
Login as ec2-user instead of root
There are 2 security update(s) out of 19 total update(s) available
sudo yum update

mongo
    MongoDB shell version: 2.0.4
    connecting to: test
    Thu Apr 19 12:29:36 Error: couldn't connect to server 127.0.0.1 shell/mongo.js:84
    exception: connect failed
 
http://www.mongodb.org/display/DOCS/Starting+and+Stopping+Mongo
 
sudo mkdir -p /data/db
sudo chown ec2-user /data/db
 
mongod --help for help and startup options
Thu Apr 19 13:40:37 [initandlisten] MongoDB starting : pid=19836 port=27017 dbpath=/data/db/ 64-bit host=ip-10-128-75-170
Thu Apr 19 13:40:37 [initandlisten] db version v2.0.4, pdfile version 4.5
Thu Apr 19 13:40:37 [initandlisten] git version: 329f3c47fe8136c03392c8f0e548506cb21f8ebf
Thu Apr 19 13:40:37 [initandlisten] build info: Linux ip-10-110-9-236 2.6.21.7-2.ec2.v1.2.fc8xen #1 SMP Fri Nov 20 17:48:28 EST 2009 x86_64 BOOST_LIB_VERSION=1_41
Thu Apr 19 13:40:37 [initandlisten] options: {}
Thu Apr 19 13:40:37 [initandlisten] journal dir=/data/db/journal
Thu Apr 19 13:40:37 [initandlisten] recover : no journal files present, no recovery needed
Thu Apr 19 13:40:38 [initandlisten] preallocateIsFaster=true 6.16
Thu Apr 19 13:40:38 [initandlisten] preallocateIsFaster=true 7.12
Thu Apr 19 13:40:40 [initandlisten] preallocateIsFaster=true 7.76
Thu Apr 19 13:40:40 [initandlisten] preallocateIsFaster check took 3.192 secs
Thu Apr 19 13:40:40 [initandlisten] preallocating a journal file /data/db/journal/prealloc.0
Thu Apr 19 13:42:09 [initandlisten] waiting for connections on port 27017
Thu Apr 19 13:42:09 [websvr] admin web console waiting for connections on port 28017
Thu Apr 19 13:43:09 [clientcursormon] mem (MB) res:15 virt:612 mapped:0
 
Browse to http://**.ap-southeast-1.compute.amazonaws.com:28017/
The connection has timed out
The server at **.ap-southeast-1.compute.amazonaws.com is taking too long to respond.

 
sudo chmod 777 /var/log
mongod --fork --logpath /var/log/mongodb.log --logappend
 
_______________________________

There's a great tutorial introducing the basic MongoDB commands here: http://www.mongodb.org/display/DOCS/Tutorial

mongo
> use mydb
> j = { name : "mongo" };
{"name" : "mongo"}
>
> t = { x : 3 };
{ "x" : 3  }
>
> db.things.save(j);
> db.things.save(t);
> db.things.find();
{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }
{ "_id" : ObjectId("4c2209fef3924d31102bd84b"), "x" : 3 }
> 
> for (var i = 1; i <= 20; i++) db.things.save({x : 4, j: i});                              
> db.things.find();
{ "_id" : ObjectId("4f91383aba111a1e6815b62d"), "name" : "mongo" }
{ "_id" : ObjectId("4f913840ba111a1e6815b62e"), "x" : 3 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638f9"), "x" : 4, "j" : 1 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638fa"), "x" : 4, "j" : 2 } 
...
{ "_id" : ObjectId("4f913a34d8aa5ac12a163909"), "x" : 4, "j" : 17 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a16390a"), "x" : 4, "j" : 18 }
has more
> 
> it
{ "_id" : ObjectId("4f913a34d8aa5ac12a16390b"), "x" : 4, "j" : 19 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a16390c"), "x" : 4, "j" : 20 }
> it
no cursor
> 
> 
> var cursor = db.things.find();
> 
> while (cursor.hasNext()) printjson(cursor.next());                                        
{ "_id" : ObjectId("4f91383aba111a1e6815b62d"), "name" : "mongo" }
{ "_id" : ObjectId("4f913840ba111a1e6815b62e"), "x" : 3 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638f9"), "x" : 4, "j" : 1 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638fa"), "x" : 4, "j" : 2 }
...
{ "_id" : ObjectId("4f913a34d8aa5ac12a16390b"), "x" : 4, "j" : 19 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a16390c"), "x" : 4, "j" : 20 }
> 
> // same: db.things.find().forEach(printjson);
>
> printjson(cursor[4]);                                                                     
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638fb"), "x" : 4, "j" : 3 }
> 
> var arr = db.things.find().toArray();
> arr[5];
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638fc"), "x" : 4, "j" : 4 }
> 
> // SELECT * FROM things WHERE name="mongo"
> db.things.find({name:"mongo"}).forEach(printjson);
{ "_id" : ObjectId("4c2209f9f3924d31102bd84a"), "name" : "mongo" }
>
> // SELECT * FROM things WHERE x=4
> db.things.find({x:4}).forEach(printjson);
{ "_id" : ObjectId("4c220a42f3924d31102bd856"), "x" : 4, "j" : 1 }
{ "_id" : ObjectId("4c220a42f3924d31102bd857"), "x" : 4, "j" : 2 }
...
{ "_id" : ObjectId("4c220a42f3924d31102bd868"), "x" : 4, "j" : 19 }
{ "_id" : ObjectId("4c220a42f3924d31102bd869"), "x" : 4, "j" : 20 }
>
> // SELECT j FROM things WHERE x=4
> db.things.find({x:4}, {j:true}).forEach(printjson);
{ "_id" : ObjectId("4c220a42f3924d31102bd856"), "j" : 1 }
{ "_id" : ObjectId("4c220a42f3924d31102bd857"), "j" : 2 }
...

{ "_id" : ObjectId("4c220a42f3924d31102bd868"), "j" : 19 }
{ "_id" : ObjectId("4c220a42f3924d31102bd869"), "j" : 20 }
>
> printjson(db.things.findOne({name:"mongo"}));
{ "_id" : ObjectId("4f91383aba111a1e6815b62d"), "name" : "mongo" }
> 
> printjson(db.things.findOne({x:4}));                                                      
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638f9"), "x" : 4, "j" : 1 }
> 
> db.things.find().limit(3);                                                                
{ "_id" : ObjectId("4f91383aba111a1e6815b62d"), "name" : "mongo" }
{ "_id" : ObjectId("4f913840ba111a1e6815b62e"), "x" : 3 }
{ "_id" : ObjectId("4f913a34d8aa5ac12a1638f9"), "x" : 4, "j" : 1 }
> 


______________________________________

Notes:
Mongo is a full JavaScript shell, so any JavaScript function, syntax, or class can be used in the shell.  In addition, MongoDB defines some of its own classes and globals (e.g., db).  You can see the full API at http://api.mongodb.org/js/.


http://www.mongodb.org/display/DOCS/BSON

http://www.mongodb.org/display/DOCS/SQL+to+Mongo+Mapping+Chart

MySQL term Mongo term/concept
database database
table collection
index index
row BSON document
column BSON field
join embedding and linking
primary key _id field
group by aggregation

http://www.mongodb.org/display/DOCS/Manual

Mongo 1-* Database 1-* Collection 1-* Document (BSON)
document-oriented, dynamic schema

Documents are limited to 16MB to save bandwidth for queries (and keep schema sizes ideal).  For larger document sizes, use GridFS.

http://www.mongodb.org/display/DOCS/GridFS
GridFS is a specification for storing large files in MongoDB.

http://www.mongodb.org/display/DOCS/GridFS+Tools
mongofiles is a tool for manipulating GridFS from the command line
mongofiles [list | put | get]

http://www.mongodb.org/display/DOCS/Data+Types+and+Conventions
Mongo uses special data types in addition to the basic JSON types of string, integer, boolean, double, null, array, and object. These types include date, object id, binary data, regular expression, and code.

http://www.mongodb.org/display/DOCS/Tutorial