Fancy UI

All About Crazy IT Stuff

Archive for the ‘Ubuntu’ Category

Installing MongoDB 2.0 on Ubuntu 11.10

with 13 comments

Ubuntu 11.10 ships with an older version of MongoDB, in this article I’m going to show how to install properly the latest version MongoDB which the 2.0.1.

First you have to download the 32bit or 64bit Linux binaries from here and unzip the contents to /usr/local.

cd /tmp
wget http://fastdl.mongodb.org/linux/mongodb-linux-i686-2.0.1.tgz
sudo tar -zxf /tmp/mongodb-linux-i686-2.0.1.tgz -C /usr/local

Then you need to configure some symbolic links.

sudo ln -s /usr/local/mongodb-linux-i686-2.0.1 /usr/local/mongodb
sudo ln -s /usr/local/mongodb/bin/bsondump /usr/local/bin/bsondump
sudo ln -s /usr/local/mongodb/bin/mongo /usr/local/bin/mongo
sudo ln -s /usr/local/mongodb/bin/mongod /usr/local/bin/mongod
sudo ln -s /usr/local/mongodb/bin/mongodump /usr/local/bin/mongodump
sudo ln -s /usr/local/mongodb/bin/mongoexport /usr/local/bin/mongoexport
sudo ln -s /usr/local/mongodb/bin/mongofiles /usr/local/bin/mongofiles
sudo ln -s /usr/local/mongodb/bin/mongoimport /usr/local/bin/mongoimport
sudo ln -s /usr/local/mongodb/bin/mongorestore /usr/local/bin/mongorestore
sudo ln -s /usr/local/mongodb/bin/mongos /usr/local/bin/mongos
sudo ln -s /usr/local/mongodb/bin/mongosniff /usr/local/bin/mongosniff
sudo ln -s /usr/local/mongodb/bin/mongostat /usr/local/bin/mongostat

All you need to do now is to setup the Linux service which will be used to start MongoDB server, to do so download this script from here.

wget https://github.com/ijonas/dotfiles/raw/master/etc/init.d/mongod
sudo mv mongod /etc/init.d/mongod
sudo chmod +x /etc/init.d/mongod

After this, you’ll need to create a new system user ‘mongodb’ and prepare some folders.

sudo useradd mongodb
sudo mkdir -p /var/lib/mongodb
sudo mkdir -p /var/log/mongodb
sudo chown mongodb.mongodb /var/lib/mongodb
sudo chown mongodb.mongodb /var/log/mongodb

And finally you need to activate you MongoDB service’s by adding it to your system’s run-level. That way the service will startup during the boot sequence and stop nicely during the OS shutdown procedure.

sudo update-rc.d mongod defaults

That’s all, now you have a cleanly installed MongoDB on your server.

Written by imrabti

October 31, 2011 at 8:23 pm

Posted in MongoDB, Ubuntu

Tagged with , ,

Setting up Gitorious on Ubuntu 11.04 Server

with 2 comments

Gitorious is the name of  a web based project host for collaborative open source projects using the Git distribution revison control system. It’s is an open source clone of the well know GitHub platefom. Gitorious is written on Ruby On Rails and helps managing your projects: Git repositories, wikis, timeline and more.

Since it’s an open source application you can set up you own Gitorious in your company and host in it all your projects, this article will help step by step on how set up and configure Gitorious on Ubuntu server 11.04. Instead of providing a big apt-get install command that contain all required stuff, I prefer to group them by libraries for better understanding.

Install Git and system essentials

sudo apt-get install -y git-core git-svn
sudo apt-get install -y apg build-essential libpcre3 libpcre3-dev sendmail make zlib1g zlib1g-dev ssh

Ruby and required libraries

Gitorious runs on a variety of Ruby version (you’ll need at least Ruby 1.8.6), during this part I’ going to use the Ruby 1.8.7 from apt which more is than enough, and then I’ll create a symbolic link of /usr/bin/ruby1.8 to /usr/bin/ruby.

sudo apt-get install -y ruby1.8 libbluecloth-ruby libopenssl-ruby1.8 ruby1.8-dev ri rdoc irb
sudo ln -s /usr/bin/ruby1.8 /usr/bin/ruby

RubyGems

Since we are using Ruby 1.8.7 from apt, we need to set up RubyGems from source (Do not install RubyGems from apt !!). Get the 1.5.3 version of RubyGem on this page.

wget http://rubyforge.org/frs/download.php/74343/rubygems-1.5.3.tgz
tar xvzf rubygems-1.5.3.tgz
cd rubygems-1.5.3
sudo ruby setup.rb
sudo ln -s /usr/bin/gem1.8 /usr/bin/gem

Common libraries & RMagic

Regardless of your Ruby version, you’ll have to install theses tools :

sudo apt-get install -y libonig-dev libyaml-dev geoip-bin libgeoip-dev libgeoip1
sudo apt-get install -y imagemagick libmagickwand-dev

Setting Up the database

Gitorious can use a variety of Relational Database (MySQL, PostgreSQL…), during this article I’m going to install MySQL 5.1 from ubuntu apt.

sudo apt-get install -y mysql-client-5.1 mysql-server-5.1 libmysqlclient16-dev

Setting Up Sphinx search engine

If you want to have the ability to search you gitorious content you need to install sphinx.

wget http://www.sphinxsearch.com/downloads/sphinx-0.9.8.tar.gz
tar xvfz sphinx-0.9.8.tar.gz
cd sphinx-0.9.8
./configure
make && sudo make install

Memcache

setting up Memcache is not required to run Gitorious, but it’s recommended to improve performance :

sudo apt-get install -y memcached
sudo update-rc.d memcached defaults

Installing Gitorious

First we are going to create a new users group called gitorious. Everyone that needs to work on the Gitorious code can be added to this group to gain access.

sudo groupadd gitorious
sudo usermod -a -G gitorious <username>

username is the login of the user that you wish to add to the group.

Now lets create a base directory to hold Gitorious, the logs and Apache configuration files.

sudo mkdir -p /var/www/gitorious.server.com
sudo chown <username>:gitorious /var/www/gitorious.server.com
sudo chmod -R g+sw /var/www/gitorious.server.com

This will set the setgid bit, ensuring the gitorious group follows new files and directories inside /var/www/gitorious.server.com, now create a home for Gitorious :

cd /var/www/gitorious.server.com
mkdir log
mkdir conf
git clone git://gitorious.org/gitorious/mainline.git gitorious
sudo ln -s /var/www/git.server.com/gitorious/script/gitorious /usr/local/bin/gitorious
cd gitorious
rm public/.htaccess
mkdir -p tmp/pids
sudo chmod ug+x script
sudo chmod -R g+w config/ log/ public/ tmp/

/var/www/gitorious.server.com/conf and /var/www/gitorious.server.com/log are for Apache configuration and log files. We’ll create those later.

Configuration of git-daemon, git-ultrasphinx and git-poller services

Gitorious ships with ready to use git-ultrasphinx, git-daemon and got-poller scripts wich are used to start these services automaticly when the system starts. Be aware that these script has hardcoded paths to both Ruby Enreprise Edition and Gitorious and need to be change befor linking them to /etc/init.d.

Open the git-daemon, git-poller, stomp and git-ultrasphinx and change the value of RUBY_HOME to /usr and GITORIOUS_HOME to use your path.

RUBY_HOME="/usr"
GITORIOUS_HOME="/var/www/git.server.com/gitorious"

The next step is to create a symbolic link of these scripts to /etc/init.d, configure the permission and finally set them to start automatically.

sudo ln -s /var/www/gitorious.server.com/gitorious/doc/templates/ubuntu/git-ultrasphinx /etc/init.d/git-ultrasphinx
sudo ln -s /var/www/gitorious.server.com/gitorious/doc/templates/ubuntu/git-poller /etc/init.d/git-poller
sudo ln -s /var/www/gitorious.server.com/gitorious/doc/templates/ubuntu/git-daemon /etc/init.d/git-daemon
sudo ln -s /var/www/gitorious.server.com/gitorious/doc/templates/ubuntu/stomp /etc/init.d/stomp
sudo chmod +x /etc/init.d/git-daemon
sudo chmod +x /etc/init.d/git-poller
sudo chmod +x /etc/init.d/git-ultrasphinx
sudo chmod +x /etc/init.d/stomp
sudo update-rc.d git-daemon defaults
sudo update-rc.d git-poller defaults
sudo update-rc.d git-ultrasphinx defaults
sudo update-rc.d stomp defaults

Installing all required Gems for Gitorious

Gitorious uses a bunch of gems, that are now easily managed by Bundler, first you’ll have to install bundler, and finnally install all required gems.

sudo gem install -v 0.8.7 rake
sudo gem install -v 1.1.0 daemons
sudo gem install -b rmagick stompserver passenger bundler
cd /var/www/gitorious.server.com/gitorious
sudo bundle install

Setting up home for git repositories

You’ll need to create a new user in charge of your git repositories, the git daemon and message queue poller, normally this user is called git but you can named whatever you want. The git repositories will be stored on a folder inside the home of the newly created user.

sudo adduser git
sudo usermod -a -G gitorious git
sudo mkdir /home/git/repositories
sudo mkdir /home/git/tarballs
sudo mkdir /home/git/tarballs-work
sudo chown -R git:git /home/git

Now we need to login as git user and to some work, when someone need to push throught Gitorious, the user wil be looked up in the git user’s ~/.ssh/authorized_keys. we have to create this file, but Girtorious will maintain it for us.

su git
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys

Configure Gitorious and migrate the database

In this part we are going to configure Gitorious, and create the database. We need to configure the database (database.yml file), the borker (broker.yml file) and Gitorious itself (gitorious.yml file). First if you are still logged as git you have to logout.

cd /var/www/gitorious.server.com/gitorious
cp config/database.sample.yml config/database.yml
cp config/gitorious.sample.yml config/gitorious.yml
cp config/broker.yml.example config/broker.yml

Configuring gitorious.yml

First we need to generate a secret key with apg command:

apg -m 64

Copy the key generated and clean it by removing every newlines (this key we be used lately on the gitorious.yml file). I’ll make a brief commentary on some properties here, and then I’ll show you what my config looks like.

  1. gitorious_host: This should match the hostname you access Gitorious with.
  2. gitorious_client_host and gitorious_client_port: These are the values used by script/server. However, they’re also used by the git-daemon, so be sure to set these correctly, or you’ll have trouble pushing.
  3. public_mode: If you’re setting up a closed Gitorious, you’ll want to set this to true but if you want people to be able to browse your projects without logging in, you can keep it true.

Here’s an example configuration. In this example I configured only the production environment, but the other environment should be the same thing.

production:
  cookie_secret: <Here the Key generated with apg -m 64 without EOL>
  repository_base_path: "/home/git/repositories"
  extra_html_head_data:
  system_message:
  gitorious_client_port: 80
  gitorious_client_host: gitorious.server.com
  gitorious_host: gitorious.server.com
  gitorious_user: git
  exception_notification_emails: support@gitorious.org
  mangle_email_addresses: true
  public_mode: false
  locale: en
  archive_cache_dir: "/home/git/tarballs"
  archive_work_dir: "/home/git/tarballs-work"
  only_site_admins_can_create_projects: true
  hide_http_clone_urls: false
  is_gitorious_dot_org: false

Configuring broker.yml

You can check the broker configuration in config/broker.yml, but you don’t need to change anything.

Configuring database.yml

Edit your config/database.yml to suit your database configuration, then open mysql and create the production database and the user :

mysql -uroot -p
create database gitorious;
create user gitorious@localhost identified by 'PASSWORD';
grant all privileges on gitorious.* to gitorious@localhost identified by 'PASSWORD';

If your user already exists and has create privileges, you can do rake db:create instead of the above. But I prefer to give database users as limited privileges as possible.

Gitorious can trip you up if you don’t pay close attention to environments: the git-daemon uses the production environment by default. However, forgetting to specify an environment will cause other scripts to use the development environment. This also goes for script/poller, which polls the queue for tasks. Since I’m configuring a production environment I need to adjust the production configuration on database.yml file.

production:
  adapter: mysql
  database: gitorious
  username: gitorious
  password: <Your password>
  host: localhost
  encoding: utf

The next step is to tell Rails to migrate the production database, but before that we need to fix an incompatibility issue of RubyGems with Rails < 2.3.11 and to do that you need to add the following line at the top of config/boot.rb :

require 'thread'

Now we let rake do all the work for us:

rake db:setup RAILS_ENV=production

The gitorious must have an administrator, to do so execute the following commands :

cd /var/www/gitorious.server.com/gitorious
env RAILS_ENV=production ruby script/create_admin
script/console production
  user = User.first
  user.login = "admin" #Change to your desired username
  user.activate
  user.accept_terms
  user.save

You’ll be prompted for credentials, and the user will be created and automagically activated. You then activate the user by hand by using script/console.

Get Ultrasphinx running

To get set up Ultrashpinx you’ll have to run theses commands :

cd /var/www/gitorious.server.com/gitorious
rake ultrasphinx:bootstrap RAILS_ENV=production

You also need to create a Linux CRON job to keep UltraSphinx indexes updated using the following command :

crontab -e * * * * * cd /var/www/gitorious && /usr/bin/bundle exec rake ultrasphinx:index RAILS_ENV=production

Permissions

Before running Gitorious, you need to get your permissions right.

  • Whatever user owns config/environment.rb is the user who’ll own the Rails process under Passenger
  • The user who runs the application needs full access to the git repositories
  • the git-daemon process needs full access to the git repositories
  • the git-poller process needs full access to the git repositories
/var/www/gitorious.server.com/gitorious
sudo chown -R git:gitorious config/environment.rb script/poller log tmp
sudo chmod -R g+w config/environment.rb script/poller log tmp
sudo chmod ug+x script/poller

Now, let’s start every required service to see if our Gitorious works, first we need to start stompgit-daemon and git-poller.

sudo service stomp start
sudo service git-daemon start
sudo service git-poller start

Now let’s start our Gitorious embedded Web server (the one that runs on the 3000 port) to be sure that Gitorious works.

su git -c "cd /var/www/gitorious.server.com/gitorious && script/server -e production"

This should start the server successfully. Now head over to http://gitorious.server.com:3000, and with any bit of luck, you didn’t miss a detail and the Gitorious splash screen will great you. Log in with the user you created earlier.

To be continued, On The next version of this article I’ll show you how to use Apache HTTP server instead of the embedded one…

Written by imrabti

September 18, 2011 at 7:34 pm

Posted in Git, Gitorious, Ubuntu

Tagged with , ,