Topic: Preparing a Debian Etch Host to be Deployed to for Kete
Topic type:
A guide for how to install all the required software on Debian for a production server. Covers configuring software, adding the user necessary, etc.
This is now superseded by the Installation Guide
Keeping around for historical reference.
Note: I assume you have a fresh and up to date Debian Etch host that you have set up your own user account with sudo on.
Also note: I won't be covering all the security measures you should be taking. Security, besides common practices, is beyond the scope of this guide. I also won't be covering things related to mail or DNS, I assume you have a handle on both.
Install Required Software for Running Kete
We'll jump right into what we can grab from debian packages and ruby gems. You may encounter something like "could not find blah..." when doing a gem install. Rubyforge and the gem mirros can be flaky, just try again and it will usually come right.
me@host: ~ sudo -s
root@host: # apt-get update
root@host: # apt-get install apt-utils build-essential
root@host: # apt-get install subversion subversion-tools libsvn-dev libsvn1
root@host: # aptitude update
root@host: # aptitude upgrade # you may have to reboot after this step, if so start up with next command after another "sudo -s"
root@host: # aptitude -y install rubygems # this will install all the ruby stuff we need
root@host: # apt-get install libmysql-ruby1.8 libmysqlclient15off mysql-common mysql-client libmysqlclient15-dev mytop zlib1g-dev mysql-server
root@host: # gem update --system
root@host: # gem install -y rails
root@host: # apt-get install imagemagick librmagick-ruby1.8 librmagick-ruby-doc
root@host: # apt-get install ruby1.8-dev libxslt1-dev # <= need to this on ubuntu libxslt is needed for zebra
root@host: # gem install -y mongrel mongrel_cluster # choose latest ruby versions
root@host: # quit
me@host: ~ mysqladmin -u root password your_new_password # set a mysql root password!
Kete Specific Setup
That's everything you need to support Rails proper. However, Kete has a few more needs. For one, it supports unicode characters, so we'll want to adjust mysql to make unicode the default for new databases. Make utf8 the default by editing /etc/mysql/my.cnf and adding this in the [mysqld] section:
# making utf8 the default
init-connect = 'SET NAMES utf8'
character-set-server=utf8
collation-server=utf8_general_ci
Restart mysql with /etc/init.d/mysql restart.
Kete uses Memcached for Sessions
We'll probably expand use of Memcached in the future, too. Thankfully it's easy to install on Debian. This wil turn on the daemon, too.
me@host: ~ sudo apt-get install memcached
YAZ and Zebra
Kete uses Zebra (and thus YAZ, too) for search support. It has a slightly different installation procedure. If you aren't using a i386 platform, skip to the "For non-i386 platforms".
As mentioned in the README for Zebra, you need to update your /etc/apt/sources.list (as root) to add IndexData's packages with the following:
deb http://ftp.indexdata.dk/debian etch main
deb-src http://ftp.indexdata.dk/debian etch main
Then...
me@host: ~ cd
me@host: ~ mkdir tmp
me@host: ~ cd tmp
me@host: ~ wget http://ftp.indexdata.dk/debian/indexdata.asc
me@host: ~ sudo -s
root@host: # apt-key add indexdata.asc
root@host: # apt-get update
root@host: # apt-get upgrade # optional
root@host: # apt-get install idzebra-2.0-doc idzebra-2.0 libyaz-dev apache2-utils xsltproc
root@host: # quit
YAZ and Zebra for non-i386 Platforms
As noted in this comment, non-i386 don't have regular Debian packages available. There process is slightly different. If you have problems with the instructions above, this should work for you'll have to compile from source. We're going with versions of YAZ and Zebra that we know to work with Kete. Later versions at this point, do not.
If you haven't in the past, create a /usr/local/src directory, untar the file there, and prepare to build it, like so:
$ sudo mkdir /usr/local/src
$ sudo chmod 777 /usr/local/src
$ cd /usr/local/src
$ wget http://ftp.indexdata.dk/pub/yaz/yaz-2.1.54.tar.gz
$ tar xfz yaz-2.1.54.tar.gz
$ cd yaz-2.1.54
I'm going to use standard *nix installation process with YAZ. However, I'm going to install under /usr to mimic where a Debian package would be installed rather than default /usr/local.
$ ./configure --prefix=/usr
$ make
$ sudo make install
Kete uses the Zebra search engine for all it's searches and a number of other things. Let's install it now (held back to 2.0.6 that we know works, this is also why we require the --with-yaz option):
$ cd /usr/local/src
$ wget http://ftp.indexdata.dk/pub/zebra/idzebra-2.0.6.tar.gz
$ tar xfz idzebra-2.0.6.tar.gz
$ cd idzebra-2.0.6
$ ./configure --prefix=/usr --with-yaz=../yaz-2.1.54
$ # on ubuntu use the following to make sure you have the alvis dependencies installed
$ ./configure --prefix=/usr --with-yaz=../yaz-2.1.54 --enable-mod-alvis
$ make
$ sudo make install
<!--
<p>Broken!!
<p>First, delete the non-src indexdata line from your /etc/apt/sources.list.</p>
<code>
<pre>
$ sudo dpkg -P --force-depends libidzebra-2.0-mod-grs-marc libidzebra-2.0-mod-grs-regx libidzebra-2.0-mod-grs-xml libidzebra-2.0-mod-text libidzebra-2.0-modules libidzebra-2.0-mod-alvis libidzebra-2.0 libidzebra-2.0-dev idzebra-2.0 idzebra-2.0-common idzebra-2.0-doc idzebra-2.0-utils libyaz libyaz-dev yaz yaz-ziffy yaz-doc
$ sudo apt-get update
$ sudo apt-get upgrade # just for the sake of it
$ sudo apt-get build-dep libyaz-dev
$ sudo apt-get source --compile libyaz-dev
$ sudo dpkg -i libyaz*.deb yaz*.deb
$ sudo apt-get build-dep libyaz3-dev
$ sudo apt-get source --compile libyaz3
$ sudo dpkg -i libyaz3_3.0.8-1_amd64.deb
$ sudo apt-get build-dep idzebra-2.0
$ sudo apt-get source --compile idzebra-2.0
$ dpkg -i idzebra-2.0.6*.deb libidzebra-2.0.6*.deb
</pre>
</code>
<p>You may need to adjust version numbers in file names
accordingly.</p>
<p><i>Important Note: DO NOT INSTALL THE libnet-z3950-perl package!
This will uninstall the packages you just built from source because of
a dependency. It is also not needed for Kete.</i></p>
-->
Nginx
Grab the latest tarball from http://sysoev.ru/nginx/download.html and do the following (adjust version numbers accordingly).
me@host: ~ wget http://sysoev.ru/nginx/nginx-0.5.30.tar.gz
me@host: ~ tar xvfz nginx-0.5.30.tar.gz
me@host: ~ cd nginx-0.5.30/
me@host: ~ ./configure --prefix=/usr/local/nginx --sbin-path=/usr/local/sbin --with-http_ssl_module
me@host: ~ make
me@host: ~ sudo make install
You may want to add /usr/local/sbin to your PATH.
Here's a starter nginx config file, including ssl. You'll definitely want to adjust to this after you have done your first deployment and decided on the size of your Mongrel cluster which has a lot to do with what type of resources your server has, like RAM available. Very Important: adjust this to point at where your Kete application actually ends up in the directory structure!
Web requests will throw an error until you have actually deployed your Kete application and started a mongrel cluster to match your seetings in /usr/local/nginx/conf/nginx.conf
After you have your configuration in place it's time to setup the /etc/init.d/nginx script for managing the daemon. You can grab one by doing this:
me@host: ~ sudo wget http://notrocketsurgery.com/files/nginx -O /etc/init.d/nginx
me@host: ~ sudo chmod 750 /etc/init.d/nginx
If that works, you'll probably want to do this to get nginx to start up automatically at boot:
me@host: ~ sudo update-rc.d nginx defaults
Application User Set Up
This is a matter of taste, but I like to keep things by the user that is going to "own" the application. For Kete apps, unsurprisingly I have settled on a "kete" user. So Kete apps are deployed to "/home/kete/apps/your_app", where Capistrano then will create a current directory. The full path to the nuts and bolts of your app lives is thus "/home/kete/apps/your_app/current" and "/home/kete/apps/your_app/shared" for things like logs. More about that when we deploy, but first let's create the user and directories we need:
me@host: ~ sudo -s
root@host: # adduser kete # answer prompts
root@host: # cd /home/kete
root@host: # mkdir apps
root@host: # chown -R kete.kete apps
root@host: # quit
Now, you should give sudo rights to your kete user. The main thing is that your kete user needs to be able install gems, but the ability to also create configuration files under /etc/mongrel_cluster/ is good to have to. So edit (as root) /etc/sudoers to suit.
You'll also probably want to set up ssh keys for your kete user. A runthrough of the process can be found here: https://support.railsmachine.com/index.php?pg=kb.page&id=33
Create Your App's Databases
After you solidified your app's config/database.yml settings, but
before your first deploy, you will need to create your mysql
databases. This is exactly the same as what we outlined in the
"Kete Mac OS X Development Installation" guide, except on your host
that you are deploying to. Here are the details:
$ mysql -u root -p mysql # enter the mysql root password you set above
...
mysql> create database your_app_name_production;
mysql> grant all on your_app_name_production.* to 'the_username_you_chose'@'localhost' identified by 'the_password_you_chose';
mysql> create database your_app_name_development;
mysql> grant all on your_app_name_development.* to 'the_username_you_chose'@'localhost';
mysql> create database your_app_name_test;
mysql> grant all on your_app_name_test.* to 'the_username_you_chose'@'localhost';
mysql> quit;
Set Up Directory for Mongrel Config Files and to Launch at Boot
me@host: ~ sudo -s
root@host: # mkdir /etc/mongrel_cluster
root@host: # # cp
/usr/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.2/resources/mongrel_cluster /etc/init.d/ # mongrel_cluster version number may vary, adjust to suit
root@host: # chmod +x /etc/init.d/mongrel_cluster
root@host: # /usr/sbin/update-rc.d -f mongrel_cluster defaults
root@host: # quit
Note that we when we deploy we'll populate the /etc/mongrel_cluster directory with a config file.
Statistics and Log Rotation for Nginx and Kete (or any Rails app)
Beyond the scope of this guide, but it's covered extensively in this article. Note that there will be some differences as far as paths to things in scripts related to Capistrano's directory layout. You probably want to tackle this after your first deployment.
Congratulations, You Now Ready to Deploy to this Host
<!-- hhmts start --> Last modified: Tue Aug 14 00:02:03 NZST 2007 <!-- hhmts end -->
/usr/loca/nginx/conf/nginx.conf
# Walter McGinnis, 2007-08-13
# updated based on http://brainspl.at/nginx.conf.txt
# see there for comments on each option
user www-data www-data;
worker_processes 6;
pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include conf/mime.types;
default_type application/octet-stream;
# configure log format
log_format main '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log logs/access.log main;
# main error log
error_log logs/nginx_error.log debug;
# rewrite_log on; # I have yet to find where this gets saved to :(
sendfile on;
tcp_nopush on;
tcp_nodelay off;
# output compression saves bandwidth
gzip on;
gzip_http_version 1.0;
gzip_comp_level 2;
gzip_proxied any;
gzip_types text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
# your_app ------------------------------------------------------
server {
listen 80;
server_name www.your_app.com;
location / {
rewrite ^(.*)$ http://your_app.com$1 last;
}
}
server {
listen 443;
server_name www.your_app.com;
location / {
rewrite ^(.*)$ https://your_app.com$1 last;
}
}
# add listeners on ports here as needed for your mongrel cluster
# i.e. server 127.0.0.1:8001;
# server 127.0.0.1:8002;
# etc
upstream your_app {
server 127.0.0.1:8000;
}
# TODO: make sure private data is served dynamically
# see thread in wellrailed list
server {
listen 80;
server_name your_app.com;
# Set the max size for file uploads to 50Mb
client_max_body_size 50M;
access_log logs/access.your_app.log main;
error_log logs/error.your_app.log debug;
# doc root
root /home/kete/apps/your_app/current/public;
# this rewrites all the requests to the maintenance.html
# page if it exists in the doc root. This is for capistrano's
# disable web task
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
location / {
# needed to forward user's IP address to rails
proxy_set_header X-Real-IP $remote_addr;
# needed for HTTPS
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
proxy_max_temp_file_size 0;
# If the file exists as a static file serve it directly without
# running all the other rewite tests on it
if (-f $request_filename) {
break;
}
# check for index.html for directory index
# if its there on the filesystem then rewite
# the url to add /index.html to the end of it
# and then break to send it to the next config rules.
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
# this is the meat of the rails page caching config
# it adds .html to the end of the url and then checks
# the filesystem for that file. If it exists, then we
# rewite the url to have explicit .html on the end
# and then send it on its way to the next config rule.
# if there is no file on the fs then it sets all the
# necessary headers and proxies to our upstream mongrels
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://your_app;
break;
}
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/kete/apps/your_app/current/public;
}
}
server {
listen 443;
server_name your_app.com;
# Set the max size for file uploads to 50Mb
client_max_body_size 500M;
access_log logs/access.your_app.log main;
error_log logs/error.your_app.log debug;
# doc root
root /home/kete/apps/your_app/current/public;
# this rewrites all the requests to the maintenance.html
# page if it exists in the doc root. This is for capistrano's
# disable web task
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
location / {
# needed to forward user's IP address to rails
proxy_set_header X-Real-IP $remote_addr;
# needed for HTTPS
proxy_set_header X_FORWARDED_PROTO https;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect false;
proxy_max_temp_file_size 0;
# If the file exists as a static file serve it directly without
# running all the other rewite tests on it
if (-f $request_filename) {
break;
}
# check for index.html for directory index
# if its there on the filesystem then rewite
# the url to add /index.html to the end of it
# and then break to send it to the next config rules.
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
# this is the meat of the rails page caching config
# it adds .html to the end of the url and then checks
# the filesystem for that file. If it exists, then we
# rewite the url to have explicit .html on the end
# and then send it on its way to the next config rule.
# if there is no file on the fs then it sets all the
# necessary headers and proxies to our upstream mongrels
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://your_app;
break;
}
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/kete/apps/your_app/current/public;
}
}
}
was the last to edit Preparing a Debian Etch Host to be Deployed to for Kete
on Aug 4th, 2008 at 5:04 PM
created Preparing a Debian Etch Host to be Deployed to for Kete
on Aug 29th, 2007 at 11:27 PM
