capistrano deploy and multistage

Just a check list :

cap deploy:setup - use for the first time
cap deploy:check - check that the system is ok and ready for deploy

multi stage example deploy.rb :

set :stages, %w(staging production)
set :default_stage, "staging"
require 'capistrano/ext/multistage'
# since :domain is defined in another file (staging.rb and production.rb),
# we need to delay its assignment until they're loaded
set(:domain) { "#{domain}" }
role(:web) { domain }

And in deploy/staging.rb
set :domain, '111.111.210.211'

set :user, "username"
set :deploy_to, "/home/rubyapps/#{application}"
set :rails_env, "staging"

Use Bitnami Redmine for deploy of my redmine version

I need to deploy Redmine on windows ,to minimize work and issues, I use Bitnami Redmine for windows installer.
I will update the installtion to use my own custom version of redmine , that will be stored on a remote git branch (currently github).

The steps

1. Install redmine stack from bitnami , make sure that every thing is working (browser), then stop all services of Redmine stack.
2.  Goto BitNami\RedmineStack\apps\redmine folder , rename the "htdocs" folder to "htdocs_org" and create a new blank "htdocs" folder.
3. In the blank "htdocs" folder , we will init a new git repo and link it to our remote branch by doing :

cd htdocs
git init
git remote add -t my_work -f origin git://github.com/haimlankry/redmine-1.git
git checkout my_work


At this stage you should have a full redmine project from your git, now will copy some data from the original folder.


4. Copy the folowing folders and files from "htdocs_org" to "htdocs" in the relevant location
root: .bundle, bin , Gemfile.lock 
config/database.yml configuration.rb and additional_environment.rb
tmp/ cache sessions and sockets folder
config\initializers\secret_token.rb

5. Add to the Gemfile the line: gem 'thin' #(if it is not allready in the gemfile)

Test your settings buy starting the thin server, (takes a while on windows) :
start mysql using the service manager utility (or any other way) then type:
bundle exec thin start -e production -p 3001 --prefix /redmine

This should start a thin webserver listens on port 3001 , you can check on the browser for :
http://localhost:3001/redmine
This should result in the redmine home page.

Thats it , now you can develop on your machine , push to git hub, then "git clone" on deploy to get latest update.

Git And ssh on server

Adding a user to existing server to have git access sup

$ sudo groupadd developers
$ sudo useradd -G developers -d /home/john -m -s /bin/bash john
$ sudo useradd -G developers -d /home/andrew -m -s /bin/bash andrew

$ sudo passwd john
$ sudo passwd andrew

connect users toserver withut pssword , using public key that is in the .ssh/authorized_keys ander the home user folder
Verify that user is owner of /home/username and .ssh and authrization_keys

cd .ssh
chmod og-rw authorized_keys
chmod a-x authorized_keys
cd ~
chmod 700 .ssh

the user public key should have strict access so we need to :

chmod u+rwx /home/ec2-user/.ssh/ -R

chmod a-rwx /home/ec2-user/.ssh/ -R ??? not sure

Install apache & passenger for rails on amazon linux

First we need to have apache and curl-dev installed  using root..
sudo -s
yum install httpd httpd-devel
yum install libcurl-devel
exit

# now install passenger , if you have problems with gemfiles , fix them first. (check bundle )
gem install passenger
passenger-install-apache2-module

if all goes good you should get instructions like this :
---------------------------------------------------------------
Please edit your Apache configuration file, and add these lines:

   LoadModule passenger_module /usr/local/rvm/gems/ruby-1.9.3-head@smifr/gems/passenger-3.0.12/ext/apache2/mod_passenger.so
   PassengerRoot /usr/local/rvm/gems/ruby-1.9.3-head@smifr/gems/passenger-3.0.12
   PassengerRuby /usr/local/rvm/wrappers/ruby-1.9.3-head@smifr/ruby

After you restart Apache, you are ready to deploy any number of Ruby on Rails
applications on Apache, without any further Ruby on Rails-specific
configuration!


<VirtualHost *:80>
      ServerName www.yourhost.com
      # !!! Be sure to point DocumentRoot to 'public'!
      DocumentRoot /somewhere/public  
      <Directory /somewhere/public>
         # This relaxes Apache security settings.
         AllowOverride all
         # MultiViews must be turned off.
         Options -MultiViews
      </Directory>
   </VirtualHost>

And that's it! You may also want to check the Users Guide for security and
optimization tips, troubleshooting and other useful information:

  /usr/local/rvm/gems/ruby-1.9.3-head@smifr/gems/passenger-3.0.12/doc/Users guide Apache.html

Enjoy Phusion Passenger, a product of Phusion (www.phusion.nl) :-)
http://www.modrails.com/
------------------------------
follow the instractions and update the httpd.conf file , to find it use the command :
apachectl -V

then restart apache using

apachectl -k restart

# After configuring httpd.conf the restart has an error, you need to provide more rights I did:

passenger-config --root
#Returns path_to_root
chcon -R -h -t httpd_sys_content_t /path_to_root

403 error :
You should set permissions for the rails app , so apache can access these files
I am runing:
chmod 755 /home/ec2-user/ -R

If still not working ..
in file: /etc/sysconfig/selinux
change to SELinux=enforced to SELinux=disabled

install sphinx from source on ec2 linux

sphinx install needs my sql location .
download the sources and extract to folder then installed using :

./configure --prefix=/usr/local/sphinx --with-mysql-includes=/usr/include/mysql51/
make
make install 

PostgreSQL 9 install for rails on centos

sudo -s
# follow instructions in the next link, to install the needed RPM
http://wiki.postgresql.org/wiki/YUM_Installation
#change the name to the version you need
yum install postgresql9
yum install postgresql9-devel
yum install postgresql9-server

service postgresql initdb
service postgresql start
# start on boot
chkconfig --levels 235 postgresql on

# create user that already exists in the system, and should have a database create rights
sudo -u postgres createuser ec2-user -d


# exit from the sudo to the user that is going to be owner of the db (ec2-user in here)
exit

createdb smifr_prod

login to that db with the current user :

psql -d smifr_prod

you should be in the psql console

\q
to exit


set up access to this user throgh rails , it is acting as a loclhost connection so we need to edit :
edit /var/lib/pgsql9/data/pg_hba.conf using root user and set:
local    all         all                                     trust

host    all         all         127.0.0.1/32          trust
host    all         all         ::1/128               trust


TODO : better security method needed here

sudo service postgresql restart

check in the application folder for db connection :

Add local git repo to remote server

On server with a folder named "repos" accessible for the user

mkdir -p repositories/project.git
cd repositories/project.git
git init --bare --shared=group
chgrp -R <developers group> .


On local machine:

git remote add origin haim@IPADDRESS:repos/project.git
git push -u origin master

EC2 linux rvm ruby rails install steps

Steps to install ec2 amazon linux with rails

An old ruby version is required, currently it is intalled by default (v1.8.7), if not, install it using "yum install ruby" .

Steps

sudo -s

yum install -y git gcc-c++ autoconf automake make patch
yum install -y bzip2 readline readline-devel zlib zlib-devel libyaml-devel libffi-devel openssl-devel iconv-devel

curl -L get.rvm.io | bash -s stable

gpasswd -a ec2-user rvm
source /etc/profile.d/rvm.sh
yum install -y bison

run the follwing to see that things are OK:

rvm requirements 

## for problems with rvm and kernel header file:

vi /etc/yum.conf

change the line to comment : #exclude=kernel*

yum install kernel-headers  

======

Install ruby

rvm install 1.9.3-head

## create a new gemset and set the ruby and gemset to be default by rvm

rvm gemset create mygemset

rvm use 1.9.3-head@mygemset --default

# ruby is ready with rvm , now install rails (without documentation)

gem install rails -v 3.2.2 --no-rdoc --no-ri

next steps:

- install database 

- install passanger with apache

add sphinx to linux boot ( init.d )

add a init file from your app to
/etc/rc.d/init.d/searchd
with the name of the service (for sphinx it is : searchd)

file example :


#!/bin/bash
#
# Init file for searchd
#
# chkconfig: 2345 55 25
#
# description: searchd 
#
# USE "chkconfig --add searchd" to configure Sphinx searchd service
#
# by Vladimir Fedorkov Mar 1, 2006, info@astellar.com
# public domain


BASE_PATH=/usr/local/sphinx/
PID_FILE=/home/rubyapps/smifr/shared/pids/searchd.pid


# point the conf to rails smifr version
# CONFIG_FILE=$BASE_PATH/etc/sphinx.conf
CONFIG_FILE=/home/rubyapps/smifr/current/config/production.sphinx.conf
USER=haim
EXEC_PATH=$BASE_PATH/bin/
LOG_PATH=$BASE_PATH/var/log/
DATA_PATH=$BASE_PATH/var/data/
RETVAL=0
prog="searchd"


do_config() {
mkdir -p $EXEC_PATH
mkdir -p $DATA_PATH
mkdir -p $LOG_PATH


chmod 600 $CONFIG_FILE
chmod u+rwx $EXEC_PATH/*
chmod -R u+rw,go-rwx $DATA_PATH
chmod -R u+rw,go-rwx $LOG_PATH
}


do_start() {
echo "Starting $prog"
sudo -u $USER $EXEC_PATH/$prog --config $CONFIG_FILE
RETVAL=$?
echo
return $RETVAL
}


do_stop() {
echo "Stopping $prog"
if [ -e $PID_FILE ] ; then
kill -15 `cat $PID_FILE`
sleep 5
if [ -e $PID_FILE ] ; then
kill -9 `cat $PID_FILE`
fi
fi
RETVAL=$?
echo
return $RETVAL
}


case $* in


config)
do_config
;;


start)
do_start
;;


stop)
do_stop
;;


*)
echo "usage: $0 {start|stop|config}" >&2


exit 1
;;
esac


exit $RETVAL

then chkconfig --add searchd

where searchd is the service name

then you can do

service searchd start

manage services on linux

Services info: 

chkconfig --list [name] 
chkconfig --add name 
chkconfig --del name 
chkconfig [--level levels] name <on|off|reset> 
chkconfig [--level levels] name 

start mysql service on boot :
chkconfig --levels 235 mysqld on

service utility:

service

service httpd status 

service httpd stop

runing process info :

ps aux | less
ps aux
ps aux | grep searchd 

sphinx server ec2 restart

on each deploy if the searchd is running it makes errors so we need to stop it

if rake ts:stop doesnt works then :

pgrep searchd

>> 1944

sudo kill -9 1944

will do the work

more process details:
ps -fw -C searchd

chmod linux

chmod <people><+/-><permissions>
Example: chmod o-w (deny others from editing the file)
Example: chmod u+rwx (give the owner full control)
Exmaple: chmod +rwx (give everyone full control)
Example: chmod +x (allow anyone to execute the file)


r - Read
w - Write
x - Execute

u - The owner of the file
g - The group that the file belongs to
o - Anybody who is not one of the above
a - All users

+ - Add permissions
- - Remove permissions

sphinx and delayed job on dev after startup

Steps to do to start the delayed job worker
stop all workers :
script/delayed_job stop
Start a new worker with monitoring to screen ( use start instead run to lunch in background)
script/delayed_job run

stop sphinx server
sudo service sphinxsearch stop
build the index and update configuration
rake ts:index
run the search server with the rails configuration
rake ts:start

comment visual block vim

select your lines using visual block: CTRL+v
type I to go to insert mode before the block
type the comment char (e.g. #)
press ESC

update date from internet on linux

shut down ntp server if runing by :
sudo service ntp stop
update the date and time from internet server 
sudo ntpdate pool.ntp.org
start back the ntp server 
sudo service ntp start

Delayed job startup on rails boot

DELAYED_JOB_PID_PATH = "#{Rails.root}/tmp/pids/delayed_job.pid"

def start_delayed_job
  Thread.new do
    `ruby script/delayed_job start`
  end
end

def process_is_dead?
  begin
    pid = File.read(DELAYED_JOB_PID_PATH).strip
    Process.kill(0, pid.to_i)
    false
  rescue
    true
  end
end

if !File.exist?(DELAYED_JOB_PID_PATH) && process_is_dead?
  start_delayed_job
end

mysql utf support

need to set in the conf file before creating the db :

unders mysqld section :

character-set-server=utf8

collation-server=utf8_general_ci

deploy sphinx rails

using thinking-sphinx gem Config file at :
App/config/sphinx.yml

rake tasks

rake ts:index # create the configuration file based on sphinx.yml and create the index

Vim folding commands

In visual mode ty zf to fold the visual area.

zf#j creates a fold from the cursor down # lines.
zf/string creates a fold from the cursor to string .
zj moves the cursor to the next fold.
zk moves the cursor to the previous fold.
zo opens a fold at the cursor.
zO opens all folds at the cursor.
zm increases the foldlevel by one.
zM closes all open folds.
zr decreases the foldlevel by one.
zR decreases the foldlevel to zero -- all folds will be open.
zd deletes the fold at the cursor.
zE deletes all folds.
[z move to start of open fold.
]z move to end of open fold.

Passenger monitoring

passenger-memory-stats

passenger-status --verbose

Linux commands

w  list of connected users
tty  my connection name
pkill -9 -t pty/0  kill connection pty/0
path global info

global system profile (including ssh session) at :
/etc/bashrc (ssh is non interactive mode, so check for this condition)
/etc/profile
/etc/environment

search for files:  find / -name 'program.c'

MYSSQL on amazon ec2

yum install mysql mysql-server mysql-libs 
service mysqld start
chkconfig --levels 235 mysqld on
mysql_secure_installation

setting to load on boot (as service):

$> chkconfig --level 235 mysqld on

Create the database and user:

mysql -u root -p

create database smifr_prod

grant all privileges on smifr_prod.* to 'smifr'@'localhost' identified by 'thepass';

Gem for mysql : mysql2 , support utf8 , install :

gem install mysql2 -- --with-mysql-dir=/usr/bin --with-mysql-include=/usr/include/mysql --with-mysql-lib=/usr/lib64/mysql --with-mysql-config=/usr/bin/mysql_config 

ssh client settings for not freezeing

add the folowing to the /.ssh/config file :

ServerAliveCountMax 3
ServerAliveInterval 10

make the file owned and accessible by the use