Category Archives: Tools

apt get install libapache2 mod evasive

apt-get install libapache2-mod-evasive

cat > /etc/apache2/mods-enabled/mod-evasive.conf <<EOF
<IfModule mod_evasive20.c>
 DOSHashTableSize 3097
 DOSPageCount 2
 DOSSiteCount 50
 DOSPageInterval 1
 DOSSiteInterval 1
 DOSBlockingPeriod 60
</IfModule>
EOF

Similar Posts:

Tagged , ,

Tweaking GitLab Setup

What?

GitLab is your own GitHub and more (or less). They have pretty good introduction on the home page, so I won’t repeat that here.

The recommended installation method for GitLab is using the Omnibus package. Head to the downloads page and follow the instructions. You should have a GitLab setup in no time, who needs GitHub! oh well, many many people…

Now to the tweaks.

Why?

If you’re like me trying to hide the ports on your server from the bots and prying eyes, they you would have SSH on a different port and your other services all bound to localhost and facing the Internet bravely from behind a proxy server. I use Apache on my personal server, it’s pretty robust and gets the job done.

So let’s say SSH is on port 2022, and apache is taking firm hold on ports 80 and 443. So GitLab’s NGINX should take port 8088.

And the domain you’re using for gitlab is not the machine’s hostname, so hostname is ‘host4339.moodeef.com’ and gitlab’s URL is ‘gitlab.deeb.me’

How?

Edit the “/etc/gitlab/gitlab.rb” file with the following changes/additions:

gitlab_rails['gitlab_host'] = 'gitlab.deeb.me'
gitlab_rails['gitlab_ssh_host'] = 'gitlab.deeb.me'
gitlab_rails['gitlab_port'] = 8088
gitlab_rails['gitlab_email_from'] = 'git-no-reply@deeb.me'
gitlab_rails['gitlab_support_email'] = 'git-no-reply@deeb.me'
gitlab_rails['gitlab_shell_ssh_port'] = 2022
external_url = 'https://gitlab.deeb.me'

Then run gitlab-ctl reconfigure and see how it goes from there.

If things seem to be too complicated, you can always get a subscription option with full support from the GitLab folks. Or hire me to fix it for you!

Similar Posts:

Tagged , , , , ,

IP Location

We rely on IP location for many things as sysadmins, like checking where that flood of logins to wordpress is coming from (China?!). Or even checking if a new order is coming from downtown Dubai or from the outskirts of Jakarta…

I usually use ip2location.com/demo/ and it works great from the browser. Recently however I noticed one of my colleagues using db-ip. They have a rest API. So I wrote this small wrapper, that needs plenty of fixes:
#!/bin/bash
ADDR=$1
KEY=GETYOURKEYFROMhttp://db-ip.com/api/#key
wget -qO- "http://api.db-ip.com/addrinfo?addr=${ADDR}&api_key=${KEY}"

Here’s a sample session:
$ ipinfo 199.59.150.7
{"address":"199.59.150.7","country":"US","stateprov":"California","city":"San Francisco"}

Very simple, but seems enough if I want to check the IP location from the shell. Maybe I’ll add more features (if I need them!)

Similar Posts:

Tagged , , ,

Quickly Start a Django Project

There are many things I like to do when starting a django project. I started to compile those in a script I’m using. The gist is linked below.

New Django Project.sh

#!/bin/bash
# 
# Author: Abdallah Deeb 
# Requirements: python, pip, virtualenv, virutalenvwrapper, git
#
# Edit the following 2 lines
PROJECTNAME=proj
APPNAME=myapp
if [ -n "$2" ]
then 
  APPNAME="$2"
fi
if [ -n "$1" ]
then 
  PROJECTNAME=$1
fi

source `which virtualenvwrapper.sh` 

echo "Starting a new Django project: $PROJECTNAME"
# Make a virtualenv
mkvirtualenv $PROJECTNAME

# Install latest django and start the project/app
pip install django
django-admin.py startproject $PROJECTNAME
cd $PROJECTNAME
django-admin.py startapp $APPNAME
mv $PROJECTNAME conf

# conf is much nicer than projname
replace $PROJECTNAME conf -- manage.py conf/settings.py conf/wsgi.py conf/urls.py
chmod +x manage.py
workon $PROJECTNAME

# Initialize and use git
git init
git add .
git commit -a -m 'initial commit'

and the raw gist download

Similar Posts:

Tagged , , , ,

Django on Codio

Codio is awesome. Coolio would have been a better name, but I guess that’s taken!

It’s free, but don’t let that fool you. What you get is a beautifully simple yet powerful IDE in your browser. It works great even on my tablet, so I can edit/update/deploy even on the road (or on the couch half asleep, and here’s where git comes in handy!). It supports github out of the box. And has a cool terminal with SSH and everything. Yay!

Now if that was not awesome enough, these guys went out of their way and added an even cooler feature:

Every project gets its own Box: an instantly available server-side development environment with full terminal access.

So you get to do it all without leaving their site. I wanted to try playing with a pet project I’m working on using django. It was pretty easy to get it all setup (especially that the code was on github, but you could as simply upload the code).

Here’s a step by step tutorial to get django up and running on codio.

Open a terminal:
Tools > Terminal

I’m using MySQL, so:

parts install mysql
parts mysql start
mysqladmin -u root password StrongPassword
mysql -u root -pStrongPassword
mysq> create database dbname;
mysql> grant all privileges on dbname.* to 'dbuser'@'localhost' identified by 'dbpasswd';
mysql> flush privileges;
mysql> \q

Also make sure to install python using the ‘parts’ command:

parts install python2

(see the related issue on github.com: https://github.com/codio/boxparts/issues/80)

Create the requirements.txt file if you don’t have that already. Hint: run ‘pip freeze > requirements.txt’ to save those. So in requirements.txt:

Django
MySQL-python
#distribute
#wsgiref

Run the following in the codio.com terminal:
parts install pip
pip install -r requirements.txt

Upload the django project, or import it from GitHub then:
python manage.py syncdb
python manage.py runserver 0.0.0.0:8009

I still need to get it running through apache. I’ll get on that later tonight.

parts install apache2_mod_wsgi # also installs apache2 and apr_util
============ apache2 ============                  
To start the Apache server:
  $ parts start apache2 
To stop the Apache server:
  $ parts stop apache2    
Apache config is located at:
  $ /home/codio/.parts/etc/apache2/httpd.conf                                               
Default document root is located at:  
  $ /home/codio/workspace 
     
============ apache2_mod_wsgi ============                
If Apache2 httpd is already running, you will need to restart it: 
  $ parts restart apache2
Default configuration for wsgi is:  
  WSGIScriptAlias / /home/codio/workspace                                                  
  You can change default in /home/codio/.parts/etc/apache2/config/wsgi.conf file

So I changed the file in /home/codio/.parts/etc/apache2/config/wsgi.conf to:

LoadModule wsgi_module  /home/codio/.parts/packages/apache2_mod_wsgi/3.4/mod_wsgi.so
WSGIScriptAlias / /home/codio/workspace/conf/wsgi.py # <-- this is where I put my wsgi.py file               
WSGIPythonPath /home/codio/workspace                         
                          
                                             
Order deny,allow                                           
Require all granted                                       
                                                  

Then start apache, and visit your site at http://UNIQUE-NAME.codio.io:3000/:
parts start apache2

You will probably see a ‘500 Internal Server Error’ message. So tail the apache ErrorLog file in the terminal to see what went wrong:
tail -f /home/codio/.parts/var/apache2/log/error_log

Here’s what I see (and I haven’t figured the fix yet):

[] mod_wsgi (pid=912): Target WSGI script '/home/codio/workspace/conf/wsgi.py' cannot be loaded as Python module.                                                                  
[] mod_wsgi (pid=912): Exception occurred processing WSGI script '/home/codio/workspace/conf/wsgi.py'.                                                                             
[] Traceback (most recent call last):                                                  
[]   File "/home/codio/workspace/conf/wsgi.py", line 5, in                     
[]     from django.core.wsgi import get_wsgi_application                               
[]   File "/home/codio/.parts/packages/python2/2.7.6/lib/python2.7/site-packages/django/core/wsgi.py", line 1, in                                                          
[]     from django.core.handlers.wsgi import WSGIHandler                               
[]   File "/home/codio/.parts/packages/python2/2.7.6/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 6, in                                                 
[]     from io import BytesIO                                                          
[]   File "/home/codio/.parts/packages/python2/2.7.6/lib/python2.7/io.py", line 51, in                                                                                     
[]     import _io                                                                      
[] ImportError: /home/codio/.parts/packages/python2/2.7.6/lib/python2.7/lib-dynload/_io.so: undefined symbol: PyUnicodeUCS2_Replace 

to be continued…
… issue fixed by installing python using the ‘parts’ command above

Similar Posts:

Tagged , ,

Exim mail queue cleanup

I wrote before about cleaning up the mail queue. I had a large queue today though and the exim tools “felt” slow. So I did it manually (which wasn’t much faster):
cd /var/spool/exim/input/
for d in *; do echo in $d; cd /var/spool/exim/input/$d; C=0; for x in *H; do grep -q example.com $x; if [ $? -eq 0 ]; then f=${x%H}; (( C++ )); rm ${f}{H,D}; fi; done; echo $C mails deleted; echo "remaining: "; ls -l | wc -l; echo ---; done

Similar Posts:

Tagged , , , , ,

Email Tracking

Most email clients can request a return receipt. One way to set this up in Thunderbird for example is to go to “Preferences > Advanced > General > Return Receipts …” and check the “When sending messages, always request a return receipt”

The problem with this approach is that when the person you’re emailing has set his email client to reject that request (or does that manually every time), you will not get a receipt back. And honestly speaking there is no way to force that. Now there are some services out there that promise to track your mail when that’s read, and those rely on embedding a transparent gif that will call home when the message is opened. I tried a few of these services: the free ones did not deliver (or seemed too shady for my taste) and the paid ones did not look too good either. So I cooked up a quick solution that I can use when needed (like when I’m tracking my brother on his honeymoon trip *evil grin*)

The code is below, pretty self explanatory. You will need a transparent gif/png. That’s easy too 😉

Edit: Looks like some services are not so bad. Checkout bananatag and YesWare Email Tracking

Similar Posts:

Tagged , , , , ,

Clearing the Queue

Talking about mail queues here. Especially when the queue gets filled with hundreds of thousands of spam emails.

Stop the MTA

The first thing you do is stop the MTA before it gets worse. On most Linux servers:
~: # /etc/init.d/exim stop # most WHM/cPanel based servers
~: # /etc/init.d/postfix stop # if you have postfix

Then check the queue

The commands to use are
~: # exim -bp
~: # mailq

I suggest keeping CTRL+C ready to stop the flow. If you have an infestation scrolling through the queue will take a very very long time.
But checking out the first few lines (pages) of the queue may reveal many things. For example, what user account is being exploited for sending out the spam. Sometimes that’s obvious, other times you may need to do some digging. Here’s a sample:

24h  1.1K 1VeKKb-00030t-42 <noreply@yahoo-inc.com>
        D alena@example.com
        D alenka@example.com
        D alepp@example.com
        D alerei@example.com

24h  1.1K 1VeKKb-00030u-5R <noreply@yahoo-inc.com>
        D alex@example.com
        D alex.b@example.com
        D alex.d@example.com
        D alex.f@example.com

24h  1.1K 1VeKKb-00030v-1R <noreply@yahoo-inc.com>
        D alex.t@example.com
[...]

The lines above are from an exim queue output. Obviously noreply@yahoo-inc.com is not an account on this server. Let’s find the username then clean up. A nice Exim utility is exigrep (basically it’s grep with exim ties!)
:~# exigrep 1VeKKb-00030t-42 /var/log/exim/mainlog
+++ 1VeKKb-00030t-42 has not completed +++
2013-11-08 21:18:14 1VeKKb-00030t-42 <= noreply@yahoo-inc.com H=(User) [XXX.XX.XX.XXX] P=esmtpa A=login:test@mydomain.com S=1193 T="Authenticate Your Email" from <noreply@yahoo-inc.com> for alena@example.com alenka@example.com alepp@example.com alerei@example.com [...]

2 things interest us here: the H=(User) [XXX.XX.XX.XXX] part tells us where that intruder is logging from (probably another exploited server, so it’s nice to alert the owners of that server as well. And A=login:test@mydomain.com tells us which user they are logged in as, so we can lock down that user, change their password, etc.

Cleaning up

Emptying the queue is usually an easy way out:
~: # postsuper -d ALL # for postfix
~: # exim -bp | exiqgrep -i | xargs exim -Mrm # for exim

A more subtle approach is needed when some of that mail in the queue is actually legit. If you checked out the queue earlier you might do something like the following:
~: # exiqgrep -i -f noreply@yahoo-inc.com | exim -Mrm # that will remove all mail sent by noreply@yahoo-inc.com

It’s a bit harder in postfix, here’s one recipe:
~: # mailq | tail +2 | awk 'BEGIN { RS = "" } { if ($7 == "noreply@yahoo-inc.com") print $1 }' | tr -d '*!' | postsuper -d -
remove the tail +2 | if that doesn’t work for you.

There are plenty of other things to check and do to fix your server when someone is abusing it to spam. Cleaning up is one way to start.

Ah, before I forget, here’s a link to a very nice cheatsheet for exim

Similar Posts:

Tagged , , , , ,

Protecting Against WordPress Brute-Force Attacks

One of our customers was experiencing very high load today. Checking his logs showed too many IPs trying to hack at the wp-admin.php login page. Here’s a snapshot of what I saw:
# tail -f /var/www/vhosts/*/statistics/logs/*_log
==> /var/www/vhosts/example.com/statistics/logs/access_log <== 10.0.1.169 - - [03/Oct/2013:05:50:17 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0" 10.1.1.206 - - [03/Oct/2013:05:50:21 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0" 10.0.2.197 - - [03/Oct/2013:05:50:23 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0" 10.0.55.117 - - [03/Oct/2013:05:50:24 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0"

That went on forever!

The first thing I did was use the fail2ban filter I found here.

Unfortunately this did not work as the attack was widely distributed. Fail2ban is ineffective against such attacks. So the next thing I tried was password protect that page. I added a few lines in the apache configuration for that VirtualHost for basic authentication. That worked and the load instantly dropped to normal.

In case the files got lost or misplaced here's the gist:

Similar Posts:

Tagged , , , ,