apache (9)


Amazon SES Dashboard

At work, we wanted to switch from Mandrill/Mailchimp to Amazon SES for a long time. But that was not happening mainly because the tools SES offered to monitor sent mail were, how should I say, DIY.
So, after some deliberation and when I found some time to tackle it, I did it 🙂

The setup is not too complex? Well, it is. But once you understand it, it’s pretty basic.

Let’s start at the source: Amazon

You will see this notice under Notifications for each Email Address you create/verify in SES:

Amazon SES can send you detailed notifications about your bounces, complaints, and deliveries.
Bounce and complaint notifications are available by email or through Amazon Simple Notification Service (Amazon SNS).

Next step is to create the SNS Topic, it’s just a label really.

You will also need an Amazon SQS queue. A standard queue should be good. Once it’s there, copy the ARN as you will need that for the SNS subscription.

Let’s go back to the SNS Topic we created and click on the Create subscription button. Choose Amazon SQS for the Protocol and paste the ARN of the SQS queue you created earlier. You may need to confirm that too? Just click the button if it’s there.

That’s all on the Amazon side! See how easy that was?!

Next you need a Graylog setup.

Where do I start? Well, first choose where do you want to put that Graylog “machine”. For Amazon EC2 I would just go with their ready-made AMIs. Here’s the link/docs to follow: http://docs.graylog.org/en/latest/pages/installation/aws.html (but and I quote: The Graylog appliance is not created to provide a production ready solution)

Another way to get started quickly is an Ansible role you can pick/install from Ansible Galaxy. Check out the QuickStart in the README per https://galaxy.ansible.com/Graylog2/graylog-ansible-role/#readme

But since I like doing things the “easy” way, I went with the Ubuntu 16.04 package per http://docs.graylog.org/en/latest/pages/installation/operating_system_packages.html
Seriously, it’s much easier to use and maintain since I know where everything is. Maybe it’s just me …
Anyway, here’s my bash session:

apt update && apt upgrade
sudo apt-get install apt-transport-https openjdk-8-jre-headless uuid-runtime pwgen
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2930ADAE8CAF5059EE73BB4B58712A2291FA4AD5
echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.6 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.6.list
apt update && apt install -y mongodb-org
systemctl daemon-reload
systemctl enable mongod.service
systemctl restart mongod.service
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list
apt update && apt-get install elasticsearch
vi /etc/elasticsearch/elasticsearch.yml)
vi /etc/elasticsearch/elasticsearch.yml
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl restart elasticsearch.service
wget https://packages.graylog2.org/repo/packages/graylog-2.4-repository_latest.deb
dpkg -i graylog-2.4-repository_latest.deb
apt-get update && sudo apt-get install graylog-server
vi /etc/graylog/server/server.conf
systemctl daemon-reload
systemctl enable graylog-server.service
systemctl start graylog-server.service

I followed the instructions there, and installed Apache on top of that with the following configuration for the VirtualHost


ServerName example.com

# Letsencrypt it
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf

# The needed parts start here
ProxyRequests Off
Order deny,allow
Allow from all

RequestHeader set X-Graylog-Server-URL "https://example.com/api/"
ProxyPass http://127.0.0.1:9000/
ProxyPassReverse http://127.0.0.1:9000/

This will leave you with a Graylog server ready to receive the logs. Now, how do we get the logs over to Graylog? Easy! Pull them from SQS.

Start by adding a GELF HTTP Input in Graylog (System > Inputs > Select Input: GELF HTTP > Launch new input)
Make sure to get the port there right, you will need to configure the script below.
Then download the script, make sure it’s executable. Do run it manually, that way it will tell you what’s missing (BOTO3)
Make sure to configure AWS credentials. The quickest way is:
* to install awscli: apt-get install awscli
* and run its configuration: aws configure

Edit the script with the right configuration vars, add it to cron to run as much as you feel necessary (I use it @hourly)

Enjoy the dashboard! Oh, there’s plenty to learn about Graylog if it’s your first time, but it’s pretty good once you get the hang of it.

Similar Posts:




apt get install libapache2 mod evasive

apt-get install libapache2-mod-evasive

cat > /etc/apache2/mods-enabled/mod-evasive.conf <<EOF
<IfModule mod_evasive20.c>
 DOSHashTableSize 3097
 DOSPageCount 2
 DOSSiteCount 50
 DOSPageInterval 1
 DOSSiteInterval 1
 DOSBlockingPeriod 60
</IfModule>
EOF

Similar Posts:




Protecting Against WordPress Brute-Force Attacks

One of our customers was experiencing very high load today. Checking his logs showed too many IPs trying to hack at the wp-admin.php login page. Here’s a snapshot of what I saw:
# tail -f /var/www/vhosts/*/statistics/logs/*_log
==> /var/www/vhosts/example.com/statistics/logs/access_log <== 10.0.1.169 - - [03/Oct/2013:05:50:17 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0" 10.1.1.206 - - [03/Oct/2013:05:50:21 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0" 10.0.2.197 - - [03/Oct/2013:05:50:23 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0" 10.0.55.117 - - [03/Oct/2013:05:50:24 -0500] "POST /wp-login.php HTTP/1.0" 200 4479 "example.com/wp-login.php" "Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0"

That went on forever!

The first thing I did was use the fail2ban filter I found here.

Unfortunately this did not work as the attack was widely distributed. Fail2ban is ineffective against such attacks. So the next thing I tried was password protect that page. I added a few lines in the apache configuration for that VirtualHost for basic authentication. That worked and the load instantly dropped to normal.

In case the files got lost or misplaced here's the gist:

Similar Posts:




Images not displayed unless site is refreshed

I faced a bit of puzzle today with Tomcat/Apache setup.
Tomcat is running in the background with Apache as frontend via mod_proxy_ajp. The site loads ok except for static files that return a 404 (File Not Found) on first load, then show up normally on refresh!

The apache configuration looks like the following:

Alias /static /var/www/static
ProxyPass /static !
ProxyPass / ajp://127.0.0.1:8009/
ProxyPassReverse / ajp://127.0.0.1:8009/

An example failing URL: http://example.com/static/images/email.png;jsessionid=3892BC4B4C26073338268AF98ECA73D6
And in the error log I see the following:
[Fri Sep 06 10:18:14 2013] [error] [client 00.00.0.000] File does not exist: /var/www/static/images/email.png;jsessionid=3892BC4B4C26073338268AF98ECA73D6, referer[…]

Then it dawned on me that apache wouldn’t know about the jsessionid if it was not sent over to tomcat for processing. Since apache was handling static files the session id needed to go.

Solution: I added the following rewrite rule


        RewriteEngine On
        RewriteRule static/(.*);jsessionid=.* /static/$1 [R,L]

related searches I came across while googling:
page not found when including jsessionid in URL
;jsessionid and 404 File Not Found
Apache getting confused by encoded jsessionid’s (404 Not Found)

Similar Posts:




Password Protect Web Directory Using htaccess

This is a trivial matter for most admins. Here’s a quick recipe:

AuthType Basic
AuthName "Restricted Area"
AuthBasicProvider file
AuthUserFile /etc/apache2/.htpasswd
Require valid-user

Similar Posts:




Stop Apache VirtualHost Spoofing

There are many techniques to stop spoofing websites. One of the most straight forward is to create a default NameVirtualHost to capture and block all spoofs. Here’s a recipe I’m using:

ServerName stop.spam
DocumentRoot /var/www/stopspam
ErrorLog /dev/null

Deny from all

Similar Posts:




Apache Reverse Proxy + SSH Reverse Tunnel

Disclaimer: This setup is just a bit crazy, and I wouldn’t recommend it for a production site. Works for me.
I needed to allow some clients to test a web app I’m working on. But I didn’t want to deploy the code to a server for now for different reasons (mainly memory constraints). So the solution was to setup a reverse proxy using apache to my local machine. Check out the code after the break

Similar Posts:




Redirect all traffic to https

RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^(.*)$ https://%{HTTP_HOST}/$1 [R=301,L]

Similar Posts:




List top 10 visitors by IP

The quick and dirty way:
cat $LOGFILE | awk '{ print $1 }' | uniq -c | sort -nr | head -n10

Similar Posts: