Mail Server

Mail Server


SysAdmin, Linux, Tech

by on 05 May 2013 - 00:44  

Notes from when I set up a mail server, cause you never know when notes like this will come in handy...

First a lesson about mail servers. A mail server is a combination of a bunch of software working together. The two main parts are the email server itself, for which I use postfix, and the imap server (or pop server), for which I use cyrus. Postfix sends and receives mail, and cyrus sorts and stores the mail. And then there are other pieces, depending on what you want to do. I use a combination of Amavisd-new, SpamAssassin, Razor, DCC, Pyzor and ClamAV for spam/virus filtering. I use Squirrelmail (I have actually since started using RoundCube, which I think has a nicer interface, and some users have found is less buggy) as a web interface to the mail server, doing basically the same thing that Outlook, Mozilla Mail, or Thunderbird do on your local machine, but it does it on the server through a web interface. Obviously, you also need a web server for that, which I am already running for our web pages. I use Apache. I have also set up Mailman for our mailing lists. For security, I use Openldap as the authentication system, and certificates, which is a whole 'nother topic.

Notes from setting up new mail server.

I made a special mail account named test. I am having all spam that gets a score greater than 10 + mails with banned contents or viruses sent to this account. This account has weekly rotation set up. Mail in this account is deleted after 4 weeks. Made a python script to rotate the spam and delete.

Tested getting rid of mail older than 20 days in user test using ipurge. Have to use -f option, but this also means it checks all folders under level requested, and if you are cleaning the inbox, this is all folders, so be careful! Must do as user cyrus if doing from command line: /usr/sbin/ipurge -d 20 -f user.test Worked fine, so added this to cyrus.conf in Events section:

purgetrash cmd="/usr/sbin/ipurge -f -d 14 *.Trash" at=0301

Which purges all messages older than 14 days, in all users' Trash folders and runs every morning at 3:01am. See the man pages for ipurge and cyrus.conf for more details.

mail set up based on:
+ cyrus instead of outside mail delivery
software needed to be configured for mail/web server:

  • backup - bacula
  • web:
    • apache
    • munin
      • /var/lib/munin
      • /var/log/munin
      • /var/run/munin
    • webalizer
    • squirrelmail
    • pmwiki
  • mail:
    • postfix
    • cyrus
      • to change logging for cyrus: /etc/default/cyrus2.2/
    • spam stuff:
      • amavisd-new
      • pyzor
      • razor
      • spamassassin
      • DCC
      • clamav
    • mailman - mailman is a pain moving from one machine to another, be careful of these directories:
      • /var/lib/mailman/archives /var/lib/mailman/data /var/lib/mailman/logs /var/lib/mailman/lists /var/lib/mailman/archives
  • security
    • denyhosts
  • dns server
    • bind
  • ldap
    • configure to use ldap server

good to know:

  • dealing with aliases: If you change the alias database, run newaliases
  • to deal with modules in apache use a2enmod module and a2dismod module

tweaking settings: If you want to configure your system to use more instances of amavisd-new, allocate at least 60MB for each additional instance. It you wanted to double the number of child processes from 2 to 4, you would edit amavisd.conf and change: $max_servers = 2; to $max_servers = 4; Then edit and change: smtp-amavis unix - - - - 2 smtp to smtp-amavis unix - - - - 4 smtp

Amavisd-new (SpamAssassin actually) will be the biggest bottleneck in the system. On a busy server you will probably want 2GB RAM so you can accommodate somewhere around 12 $max_servers.

If you run sa-learn --force-expire or spamassassin --lint -D or other spamassassin commands from the root account, SpamAssassin may change the owner of the Bayes files to 'root'. If it does, amavis will no longer be able to read those files. You would need to run chown -R amavis:amavis /var/lib/amavis to regain ownership. In general, if you do any spamassassin maintenance from the command prompt as root, the best thing to do is run chown -R amavis:amavis /var/lib/amavis afterwards; just to make sure. You can avoid these problems by remembering to run spamassassin commands as the amavis user. For example su amavis -c 'sa-learn --sync --force-expire'

This script does have some entries that are dependent on the version of SA. If you are not running SA 3.2.5, the script may need to be edited, and you must remember to edit this file when a new version of SA comes out: vi /usr/sbin/

Notice the lines that may need to be changed. Change 3.002005 if needed (3.3.0 might be 3.003000 for example):
rm -f /var/lib/spamassassin/3.002005/saupdates_openprotect_com/
rm -f /var/lib/spamassassin/3.002005/saupdates_openprotect_com/
rm -f /var/lib/spamassassin/3.002005/saupdates_openprotect_com/loadplugins.pre

Exit (or save) the file and run the script:

Bind: using jail now
munin: can change munin frequency in /etc/cron.d/munin,


testing and how-to stuff:

This excludes much the server says back to you...

server1:~# telnet 25
Connected to localhost.
Escape character is '^]'.
220 ESMTP Postfix (Debian/GNU)
ehlo localhost
250-SIZE 10240000
250 DSN
mail from:<>
rcpt to:<>
Hi John,

just wanted to drop you a note.

look at log file for postfix sending. This should not involve spam filtering. spam filtering is only through (14) not actually sure about port 25 on check to inbox

At the bottom of the above link are also hints about dealing with logfiles and backing up config files.

spamassassin -t < message.eml

to see more infos (what SA is actually doing)

spamassassin -D -t < message.eml

check website check squirrel mail

Mail Server ~ Comments: 0

Add Comment

openldap configuration


Linux, SysAdmin, Ldap, Tech

by maria on 08 Mar 2013 - 01:27  

It use to be you edited slapd.conf to change configurations, but now openldap has its configuration in the database itself. Which means you can change the configuration without having to restart ldap. Or you can totally screw it up while its running. Cool. At any rate, the guides all helpfully say, now you change your configs just like you change anything else in your database by ldapmodify. Great. But, wait, how do I do that exactly? Well, first you need to know the cn, which isn't going to be the same as the changes you usually make. Fortunately, it is easy, cn=config. For an example, what if you want to check what schemas are loaded?

ldapsearch -LLLQY EXTERNAL -H ldapi:/// -b cn=schema,cn=config dn

The -L option used here causes the results to be displayed in LDAP Data Interchange Format (LDIF). A second -L will disable comments, and a third one will prevent the LDIF version from being printed. The default is to use extended LDIF. The -Q will enable SASL Quiet mode. Never prompt. The -Y will specify we want the EXTERNAL (usually TSL) authentication mechanism. If you want to see the entire structure of the schemas, omit the dn from the end.

To get a nice overview of the configurations:

ldapsearch -LLLQY EXTERNAL -H ldapi:/// -b cn=config "(|(cn=config)(olcDatabase={1}hdb))"

When I first started using the new configuration scheme, I could not authenticate to make any configuration changes, even though I could still make changes to our database. Something happened when I converted from ldapd.conf to the new configuration, so that I got this error message when trying to authenticate:

LDAP "Invalid credentials (49)" for cn=config (10.04 svr)

The problem is that normally I use cn=admin for making database changes, but to make configuration changes you have to use the olcRootDN, cn=admin,cn=config.

If you search by authenticating as config, you can tell if you will have this problem:

ldapsearch -xLLL -b cn=config -D cn=admin,cn=config -W olcDatabase={1}hdb

To fix this I re-created the olcRootDN and password. First get the encrypted version of your password by running this command:

slappasswd -h {MD5}

Type your password twice and copy the result in to a file with the following contents (I used emacs config.ldif):

dn: cn=config
changetype: modify

dn: olcDatabase={0}config,cn=config
changetype: modify
add: olcRootDN
olcRootDN: cn=admin,cn=config

dn: olcDatabase={0}config,cn=config
changetype: modify
add: olcRootPW
olcRootPW: {MD5}your password here

dn: olcDatabase={0}config,cn=config
changetype: modify
delete: olcAccess

Include the {MD5} part before the actual password. The delete: olcAccess is so that users other than root can have administrative access. Now load the config file:

ldapadd -Y EXTERNAL -H ldapi:/// -f config.ldif

One thing I wanted to do was change what was being indexing, so I created a file index.ldif

dn: olcDatabase={1}hdb,cn=config
changetype: modify
add: olcDbIndex
olcDbIndex: uid eq
add: olcDbIndex
olcDbIndex: cn eq
add: olcDbIndex
olcDbIndex: uidNumber eq

and loaded that:

ldapmodify -QY EXTERNAL -H ldapi:/// -f index.ldif

I decided what to index on by looking at the log files to see what was being searched on, but wasn't indexed.

There was something else strange that happened when I upgraded, the shadowLastChange attribute was missing from all of the people. So, I tried adding it back using this in my change.ldif file:

dn: uid=jd,ou=people,dc=example,dc=com
changetype: modify
replace: shadowLastChange
shadowLastChange: 15771

I got the following message:

add shadowLastChange:
modifying entry "uid=jd,ou=people,dc=example,dc=com"
ldap_modify: Constraint violation (19)
        additional info: attribute 'shadowLastChange' cannot have multiple values

That's strange. So, maybe it thinks it already has that attribute. let's see what happens if we try to modify it instead of add it:

replace shadowLastChange:
modifying entry "uid=jd,ou=people,dc=example,dc=com"
modify complete

Huh, well that seemed to have worked. Let's see what the value is now.

annette:~# ldapsearch -x "uid=jd" shadowLastChange
# extended LDIF
# LDAPv3
# base <dc=example,dc=com> (default) with scope subtree
# filter: uid=jd
# requesting: shadowLastChange 

# jd, people,
dn: uid=jd,ou=people,dc=example,dc=com

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

Um, so where is it? Interestingly, if I do stop ldap and do a slapcat, it shows up in the ldif file. So, it is there, I just can't see it using ldapsearch. Let's check permissions:

ldapsearch -LLLQY EXTERNAL -H ldapi:/// -b cn=config "(|(cn=config)(olcDatabase={1}hdb))"

olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymous auth by dn=
 "cn=admin,dc=example,dc=com" write by * none
olcAccess: {1}to dn.base="" by * read
olcAccess: {2}to * by self write by dn="cn=admin,dc=example,dc=com" write by * read

So, for userPassword and shadowLastChange the last permission is by * none, which effectively means that no one can read it. That is what we want for userPassword, but not what we want for shadowLastChange. Hmmm. So, now we learn how to change permissions. Probably it is easiest to just make separate entries for userPassword and shadowLastChange. So, I created this file:

dn: olcDatabase={1}hdb,cn=config
changetype: modify
delete: olcAccess
olcAccess: {0}
add: olcAccess
olcAccess: {0}to attrs=shadowLastChange by self write by anonymous auth by dn="cn=admin,dc=example,dc=com" write by * read
delete: olcAccess
olcAccess: {1}
add: olcAccess
olcAccess: {1}to attrs=userPassword by self write by anonymous auth by dn="cn=admin,dc=example,dc=com" write by * none
delete: olcAccess
olcAccess: {2}
add: olcAccess
olcAccess: {2}to dn.base="" by * read
add: olcAccess
olcAccess: {3}to * by self write by dn="cn=admin,dc=example,dc=com" write by * read

Order is important, which is why I rewrote all of the olcAccess attributes. Also syntax is important. Make sure you are using the correct dn, have the changetype, etc. Now implement your changes:

ldapmodify -QY EXTERNAL -H ldapi:/// -D cn=admin,cn=config -f change.config

And that should do it. But, no, now even though I have these permissions:

olcAccess: {0}to attrs=shadowLastChange by self write by anonymous auth by dn=
 "cn=admin,dc=example,dc=com" write by * read
olcAccess: {1}to attrs=userPassword by self write by anonymous auth by dn="cn=
 admin,dc=example,dc=com" write by * none
olcAccess: {2}to dn.base="" by * read
olcAccess: {3}to * by self write by dn="cn=admin,dc=example,dc=com" write by * read

I still can't read the shadowLastChange.

Aha! And now another lesson in openldap permissions. Order is important. I have stated in my permission for shadowLastChange 'by anonymous auth', so if I try to look at shadowLastChange anonymously, we come to this directive and stop, cause it looks like we don't have permission to read. Doesn't matter that later on, we say that anyone can read. So, be careful when you are adding a new directive that you put it in the correct place, and that there isn't a directive there already that contradicts it. In this case, the way I added the new rule 'by read *', I was effectively saying, everyone but anonymous has read permission. If we move the directive 'by read *' earlier, we give anonymous read permission by giving everyone read permission, and it makes no sense to then say they have auth permission, which is more restricted access. So, we drop that directive altogether. Here is my new configuration:

olcAccess: {0}to attrs=shadowLastChange by self write by dn="cn=admin,dc=example,dc=com" write by * read
olcAccess: {1}to attrs=userPassword by self write by dn="cn=admin,dc=example,dc=com" write by anonymous auth by * none
olcAccess: {2}to dn.base="" by * read
olcAccess: {3}to * by self write by dn="cn=admin,dc=example,dc=com" write by * read

And, seems like a good idea to post the who and the what:

*All, including anonymous and authenticated users
anonymousAnonymous (non-authenticated) users
usersAuthenticated users
selfUser associated with target entry
dn[.<basic-style>]=<regex>Users matching a regular expression
dn.<scope-style>=<DN>Users within scope of a DN

Access Levels:

none=0no access
disclose=dneeded for information disclosure on error
auth=dxneeded to authenticate (bind)
compare=cdxneeded to compare
search=scdxneeded to apply search filters
read=rscdxneeded to read search results
write=wrscdxneeded to modify/rename
manage=mwrscdxneeded to manage

Of course, all of this information and more is in the docs

openldap configuration ~ Comments: 0

Add Comment

Apache and SSL


Tech, Apache, SSL, PHP,Linux,SysAdmin

by maria on 01 Mar 2013 - 22:51  

Once upon a time we had set up our web server so that we had a secure VirtualHost pointing to a subdirectory of /var/www, so the DocumentRoot for the secure site was /var/www/https. This worked just fine. Some content was just for our own use, and required secure login, and some content was for the general public, and did not. Then we decided it would be better if our website was a wiki, so that everyone in the lab could update the website. Since our website was initially not a wiki, and we wanted to save some content from the old site, we had set up the wiki as a subdirectory of /var/www. The wiki had some links to the old content. Once we got the wiki up and running, we decided it would be good if the login to the wiki was over SSL.

To do this, we needed the main /var/www directory to allow https access sometimes, but not always. If you always run everything over SSL, there is larger overhead, and pages are likely to load more slowly. So, we can just make the DocumentRoot for both /var/www and use code to switch between SSL and non-SSL, right? I assumed that pages would be served by HTTP unless SSL was requested, but it turns out at least one browser I know of will choose SSL over non-SSL if both are offered by the server. Which means that the dumb solution of just checking to see if the person requesting a page is trying to edit or login and then requiring SSL wasn't going to work, as general public looking for our website was just as likely to be given a login window as site content. This could be dealt with on the wiki, since it runs on PHP, but was not clear how to do deal with the rest of the website.

So, maybe we just want to worry about the wiki having SSL login, since that is the only place needing it. Anything else on the website has to be edited directly on the server. The alias directive seemed a good solution. This allows you to add content not under the document root to be served as part of the document tree. You enable the mod_alias module by the following commands

a2enmod alias
service apache2 restart

Now, I am using CleanURLs for my wiki, so I needed to figure out how to set this up when using SSL. I don't know of a way to do and if statement in htaccess to check if someone is using SSL, but a side effect of having different root directories for the normal and SSL site is that I could just use 2 different htaccess files. One is for the non-SSL site:

RewriteCond %{HTTP_HOST} ^
RewriteRule (.*)$1 [R=301,L]

and the other for the SSL site:

RewriteCond %{HTTPS_HOST} ^
RewriteRule (.*)$1 [R=301,L]

And the Rewrite Base and Rewrite Rules are going to be slightly different.

So, now for the PHP solution. I started by using a recipe on the PmWiki website for enablingSSL for the initial log in. Really, this seemed to be the only time that SSL was really necessary. But, there seemed to be no memory in the PHP code of logging in via SSL when we returned to HTTP, so after the initial logging in, if you tried to edit a different page, you were asked for a password again. Well, that got old fast, and hinted that there was a possible security hole. The most sensible solution seemed to be to have the whole session using SSL, but there was no obvious way to do this from the recipes available on the PmWiki site. So, I went on the PmWiki user mailing list to try to figure out how to adapt one of the recipes for my purpose. In the end, I used a combination of a hint from Patrick Michaud and a recipe by jtankers. If your interested, you can see my code pmwiki ssl

And, indeed, it seemed to work fine, mostly. But, remember how alias adds content not under the document root to be served as part of the document tree? This means that if you try to see content above the alias (ie. something in /var/www), according to the secure site configuration, apache should look in /var/www/https, since this is the root for the secure site. So, stuff that had been linked to in the wiki from other directories in /var/www was not showing up in the secure site. I managed to solve this by changing paths in the PHP code that runs our wiki.

More Apache hints, in no particular order:

  • MultiViews requires that files are owned by the group that apache runs as, in my case www-data, and that permissions are set to 770:
    1. chgrp www-data /var/www/ -R
    2. chmod 770 /var/www/ -R
    When you edit files, make sure you are a part of the correct group (newgrp www-data) or that you change the group after you edit.
  • if you are using Clean URLS, you need to have AllowOverride set to All
  • not strictly an Apache hint, but if you want to post an email address on a website in such a way it is unlikely to be found by bots, use an ascii to html code converter (converter apps available on the web, just search), and enter the email address with html code (looks like test). To be extra careful, add spaces around the @ symbol.
  • mod_rewrite can do everything mod_alias can do, and a lot more.
    • Use mod_alias (Redirect in htaccess) when you can because it is cleaner, uses less cpus and overhead, and easier to figure out what is going on when looking at configuration after the fact.
    • Use mod_rewrite when you are trying to stop things from displaying in the url bar.
    • rules in .htaccess are executed in order, however, Rewrite has priority over Redirect.
    • more excellent hints on mod_rewrite and mod_alias can be found here

Apache and SSL ~ Comments: 0

Add Comment