Exchange - Basic Email Gateway Server

Exchange Server
Compatibility Information

Exchange
Server Version
Compatibility 5.5 Yes

2000 Yes

2003 Yes

2007 Yes

Basic Email Gateway Server
Author: Simon Butler, Exchange MVP, MCSE
Last Page Review: 09/03/2008

One of the features of Exchange 2007 is the new "Edge Services" role. To use this new role you need to have an additional Exchange server license as Exchange licenses are per physical machine. The Edge server cannot have any additional roles such as Mailboxes, Client Access etc, so you will need a dedicated machine to hold the Edge Server role.

However it is possible to reproduce most of the benefits and features of the Edge Services server without having to purchase another Exchange license. This uses tried and tested 32 bit applications, which means you can use an old server or workstation to provide the hardware. Windows 2003 will install on anything above PIII 500 with 256mb of RAM, although 512mb makes the server more comfortable to work with.

With care - you could probably combine this technique with an ISA server, therefore having a single server in your DMZ protecting both http based services (OWA, Outlook Anywhere/RPC over HTTPS, ActiveSync etc) and SMTP traffic. However this article explains how to setup a gateway only machine.

While this article has been written to protect an Exchange server, it could be used to protect almost any SMTP server, particularly if that server has LDAP lookup capability that can be used for recipient filtering.

Objective

This Gateway or Edge replacement server will be setup to do three basic tasks.

recipient filtering
tar pit on SMTP to secure recipient filtering
greylisting

You could also introduce antivirus or anti-spam products if they can operate in a gateway format without running their own SMTP server. Look for IIS integration.

For antivirus, you may want to leave that on the Exchange server. Greylisting is actually quite effective at stopping virus infected messages coming in as well.

Recipient filtering requires a query to AD
Tar pit is available as part of Windows 2003 SP1 and higher.
Greylisting needs to use a third party tool. You can read about greylisting here: http://www.sembee.co.uk/archive/2006/09/18/24.aspx It remains an effective way of dealing with spam, if you can tolerate the delay in email delivery that it can cause.

A gateway server is ideal if you have older versions of Windows and Exchange. If you have Exchange 2000 machines then you do not have recipient filtering or tar pit available to you. Vamsoft ORF can provide that functionality. While you can install Vamsoft directly on to Exchange with E2000/E2003, you may prefer not to.

Shopping List

For this, you will need two things.

At least Windows 2003 SP1 machine (R2 or original release). If you are placing this machine in the DMZ then it should not be a member of your domain. Ensure that it is fully patched and automatic updates are set. For this machine you could use the automatic update and restart mechanism, as once the machine is built it doesn't need much maintenance.
A third party tool - Vamsoft ORF. This is a spam filtering application which can provide greylisting, tar pit, recipient filtering and also allow you to use Real time Blacklists and other internet based resources. In this scenario it is being used for greylisting and recipient filtering only. You can download a demo version of the tool here: http://www.shareit.com/product.html?productid=169362&affiliateid=200023740

Vamsoft ORF isn't free, but it is much cheaper than an Exchange license. It is also priced on a per server basis, making it ideal for this task.

Initial Setup of the Server

Most of this configuration can be done offline without affecting your production system. If you have a test environment (such as virtual machines with the 32 bit evaluation of Exchange 2007) then you can use it with those, as the configuration changes required to switch it to the live domain are minimal.

NOTE: As with other articles on this web site, it is presuming that you have knowledge of how to do the following tasks without simplistic "click next" type instructions.

Install Windows 2003, SP1 or SP2 (if not integrated) and update. DO NOT add to your domain.
Install IIS. The only components that you need are SMTP and the core files. You don't need the web components unless you are going to host a web site on this server as well.
Make the tar pit registry change. You can find instructions on that change here: http://www.amset.info/exchange/filter-unknown.asp
Lock down the machine - at the very least run the Security Configuration Wizard.
Install Vamsoft ORF - but don't touch its configuration.

If you are not using Windows 2003, then you will need to use Vamsoft to provide the tar pit option.

Firewall Configuration

Before attempting to configure the server further, you need to look at your firewall. If you are building a test site that is behind the firewall, then you can skip this part until you are ready to go live.

Two ports need to be open from your production network to the DMZ for this server.

25 for SMTP traffic to the internal Exchange server.
389 for LDAP traffic to a global catalog domain controller.

If you are going to use an alternative port for sending email between the gateway server and the Exchange server, such as the TLS port of 465, then that one needs to be open as well.

When you go live, port 25 from the internet needs to be pointed at this server. No other port is required for email - but if you are also offering OWA and other web services to the end users, that port still needs to be open.

IIS SMTP Server Configuration

Before looking at Vamsoft, you need to configure the SMTP functionality of the server. This falls in to two sections - inbound email and outbound email.

This server is effectively a relay server. It will accept email and pass it on to the internal server. Therefore you have to configure it for relaying.

Open IIS Manager and find the SMTP server section.
Create a new Remote domain, which matches the primary domain that your Exchange server accepts email for. If your Exchange server accepts email for more than one domain, then each domain needs to be listed separately. Mirror what you have in the Accepted Domains list in Exchange 2007, or the domains listed in recipient policy for Exchange 2003 or older. Local domains (domain.local) do not have to be added.
Open each domain and on the first tab enable the option "Allow incoming email to be relayed to this domain"
Under Route domain, change the setting to use a smart host and enter the internal IP address of your Exchange server in [ ] - for example [192.168.11.1]
Repeat for all the domains.

On the properties of the server itself (Default SMTP Virtual Server) there are a couple of options to review and change.

On the delivery tab review the limits to ensure that they match what you are using internally.
While on the delivery tab, also choose the Advanced button. In the box marked Fully Qualified Domain name, enter the name that this server is known as on the internet. This should ideally be the name in the MX records and match the forward and reverse DNS for the external IP address. Remember that this is the server that will now be exposed to the internet - not your Exchange server.
On the access tab, there are more options to consider.
To allow outbound email to flow through the server, you need to allow the Exchange server to relay. This can be controlled in one of two ways
1. By granting access to its IP address.
2. To authenticate.
Both have their positives and their negatives.

Relaying by IP address could potentially be bypassed by IP address spoofing.
Relaying by authentication would make the server vulnerable to an attack on the SMTP server to find out the administrator password. Exchange 200x allows you to control which user accounts can be used for authenticated relaying. IIS SMTP does not.

If you decide to use the IP address method, then set it in the relay option.
If you decide to use the authentication method then you need to enable basic authentication. You cannot use integrated authentication as the machine is not a member of the domain. Don't forget to create a user account in Computer Manager to authenticate with. Ensure that the account has a very strong password to defeat most password attacks - it does not have to be a member of any group other than users.

If you are using the authentication method then you may also consider using an SSL certificate. This will need to be trusted by the Exchange server so you either have to generate your own and import it, or purchase a cheap SSL certificate (such as http://www.certificatesforexchange.com).

Vamsoft ORF Configuration

To begin with, you should remove most of the tests that Vamsoft has enabled by default. The only options that you want to enable are:

greylisting (before arrival)
Active directory (before arrival)
Auto sender white list (both).
If you are not using Windows 2003 then also enable the tar pit option.

Bind the application to the server for both inbound and outbound.

Auto sender white list and greylisting do not need any configuration. However you do need to configure the Active Directory test.

Under Tests, choose Active Directory and then settings.

On the tab "Integration Mode" leave it to "Live Query Mode"
On the tab "Directory" set the LDAP root to use a specific tab. The format of the address is as follows:

LDAP://192.168.22.2/DC=domain,DC=local

Where…
- 192.168.22.2 is the IP address of your global catalog domain controller
- DC=domain,DC=local is the format of your ad domain.
For example, if your domain was exch.domain.local, then the path would be:

LDAP://192.168.22.2/DC=exch,DC=domain,DC=local

Testing

Before making any changes to Exchange which affect live email, you should test the server using telnet. However, Vamsoft ORF by default will auto white list any connections coming from internal machines. Therefore to test the recipient filtering and greylisting you will need to use a machine that is coming in from an outside connection.

If you don't have a spare IP address, use port translation on your firewall to allow you to connect on an alternative port.

Exchange Configuration Changes

When your testing is complete, you will need to change the configuration of your Exchange server to use the new server for outbound email. This change will affect outbound email almost immediately, so ensure that the new gateway server is ready before making the changes.

The reason that you ask Exchange to send email out through this gateway server is that Vamsoft ORF can build an automatic white list. This means that when someone replies to an email message that one of your end users has sent, it is not subject to greylisting and will arrive immediately.

To go live with this for Exchange, make the following changes.

Exchange 2000/2003
For the older versions of Exchange, configure a new SMTP connector to use a smart host. Enter the IP address of this server as the smart host, remembering to include it in [ ] - [192.168.22.2]
If you are using authentication then you need to enable that as well and enter the account you created earlier.
If you are using the SSL certificate, then enable TLS on the connector as well.

Exchange 2007
For Exchange 2007, you need to configure a new Send Connector. This is set under Organisation Configuration in Hub Transport. Set the type as Internet, not internal. Set the domain as * and then set the smart host.
Authentication should be set if you are using it, along with TLS if you have put an SSL certificate on to the other server. Finally set the source server as required. In a single server site this would be your only Exchange server.

Specifying Different Routes for Different Domains

If you have domains where you need to send email via another host (your ISP for example) these will have to be sent using SMTP/Send Connectors as before, therefore bypassing the server in the DMZ and Vamsoft. This could mean that replies are delayed as they are not white listed.

While it is possible to set a smart host on IIS SMTP, you are faced with two settings.

a smart host on the SMTP virtual server itself (which will cause problems with email delivery coming to your server)
setting up each domain in IIS SMTP and then setting the smart host (this is a problem as you need to set the option to allow emails for that domain to be relayed, meaning that anyone could bounce emails off your server for that domain).

One option round that problem would be use a second SMTP virtual server on this relay server and disable anonymous authentication on the SMTP virtual server. Use authentication for receiving email and disable anonymous authentication.

On Exchange set up an SMTP/Send Connector to send emails for those specific domains to the additional SMTP server. Remember to bind Vamsoft ORF to the second SMTP virtual server so that the outbound emails are white listed automatically.

Related Articles

SMTP Connector (Exchange 2003)
Tarpit (part of the recipient filtering article)
Greylisting (external blog posting - new Window)

Sponsored Links

Last Page Update:
09/03/2008

Back to the Top
Contact Us - Advertise on amset.info
Exchange Index - Home Page

Broadband - Exchange - Login Scripts - Network Admin - Outlook - Windows Mobile - Windows

Â© Amset IT Solutions Ltd. 1998 - 2008. All rights reserved. Reproduction of any content on this web site is prohibited without express written consent. Use of this web site is subject to our terms and conditions. All trademarks and registered trademarks are property of their respective owners.Â This site is not endorsed or recommended byÂ any company or organisation mentioned on this site. This site is to provide guidance only and as such we cannot be held responsible for any consequences of following the advice given.