Configure asterisk with wildcard extensions

This post covers a common use case when deploying a phone system: Somebody in the business needs to be able to record voicemail greetings for other people.

This assumes that you are already running asterisk, and that people already have something in your Dialplan (extensions.conf) for people to record a greeting:

; Record voicemail greeting
exten => *,1,AGI(scripts/record-voicemail-greeting.pl);

To run this script as somebody else, in the phone sense, we just need to change the CallerID before we run it. Some things you’ll need to know to do this:

  1. An “X” in the extension matches any digit
  2. EXTEN is a variable holding the current extension
  3. CALLERID(num) is another variable, which holds the CallerID number
  4. ${EXTEN:2} is a “substring”, which cuts the first two letters off the extension

With that in mind, if * records your own voicmail, then **4567 would record 4567’s voicemail using this snippet:

; Record other person's voicemail greeting
exten => **XXXX,1,Set(CALLERID(num)=${EXTEN:2})
exten => **XXXX,2,Goto(*,1)

Of course, it would be a terrible idea to enable this for the whole business, which is why you can also check the CallerID before you change it. This alternative snippet allows you to record any voicemail greeting, but only if you are calling from 1234.

; Record other person's voicemail greeting (if calling from phone 1234)
exten => 1234/**XXXX,1,Set(CALLERID(num)=${EXTEN:2})
exten => 1234/**XXXX,2,Goto(*,1)

Locking down your VOIP setup with a SIP Threat Manager

If you run a Voice over IP network which is available from the Internet, then it’s quite important to lock it down properly, so that it isn’t hijacked for relaying spam calls at your expense.

This article will cover the steps you need to deploy the SIP Threat Manager from Allo, which you can think of as a security-focused SIP proxy & firewall.

Topology

For example, you might have SIP clients both on-site with your server, and also allow people with their own devices to connect directly from home. One of the simplest ways to harden this setup is to add a specialised SIP router between your server and the Internet, to filter connections and log security-relevant events:

VOIP network with STM

VOIP network with STM

This post will show you how to migrate your VOIP network to this more secure topology using Allo’s SIP Threat Manager..

The Allo STM Box

Allo STM front
Allo STM back

The STM itself is only small box, with two 100 Mbit/s Ethernet ports, and two USB ports. It is USB-powered, so one of these ports is for powering the box, and the other is for connecting external storage for log files. It is not power-hungry, and I was able to run it from a laptop USB port without any issues. Other than this, it’s externally a typical embedded network device: it has indicator lights, a power switch, a factory reset button, and a console port.

Allo STM internal

Internally, it runs a MIPS processor, which appears to host Snort on embedded Linux.

Initial Setup

Although the box advertises that it will work out-of-the-box, I found that it was easier to configure the box to match my network, than to re-work my network around the box. This section will simply show you how to get logged in and change the box’s IP address.

First up, I took a look at the console, which is accessible at a baudrate of 38400. It’s far from the most functional CLI around, so I only made use of the factoryreset function to get a clean slate. It showed a few of the open source packages running, such the dropbear sshd, lighttpd and crond among other familiar programs:

Please press Enter to activate this console. 
Starting pid 942, console /dev/ttyS2: '/usr/bin/maincli'
Shield STM Appliance Appliance
shield> factoryreset

Will output 1024 bit rsa secret key to '/etc/dropbear/dropbear_rsa_host_key'
Generating key, this may take a while...
Public key portion is:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAgwCKBcVlWK+UiiELbg2CNfOt9rNmj51dmyz7d10MgRfAk9XU9x+kmlMueCFEBMTchsaywigLw0yFqeMZ
Fingerprint: md5 50:5b:c2:64:d4:87:f8:86:ab:c6:e1:59:e4:16:c2:cf
Generating a 1024 bit RSA private key
...++++++
...................++++++
writing new private key to '/etc/lighttpd/webserver.pem'
-----
ip: RTNETLINK answers: No such process
Jan  1 00:40:41 crond[875]: crond 2.3.2 dillon, started, log level 8

mount: mounting /dev/sda1 on /cf/disk failed

The CLI command show ip confirmed that the default IP of the box is 192.168.100.1, netmask 255.255.255.0. From a Linux laptop, you can change IP to something nearby and confirm that you can see the device with these commands:

# ifconfig wlan0 192.168.100.2 netmask 255.255.255.0
# arp-scan -l
Interface: wlan0, datalink type: EN10MB (Ethernet)
Starting arp-scan 1.8.1 with 256 hosts (http://www.nta-monitor.com/tools/arp-scan/)
192.168.100.1	00:17:f7:00:9b:0a	CEM Solutions Pvt Ltd

The STM is then accessible via the web address https://192.168.100.1, with the default credentials admin / admin.

STM login

The IP setup is located under Device → General Setup. Change this to DHCP or a spare address on your network:

STM IP config

Configuration

Now that you can access the STM from any device, your first task is to change the admin password. The button for this is in the top-right:

2015-01-stm-screen21

The STM only allows one session at a time- whilst it’s a good idea not to log in twice, this was a surprising limitation. At the STM does not act as its own SIP endpoint, my server was already able to contact the Internet through it at this point.

I quickly screen captured the available settings so that you can click through them. Some of these are SIP-specific, and others of which are general firewall features. One of the more interesting features which you can’t set up with iptables is location-based IP filtering. This could, for example, block problematic SIP calls coming from fraud hotspots in areas where your organisation doesn’t operate.

So once the network is set up on the STM, no changes need to be applied to your SIP server, other than its gateway or IP.

Notes

Whilst this box works as it’s supposed to, I found it to have an un-polished user experience.

  • The network interface labels on the box had a label over them with the opposite information.
  • The box is closed on port 80: It doesn’t reply to HTTP requests, even to redirect them to HTTPS.
  • The command prompt wasn’t as useful as other network devices.
  • I couldn’t get SSH login or NTP to work, although I didn’t investigate these in great detail.
  • The LAN interface (but not the WAN interface) did not light up when connected to a Gigabit POE network, but did work on a 100 megabit network.

However, there are some positives: The 100 Mbit/s interfaces are more than sufficient for voice traffic, the configuration was simple, and USB is a good choice of power supply for equipment which can be connected directly to a server.

Do you really need another box?

This depends on your setup. If your VOIP server doesn’t speak to the Internet, then this box wont fit into your topology.

If it only sees the outside world via an ISP-run SIP trunk, then this type of security is probably not necessary either. Security measures you would use instead are:

  • Use firewall rules to restrict connections so that only the SIP trunk can speak to your VOIP server.
  • Configure your VOIP server to “stay on the line” for calls (directmedia=no in Asterisk) so that the phones do not speak directly to the trunk, and disallow registration from the Internet.

If your VOIP server accepts connections from the public internet throgh SIP, then some sort of separate, SIP-aware firewall or proxy is highly advisable.

Acknowledgement

Thanks to Allo (allo.com) for sending in the box which is used for this example setup.

How to set up Asterisk in 10 minutes

Asterisk is an open-source IP PABX, meaning it lets you run a phone system over your computer network. Whilst IP telephony has been gaining the upper hand over traditional PABX’s for years, few people outside the industry realise just how easy it is to set up your own phone server.

With this guide, you can turn any Linux device into your own PABX for free. We’ll set up two SIP devices on a small network, so that they can dial each other.

Diagram showing how the two example users will connect to the server

Prepare the environment

First, you should get something with Linux. A virtual machine, a spare laptop, a Raspberry pi- anything.

Install asterisk with one of the following commands, depending on your distribution:

apt-get install asterisk
yum install asterisk

If you don’t have an IP phone handy, then you need a program on your computer which speaks SIP (Session Initiation Protocol). This guide uses Linphone (available for Linux and Windows among other platforms) and the Polycom 331 as examples, but any two SIP endpoints will work just as well for testing.

Make users on the server

Asterisk keeps its configuration in /etc/asterisk. The file we need to edit for this setup is users.conf. Open it up with your favourite text editor:

nano /etc/asterisk/users.conf

The syntax is similar to .ini files. Add two users to the bottom of the file:

[6001]
fullname = Example Bob
secret = 1234
hassip = yes
context = users
host = dynamic

[6002]
fullname = Example Joe
secret = 1234
hassip = yes
context = users
host = dynamic

What does this mean?

[600..]
This is the username, and will become the user’s extension on our small network. Dial 6001 for Bob or 6002 for Joe. Whilst 'bob' and 'joe' could also be used here, numeric usernames are more common.
fullname = ...
Used in the Caller ID.
secret = ...
The password used to log in. In a secure system, you would use something other than 1234!
hassip = yes
This tells Asterisk to make a SIP account for the user. Asterisk supports a few other account types, but SIP is the most widely implemented.
context = users
A context is a bit like a category for the user. The extensions which they can dial depend on this.
host = dynamic
This tells Asterisk that the users don’t have a fixed IP address. This means that they must register periodically with the SIP server so that their IP is known.

To activate these changes, save the file, and reload the configuration through the Asterisk console:

mike# asterisk -r -vvvvvvvvv
CLI> reload
CLI> sip show users
Username                   Secret           Accountcode      Def.Context      ACL  ForcerPort
6002                       1234                              users            No   Yes       
6001                       1234                              users            No   Yes       

All of those v‘s stand for verbose, meaning that the asterisk console will give you more information.

Configure the clients

First you should find out your server’s IP address. From the terminal, you can find this with:

ifconfig

Setting up “Example Joe” on a Linphone instance only takes a few clicks. Add a new account, with 6002 as the identity, and your asterisk server as the proxy address (eg: sip://voip.example.com). Click the image below for an example:

Linphone account setup

Linphone account setup

Meanwhile, the Polycom 331 can be configured as “Example Bob” by navigating the menus on the phone itself, or via the web (suggested). The default login is:

Username:
Polycom
Password:
456
Polycom 331 configuration page

The Polycom 331 web interface.

Your asterisk server address needs to be added under SIP -> Servers -> Server 1, while Example Bob’s identity is added under Lines -> Line1. Click the below images for an example.

Polycom 331 configuration page
Polycom 331 configuration page

Server (SIP) configuration on the left, and line configuration on the right.

Once these are saved, the two clients will register with the server. In SIP, clients periodically register so that the server knows where to find them.

In the asterisk console, you will see something like this:

-- Registered SIP '6001' at 192.168.1.4:5060
   > Saved useragent "PolycomSoundPointIP-SPIP_331-UA/3.3.3.0069" for peer 6001

If registration fails, the console will tell you why, provided that you have set the verbosity high enough

You can check which users have registered with this command:

CLI> sip show peers
Name/username              Host                                    Dyn Forcerport ACL Port     Status     
6001/6001                  192.168.1.4                              D   N             5060     Unmonitored 
6002                       (Unspecified)                            D   N             0        Unmonitored 
2 sip peers [Monitored: 0 online, 0 offline Unmonitored: 1 online, 1 offline]

Unfortunately, even after both users have registered, they aren’t ready to communicate yet.

Add extensions to the server

In the world of VOIP, an extension is not a real loop of copper, but a sequential list of things to do when a number is dialled.

This extra step is where Asterisk gets its flexibility. With your extensions.conf setup, you could set your instance to redirect numbers, or dial for 12 seconds before going to voicemail.

We haven’t told the server what to do, so if “Example Bob” makes a call, it wont work yet:

  == Using SIP RTP CoS mark 5
[Nov 15 07:59:30] NOTICE[6070]: chan_sip.c:22753 handle_request_invite: Call from '6001' (192.168.1.4:5060) to extension '6002' rejected because extension not found in context 'users'.

To add extensions, open extensions.conf up:

nano /etc/asterisk/extensions.conf

The syntax is still INI-like. Under [users], we add the steps for each extension, numbered sequentially. In this case, there is only 1 step for each extension: to dial a SIP user.

[users]
exten => 6001,1,Dial(SIP/6001)
exten => 6002,1,Dial(SIP/6002)

In the Asterisk console, type reload to activate the changes.

Now, as planned, both users on the network can dial each-other and have a chat.

More advanced setups

Ok, time to do a reality check. You’ve built an intercom, but not a full phone system! For a start, you need a way to dial the outside world, and let the outside world dial you. For this, you’ll need to work with hardware and service providers.

From SIP to speaker: Setting up a VOIP Zone Controller

In many scenarios where you find a phone system, you’ll also find a PA system. Ideally, we want the audio from the phones to be able to reach the PA system when a special number is dialled:

Ideal VOIP Zone Controller setup

With analog phone systems, this is a common feature. IP-based systems can do this too, with the help of a VOIP Zone Controller.

I think of a Zone Controller as an “Ethernet to RCA” adapter. This article will show you how to get a CyberData controller set up with Asterisk.

The equipment

I settled on this CyberData 4-port Zone Controller, which is quite small, POE-powered, and is of course a reliable and configurable embedded SIP endpoint:

Front view - CyberData 4 port Zone ControllerBack view - CyberData 4 port Zone Controller

In a typical configuration, the controller would continually receive audio, such as background music, and then silence it when an announcement is made (this is called “night ringer”).

This device has 4 different audio outputs, and waits for a DTMF tone by default, which is then mapped to one or more outputs. I had a very simple use case, and disabled this, so that dialing the box simply caused audio output.

On the VOIP server

Hopefully, you’re using Asterisk to run your phone network! This is the usual platform for VOIP enthusiasts. If you’re dealing with a proprietary system, then you’ll need to skip this section.

First, sip.conf needs to have an entry for the zone controller. This should look like a regular phone. Without going into the intricacies of Asterisk’s SIP configuration, this snippet adds the zone controller as user 1234. It’s given the caller ID “PA System”, and is allowed it to connect from any IP address with the specified secret:

[1234]
type = peer
host = dynamic
context = users
hassip = yes
directmedia = no
fullname = PA System
callerid = PA System
secret = ... (something random here) ...
nat = no

In the extensions.conf, you can then make the device contactable by all phones by adding a line to the users context:

[users]
exten => 1234,1,Dial(SIP/1234)

Setting up the controller

First, you need to plug the audio out into some sort of speaker, and the Ethernet into a POE network with an IP phone system.

In operation - CyberData 4 port Zone Controller

The CyberData devices have a web interface, so you need to find it on the network. I suggest filtering the output of arp-scan -l eth0 on GNU/Linux, which will find the IP address corresponding to the MAC address printed on your device.

Once you’ve found it, the configuration is dead simple. Type the IP address into a web browser, and log in as user “admin”, password “admin”.

Main configuration- CyberData 4 port Zone Controller

The main configuration page of the zone controller.

The web interface is extensive, and shows the depth of options which are used in this niche application, such as custom audio snippets and test routines.

Of course, you may want to adjust the network configuration, which is on the “Network Configuration” page. Once you adjust anything, no changes will take effect until you save and reboot.

IP configuration- CyberData 4 port Zone Controller

IP configuration panel

Remembering the login and password you set up in sip.conf, you will need to fill in the SIP configuration as well.

SIP configuration- CyberData 4 port Zone Controller

SIP configuration panel

Once you reboot, you should see the device register from the Asterisk console, and it will then be reachable. More configuration options to explore include:

  • Tick “Bypass SIP DTMF Entry” in “Zone Config” if you don’t have zones.
  • Set the admin password (!)
  • Take a backup by exporting the configuration from the main page

Good luck!

Patton CLI commands

Patton SN4114 and SN4120

Patton makes a bunch of gateways to connect computer networks to telephone networks. They are some of the most amazingly configurable boxes in the world (once you figure out how the damn things work!)

Setting the admin password via the web interface has never worked on any of the 5 SmartNodes which I use, but rather than try a firmware upgrade, I thought I’d try the CLI.

Whilst I was logged in, I noticed that they all support SNMP, which I also can’t find out how to enable via the web interface.

enable
    configure
        administrator <username> password <password>
        snmp community <public> ro
        copy running-config startup-config
        exit
    exit
exit

Why SNMP?

SNMP lets you check just about anything. In my case, I use it to find out if the ISDN lines are down, via the check_snmp plugin for Icinga.

Loading Asterisk CDR into a database

In the interests of accurate accounting, Asterisk creates a Master.csv, logging all calls and a few things about them.

This page on wiki.asterisk.org has a good breakdown of what all the fields mean.

I put together this MySQL table so that the data can be processed for data-analysis, accounting, or whatever it is that the data is needed for.

-- Code to create MySQL table for Asterisk CDR data
-- Field descriptions from:
--   https://wiki.asterisk.org/wiki/display/AST/CDR+Fields
CREATE TABLE cdr (
    accountcode VARCHAR(256) COMMENT 'What account number to use, (string, 20 characters)',
    src VARCHAR(80) COMMENT 'CallerID number',
    dst VARCHAR(80) COMMENT 'Destination extension',
    dcontext VARCHAR(80) COMMENT 'Destination context',
    clid VARCHAR(80) COMMENT 'CallerID with text',
    channel VARCHAR(80) COMMENT 'Channel used',
    dstchannel VARCHAR(80) COMMENT 'Destination channel if appropriate',
    lastapp VARCHAR(80) COMMENT 'Last application if appropriate',
    lastdata VARCHAR(80) COMMENT 'Last application data (arguments)',
    tsstart DATETIME COMMENT 'Start of call (date/time)',
    tsanswer DATETIME COMMENT 'Answer of call (date/time)',
    tsend DATETIME COMMENT 'End of call (date/time)',
    duration INT(11) COMMENT 'Total time in system, in seconds (integer), from dial to hangup',
    billsec INT(11) COMMENT 'Total time call is up, in seconds (integer), from answer to hangup',
    disposition ENUM('ANSWERED', 'NO ANSWER', 'BUSY') COMMENT 'What happened to the call',
    amaflags ENUM('DOCUMENTATION', 'BILL', 'IGNORE') COMMENT 'What flags to use, specified on a per channel basis like accountcode.',
    uniqueid VARCHAR(32) COMMENT 'Unique Channel Identifier',
    userfield VARCHAR(256) COMMENT 'user field: A user-defined field, maximum 255 characters'
) ;

The second-last field is not present on all installations, but it is on mine. The version I’m on is:

asterisk:~# asterisk -V
Asterisk 1.8.10.1~dfsg-1ubuntu1

Australian telephone tones

I made this set of telephone tones for an IP-PBX running asterisk. A lot of telephone-related audio files on the Internet seem to be recordings of actual beeps, so the quality is not great. Given that they are trivial to synthesise with Audacity, I made these:

To convert them to .ulaw format for asterisk (click here to download archive), I saved them as a .wav file and ran this bash script over the directory:

#!/bin/bash
for a in *.wav; do
    sox $a -r 8000 -t ul $a.ulaw
done

Making tones for other countries

Get a copy of the document "Various tones used in national networks (According to ITU-T Recommendation E.180) (03/1998)". It describes the tones used in most of the world. There is also a Cisco tones database, which has slightly different numbers.

Open up Audacity and use the Generate → Tones feature to make the first beep. Where there are multiple frequencies, make each frequency in different tracks, then merge them into a single track.

Use Generate → Silence to add the delay on the end, then continue from there

Afterward, you need to normalise the volume. Read the number from the volume Cisco tones database, then use Effect → Amplify, and set the new peak volume to the value shown — for the above tones this is -10 or -13dB.

Alphanumeric phone numbers

Some popular phone numbers (eg. 1300-FOOBAR) are not numbers at all. If you are running a VOIP server then you may be interested in this snippet of PHP code to convert them into actual numbers, allowing users to dial by typing the more familiar form.

function normaliseTelephoneNumber($start) {
	/* Return an extension with numbers substituted in place of letters for dialling */
	$map = array(	"A" => "2", "B" => "2", "C" => "2",
			"D" => "3", "E" => "3", "F" => "3",
			"G" => "4", "H" => "4", "I" => "4",
			"J" => "5", "J" => "5", "L" => "5",
			"M" => "6", "N" => "6", "O" => "6",
			"P" => "7", "Q" => "7", "R" => "7", "S" => "7",
			"T" => "8", "U" => "8", "V" => "8",
			"W" => "9", "X" => "9", "Y" => "9", "Z" => "9",
			"+" => "+", "*" => "*", "#" => "#");
	$new = "";
	$hasnumber = false;
	$ext = strtoupper($start);
	for($i = 0; $i < strlen($ext); $i++) {
		$c = substr($ext, $i, 1);
		if(isset($map[$c])) {
			$new .= $map[$c];
			if($hasnumber == false) {
				/* No numbers before letters */
				return $start;
			}
		} else if(is_numeric($c)) {
			$new .= $c;
			$hasnumber = true;
		}
	}

	if($hasnumber == true) {
		return $new; /* Return numeric version as appropriate */
	} else {
		return $start; /* Leaves full words like "joe" or "bazza" unchanged */
	}
	return $new;
}

Note that this will only alter the number if it begins with numbers. This is to make sure that (at least in my case) the local network extensions don't get messed with:

echo normaliseTelephoneNumber("1300-FOOBAR")."n"; /* 1300366227 */
echo normaliseTelephoneNumber("mike")."n"; /* mike */

Using speech synthesis in AGI apps

For some of my telephone apps, stringing together pre-recorded messages is a pain, and it’s not even useful for more dynamic content. This is how I got PHP to do sound output via festival whilst using the asterisk AGI.

First, I made a folder called agi in my /var/lib/asterisk/sounds/en/. This is where we will cache sound files.

This shell script makes a sound file with your text, and saves it in that directory:

#!/bin/sh
echo "$1" | text2wave -otype ulaw -o /var/lib/asterisk/sounds/en/agi/$2.ulaw

For the PHP side, we just use MD5 to name the files based on their contents, and only run the shell script if no file has been created with the text we want.

In this project, $agi refers to an instance of PHP AGI. (highly recommended)

function sayNow($string, $escape = "") {
	global $agi;
	$sounds_path	= "/var/lib/asterisk/sounds/en/agi/";
	$agi_path	= "/var/lib/asterisk/agi-bin/"
	$fn = md5($string);
	$string = str_replace(""", "", $string);
	$string = str_replace(";", ",", $string);
	$string = str_replace("'", "", $string);
	$string = str_replace(""", " slash ", $string);
	if(!file_exists("$sounds_path$fn")) {
		system("$agi_path$common/sound.sh "$string" "$fn"");
	}
	return $agi -> stream_file("$sounds_path$fn", $escape);
}

This makes sound output at least a million times easier. We use it like this:

$a = sayNow("Enter the number, then press hash");

That’s all there is to it. I should probably also set up a cron job to clear the sound files at the end of each week.

Parsing Asterisk Configuration

If you are writing for Asterisk PBX, you may feel the need to create a public telephone directory.

Today we’ve been putting together Phonebook, a web-based phonebook which pulls data from an asterisk server. We used JQTouch for the interface, with PHP to process data files. We also use a CSV file exported from our hosted gmail to ensure that people’s names are spelled correctly.

The configuration

Asterisk’s configuration files are INI-like, and you’ll find them in /etc/asterisk/ on most systems. Just one important thing though, asterisk lets you use brackets ( ) in caller-IDs, etc, which will cause PHP’s parse_ini_file() function to fizzle out and die due to an alleged syntax error.

To save yourself some trouble, use this little class I wrote, which will load INI data into an associative array:

class ns_ini_parser {
	/* Mike's non-standard INI parser for asterisk files. https://mike42.me
		Note that PHP's parse_ini_file will die with a syntax error on key = value (bracket), which is unacceptable */

	function parse_string($string) {
		$lines		= explode("n", $string);
		$section	= "0";
		$result		= array();
		foreach($lines as $line) {
			$line = trim($line);
			if($line == "" || substr($line, 0,1) == ";") {
				/* Comment, no action */
			} elseif(substr($line, 0,1) == "[") {  /* [section] */
				$l		  = strlen($line);
				$line		  = trim(substr($line, 1, $l - 2));	/* Strip brackets */
				$section	  = $line;
				if(!isset($result[$section])) {
					$result[$section] = array();
				}
			} else {				/* key = val */
				$parts = explode("=", $line);
				$key = trim($parts[0]);	/* The key is everything left of the equal */
				unset($parts[0]);	/* Got that, unset it */
				$val = trim(join("=", $parts));	/* Value is everything on the righht */
				$result[$section][$key] = $val;
			}
		}
		return $result;
	}

	function parse_file($path) {
		$string = file_get_contents($path);
		return $this -> parse_string($string);
	}
}

The example usage below will list user’s extensions next to their caller ID, which you could use for a web phonebook:

/* Example PHP code to parse asterisk configuration */
$config_parser = new ns_ini_parser;
$users  = $config_parser -> parse_file("/etc/asterisk/users.conf");

foreach($users as $id => $user) {
	if(is_numeric($id)) { /* Only show numbers, not other sections */
		echo "<a href="tel:$id">$id</a> ".$user['fullname']."<br />";
	}
}

I don’t think it gets easier than that! I’ll post the rest once you can manage the contacts as well.