Caller ID Superfecta: THE MODULE

[tshif]

NOTE: This thread covers the early released versions of the Caller ID Superfecta module for FreePBX.
THIS INFORMATION IS OBSOLETE and is maintained here for historical and reference purposes only.
Go here for the current CID Superfecta thread. (tshif)

Caller ID Superfecta: THE MODULE
If there is one thing that is universally expected when people set up a PBX, it’s got to be Caller ID. And, as we all know – Caller ID is the responsibility of the receiving party’s telephone providers. Um, in simpler terms, that's US, and our PBX's.

There’s at least two very good examples of this function right here in the forum. The Ultimate CNAM, originally created by former forum participant Titanous and the Caller ID Superfecta, originally written by our own Nerd Uno, Ward Mundy.

As a testimony to open software – both of these creations have now exceeded their original programmers’ vision because the community here at the PBXIAF forums has contributed to their growth. Today – each of these Caller ID look up programs has been extended to include more sources of information than the original authors would have imagined.

There’s just a couple little hiccups with getting all this good stuff working right. The install scripts have to be run from a CLI, and their configuration parameters have to be edited in a text file editor. I always thought it should be possible to install, configure and use these great tools without having to do all that. And - when its time to upgrade – its just to messy for an end user to attempt.

Since I’ve been on a module kick of late – it occurred to me that all that could be solved by “modularizing” the Caller ID function into a FreePBX Module.

Open Software Should Be Open
It’s a rare and beautiful thing to see folks contributing to a coding project. But have a look around here at the PBXIAF forums and you see it; A lot of it. It may not be unique – buts its rare enough. I most sincerely hope that all of you who have contributed to the Superfecta code upgrade along the way will consider updating this module in the same way. I will certainly keep track of its status – but everyone here who has been contributing should feel as welcome to provide upgrades to the module as they have for the non module version of the Superfecta. I hope to have it in a svn soon – so that all can have easy access.

The module still installs and uses callerid.php. The only changes I have made were to comment out the configuration section in the script, and add a few lines of code to cause it to gather those same configuration items from a database. The changes are well commented, and highly localized, just as the previous block of config data was at the top of the script. For the typical upgrade, such as when a provider changes their output, modifying the script will feel identical to how you have done so in the past.

To add additional sources can be accomplished almost as easy with more changes to the supporting module code.

Install and Upgrade
This module should co reside with any and all other CID solutions you maybe running. It will take about 1 minute to complete the install. Upgrading will not wipe out your settings.

Installation:
(This download link was updated on 5-3-2009 to point to the most current version of this module - tshif)
Step 1: Download the module to your desktop From here. (Generally, you'll want to select the most recent released (non-dev) version.)
Don’t decompress the archive!
Step 2: Use a web browser to access FreePBX on your PBX. Choose Tools, Module Admin, Upload Module.
Now, browse to the superfecta-1.0.3.3.tgz file on your Desktop and click the Upload button. Now click local module administration, scroll down and click on CID Superfecta and then the Install radio button. Now click Process, and complete the usual FreePBX install and reload process.
Step 3: Use a web browser to access FreePBX on your PBX. Choose Setup, CID Superfecta.
You will be presented with configuration screen. Establish settings that make sense for your PBX, and Save your changes.
Step 4: Use a web browser to access FreePBX on your PBX. Choose Setup, Caller ID Look Up Sources.
The module installer has already created the entry for Caller ID Superfecta. Select if from the list. In the USERNAME enter MAINT, PASSWORD field enter your current PBXIAF MAINT password.

NOTE: Yes, I know the password is visible. It’s a FreePBX thing. If it means anything to you, I think the password box should be obfuscated (stars) by FreePBX. But until they come around to my way of thinking (lol), consider this. If the person is already viewing this screen, they have to have the password already – or they couldn’t be here. If this STILL drives you crazy - look here for a fix. (4/1/09: (http://freepbx.org/trac/ticket/3613. Sorry - I tried. I guess I didn't do the naked dance around the fire well enough to appease the "great ones". If I ever get so closed-eared and arrogant, please somebody shoot me.)

Conditions/Prerequisites:
TThis module is compatible with the security models used in, and has been tested with the following distributions:
Fonicatec PABX
Foncordiax
PBX In A Flash
TrixBOX CE
The Module’s wiki Page is here.
(If you use a distro other than PBXIAF, read the wiki for more installation help.)

Asterisk Version Compatibility
This module has been tested and found to work correctly with Asterisk versions 1.4 and 1.6.

Credits:
The CallerID Superfecta was written originally by Ward Mundy, as the Caller ID Trifecta. It has evolved to support more sources, and is now maintained by the community of forum users at PBX In a Flash.
The Superfecta was converted to FreePBX Module format by Tony Shiffer & Jerry Swordsteel.

Notice:
The Developement Home page for this module is here.

The Caller ID code undergoes regular maintenance - it is our hope that by placing the module in an svn that everyone who wants to contribute can and still will do so. Besides, this a great way for you to start getting familiar with modules in general!

I’ll post the link to the svn back to this article as soon as its ready.

Until next time
-tshif

 

[wardmundy]

Niiiiiiiiiiiice, Tony!


Would you mind adding the new SugarCRM module, too?

 

[tshif]

We finished adding the SugarCRM piece to Superfecta Module. I don't have any way to test it. Here is the module with the SugarCRM Part. Would some one with SugarCRM please try this out for me? (This code is obsolete. This thread is still here for archival purposes only.5-3.090)

NOTE: Because there is a database change - if you have the previous version of this module installed, it will be necessary to use FreePBX to UNINSTALL the previous module, and then install this new one. Your configuration settings for Superfecta will have to be re-entered.

Somebody - PLEASE let me know if this works with SugarCRM -

Tony

 

[Lost Trunk]

This module depends up the Asterisk DB AMP user ID and password being set at their default values. The module script may be edited to reflect your actual id and passwords if you have changed them.

Any particular reason you can't simply read those vales from the /etc/amportal.conf file? There are distributions based on FreePBX where these passwords are not set to the FreePBX default, but you can always get the correct AMPDBUSER and AMPDBPASS values (if that's what you're using) from amportal.conf.

 

[tshif]

It's something we are working on for future updates.

Due to the strength of the PBXIAF security methods - very few people change that ID/PW. I'm sure you've read Wards postings on the topic of changing them before.

 

[tshif]

Obfuscating the Password Field Contents in FreePBX Caller ID Lookup Sources

Obfuscating the Password Field Contents in FreePBX Caller ID Lookup Sources.

For this project - we had to enter our MAINT ID and Password into the Caller ID lookUp Sources section of FreePBX.

I discovered that the field displays plain text. (http://freepbx.org/trac/ticket/3613. Sorry - I tried. I guess I didnt do the naked dance around the fire well enough to appease the "great ones".)

I KNOW that anyone who has gotten this deep into the system would already have to KNOW that password - but I *hate* that it displays it in plain text.

Heres a quick fix for that. It should last until you update the CIDLookup module of FreePBX at some time in the future.

Using your favorite editor, open this file:

/var/www/html/admin/modules/cidlookup/page.cidlookup.php

On line 145, locate the section that reads: type="text" and change it to read: type="password"

Save your changes. Now, go back and view one of your caller ID look up sources. The password field is not displaying plain text any more.

This process has been tested with FreePBX v 2.5.1.1, and should work with all versions of the cid lookup module.

-tshif

 

[Lost Trunk]

It broke

Doesn't work here - I get this in the CLI (changed the incoming number, otherwise this is exactly how it appeared):

    -- Executing [s@macro-user-callerid:19] NoOp("SIP/1-pstn-b7e65348", "Using CallerID "<!DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN>
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href=https://127.0.0.1/admin/modules/superfecta/callerid.php?thenumber=9999999999>here</a>.</p>
<hr>
<address>Apache/2.2.3 (CentOS) Server at 127.0.0.1 Port 80</address>
</body></html>" <9999999999>") in new stack

That html code actually appears in my CLI output, and as shown, the CID name is set to

<!DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN>

I did change the password in the script, etc. and the weird part is that if I execute the url directly from a browser, using

[U]https://serveripaddress/admin/modules/superfecta/callerid.php?thenumber=9999999999[/U]

Then it works. The CID lookup source "Path" is set to /admin/modules/superfecta/callerid.php which seems correct, so I don't get the problem.

EDIT: If I try to use this URL with WGET on the server itself, using an ssh connection, I get this error:

--01:13:57--  http://127.0.0.1/admin/modules/superfecta/callerid.php?thenumber=9999999999
Connecting to 127.0.0.1:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://127.0.0.1/admin/modules/superfecta/callerid.php?thenumber=9999999999 [following]
--01:13:57--  https://127.0.0.1/admin/modules/superfecta/callerid.php?thenumber=9999999999
Connecting to 127.0.0.1:443... connected.
ERROR: cannot verify 127.0.0.1's certificate, issued by `/C=--/ST=SomeState/L=SomeCity/O=SomeOrganization/OU=SomeOrganizationalUnit/CN=localhost.localdomain/[email protected]':
  Self-signed certificate encountered.
ERROR: certificate common name `localhost.localdomain' doesn't match requested host name `127.0.0.1'.
To connect to 127.0.0.1 insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.

Two other questions:

At one time, in an earlier incarnation, this also had a lookup source for Canadian area codes - did that stop working, or did the code for that just get lost somewhere along the line?

Also, if the call does come in with valid Caller ID, is this smart enough to not do a lookup anyway, or does it just assume that any Caller ID name received must be wrong? It would be nice if there were a way to make this trigger only if there is no name received, or if the name received matches certain patterns (such as "Unknown Caller" or whatever the provider sends when it can't do the lookup).

 

[tshif]

You missed a step

Those symptoms:

<!DOCTYPE HTML PUBLIC -//IETF//DTD HTML 2.0//EN>

usualy mean only one thing - you missed this step (or perhaps didn't save):

Step 4: Use a web browser to access FreePBX on your PBX. Choose Setup, Caller ID Look Up Sources.
The module installer has already created the entry for Caller ID Superfecta. Select if from the list. In the USERNAME enter MAINT, PASSWORD field enter your current PBXIAF MAINT password.

At one time, in an earlier incarnation, this also had a lookup source for Canadian area codes - did that stop working, or did the code for that just get lost somewhere along the line?

Also, if the call does come in with valid Caller ID, is this smart enough to not do a lookup anyway, or does it just assume that any Caller ID name received must be wrong? It would be nice if there were a way to make this trigger only if there is no name received, or if the name received matches certain patterns (such as "Unknown Caller" or whatever the provider sends when it can't do the lookup).

Those are great questions - I have no idea. The Module incorporates the Superfecta callerid.php script - theres a whole thread devoted to its care and maintenance. Check it out when you get time.

I do think the script is smart enough to NOT check each source once it finds a Hit, it stops looking - but I could be wrong.

- tshif

 

[Hat]

Tony,

Thanks for yet another add-on that installs cleanly. So far this works as advertized. I do not have Sugar CRM, soooo...I can't help you with that one.


Tom

 

[tshif]

Tony,

Thanks for yet another add-on that installs cleanly. So far this works as advertized. I do not have Sugar CRM, soooo...I can't help you with that one.


Tom

Our first success report! Thanks for letting me know Tom

 

[Lost Trunk]

Those symptoms:


mean only one thing - you missed this step (or perhaps didn't save):

Step 4: Use a web browser to access FreePBX on your PBX. Choose Setup, Caller ID Look Up Sources.
The module installer has already created the entry for Caller ID Superfecta. Select if from the list. In the USERNAME enter MAINT, PASSWORD field enter your current PBXIAF MAINT password.

No, that wasn't it - I edited my post after I figured out that it's a certificate problem. The reason it works from my web browser is that long ago I told it to ignore the invalid certificate warning that comes when you try to use an https url on the server. But whatever method the Superfecta uses to make the http call gets automagically turned into an https link (this happens even when I try to use wget) and then fails because it's not set to ignore the certificate problem. I looked through the script but could not find where the actual call is being made (do you use a wget or curl or what, and where is it?) but anyway that is the problem.

 

[tshif]

Fascinating - Im surprised to find its trying https -

It's even more surprising to find that happens with wget. It will be interesting to see if others who use the certificate script has the same issue.

Im sure some of the high IQ guys will be able to explain this behavior. Anyone?

Edited:
I had an idea.
First, make sure your system can ping itself at localhost.localdomain. If not, make an entry in the hosts file so that it WILL.
then, in the FreePBX Caller ID Lookup sources, in the HOST field, instead of 127.0.0.1, try putting localhost.localdomain.

This way the certificate will match the address being called - it might just relax and work. Let us know if this works for you ok?
Tony

 

[Lost Trunk]

With regard to the Canadian area codes

I found an old copy of the lookup script from way back when but the one I have was written in perl, not php, so I'm not even sure it's the same thing. However it did have the ability to go to Canada 411 to do a reverse lookup. The syntax for the call seems to have changed since then, this works now:

http://www.canada411.ca/search/re/1/9999999999

(9999999999 is the 10-digit number to look up)

The trick is to make sure that you only call this if the area code is one of the Canadian codes, e.g. 204, 226, 250, 289, 306, 343, 403, 416, 418, 438, 450, 506, 514, 519, 581, 587, 604, 613, 647, 705, 709, 778, 780, 807, 819, 867, 902, 905. Well, that and parsing the name out of a much too large page.

A comment in the perl script indicated that Whitepages would also accept Canadian area codes (don't know if that is still true). Just thought you might want to know.

 

[Lost Trunk]

Nope...

I still get the same error

    -- Executing [cidlookup_3@cidlookup:1] Set("SIP/1-pstn-b7e12320", "CALLERID(name)=<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="https://localhost.localdomain/admin/modules/superfecta/callerid.php?thenumber=9999999999">here</a>.</p>
<hr>
<address>Apache/2.2.3 (CentOS) Server at localhost.localdomain Port 80</address>
</body></html>") in new stack

I'm too tired to mess with this any more tonight, but just thought I'd let you know that using localhost.localdomain apparently made NO difference.

 

[Lost Trunk]

It's probably not your module, it's the CallerID Lookup Sources module.....

In giving this a bit more thought I realized that there are two problems, neither of which you can do much about (unless you want to patch the Caller ID Loopup Sources module). The first is that the Caller ID Loopup Sources module allows for http connections but not https - if the web server has been configured to not allow insecure connections, a call to http://127.0.0.1/admin/modules/superfecta/callerid.php?thenumber=[number] will fail with a message that "this document has moved" and a link to the https equivalent. That's what's been giving the message I quoted previously.

But even if HTTPS was listed as a source type in addition to HTTP, you still have to take into account that many if not most users will have an unsigned certificate and the connect attempt will still fail. The solution to this would be when allowing https, they should add the -k option to the CURL invocation (assuming they are using curl to make the call) so it looks like this (as an example):

curl https://127.0.0.1/admin/modules/superfecta/callerid.php?thenumber=[number] -k
or
curl -k https://127.0.0.1/admin/modules/superfecta/callerid.php?thenumber=[number]

Anyway that is the problem as best I can get a handle on it. Must. Get. Sleep.

 

[tshif]

Lost Trunk -
THANK YOU for all the work you put in to trouble shoot the issue. I think your take on the situastion is spot on.

The first is that the Caller ID Loopup Sources module allows for http connections but not https - if the web server has been configured to not allow insecure connections, a call to http://127.0.0.1/admin/modules/super...php?thenumber=[number] will fail with a message that "this document has moved" and a link to the https equivalent. That's what's been giving the message I quoted previously.

Im not entirely sure if we can fix it - anyone else have thoughts? How common is it among us to have the web server configured to not allow insecure connections, I wonder?

-tshif

 

[Lost Trunk]

Im not entirely sure if we can fix it - anyone else have thoughts? How common is it among us to have the web server configured to not allow insecure connections, I wonder?

I think it's the default under Elastix, just for one example, but don't quote me on that. Anyway, it SEEMS like it should be a simple fix. I tried specifying port 443 as the port but that produced a nice error message:

    -- Executing [cidlookup_3@cidlookup:1] Set("SIP/2-pstn-b7e191b0", "CALLERID(name)=<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
Reason: You're speaking plain HTTP to an SSL-enabled server port.<br />
Instead use the HTTPS scheme to access this URL, please.<br />
<blockquote>Hint: <a href="https://localhost.localdomain/"><b>https://localhost.localdomain/</b></a></blockquote></p>
<hr>
<address>Apache/2.2.3 (CentOS) Server at localhost.localdomain Port 443</address>
</body></html>") in new stack

So, okay, I thought, let's fix that. I went into /var/www/html/admin/modules/cidlookup/functions.inc.php and found line 161 currently reads as follows:

                            $url = sprintf('http://%s%s/%s?%s', $auth, $host, $path, $query);

Now I don't know a bit of PHP, but this SEEMED like a simple text replacement. So I tried changing it to this:

                            $url = sprintf('-k https://%s%s/%s?%s', $auth, $host, $path, $query);

I tried a few other variations as well, and the effect was the same - no name returned at all. I have no idea why it's not working (I am not a coder so it's probably some quirk of PHP that I'm not aware of). Had it worked, my suggestion would have been a simple if-then-else test - if the port is 443 it executes the second line (the -k https line) and if any other port then the original line. But it didn't work, and then I found that even trying to execute the curl command from the command line wouldn't work anymore (it complained it couldn't access the database, or something to that effect) so I finally had reached my frustration level and uninstalled the Superfecta module and undid my test changes in /var/www/html/admin/modules/cidlookup/functions.inc.php.

Sorry, I tried, but I have other things I need to do that are far more pressing than this.

 

[aster1sk]

Thanks, works great!

 

[tshif]

Thanks, works great!

THANKS for letting me know - anyone else has success for failures to report?

 

[jpe]

My bad... it working for me.