FOOD FOR THOUGHT Caller ID Number/Name Routing (Personal Whitelist/Blacklist/Graylist etc.) module

Cam__

Member
Joined
May 22, 2013
Messages
43
Reaction score
8
I have an idea for a module. Before I propose it, please note that this is simply a suggestion if someone would like a project that might actually be useful. Personally, I would only be using this on a home system, and I don't actually need this module because I can do the same thing using custom dialplan, but that's possibly not something the average user would be able to do. As far as I am concerned, this is just an idea that I freely offer to whoever might want to run with it.

The basic problem we have today is that we can only blacklist calls by number, and only for the entire system. Before the big forum crash, with the help of lgaetz, TSM, and tm1000, I wrote some modifications to the Swiss Army Knife module to also allow blacklisting by Caller ID name. You can see these modifications by going to this page on Github but the reason it is labeled as NON WORKING CODE is because it requires a patch to FreePBX that most users won't have, though it hopefully will be available in future versions. But even that applied to all calls into a system.

BUT - there may be times that you want to give specific treatment to individual callers, but only in certain circumstances - for example, when they have called a specific DID or group of DID's.

So my idea is this: Build module that can create individual "special routing lists" that are themselves destinations. They would conceptually be somewhat similar to Time Conditions, in that you could create as many as you need and give each one a name. Each "special routing list" would contain either a list of caller ID names, caller ID numbers, or caller area codes - the latter would only compare the first three digits of a ten digit caller ID, or the three digits after the "1" in an eleven digit caller ID number that starts with "1".

Each time you create a new list, you would specify the name of the list, and the type of list - one of the three I just mentioned, though maybe other types could be added later. You could also optionally specify an extension number to associate with the list - more on that in a moment - and a description.

Then after creating the list you would populate it with numbers, names, or area codes. At the bottom of the page, just as with a time condition, there would be two sets of destinations - one to be taken if there is a match with an item on the list, and a default destination to be used if the number is not on the list.

Note that you could chain lists together, that is, make one a destination for another. So, just as an example, you could have a whitelist - calls that are put through immediately. You could then chain that to a blacklist - calls that are never to be put through. That could in in turn chain to a graylist, which might be a list of all toll free area codes, which you might send to something that makes the caller prove it's a human and not a robocall. And if it passes all those tests, meaning it's an unknown caller but not from a toll-free number, you might send them first to an announcement that says "This call may be monitored or recorded" before sending it to the desired extension, because that tends to scare off sales droids and others that wouldn't want you to have recorded evidence of the call. These are just examples, of course - you could pick any destinations you want for any of the lists.

Each list would itself be a destination, just like any other destination. Normally you might send one or more inbound routes to such a list, and then chain the call flow through as many lists as you need, with the last one having a default destination of your choosing.

I mentioned optionally associating each list with an extension number. Doing so would have no effect on call flow. Instead, what it would do is make the number, name, or area code list visible and available for editing in the user portal. Each user would see a list of the routing pages associated with their extension and the descriptions, and could add, remove, or edit numbers on the list. I don't know if it would be a good idea to allow them to modify the destinations (probably not) but really, that is up to whoever might create such a module.

If I were writing it I would store the lists in the Asterisk database, not in MySQL, and also I would avoid using AGI calls like the plague in order to minimize lookup time, which could be important in situations where more than one of these are chained. But, that is just me. If you write it, you can do it however you want. Again, it's just a suggestion, in case anyone might be looking for a project. And if by chance such a feature is already present in a new or upcoming release, then I apologize for wasting everyone's time.
 

Cam__

Member
Joined
May 22, 2013
Messages
43
Reaction score
8
I certainly appreciate performance, but personally I rank usability first, probably because I only run low traffic systems. The complaint I have with using the Asterisk DB is that of portability. If I want to restore a MySQL database from a backup, it is easy. If I want to manually delve into a live database, there are two graphical tools in PIAF alone to facilitate this. All of the backup utilities grab a dump of the asterisk MySQL database. If I want to move an Asterisk DB or recover it, I am on my own or out of luck.

The thing I have observed is that when you are watching the CLI as a call comes in, most of the time the lines scroll by so fast you can't read them. The exception is when there is an AGI call or some large database lookup involving MySQL. In some cases it's unavoidable because of what is being done - a Caller ID Superfecta lookup would be one example - but I've noticed other cases where I believe the delay could have been avoided with a slightly different approach. If you have a delay of, say, one half second, that doesn't seem like much but then of you start doing multiple things that cause such delays, that time is cumulative and could be the difference between a caller abandoning the call or hanging in there until it is answered, especially once you have added two or three seconds or more. And people these days seem to be getting less patient all the time. This is also why I sometimes get a bit nit-picky about gratuitous Wait statements thrown into dialplan when they aren't absolutely essential.

As for moving or recovering the Asterisk database, there actually is a way to do it while the system is running, but it's a bit of a manual process, though someone could probably create a way to make it more automatic pretty easily. Let's say you wanted to me the blacklist form one machine to another. On the machine that already has the blacklist, you can do database show blacklist, which will give you a list of entries in the form:

/blacklist/8005551212 : 1

The trick is that in order to put those to a new database, you have to write them as:

database put blacklist 8005551212 1

In other words you need to remove the forward slashes, multiple spaces, and colon, so that for each line you do database put blacklist number description. Those lines can then be pasted to the CLI on the target system. This is simple string manipulation and my guess is that if someone wanted to they could code a bash script or a script in some higher level language, or even a FreePBX module that would take care of this. The only thing I don't know is how Asterisk stores the database internally. It has to be in a file somewhere, saved in some format that may or may not be decipherable when Asterisk isn't running, but I have no idea where that file is. You can dump the entire Asterisk database using just database show from the CLI, but Asterisk does need to be running for that to work.

All I really know is that lookups in the Asterisk database appear to be blindingly fast, whereas other lookup methods - not so much.
 

tm1000

Schmoozecom INC/FreePBX
Joined
Dec 1, 2009
Messages
1,360
Reaction score
78
Well of course asterisk lookups are faster. Its a sqlite database. Meaning file based database and it's written into Asterisk. You won't find anything faster than a file based database with no remote connection.

I tend to agree with lgaetz If you want to make it all about the asterisk internal DB then you will be spending a lot of time making extremely complex dialplans and honestly it will never happen because eventually you'll get bored or fed up with it or realize it's not worth your time.

This is why superfecta uses mysql.
 

Cam__

Member
Joined
May 22, 2013
Messages
43
Reaction score
8
I have been thinking about how I want to respond to this. I've already acknowledged that Superfecta is sort of a special case, given what it does, although I have to admit that I wish it were a bit faster. But then, that's mostly beyond your control, given that the major delays are the lookups from the various sources on the 'net. You really can't control how long it takes those sources to respond, and in that context, the MySQL lookups are only a minor part of that.

But for what I am proposing, especially if you were to build a chain of more than one instance, the lookup times could become pretty significant, and to me that would be unacceptable. Maybe it would not be to someone else, but I really don't like wasting callers' time.

Of course, as a programmer you would have the option to create code that is fast, or code that is easiest to write. But as a user, I would have the option to not use code that I think is too slow. And that is really all I will say about it.
 

tm1000

Schmoozecom INC/FreePBX
Joined
Dec 1, 2009
Messages
1,360
Reaction score
78
I will bet you I could write up something you want that uses MySQL. Makes a single query to it AND would be able to process under 500ms. Probably less than that. You doubt a local connection to MySQL too much and you are judging it based off of superfecta which is bad coding to be honest and Lorne and I both work on that project and we can contest to how much of a mess it really is.

What is the max time length you are talking about? How many MS? Give me these numbers and I'll mock something up and amaze you. Using MySQL.
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,168
Reaction score
5,199
If you have any doubt about MySQL performance, turn on the AsteriDex lookup and do a simple test lookup. It's lightening fast. The problem with the basic idea is that it relies on CallerID numbers which can be spoofed by thousands of service providers and millions of users worldwide. So special routing that relies on CallerID is next to worthless if security of your system matters IMHO.
 

Cam__

Member
Joined
May 22, 2013
Messages
43
Reaction score
8
What is the max time length you are talking about? How many MS? Give me these numbers and I'll mock something up and amaze you. Using MySQL.

Unless this is something that interests you, please don't do it just to prove to me that it can be done. I say that because it appears Ward doesn't care for my idea, and you liked his post, so if you agree with him that there's some sort of security issue here then my guess is you will never design it to be an actual module that people can download and use. I'm not looking for a one-off special, I'm suggesting a new module that people could actually use. I can do one-offs in extensions_custom.conf.

The problem with the basic idea is that it relies on CallerID numbers which can be spoofed by thousands of service providers and millions of users worldwide. So special routing that relies on CallerID is next to worthless if security of your system matters IMHO.

With all due respect, I just don't see how routing based on caller ID numbers can be a security issue, seeing as how you can already do it to a limited degree in Inbound Routes. Sure, a few hackers can spoof Caller ID, maybe - so what - should we be denied the benefits of using the Caller ID for call routing just because one call in a million might have a spoofed caller ID? Frankly, Ward, and you probably won't like me for saying this, but IMHO you are sometimes waaaay too paranoid about security. To put this in perspective, there is no way that what I have proposed is going to cause someone to have a 50 grand phone bill, which I know is your big concern, unless they use it in an utterly stupid manner, and there are a lot of other stupid ways to make PiaF accessible to hackers. Maybe we should do away with DISA, that can be a HUGE security risk if someone configures it improperly.

If it sounds like I'm a bit perturbed, it's because I take it personally when you call my idea next to worthless because of some ridiculous idea that it might be a security risk. If you don't like it, that's fine, but please don't just pull a spurious reason for not liking it out of thin air. Used the wrong way, PBX in a Flash is next to worthless if security of your system matters. Used correctly and with a bit of intelligence, there is no real security risk, and that's also true of what I have suggested here. I'll bet if one of your developers had come up with this you'd be gushing about what a great feature it is, not calling it a security risk. And I had probably better shut up but I am really kind of p.o.'ed about this!

Putting things into perspective, if you are talking about a breathing person waiting on the phone they won't even notice a half second delay and you can get away with 4-5 seconds delay.

If you consider a 4-5 second delay in any way acceptable then we are not even in the same universe of thought. My test is this. I watch the CLI as a call is coming in. If it hesitates at any point in the dialplan long enough for me to read the statement it's trying to execute, that's too long - the only exception being a necessary wait, and a lot of the waits that people insert into Asterisk dialplan aren't really necessary. I don't know how to quantify that in milliseconds, but just for the sake of discussion, let's say I wanted it to happen within 100 ms - are we talking possible, maybe close, or not in any way realistic? Remember, my original idea was for a module that might be called multiple times during the call flow, but maybe at this point the entire discussion is moot anyway.

Right now I regret having ever suggested this, at least in this forum.
 

tm1000

Schmoozecom INC/FreePBX
Joined
Dec 1, 2009
Messages
1,360
Reaction score
78
Actually cam this is a module we've had in the works for a while when I was asked to code it by tony of Schmoozecom during astricon last October. It's nothing new or magical or fantastic. It's already been mocked up.

Stop over assuming things. Stop thinking I'm doing something just to prove you wrong (this is like the second time now we've run into this together)

My whole point in this thread is that MySQL is fast. Probably under 100ms to build the whole route. Not 1 second.

You started this thread with an idea and now three developers have come in saying why you are wrong in your concept (not idea) and you've just dismissed all of us because we disagree with you. This is the third time I've see you act like this. We weren't even disagreeing with your idea. Just how it works.

Can you accept criticism? I accept it daily.
 

Cam__

Member
Joined
May 22, 2013
Messages
43
Reaction score
8
I can accept valid criticism. It was Ward's comment that got me in a really pissy mood. Prior to that it was a simple disagreement over whether it could be done in a reasonable time using MySQL, and I would have been happy to have been proven wrong. Now I'm kind of like I don't give a flying fig anymore, and am sorry I ever made the suggestion. I don't intend to ever make that mistake again. I apologize if my pissiness carried over in my reply to you, even though you did "like" his post.

Just as a point of interest, you say that three developers have disagreed with me. You have, and lgaetz has. Who is the third?
 

tm1000

Schmoozecom INC/FreePBX
Joined
Dec 1, 2009
Messages
1,360
Reaction score
78
I suppose there is contention or tension between us. That's fine. I'd like to point out that I "liked" that post only because of what was said about the speed of MySQL. I have often had issues with the way pbx in a flash does security so I wasn't sticking up for the whole thing. I suppose every time I "like" a post I need to write up a reply stating why I liked the post and for what reasons otherwise you will hold it against me

As much as you don't want to agree with me ward is somewhat a developer. He writes bash scripts and modifications to asterisk and changes and hacks all using bash scripts and he is the one who got digium to start having conference calls and face to face meetings with developers and hes also the one who somehow gets the attention of malcolm davenport. if that doesn't make you somewhat of a developer then I don't know what does quite frankly.

Cam, if you want to continue this argument between you and me then feel free. There is a private message notification system here and my email is pretty public knowledge. We can talk about whatever you want there but lets try to keep it out of public sight. I really don't want this to turn into bashing left and right as I am trying to hold us all to higher standards than in 2010 and such and what happened with other forum members that left that I highly respected that have left.

I thus value your opinions and don't hold any grudges. No ill will either way.

With that said I hope to work on this addition in the next year as I promised after the last time someone asked about my module (Swiss army knife)

I can accept valid criticism. It was Ward's comment that got me in a really pissy mood. Prior to that it was a simple disagreement over whether it could be done in a reasonable time using MySQL, and I would have been happy to have been proven wrong. Now I'm kind of like I don't give a flying fig anymore, and am sorry I ever made the suggestion. I don't intend to ever make that mistake again. I apologize if my pissiness carried over in my reply to you, even though you did "like" his post.

Just as a point of interest, you say that three developers have disagreed with me. You have, and lgaetz has. Who is the third?
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,168
Reaction score
5,199
Cam__ Anyone is more than welcome to suggest/discuss anything here. But leave the personal attacks at the door, or go elsewhere.
 

Cam__

Member
Joined
May 22, 2013
Messages
43
Reaction score
8
Cam__ Anyone is more than welcome to suggest/discuss anything here. But leave the personal attacks at the door, or go elsewhere.

What personal attack?!?! You called my idea "next to worthless", apparently because you have some really strange ideas about security. If you think I'm supposed to just accept that and not respond, then please cancel my account and ban me forever because I can guarantee that we will have issues in the future - assuming I even continue to participate in this forum at all, which at this point I am not inclined to.

I realize it's your forum, but I don't believe that should insulate you from criticism, particularly when you seem more than willing to dish it out. If you don't want people to ever say anything negative about you or your beliefs about security, then maybe you should refrain from being so quick to judge others and their ideas. If you have real concerns about security, then fine, please explain how what I proposed is any more insecure than other features already in PiaF and FreePBX. I'm sure tm1000 would be interested to hear this too, since apparently he's been working on something like this.

lgaetz, thanks for the suggestion, but this whole experience has left such a bad taste in my mouth that I will not be developing it any further. I can do what I need to do in extensions_custom.conf and for my own personal use that will be sufficient. At this point I see no reason to spend one more millisecond on my "next to worthless" idea. I am truly sorry I ever suggested this, and I have learned from this experience and will never make that mistake again. And that is my last word on the topic, and quite possibly my final post in this forum.
 

kenn10

Well-Known Member
Joined
Dec 16, 2007
Messages
3,764
Reaction score
2,173
Excuse me but after The Great Crash of '13, are we morphing into a Trixbox style forum?

New ideas are always welcome but the style of delivery is important as well. Belligerent and childish responses are uncalled for.
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,168
Reaction score
5,199
There was an air of hostility on the trixbox forum that we don't want replicated here. We all do this for fun. When it stops being cordial, it's not worth the effort frankly. As previously noted, anyone that wants to discuss any issue or topic is free to do so without launching personal attacks because someone doesn't happen to agree with your particular view of the universe. :idea:

NOTE: Everyone retains his/her absolute constitutional right to go elsewhere if you find this policy to be personally unacceptable or morally offensive.

As much as you don't want to agree with me ward is somewhat a developer. He writes bash scripts and modifications to asterisk and changes and hacks all using bash scripts and he is the one who got digium to start having conference calls and face to face meetings with developers and hes also the one who somehow gets the attention of malcolm davenport. if that doesn't make you somewhat of a developer then I don't know what does quite frankly...


P.S. to tm1000: Thanks for kudos and my promotion to "somewhat of a developer." I think I earned my first lunch money from our dBase clone, WAMPUM, before some of you had even made it to diapers. WAMPUM and our later dLite TSR both had pop-up phone dialers which should be the death knell for some of today's brilliant software patents.

I'll never forget coming home from a weekend trip in the fall of 1985 after Computer Shopper published a lengthy review of WAMPUM. The contents of our mailbox had literally spilled into the street because of the number of checks we received from Saturday's mail delivery. It was $20 back then. The glorious days of the shareware revolution were upon us!

Feel free to download a copy of WAMPUM and take it for a spin with VirtualBox. Believe it or not, I still get letters from folks that still use it to run their businesses. Works great with DOS 6.22 and VirtualBox...

BNOmxmTCAAA9EM8.jpg:large


WAMPUM v4.0 Ward Mundy REGISTRATION $50.00
WAMPUM RELATIONAL DATABASE MUNDY DEC89 DATA DBASE

New Release

WAMPUM is a full-featured, menu-driven, fully-relational,
dBASE III- compatible data base management system. It
permits even novice computer users to develop powerful
applications in minutes instead of months. Virtually every
dBASE III command and function is supported using a simple,
menu-driven interface supporting 9 relational data bases.
WAMPUM can handle data bases as large as 1 BILLION records
with as many as 400 fields per record. With performance
which exceeds dBASE III, WAMPUM was acclaimed by Jim Pile of
PCM Magazine as "the best data base manager I know of." And
PC World has said that "WAMPUM is comparable to dBASE III in
features and power." Both stand-alone and network versions of
WAMPUM are available. Registration entitles a user to a
choice of current versions supporting dBASE III, Clipper, and
FoxBASE+, the three major dBASE dialects. The FoxBASE
version also supports "command mode" with a simulated dot
prompt that permits users to enter virtually any dBASE
command. Simple programs up to 500 lines also can be written
and executed without purchasing any other dBASE product.

WAMPUM 4.0 adds a host of features including NINE pop-up,
relational files to support data entry, a "ditto" key which
instantly duplicates any or all previously entered field
data, support for FIVE relational files to generate reports,
labels, and form letters, and a pop-up phone dialer. Also a
user-defined DOS window has been added which permits
execution from a tailorable menu of an unlimited variety of
your favorite DOS applications (up to 512K each!). And, an
all-new POWER BROWSE provides a spreadsheet-like environment
in which you can ADD, UPDATE, DELETE, SORT and MEMO EDIT any
selected group of records from virtually any dBASE data base.
Last but not least, WAMPUM 4.0 becomes the first fully
relational ShareWare data base providing unlimited relational
calculated fields as well as a UNIQUE NUMBER generator for
production of complete inventory and accounting systems.

License fees: $50 per single-user PC or $150 for unlimited
users on a network. Registration also entitles end-users to
90 days of free technical support and complete documentation
on diskette. A 180- page soft-bound User's Guide is
available for only $20. Shipping and handling per order is
$5 U.S. or $10 foreign.

System requirements:
DOS 2.X or higher and a 512K PC/XT/AT/286/386 PC with at
least two 720K floppy drives or one 720K or larger floppy
drive and a hard disk. A hard disk is strongly recommended.
Network versions require DOS 3.1 or higher and 640K with a
Net-BIOS compatible network.
 
Last edited:

Members online

No members online now.

Forum statistics

Threads
25,782
Messages
167,509
Members
19,202
Latest member
pbxnewguy
Get 3CX - Absolutely Free!

Link up your team and customers Phone System Live Chat Video Conferencing

Hosted or Self-managed. Up to 10 users free forever. No credit card. Try risk free.

3CX
A 3CX Account with that email already exists. You will be redirected to the Customer Portal to sign in or reset your password if you've forgotten it.
Top