PIONEERS Siri Fans: Star Has Arrived

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,168
Reaction score
5,199
Beginning a new project to bring a Siri-like service to Incredible PBX. Meet Star. Suggestions welcomed!

siri-1-600x450.jpg

Photo credit: People Magazine

1. Dial *
2. Answer: good morning, good afternoon, good evening: I think you may have me confused with Siri.
3. Say what you need:

Google Speech Recognition will handle the following responses:

  • time: what time is it, what's the time, correct time, etc.
  • weather: what's the weather, weather forecast, latest forecast, carry an umbrella, need a coat, need a raincoat, is it going to rain/snow/storm/hurricane/tornado (local zip code stored in AsteriDex under weather)
  • news: what's new, today's news headlines, what's in the news, whassup, qué pasa
  • call: call Joe, call Mary (use AsteriDex name lookup)
  • dial: dial 8005551212 (spoken number to dial)
  • text: text Joe, text Mary, send a text to xxx, send SMS to xxx, send a message to xxx (using AsteriDex names)
  • reminder: remind me to xxx at time, remind me to xxx at date/time, schedule reminder at time, xxx
  • everything else: pass the query to Wolfram Alpha
DOWNLOAD CURRENT VERSION HERE.
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,168
Reaction score
5,199
Modular design taking shape with development of triggers. We're tackling weather first so here's how that will work. First, we'll need a big list of throw-away words, e.g. "hey, what's, how's, tell, will, can, could, me, I, are, is, it..." Certain words trigger a subroutine. For Weather, these words would include "weather, umbrella, storm, rain, rainy, sun, sunny, cloud, clouds, cloudy, humid, damp, hurricane, tornado, wind, windy, etc." Once the call is passed to the Weather subroutine, then we need to figure out if the caller wants local weather or weather for some specific city. The other sub-triggers would be words like "at, in, for, near" which would tell us that a city or city/state/country accompanies the request. If there are no city triggers, then we need to figure out where local is. GPS is too hard for now so that's what the AsteriDex Weather entry with a zip code would provide.

Any other thoughts/suggestions?
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,168
Reaction score
5,199
If you'd like to play along, we think we've finished Star weather, news, time of day, AsteriDex voice calling, voice dialing, and Wolfram Alpha components.

Prerequisites (Current Version 0.010):

To use this version, you will need:

1. Certified Incredible PBX Platform

2. Google Speech Recognition Key - obtain and install a Google Speech Recognition key using this tutorial

3. Use AsteriDex to add a new entry to your phonebook:

Contact Name: Weather
Contact Phone: 947
Dial Code: your local 5-digit ZIP code

4. Wolfram Alpha APP-ID - obtain Wolfram Alpha key and insert into /var/lib/asterisk/agi-bin/4747. Detailed tutorial here.

5. AsteriDex Voice Calling and Voice Dialing both require at least one properly configured outbound trunk. Google Voice is supported.

Installation/Reinstallation (last updated at 8:45 am on 10/10):

While still logged into your server as root, issue the following commands. Rerun these same commands to update your existing setup:
Code:
cd /var/lib/asterisk/agi-bin
wget http://incrediblepbx.com/star.tar.gz
tar zxvf star.tar.gz
rm -f star.tar.gz
sed -i '\:// BEGIN Star:,\:// END Star:d' /etc/asterisk/extensions_custom.conf
sed -i '/\[from-internal-custom\]/r star.txt' /etc/asterisk/extensions_custom.conf
asterisk -rx "dialplan reload"

Weather Query:

Place a call to * from an extension on your PBX and say what you'd normally say to obtain a weather report or weather forecast for a particular city and state or country. If you don't specify a city and state/country, you'll get your local weather report. The report will be for the ZIP code you entered in Prerequisites Step #3, above.

News Query:

Place a call to * from an extension on your PBX and request today's news headlines or say, "What's Up?"

Time of Day Query: Place a call to * from an extension on your PBX and request the time.

AsteriDex Voice Calling: Place a call to * from an extension on your PBX and say "CALL" followed by any name entry in your AsteriDex database, e.g. "Call Delta Airlines." No other syntax or extra words are permitted, or the call will fail.

Voice Dialing: Place a call to * from an extension on your PBX and say "DIAL" followed by a number that matches your dialplan, e.g. "Dial 8005551212." No other syntax is supported.

Wolfram Alpha Query: For any queries that aren't matched above, the query will be passed to Wolfram Alpha for processing. For sample queries, see this tutorial.

Special Notes:

Please post problem reports/quirks.

Note: We've intentionally left quality sampling of queries turned off to make it easier to uncover quirks so... there will be some. :)
 

wardmundy

Nerd Uno
Joined
Oct 12, 2007
Messages
19,168
Reaction score
5,199
Google has pulled another fast one with their Google TTS engine so we've dropped back to FLITE for the time being. Just run through the install again and all will be well. :boxing smiley:
 

cocot

Guru
Joined
Dec 10, 2014
Messages
14
Reaction score
5
I wrote my own voice navigation a couple of years ago based on the Wolfram Alpha dialplan. It has since been a test-bed for a number of ideas about voice navigation. My effort was to try and emulate the use of natural speaking and the illusion of intelligence. This was only partially successful so far. It gets very complicated in a hurry and complication slows things down.

I am probably not going to cover much you have not already though about.

My approach begins with the use of keywords. Send out the audio. Search the returned string for specific keywords. After you find your keywords, or not, then your choices branch off. I have concluded there is no single approach which covers all the possibilities.

For example: Dialing a number. This is fairly easy to mimic a natural "Intelligent" interface.

Look for the keyword 'dial". Strip out everything which is NOT a number. Confirm the number. If you did not get the correct number ask for input...Repeat

This allows for very casual conversation such as "Please dial 123456789 for me." or "Could you dial 123456789?"

You can even say "Please dial a number for me." and then be prompted for a number. This works very well and is fairly easy.

However, it get complicated very fast. For example Weather.

I use a number of keywords here. Any will trigger the weather routine. "weather, cold, hot, humid, rain, snow, umbrella" i could have added more such as ice or blizzard but every instance slows things down.

But how do you proceed from here? The caller's speech pattern is critical if you want a simple search. The nerd vittles approach asks for the town and state as a separate voice input. This works fine but does not mimic natural speech. You could educate the user to always have the town and state on the end of their sentence. Such as "Will it be hot in Chicago Il?" But as soon as the caller alters the format you have specified it becomes less fluid. ""Will it be hot in Chicago Il today?" will mess you up.

It might be possible to have a data base of the 150 larger towns and parse the string for that information. Probably slow. In any case I did not solve this. It is well beyond my abilities. It was a lot of fun trying different solutions. There has been some success.

I am attaching a version of the voice navigation which has occupied me for a while. Please feel free to play with it, change it any way you want. Thanks to all who have made pbxinaflash possible. You have allowed me countless hours of fun end experimentation.

PLEASE Change clean-pronounceable-digits.txt to clean-pronounceable-digits.sh before you install it.

There are a number of user customization spots so take a read through before you install it. There are a lot of comments to hopefully help out. It will be a lot better when we get a voice like Google again, but this still gets the job done.

Have fun and thanks again.
 

Attachments

  • Voice Cruiser_rw2.txt
    26.7 KB · Views: 6
  • clean-pronounceable-digits.txt
    1.3 KB · Views: 7

cocot

Guru
Joined
Dec 10, 2014
Messages
14
Reaction score
5
Here are a few more thoughts...

Things like Time, Blacklist, Wake-up call, Call Forwarding etc are easy to select for in a normal, natural sentence. This is perhaps slightly easier than culling out a phone number. The advantage of having the these associated with voice navigation is obvious. It is much easier to remember a name or keyword than it is to remember a dial code. Of course, ideally, these features should be designed to be keypad free and rely solely on voice inputs.

The issue of local weather verses weather for another city could be handled the same way I handled a call for Sam on his mobile phone. A simple and/or statement.

Sam, Mobile:
exten => vc,n,ExecIf($["${utterance}" =~ "Sam"&$["${utterance}" =~ "cell"|"${utterance}" =~ "mobile"]]?Set(NUM2DIAL=NXXNXXXXXX))


As for weather, assume most people will want their local weather most of the time. If one travels a lot you could add a second instance with an additional "AND" relationship.

Weather Local would not have any "AND" relationship:
exten => vc,n,GotoIf($["${utterance}" =~ "weather"|"${utterance}" =~"storm"|"${utterance}" =~"rain"|"${utterance}" =~"snow"]?weather)
This would deliver weather for your local town/city.

Weather for a different town could have a specific "AND" relationship:
exten => vc,n,GotoIf($["${utterance}" =~ "weather"|"${utterance}" =~"storm"|"${utterance}" =~"rain"|"${utterance}" =~"snow"&$["${utterance}" =~ "Boston"|"${utterance}" =~ "Beantown"]]?weather)
This would deliver weather for Boston. This setup would allow for a very natural interface. "I would like the weather for Boston." or "Is Beantown going to be rainy today?"

Weather for an unscripted town could also be handled, almost as well, but the caller would have to remember the special keyword:
exten => vc,n,GotoIf($["${utterance}" =~ "weather"|"${utterance}" =~"storm"|"${utterance}" =~"rain"|"${utterance}" =~"snow"&$["${utterance}" =~ "prompt"|"${utterance}" =~ "special"]]?weather)
"I need a special weather report." This would prompt the caller for the town as the nerd vittles weather does now.

A module could be built to facilitate the entry of these choices and keywords. This module could also be used for building code to call, text, email specific people at work, or on their cell phones. It could also provide a set-up to allow for voice navigation to existing features. Allowing for a fully customizable user set of features and contacts. This arrangement would accommodate most instances using natural conversation and mimicking some intelligence.

Attached are a few PDFs of a mock module layout. I am well aware this was the easy half of the equation. This module assumes use of AsteriDex or similar interface.

I named the mock module 'Voice Contact' but it could be 'Star Contact' or 'Star Drive" if that is not too corny.
 

Attachments

  • Dave Thomas, Mobile.pdf
    144.7 KB · Views: 1
  • Weather, Local.pdf
    133.6 KB · Views: 1
  • Weather, Boston.pdf
    133.5 KB · Views: 1
  • Weather, Prompted.pdf
    133.6 KB · Views: 1
  • Time of Day.pdf
    133.1 KB · Views: 1
Last edited:

cocot

Guru
Joined
Dec 10, 2014
Messages
14
Reaction score
5
The voice navigation dialplan I have been working on is complete for now. It works fairly well and is packed with features. Many of the features are features you are already accustomed to, but some are improved. It parses your voice inputs for keywords and sends you to part of the dialplan based on what it matches up.

Keep in mind the goal has been to provide an interface which uses normal language as much as possible and mimics some intelligence. The results seem pretty good.

I am especially happy with the 'dial' section of the dialplan. It allows the user to say things like “Would you dial XNNXNNNNNN for me please." The utterance is sent to a script (clean-pronounceable-digits.agi) which strips out everything but the number. If you forget to add a number "Could you please dial a number for me?" the dialplan will prompt you. You can be very casual when prompted for the number and respond in a noral way. “I was hoping you would dial NXXNXXXXXX for me.” Once again the non-numeric data will be removed and only the number will remain. This mimics how you might interact with another human.

I also did some work on the weather dialplan and wrote a really nice script (parse-city.agi). The script parses your utterance for over 16,300 US cities and does this in a few seconds! Once again this allows for a very natural interface.

I also added a feature to allow you to quickly send an email or text a cell phone. This works very smoothly. It uses a new script voice-email.agi. You MUST have email installed and working before this will do you any good. If you email is not up and running, search this site for the latest installation information.

Please download ALL NEW copies of the documents below. Some things did not work before. Now it has been tested and corrected. The new dialplan has been re-written and re-organized. There are new scripts and the others have been tweaked too.

If you install things as they are the voice navigation is very usable. However, if you take a moment to customize it you will be well rewarded. Look for "Custom User Input Areas" in the dialplan such as:

; *#*#*#*#*# Custom User Input Area 1 *#*#*#*#*#

There are 3 of them. Look around lines 122, 191 & 228

The dialplan is full of examples and comments explaining how to install it and why some choices were made. Tear it apart, use what you want, throw away the rest. If you make a real improvement you should share it here.

Have fun!


Current Features (probably for a while):

See attached PDF for a full explanation of each feature

Feature: Random Welcome
Feature: Options Feature: instructions
Feature: voicemail
Feature: forward2cell
Feature: forward2assistant
Feature: forward2clear
Feature: news
Feature: weather (powerful interface)
Feature: stock
Feature: dictionary
Feature: question
Feature: unforward
Feature: forwarding
Feature: pronounce
Feature: email (hopefully, in future, lines of code for each contact will be generated by a module such as the one described in previous post.)
Feature: text (hopefully, in future, lines of code for each contact will be generated by a module such as the one described in previous post.)
Feature: dial (version 1, this version requires lines of code fore each contact you want to associate with feature) (hopefully, in future, lines of code for each contact will be generated by a module such as the one described in previous post.)
Feature: time
Feature: blacklist
Feature: wake-up
Feature: trace
Feature: dial (version 2) (this is one of my favorites)
Feature: call
Feature: Random Goodbye
Subroutine: sayit
Subroutine: sayit_secure
Subroutine: listen_up
 

Attachments

  • Voice Cruiser_rw4.txt
    35.1 KB · Views: 8
  • clean-pronounceable-digits.txt
    1.3 KB · Views: 6
  • parse-city.txt
    195.3 KB · Views: 7
  • The voice navigation dialplan.pdf
    47.8 KB · Views: 6
  • voice-email.txt
    927 bytes · Views: 6

globalweb01

New Member
Joined
Dec 28, 2013
Messages
25
Reaction score
9
The feature of Star are precisely what I'm looking for. Unfortunately, the voice recognition is not working. I get the prompts. But the system isn't responding to my voice, and I speak clearly. Any troubleshooting tips?
 

cocot

Guru
Joined
Dec 10, 2014
Messages
14
Reaction score
5
Not enough information in your post but a couple quick thoughts:


Check the 'Asterisk Logfiles' under 'Reports' in the GUI....

Use your STAR and respond to a prompt. Hang up and check the log files. Look for a 'NoOp' line like (Your line will be different but look for the NoOP which is often used to return information to the log files to help in debugging problems. I just generated the log below by using my voice navigation setup. This confirms Google is still working so the problem is most likely on your end.

Code:
 NoOp("SIP/123456789-00000324", "= Script returned: Confidence = 0.98536801 , Utterance = "the rain in Spain stays mainly on the plains" =") in new stack
 

Members online

Forum statistics

Threads
25,778
Messages
167,504
Members
19,198
Latest member
serhii
Get 3CX - Absolutely Free!

Link up your team and customers Phone System Live Chat Video Conferencing

Hosted or Self-managed. Up to 10 users free forever. No credit card. Try risk free.

3CX
A 3CX Account with that email already exists. You will be redirected to the Customer Portal to sign in or reset your password if you've forgotten it.
Top