Troubleshooting a Customized AA

As discussed in Chapter 9, "Cisco IPC Express Automated Attendant Options," Cisco UE allows you to create up to five customized AA applications or top-level scripts for each Cisco UE system. Creating and deploying such applications involves the following steps:

Even though you have followed all the necessary precautions in creating the AA, some issues may still come up. When issues occur, you should use the troubleshooting techniques discussed in this chapter.

The following sections discuss possible problems you might encounter during the configuration of an AA application. These are not runtime issues. Instead, they are problems you may experience during the configuration or deployment of the system or a new AA menu.

Ensuring a Correct Configuration

An AA application may have multiple physical script files. In this case, a main .aef file calls other .aef files using the Call Subflow step. It is possible that you uploaded the main script but forgot to upload all the subflows needed by this script. Or you might have an undefined variable in the script if you forgot to validate the script in the AA script Editor before uploading.

You can find and fix many such issues when you configure the system instead of finding them during runtime when active user calls are placed to the application. You can find and eliminate most of these issues when creating the AA application using the Cisco UE administration GUI, as shown in Figure 20-1.

Figure 20-1. Creating an AA Application

Although the GUI pop-up window shown in Figure 20-1 tells you an error has been detected, it does not tell you exactly what the error is. To understand what has gone wrong, turn on the trace config-ccn trace on Cisco UE. In the trace shown in Example 20-1, the problem is that a subflow called dialbyextension.aef has not been uploaded into Cisco UE.

Example 20-1. Missing Subflow File During Application Configuration

cue#trace config-ccn all cue#show trace buffer tail 7634 12/14 02:37:39.916 ACCN STGN 0 Preprocessing GetListMember Step 748 12/14 02:39:36.252 WFSP APP 0 WFSysdbNdApp::check enter 747 12/14 02:39:36.388 WFSP APP 0 WFSysdbNdApp::isScriptExist name=main.aef 747 12/14 02:39:36.388 WFSP APP 0 WFSysdbNdApp::writeLDAPApplication saving app to ldap 747 12/14 02:39:36.735 WFSP APP 0 WFSysdbNdApp::writeLDAPApplication saving app to ldap done 747 12/14 02:39:36.735 WFSP APP 0 WFSysdbNdApp::setLDAPApplication exit 747 12/14 02:39:36.749 WFSP APP 0 WFSysdbNdApp::remoteAppReload rmi reload app=myaa 747 12/14 02:39:36.981 WFSP APP 0 com.cisco.app.InvalidApplicationException: failed to initialize configuration; nested exception is: com.cisco.app.InvalidApplicationException: failed to load script; nested exception is: com.cisco.wfapi.WFPreprocessException: Failed to preprocess the application: main.aef; nested exception is: com.cisco.wfapi.WFPreprocessException: Failed to load the sub-workflow: dialbyextension.aef; nested exception is: com.cisco.wfapi.WFException: Failed to load application from the repository: dialbyextension.aef; nested exception is: com.cisco.wfframework.repository.NoSuchObjectException: Can't find dialbyextension.aef at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer (StreamRemoteCall.java:257) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:232) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:135) at com.cisco.app.remote.rmi.RMIApplicationManagerImpl_ Stub.reload(RMIApplication ManagerImpl_Stub.java:309) at com.cisco.wf.wfsysdb.WFSysdbNdApp.remoteAppReload(WFSysdbNdApp.java:172) ...... com.cisco.wfapi.WFException: Failed to load application from the repository: dialbyextension 747 12/14 02:39:36.984 WFSP APP 0 com.cisco.aesop.sysdb.xactSysdbException: Application created successfully, but might not be fully functional due to internal errors at com.cisco.wf.wfsysdb.WFSysdbNdApp.remoteAppReload(WFSysdbNdApp.java:180) at com.cisco.wf.wfsysdb.WFSysdbNdApp.commit(WFSysdbNdApp.java:804) at com.cisco.aesop.sysdb.nativeProvider.commit(nativeProvider.java:64) at com.cisco.aesop.sysdb.sysdbNative.node_handle_wait(Native Method) at com.cisco.aesop.sysdb.sysdbProducer$1.run(sysdbProducer.java:238) at java.lang.Thread.run(Thread.java:512)

The trace shown in Example 20-2 shows an undefined variable being used in a Menu step of the script. This might be caused by the script developer's having forgotten to run the Validation step on the AA Script Editor.

Example 20-2. Missing Variable in the Menu Step

cue#trace config-ccn application all cue#show trace buffer tail 835 01/08 11:23:14.637 WFSP APP 0 WFSysdbNdApp::getLdapApplication access ldap start 835 01/08 11:23:14.932 WFSP APP 0 WFSysdbNdApp::getLdapApplication access ldap end 835 01/08 11:23:14.933 WFSP APP 0 WFSysdbNdApp::get value=main.aef 835 01/08 11:23:14.933 WFSP APP 0 WFSysdbNdApp::get exit 834 01/08 11:23:14.937 WFSP APP 0 WFSysdbNdCfgVars::get enter 834 01/08 11:23:14.937 WFSP APP 0 WFSysdbNdCfgVars::get attribute=operator done 835 01/08 11:23:25.687 WFSP APP 0 WFSysdbNdApp::setLDAPApplication exit 835 01/08 11:23:25.925 WFSP APP 0 WFSysdbNdApp::remoteAppReload rmi reload app=customaa Jan 8 11:23:26 localhost java: WARNING ccn_config application debug com.cisco. app.InvalidApplicationException: failed to initialize configuration; nested exception is: Jan 8 11:23:26 localhost java: com.cisco.app.InvalidApplicationException: failed to load script; nested exception is: Jan 8 11:23:26 localhost java: com.cisco.wfapi.WFPreprocessException: Failed to preprocess the Jan 8 11:23:26 localhost java: WARNING ccn_config application debug com.cisco. aesop.sysdb.xactSysdbException: Application created successfully, but might not be fully functional due to internal errors 835 01/08 11:23:26.308 WFSP APP 0 com.cisco.app.InvalidApplicationException: failed to initialize configuration; nested exception is: com.cisco.app.InvalidApplicationException: failed to load script; nested exception is: com.cisco.wfapi.WFPreprocessException: Failed to preprocess the application: main.aef; nested exception is: com.cisco.wfapi.WFPreprocessException: MenuStep: Variable 'menuprompt' not defined; nested exception is: com.cisco.wfapi.expression.WFSemanticException: Variable 'menuprompt' not defined at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemote Call.java:257) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:232) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:135)

The previous two examples show possible issues that can occur during the development and configuration of AA applications. These are by no means the only problems that might be encountered. If you are facing any issues during this stage of AA development, it is advisable to turn on one or more of the trace config-ccn traces shown in Example 20-3 to look for any exceptions reported by the system.

Example 20-3. Sample config-ccn Traces

cue#trace config-ccn ? all application group http-trigger jtapi-subsystem jtapi-trigger miscellaneous prompt script sip-subsystem sip-trigger

Now that you understand how to troubleshoot and correct script development and configuration issues, the next section covers runtime troubleshooting techniques.

Understanding SIP Call Flow to the AA

Troubleshooting AA applications requires an understanding of the SIP protocol and the architecture of the CRS software component of Cisco UE, as described in Chapter 3. There you learned about the different components of Cisco UE and how the CRS software is used inside Cisco UE.

Figure 20-2 shows a SIP call flow between Cisco CME and Cisco UE. It demonstrates how an incoming SIP call's signaling and media traverse the CRS component.

Figure 20-2. SIP Call Flow to CRS

Figure 20-2 shows what happens when a SIP call is placed to Cisco UE from a Cisco CME phone or the Public Switched Telephone Network (PSTN). A SIP INVITE arrives at Cisco UE's SIP stack. This event is relayed to the application framework within the CRS component of Cisco UE. Upon receiving this call setup message, the application framework consults the configuration to determine which application is associated with the called number in the setup message. Depending on the system's configuration, the appropriate application is startedin this case, the Cisco UE AA script.

Any CRS application (for example, the AA) is a logical collection of steps programmed into a script. As soon as the application is started, control of the call is handed off to the application, which starts executing the steps in the script.

The first step in an application script is almost always the Accept step, which answers the call. Answering the call results in the appropriate SIP message (a 200 OK, in this case) being sent to the gateway, which in turn sends it to the PSTN. As a result of the exchange of the call setup and call answer messages between the PSTN gateway and Cisco UE, a Real-Time Transport Protocol (RTP) media channel is established between the two software components.

To open an RTP media channel, Cisco UE uses the services of the Cisco Media Termination (CMT) component. The application must be notified of the DTMF keypresses on the caller's phone to receive the correct events, so the application subscribes to the digit press notifications from Cisco CME or the PSTN gateway. After this protocol exchange, the application execution continues, depending on the application's events and programmed logic.

Verifying Why the AA Doesn't Answer Calls

Sometimes when calls are placed to the AA, either they don't ring or they ring but are not answered. There are many possible reasons for this behavior. One of the basic reasons is that Cisco CME or the PSTN gateway originating the call does not have a dial peer pointing to the AA pilot number (trigger). You can verify this configuration by doing a show running-config or show dial-peer voice summary command on Cisco CME or the PSTN gateway. Ensure that the dial peer has the correct configuration with respect to the SIP protocol and codec to be used (G.711 µ-law only), as shown in Example 20-4.

Example 20-4. Correct SIP Dial Peer Configuration

router#show running-config dial-peer voice 6000 voip description AA destination-pattern 6... session protocol sipv2 session target ipv4:10.19.153.37 dtmf-relay sip-notify codec g711ulaw no vad

If the Cisco CME SIP dial peer configuration is correct, but the AA still doesn't answer calls, the next step is to turn on SIP debugs on Cisco CME, and check at what point in the call flow the call is failing. One of the reasons could be that the number dialed is not associated with the AA's pilot number. This is demonstrated by the SIP debug shown in Example 20-5.

Example 20-5. Missing/Disabled Trigger or a Wrong Dialed Number

cue#trace ccn stacksip dbug cue#show trace buffer tail 2641 12/29 14:13:01.551 ACCN SIPL 0 ------- INVITE sip:6600@1.3.6.129:5060 SIP/2.0 Via: SIP/2.0/UDP 1.3.6.27:5060;branch=z9hG4bK2047 From: ;tag=66CAF308-EA1 To: Date: Mon, 29 Dec 2003 23:54:30 GMT Call-ID: 2DA98768-399111D8-A2ADAECA-10549FA0@1.3.6.27 Supported: 100rel,timer Min-SE: 1800 Cisco-Guid: 766004863-965808600-2729094858-273981344 User-Agent: Cisco-SIPGateway/IOS-12.x Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, COMET, REFER, SUBSCRIBE, NOTIFY, INFO, UPDATE, REGISTER CSeq: 101 INVITE Max-Forwards: 6 Remote-Party-ID: ;party=calling;screen=no;privacy=off Timestamp: 1072742070 Contact: Call-Info: ;method="NOTIFY;Event=telephone-event;Duration=2000" Expires: 180 Allow-Events: telephone-event Content-Type: application/sdp Content-Length: 179 v=0 o=CiscoSystemsSIP-GW-UserAgent 4326 3924 IN IP4 1.3.6.27 s=SIP Call c=IN IP4 1.3.6.27 t=0 0 m=audio 19542 RTP/AVP 0 c=IN IP4 1.3.6.27 a=rtpmap:0 PCMU/8000 a=ptime:20 2643 12/29 14:13:01.551 ACCN SIPL 0 PROTOCOL_MSG:Protocol Stack Message: Stack Message=LocalLine to process DN 6600 in url sip:6600@1.3.6.129:5060 not found, try to process request on '*' line 2643 12/29 14:13:01.552 ACCN SIPL 0 PROTOCOL_MSG:Protocol Stack Message: Stack Message=Not found LocalLine for DN=6600, url=sip:6600@1.3.6.129:5060 2643 12/29 14:13:01.552 ACCN SIPL 0 PROTOCOL_MSG:Protocol Stack Message: Stack Message=Send Not found for request for INVITE sip:6600@1.3.6.129:5060 2643 12/29 14:13:01.553 ACCN SIPL 0 --- send message --- to 1.3.6.27:5060 SIP/2.0 404 Not Found Via: SIP/2.0/UDP 1.3.6.27:5060;branch=z9hG4bK2047 From: ;tag=66CAF308-EA1 To: ;tag=9b277769-411 Call-ID: 2DA98768-399111D8-A2ADAECA-10549FA0@1.3.6.27 CSeq: 101 INVITE Content-Length: 0

As shown in the trace, Cisco UE is sending a "404/Not Found" message in response to the SIP INVITE to number 6600. One of the reasons why the called number used in the INVITE is wrong may be because of an unintentional or incorrect number translation configuration in Cisco IOS, such as an incorrect translation-rule statement.

Note

You might notice that when calling from some phones or PSTN interfaces, you don't see any SIP calls going to Cisco UE. That may be because of a Class of Restriction (COR) configuration in Cisco IOS.

Here are two other reasons why a "404/Not Found" may be sent from a Cisco UE:

• The SIP trigger (the pilot number configured for the AA) does not match the dialed number received from Cisco CME.
• The SIP trigger is disabled.

Example 20-6 shows a disabled trigger for the AA (the customaa application).

Example 20-6. Disabled Trigger

cue#show ccn trigger Name: 6800 Type: SIP Application: voicemail Locale: en_US Idle Timeout: 5000 Enabled: yes Maximum number of sessions: 8 Name: 6700 Type: SIP Application: autoattendant Locale: en_US Idle Timeout: 5000 Enabled: yes Maximum number of sessions: 8 Name: 6600 Type: SIP Application: customaa Locale: en_US Idle Timeout: 10000 Enabled: no Maximum number of sessions: 8

Sometimes you might hear a prompt saying, "I am sorry. We are currently experiencing system problems." One of the reasons for this message is that the application may be disabled, but the trigger (pilot number) is not. Example 20-7 shows an example of where the AA application (customaa) is disabled.

Example 20-7. Disabled AA Application

cue#show ccn application Name: ciscomwiapplication Description: ciscomwiapplication Script: setmwi.aef ID number: 0 Enabled: yes Maximum number of sessions: 8 strMWI_OFF_DN: 2221 strMWI_ON_DN: 2222 CallControlGroupID: 0 Name: voicemail Description: voicemail Script: voicebrowser.aef ID number: 1 Enabled: yes Maximum number of sessions: 8 logoutUri: http://localhost/voicemail/vxmlscripts/ mbxLogout.jsp uri: http://localhost/voicemail/vxmlscripts/ login.vxml Name: customaa Description: customaa Script: main.aef ID number: 4 Enabled: no Maximum number of sessions: 4 operator: 0

You can also turn on the Cisco UE command trace ccn managerappl dbug, as shown in Example 20-8, to see whether the application is enabled or disabled. This is handy in case the show ccn application command shows that the application is enabled, but it is, in fact, disabled internally because of a software error.

Example 20-8. Trace Example for a Disabled Application

cue#trace ccn managerappl dbug cue#show trace buffer tail 3463 01/08 11:46:49.430 ACCN APMG 0 APP_RELOADED:Application reloaded: Application=customaa Jan 8 11:47:21 localhost java: ERROR ccn ManagerAppl DBUG UNABLE_INVOKE_APP:Unable to invoke application: Application=customaa,Exception=com.cisco.app.ApplicationDisabledException: application 'customaa' is disable Jan 8 11:47:21 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION:com.cisco.app.ApplicationDisabledException: application 'customaa' is disabled Jan 8 11:47:21 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION: at com.cisco.app.impl.ApplicationManagerImpl.invoke(ApplicationManagerImpl.java (Compiled Code)) Jan 8 11:47:21 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION: at com.cisco.app.Application.invoke(Application.java(Inlined Compiled Code)) Jan 8 11:47:21 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION: at com.cisco.app.Application.invoke(Application.java(Inlined Compiled Code)) Jan 8 11:47:21 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION: at com.cisco.wf.subsystems.callcontrol.AppLineListener$1.run(AppLineListener. java(Compiled Code)) Jan 8 11:47:21 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION: at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(PooledExecutor. java(Compiled Code))

You can enable the application as shown in Example 20-9 or via the GUI.

Example 20-9. CLI to Enable the AA Application

cue(config)#ccn application customaa Modifying existing application cue(config-application)#enabled

If the application is not disabled and you still cannot place calls to it, the maximum number of allowed sessions for the trigger may have been exceeded. You can check this situation by tracing the SIP messages between Cisco CME and Cisco UE, as shown in Example 20-10. You see that Cisco UE returns the message "480 Temporarily Unavailable".

Example 20-10. Trace Example for Maximum Calls to Trigger Active

cue#trace ccn stacksip dbug cue#show trace buffer tail Press to exit... 3470 01/07 15:46:36.296 VMSS vxml 0x00000003b9ad1332 0 TIMEOUT 3544 01/07 15:46:53.251 ACCN SIPL 0 ------- INVITE sip:5800@1.3.6.127:5060 SIP/2.0 Via: SIP/2.0/UDP 1.3.6.27:5060;branch=z9hG4bKEF7 From: ;tag=951D3478-12C8 To: Date: Wed, 07 Jan 2004 23:46:53 GMT Call-ID: 9ABD8318-40A211D8-8907AECA-10549FA0@1.3.6.27 Supported: 100rel,timer Min-SE: 1800 Cisco-Guid: 2573390720-1084363224-2298785482-273981344 User-Agent: Cisco-SIPGateway/IOS-12.x Allow: INVITE, OPTIONS, BYE, CANCEL, ACK, PRACK, COMET, REFER, SUBSCRIBE, NOTIFY, INFO, UPDATE, REGISTER CSeq: 101 INVITE Max-Forwards: 6 Remote-Party-ID: ;party=calling;screen=no;privacy=off Timestamp: 1073519213 Contact: Call-Info: ;method="NOTIFY;Event=telephone-event;Duration=2000" Expires: 180 Allow-Events: telephone-event Content-Type: application/sdp Content-Length: 179 v=0 o=CiscoSystemsSIP-GW-UserAgent 5995 4148 IN IP4 1.3.6.27 s=SIP Call c=IN IP4 1.3.6.27 t=0 0 m=audio 17524 RTP/AVP 0 c=IN IP4 1.3.6.27 a=rtpmap:0 PCMU/8000 a=ptime:20 3546 01/07 15:46:53.252 ACCN SIPL 0 Substring > telephone-event/8000< not found 3546 01/07 15:46:53.257 ACCN SIPL 0 --- send message --- to 1.3.6.27:5060 SIP/2.0 480 Temporarily Unavailable Via: SIP/2.0/UDP 1.3.6.27:5060;branch=z9hG4bKEF7 From: ;tag=951D3478-12C8 To: ;tag=a3cfd31b-135700 Call-ID: 9ABD8318-40A211D8-8907AECA-10549FA0@1.3.6.27 CSeq: 101 INVITE Content-Length: 0

Fixing Abnormal Exits and System Problems

It is always recommended that you try a newly developed AA in a test environment before deploying it for live calls, because runtime errors may affect your office's operations. This kind of testing eliminates possible runtime errors from the AA.

This section discusses how to identify and fix these potential issues. The most common error is a message from the system that says, "I am sorry. We are currently experiencing system problems. Please try again later." The system plays this prompt when it encounters an unexpected problem in the call processing, such as a missing prompt file or a missing subflow script (.aef) file.

The trace to turn on for such problems is the trace ccn managerappl dbug command, which you used earlier, in Example 20-8, and look for any exceptions that might point to any other misconfigurations, missing prompts, or files.

Missing Prompt File

If a prompt used in the script is not present on the system, an exception is generated when running the application, as shown in Example 20-11.

Example 20-11. Missing Prompt File

cue#trace ccn managerappl dbug cue#show trace buffer tail 4131 01/08 11:38:07.969 ACCN APMG 0 APP_RELOADED:Application reloaded: Application=promptmgmt Jan 8 11:40:13 localhost java: ERROR ccn ManagerAppl DBUG TASK_ABORTED:Application task aborted: Application=App[name=customaa,type=Cisco Script Application,id=4,desc=customaa, enabled=true,max=4,valid=true, optional=[cfgVars=[Lcom.cisco.wfapi.util.WFNameValuePair;@2a44c3c4,script= main.aef]],Application Trigger=Co Jan 8 11:40:13 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION:com.cisco.prompt.UndefinedPromptException: user prompt '/usr/wfavvid/Prompts/user/ga_IE/menuprompt.wav'

There are two ways to configure system behavior when a prompt file is missing. If the system finds that a prompt referred to by a script is not present in the system, it can either ignore it and continue with the script execution, or stop the application execution and play an error prompt to the caller. You can configure this behavior in the Step Configuration window for each step using the continue on prompt errors option. Figure 20-3 shows this configuration for the Menu step.

Figure 20-3. Customizing System Behavior on Prompt Errors

Missing Subflow Script File

The trace output shown in Example 20-12 shows a missing subflow .aef file.

Example 20-12. Missing Subflow Error at Runtime

cue#trace ccn managerappl dbug cue#show trace buffer tail Jan 11 01:30:45 localhost java: ERROR ccn ManagerAppl DBUG UNABLE_INVOKE_APP: Unable to ...... Jan 11 01:30:45 localhost java: ERROR ccn ManagerAppl DBUG EXCEPTION: at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java (Compiled Code)) 3604 01/11 01:30:45.785 ACCN APMG 0 UNABLE_INVOKE_APP:Unable to invoke application: Application=App[name=customaa, type=Cisco Script Application,id=4,desc=customaa,enabled=true,max=4, valid=false,optional=[cfgVars=[Lcom.cisco.wfapi.util.WFNameValuePair;@731403c5, script=main.aef]],Exception=com.cisco.app.InvalidApplicationException: failed to load script; nested exception is: com.cisco.wfapi.WFPreprocessException: Failed to preprocess the application: main.aef; nested exception is: com.cisco.wfapi.WFPreprocessException: Failed to load the sub-workflow: dialbyextension.aef; nested exception is: com.cisco.wfapi.WFException: Failed to load application from the repository: dialbyextension.aef; nested exception is: com.cisco.wfframework.repository.NoSuchObjectException: Can't find dialbyextension.aef 3604 01/11 01:30:45.786 ACCN APMG 0 EXCEPTION:com.cisco.app.InvalidApplicationException: failed to load script; nested exception is: 3604 01/11 01:30:45.786 ACCN APMG 0 EXCEPTION: com.cisco.wfapi.WFPreprocessException: Failed to preprocess the application: main.aef; nested exception is: 3604 01/11 01:30:45.786 ACCN APMG 0 EXCEPTION: com.cisco.wfapi.WFPreprocessException: Failed to load the sub-workflow: dialbyextension.aef; nested exception is: