ServiceNow: SOAP Journal

Integration using Web Services

Category Archives: Uncategorized

SNDML Debugging

This post contains instructions for obtaining runtime diagnostics from SNDML 2.6.5.

Create a Log4J configuration file

SNDML 2.6.5 is built using Log4J 1.2. Create a file named log4j_debug.properties similar to the following:

# This file contains the Log4J configuration for the ServiceNow Datamart Loader
#
# The names of loggers in the "soap" package will be prefixed with "ServiceNow".
# The names of loggers in the "datamart" package will be prefixed with "Datamart".
#
# Most loggers for the "soap" package are NOT based on the class name.
# Instead, there are two loggers for each table, named as follows:
#   ServiceNow.Request.<tablename>
#   ServiceNow.Response.<tablename>
# Diagnostic information prior to the SOAP call will be written to the Request logger.
# Diagnostic information after the SOAP call will be written to the Response logger.
# 
# Logger levels are as follows:
# ERROR - Serious error that will result in termination of the program
# WARN  - Truncation or recoverable data conversion errors
# INFO  - One line per SOAP or Database call
# DEBUG - Display XML sent and received
# TRACE - Field level diagnostics (extremely verbose)

log4j.rootLogger=INFO, Console

log4j.appender.Console=org.apache.log4j.ConsoleAppender
log4j.appender.Console.layout=org.apache.log4j.PatternLayout
log4j.appender.Console.layout.ConversionPattern=%d{HH:mm:ss} [%t] %p %c %m%n

log4j.logger.Datamart=DEBUG
log4j.logger.ServiceNow=DEBUG
log4j.logger.ServiceNow.Request=DEBUG
log4j.logger.ServiceNow.Response=INFO
log4j.logger.ServiceNow.Request.sys_db_object=WARN
log4j.logger.ServiceNow.Response.sys_db_object=WARN
log4j.logger.ServiceNow.Request.sys_dictionary=WARN
log4j.logger.ServiceNow.Response.sys_dictionary=WARN

Note that you can set the logging at different levels for different tables.

For general information on Log4J configuration files refer to http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PropertyConfigurator.html.

Make the Log4J config file available in the class path

The Log4J configuration needs to be in the class path so that Log4J can load it as a resource. One way to do this is to place the file in the current directory and prepend the current directory (“.”) to the class path. You can specify the class path on the Java command line using the “-cp” option.

For Unix the use a colon for the path separator:

-cp .:sndml-2.6.5-mysql.jar

For Windows use a semicolon for the path separator:

-cp .;sndml-2.6.5-mysql.jar

Set log4j.configuration to the name of the config file

Use the “-D” option to set the system property log4j.configuration to the the name of the file.

-Dlog4j.configuration=log4j_debug.properties

Put it all together

You cannot use the “-jar” option with “-cp”. Instead you must specify the main class on the command line. The main class is servicenow.common.datamart.Loader.

The full Java command looks like this:

java -cp .:sndml-2.6.5-mysql.jar -Dlog4j.configuration=log4j_debug.properties servicenow.common.datamart.Loader -p <propfile> -f <scriptfile>

Where <propfile> is the SNDML properties file and <scriptfile> is the SNDML script file.

Getting Distinct Records

GarrettNow

Every once in a while, I run into a situation where I need a list of distinct values from a table. The latest situation I ran into for this, I needed to get a distinct list of every user that had an open Project Task.

The easiest way to do this of course is to go to a list view, and sort by the field you care about. For example:

https://demo013.service-now.com/nav_to.do?uri=pm_project_task_list.do?sysparm_query=active=trueGROUPBYassigned_to

Project Tasks by User Grouping Project Tasks by Assigned To

 

For my purposes though, I wanted to send out scripted email notifications to everyone that had an open task. As a result, I needed this information available to me through scripting, which caused me to look for a different solution.

The answer came in the form of “GlideAggregate”. This is a Service-Now custom JavaScript object that functions very similar to the popular “GlideRecord”, with a few key differences.

GlideAggregate is built to…

View original post 159 more words

SOAP Retrieval Limit

Someone help me out here. Did I miss something or am I just stupid?

On the wiki page Retrieving A Large Number Of Records From ServiceNow it says

ServiceNow has imposed a hard-limit of 250 records that can be retrieved at any time in a single query.

The wiki page was created in July 2009.  The “Winter 2009 Stable 2” Release Notes mention the following new “feature”

  • Limit SOAP to 250 records

ServiceNow seemed to be pretty serious about the 250 record limitation. However, in September 2012 a new property — glide.remote_glide_record.max_count — appeared on the Available System Properties wiki page with notes about controlling the “records returned when using a Perl API gliderecord query.” On the Direct Web Services wiki page there is an example of Perl code using windowing, with the following comment:

Using a windowed query mechanisms overcomes the default limitation of only getting a maximum of 250 records per query.

“Default” implies that the limitation can be overridden. I assumed that the purpose of the system property was to override the default.

This is, in fact, the behavior or the property. It raises the “hard-limit”. If you set the property, it will increase the number of records returned by the getRecords SOAP method. It is not Perl specific. It works for any language. I assumed that setting the property was the only way to get more than 250 record in a single SOAP call. There is a __limit extended query parameter documented; but I assumed that this parameter was for retrieving fewer records, and that it did not allow you to bypass the “hard-limit”.

Not so!

The “hard-limit” is not a hard-limit at all! It is a soft-limit. It turns out that there is an easy way to get more than 250 records, and it does not require changing a system property. All you need to do is ask! By simply passing in the __limit parameter, you can override the “hard-limit”.

To test this theory, I modified the Visual Basic example from my previous post. This code now returns a list of all the users in single Web Services call.

Dim objGetUsers : Set objGetUsers = New ServiceNowDirectWS
objGetUsers.SetMethod "sys_user", "getRecords"
objGetUsers.SetValue "__limit", "2000"
If objGetUsers.Post Then
  Dim nRows : nRows = objGetUsers.GetRowCount
  Dim i : For i = 1 To nRows
    WScript.Echo i & ": " & objGetUsers.GetRowValue(i, "user_name") & _
      ": " & objGetUsers.GetRowValue(i, "name")
  Next
Else
  WScript.Echo "Error: " & objGetUsers.Status
  WScript.Echo objGetUsers.StatusText
End If

Obviously we do not want to get carried away. It is fine to bump up the limit if you are trying to retrieve a list of your 1500 users. To retrieve your list of 150,000 servers, you are still strongly advised to use windowing.

The thing I am still trying to figure out is: Has it always worked this way? Was the hard-limit really a hard-limit in the beginning, and then it was reduced to a soft-limit? Or was it always a soft-limit, and the wiki page simply written to discourage developers from bypassing the limit. Please leave a comment if you know the answer.