Breaking the Search ceiling

The standard model for scripted search in Netsuite is fairly simple, but unfortunately very limited.  Thanks to the 2012.1 release we have an additional method for searching that is far more robust.  The new method has a short learning curve and is fairly flexible once you get the basics down.

First let's review the basic's of a scripted search with nlapiSearchRecord().  The basic two-step process is:

1. Compile a list of filters and columns,
   

   var filters = [
   new nlobjSearchFilter( field_name, join_record, search_type, value_1, value_2 ),
   ...
   ];
   var columns = [
   new nlobjSearchColumn( field_name ),
   ...
   ];

2. Then execute the search,
   

   var results = nlapiSearchRecord( record_name, saved_search_id, filters, columns );

Using this pattern we end up with an array of nlobjSearchResults.  It works well, but the one glaring problem is the 1,000 record limit - regardless of the number of records in our record of choice, nlapiSearchRecord will only return a maximum of 1,000 records.  And if your search pool is larger, you find the results are typically fragmented, making it VERY difficult to procure the full list.

Now we have a new pattern, the Saved Search.

Err, actually, we have saved searches already, in the Netsuite front-end.  This new search pattern ties into saved searches, but it's not the same thing.  So, lets call it Scripted Saved Search (or SearchResultSets, which will make more sense in a minute.)

The new pattern has two primary objects, nlobjSearch and nlobjSearchResultSet.  We use a three-step process to get our search results:

1. Obtain a Netsuite Saved Search.
2. Run the Saved Search to obtain a Search Result Set.
3. Use the Search Result Set to customize a list of nlobjSearchResults.

A bit more detail.

Step 1 - Obtain a Netsuite Saved Search

In our old method we had a convoluted search mechanism: nlapiSearchRecord only REQUIRED a record type.  You could then pass in the id of a saved search as a sort of stored procedure, or skip the saved search id and just pass in filters and/or columns to construct a customized search.  Or pass in the saved search id AND the filters & columns to create a customized set of results from your stored procedure.  Powerful, but clunky.

SearchResultSets are a bit more elegant: a search record of type nlobjSearch is generated by the following methods:

• nlapiCreateSearch(type, filters, columns) and 
• nlapiLoadSearch(type, id)

We're doing roughly the same thing here, saved searches can be accessed through nlapiLoadSearch, and custom searches can be "created".  Additionally there is now the ability to save the nlobjSearch and create new saved searches directly from SuiteScript!  Cor!  And the nlobjSearch type can add filters and columns after being set, so you still have all of the power from before.

Step 2 - Run the Saved Search to obtain a Search Result Set

Once the nlobjSearch is customized to your hearts content, it is used to generate a Result Set (note that while the nlobjSearch CAN be saved, it does not have to be in order to generate results).  It's a simple one-liner:

var resultset = savedsearch.runSearch();

Yep, thats it.  The Search Result Set is a powerful object for large searches, we'll examine it in...

Step 3 - Use the Search Result Set to customize a list of nlobjSearchResults

Now that we have the Search Result Set we have two options:

• perform a limited harvest with a callback function
• perform an extended harvest with "slices"

Each one is fairly useful and deserves a brief mention:

By calling forEachResult() you can define a callback function that accepts one parameter (nlobjSearchResult).  The function will be called sequentially as long as the previous instance returns true, returning false will cut the sequence short and end the forEachResult method.  It can be compared to a for-loop where returning false is analogous to a break statement, or perhaps better compared to a do...while-loop where the return value is the evaluation statement.  Either way forEachResult has a hard-limit of 4,000 records.  A bit larger than nlapiSearchRecord, but if you have larger search pools you'll need to use "slicing".

getResults() has two required parameters: start and end.  If you think of the Search Result Set as an abstracted array, start and end reference the indexes (0-based, end is exclusive) of this array.  getResults() consumes 10 governance units and only returns 1,000 records max. per call, but it can be called over and over again!  getResults() returns an array of nlobjSearchResult, just like nlapiSearchResult, but the required indices make it far more powerful.

To finally tie it all together, here is a sample using a custom search with the getResults syntax:

var filters = [...];
var columns = [...];
var results = [];
var savedsearch = nlapiCreateSearch( 'customrecord_mybigfatlist', filters, columns );
var resultset = savedsearch.runSearch();
var searchid = 0;
do {
    var resultslice = resultset.getResults( searchid, searchid+1000 );
    for (var rs in resultslice) {
        results.push( resultslice[rs] );
        searchid++;
    }
} while (resultslice.length >= 1000);
return results;

The 1,000 search limit ceiling isn't just lifted, it just disappeared.  It's all open skies now!