This article is relevant if you need a way to work a large number of records in NetSuite to drive updates, produce action, or create other records.

Background

While I love the NetSuite platform, it can be challenging to work with the system when it comes to Mass or Bulk record operations.  The major challenge is that the development platform has not exposed the built-in record generators like it has in the form-technology.  NetSuite’s bulk record generators are offered in various application areas, such as the Pay Bills, or Order Items routine.  While each provided bulk record generator may offer up a way to change the view or perhaps the filter, too often you can’t get at the information you need.

The most common generator request I get from prospective clients is related to the Purchase Order in connection with the Sales Order. The idea is that a client needs more control over the crafting of purchase orders based on the criteria of the information presented on Sales Order lines.  For example, if you have many sales orders that need to be supplied from an array of suppliers, you may need criteria such as the supplier’s pickup address or quantity price breaks to make a decision. In another perspective, perhaps you need to evaluate item lines to make better assessments about the visibility of existing stock in your warehouses to help choose using a dropship vendor or deliver from stock?  Perhaps there is a special item attribute that you want to filter by but NetSuite is not offering it up?

I get a fair number of inquiries based on my 2017 article, Learn how to Consolidate Multiple NetSuite Sales Orders to One Purchase Order, which is a basic illustration of the challenge.  In that article, we solved the challenge specifically.

Yet, the challenge does not simply exist in the Sales Order and Purchase Order domain; it can exist anywhere where bulk operation work needs to be done. To name a few, consider Invoicing Operations, Vendor Bill Payments, Customer Statements, Pay Bills, Pick Ticket Generation, or Case Record Assignment. The point is that the pattern to act on many records is pervasive in normal day-to-day business operations.

Conventional Development Approach to Custom Bulk Operations

The typical approach used to solve a bulk operation is to reach for the SuiteScript development environment and create a custom SuiteLet to solve the specific problem. I have designed many of these with my team. The pattern involves the following design consideration:

1. Source Record Evaluation: what records need to be evaluated and presented in spreadsheet-like (i.e., list) fashion so the user can evaluate and take action?
2. Source Record Filtering: how can we allow the user to filter the source records to narrow the scope of information to be evaluated so that like-kinds concerns can be acted on in a batch?
3. Information Capture: is there information that must be captured in each list row so that it can drive action? For example, must we collect a quantity, select from an options list, or collect a specific text value?  How do we validate the information provided by the user complies with rules?
4. Input Record Submission: the records need to be acted on once the user submits them.  This submission thus becomes the inputs to drive subsequent business action in a batch.
5. Business Action: the input records become the source for subsequent algorithms that will typically generate new records, update other records, or trigger functions in the environment.  This step is the action we care about and it needs to work in a fashion that can be trusted.

Indeed, NetSuite provides the general SuiteScript tools to produce applications that can address the design concerns above.  Programming usually involves simply solving the specific challenge at hand.   This approach will get the job done but it lacks any capacity for leverage; meaning, if a new challenge comes forth, we start the pattern all over again inventing every aspect of the solution.

Note, NetSuite also offers the Mass Update tool that has some capacities for development.  If you write a custom mass update script, you can attack the Business Action step. Its main limitation is that it can only work with a single saved search and the experience is rigid.  Furthermore, it may be challenging to get the permissions working right — hence, I think about the built-in Mass Update tool as a one-off Administrator feature; not a day-to-day usage tool.

Introducing the Prolecto Build Record Generator

For a fast-growing eCommerce client that has an interesting local goods pickup and delivery model, the requirements to produce a purchase order generator presented themselves in the fashion they always do. However, this time, working with our team of professionals, we stepped back and thought more carefully about a number of existing tools that we have built over the 10+ years being a NetSuite Systems Integrator.

Enjoyable (Fast) User Experience

Our client needed a very fast user experience in their Sales Order evaluation and Purchase Order generation work. Orders would be coming from an eCommerce site at a continual high frequency and multiple business users would have to make many decisions quickly to act on delivery-based Sales Orders to generate related pickup Purchase orders.   Their business is to use their own fleet of trucks to pick up products from suppliers and deliver them to customers.  NetSuite standard speed for the load-and-save operation was not going to be acceptable — and we knew it given the nature of the client’s business model.  In addition, with multiple business users using the tool simultaneously, the application needed to be responsive to work with information that has already been submitted (in the job queue) or being acted on by other business users so that there is no confusion.

These requirements forced us to think differently; our inspiration came as we reflected our own custom-built internal ticket system, called the Prolecto Ticket Manager (PTM), which sits on top of NetSuite support cases. The breakthrough was the blazing fast speed for which the application operates, yet looks and feels like NetSuite. Here, we had a technology that allowed us to bring data locally to the browser and perform all the operations client-side. Using the Model-View-Controller (MVC) design pattern, JQuery, CSS and call back SuiteLets in an intelligent manner, it changed our point of view and standard for which we could build custom NetSuite applications.

I suggest watching the video below to appreciate the speed.  While I performed edits to produce the overall video, when it came to the operation of the application between the filtering and selections (the user experience),  no video editing tricks were used to speed up the operation.

Complex Data Sources

We recognized that we had to get data from different sources that would need to be assembled together for the user experience. For example, we needed to consolidate all the sales order lines for a particular locality. We needed to also gather up all the existing inventory on hand and on order across all of our warehouse locations. We needed to get all the suppliers that may be able to fulfill these orders including their most recent price; we also had statistics on each vendors’ previous delivery performance that needed to be included.  This data is in the database but it takes multiple saved searches to get to it.

Thus, we needed to link saved searches together to get the information we needed to drive the application.  I have written before how we created a tool called the “Content Renderer Engine (CRE)” that can be used to generate a NetSuite “data universe“. See my article, Join Multiple NetSuite Saved Searches to Build a Data Universe. With CRE, we had a very easy way to describe the searches we needed and link them together.  CRE could then supply all of that data in JSON format (the preferred format for working with NetSuite JavaScript technologies) to our application.

With the linked saved searches available, we solved the Source Record Evaluation consideration as noted above.

Metadata Framework Driven Application Development

We then leveraged a design pattern that allows an analyst to describe the user application based on the source data that is supplied to the application. Using tools such as JSON Path and intelligent design structures, we can describe the list rows that are displayed to the users, offer up validation and formulas to have each column interact with other columns, produce a connection to the source data filtering system, provide for expansion and contraction of additional information (think of tooltips, mouseovers and other information element decorations) all through definition (as opposed to programming).

Naturally, to solve a major bulk operation, we need to have intelligent paging; bulk select/deselect all list rows, and other usability tricks are used to populate all the rows with the information contained in a global drop-down or other data selectors.  With all this in hand, we now had a way to allow a business analyst to drive the creation of the application based on client data requirements; that process is natural to the analyst. You can see the app emerge as you add more and more metadata structure to the framework.

With this framework, we then solved the Source Record Filtering, Information Capture, and Input Record Submission design requirements noted above.

Bulk Record Generation

In any bulk operation, even NetSuite’s, you will notice that jobs are created when you have a major record operation. When you work with jobs, the idea is that you will have a batch of work to complete and this is generally placed into a work queue as it must process in the background.  The team recognized that we have a major framework for just this kind of effort. It is called the Record Import Export Manager (RIEM) and it has been solving our most demanding client requirements for bulk record processing.  See my article, Fully Automate Complex NetSuite Data Imports to understand that tool.

While targeting the generation of business records is common, such as the creation of purchase orders, it is a simple mechanism to update existing records or trigger local or remote functions.  The point is that we can now take a batch of NetSuite work and act on it.

Demonstration Video (6:37)

In the video below (6:37) I interview Sylvain M., one of our Technical Analyst Software Engineer, on how he produced two instances of the application:

1. Purchase Order Generation from Sales Orders: consolidate sales order demand to target purchase order generation.
2. Bulk Bin Transfers: take item receipt information and target bin putaway record generation.

Thus, the same framework is used to perform two completely different application operations.  This video illustrates two applications in use from the business user’s perspective.

Technical Discussion Video (7:42)

For those that are interested in how the application was built, assuming you watched the video above, Sylvain M. discusses the technology and the metadata descriptor structure to allow the application constitution. The video (7:42) below should make sense to individuals with a programming and application design background.

Get the Prolecto Bulk Record Generator

If you have a NetSuite bulk record challenge that needs to be solved, our tools are available to all of our clients without a license charge.  By solving a specific challenge, we can help your organization learn how to use our tools to produce your own NetSuite bulk record solutions.  The key is to be clear about the source data, the user experience, and the target action.

If you found this article meaningful, feel free to sign up for notifications of future articles as I post them.  If you have a specific NetSuite bulk record challenge, let’s have a conversation.

Marty Zigman

Holding all three official certifications, Marty is regarded as the top NetSuite expert and leads a team of senior professionals at Prolecto Resources, Inc. He is a former Deloitte & Touche CPA and has held CTO roles. For over 30 years, Marty has produced leadership in ERP, CRM and eCommerce business systems. Contact Marty to set up a conversation.

More Posts - Website - Twitter - Facebook - LinkedIn - YouTube