NetSuite Bundles

The following NetSuite application is available to all active Prolecto Resources clients without license charge. Click here to get back to the bundle list.

Prolecto Record Import/Export Manager (RIEM)

Provides a framework for configuring and monitoring regular data imports and exports, and the scripts to automate the process. Allows you to set up regular import and export jobs based on various criteria, and then to monitor the status of those jobs. Each import or export is processed via a custom plug-in implementation. Drop a file in a folder or email to a specific address and let the work begin.

Also supported is scheduled NetSuite CSV exports using persistent search for large data outputs. Couple this Prolecto sFTP to drive automated file distribution.

Bundle ID

Ask for Bundle ID: 413713.

Version History

20240318.RC

Added support for the BAI2 file format as a RIEM import type. The BAI2 data is parsed to JSON which can subsequently be processed as deemed fit.

20231222.RC

  • Enhanced the "auto-generate jobs" feature of export jobs to prevent duplicates.
  • Added a parameter to the export script which allows can be used to configure a deployment to target only certain job types.
  • Added better validation on the Job Type grouping field, to reduce likelihood of user error when setting up the columns.

20230918.RC

Enhancements to RIEM exports to include the use of Queries as well as Saved Searches.  Trigger auto-generate jobs when they are in the ready state. 

20230905.RC

Expanded the enhancement from 20230825.RC so that there is a new import staging record status called ‘Failed/Duplicate” to make it clearer why the record was not processed.

Also, fixed the bug where a job that consists entirely of duplicates never moved to a final status.

20230825.RC

Job type now can optionally detect and prevent duplicate imports via staging record hashing.

20230620.RC

Added enhanced logic to prevent queues from being stuck during import errors. 

20230619.RC

RIEM File Handling (Archive, rename/timestamp) given NetSuite's file clobbering impact of moving a file into a folder with an existing file having the same name.

20230517.RC

  • Refinements to the profile record to help with defaults.
  • Grouping refinements when all options are null
  • Additional job type record to hold job-type=specific parameters to drive the plug-in

20230424.RC

  1. Bundle Update Validator function to prevent bundle updates until proper app setting is configured
  2. Added support for setting custom fields on Staging Record.

20230222.RC

  1. Enhanced Profile definition for supporting SuiteQL export definition.
  2. Added action to trigger profile processing "On Demand". On the profile page, the button "Schedule Now" immediately schedules export processing. 
  3. Ability to specify scheduling as a crontab expression.

20221020.RC

Rewrote the file importer system to use SuiteSQL, versus saved search, to overcome two concerns:
  • A larger number of folders/files in a single pass
  • The archive folder now can live under the import folder with no performance impact.

20221017.RC

Upgraded all scripts to work in 2.1 so that related plug-ins can be 2.0 to 2.1.compatible.

20220824.RC

Added support for semicolon-separated files (similar to comma and tab-separated formats)

20220124.RC

Better handling of "item" records (with its many subtypes) when creating links from a staging record to the "created" record.

20220106.RC

Refinement for the generic RIEM importer plug-in.

20211130.RC

Moved to a new development environment and thus has been deprecated distribution from Bundle ID 240913 to Bundle ID 413713.


New capacity to automatically import data that is shaped with heading names to the target with a generic plug-in. This then means no programming is required for that plugin approach.

20211019.RC

Support for scheduled profiles that drive native NetSuite persistent (large file) CSV saved search outputs.

20210726.RC

Support for SuiteQL (SQL) queries to drive RIEM Export routines.

20210719.RC

Changes for better support for Prolecto's Content Renderer Engine 2.0.

20210623.RC

  1. Support for CRE2 export jobs
  2. Convenient method to reset failed import jobs

20210525.RC

Support for 3 different script instances of the “Import Job Manager” to have control over relative priorities of job types.

20210322.RC

  1. Bugfix to new Import Job Closer script.
  2. Ensure that all scripts related to job imports have a scheduled deployment, as necessary.
  3. Added a new summary job view helpful for administrators

20210318.RC

Added a preParser function to allow processing files before they are handed to the primary engine to help clean up for intake.  Minor enhancement for noted script anomoly.

20210126.RC

Added ability to control which characters are permitted in headings of delimited files.

20201226.RC

>Refinements to installation scripts

20201215.RC

Support for calling a function after parsing is complete (and also for calling a function when the job moves to a ‘completed’ status).

20200925.RC

Support for splitting RIEM processing so that groups can be used to help with priority processing efforts.

20200911.RC

A refinement to accommodate processing under load by ensuring that only one map/reduce script is running even if scheduled or manually / programmatically triggered.

20200820.RC

Enhanced to allow for a public function to createExports and thus jobs from other programs to allow well-defined initialization.

20200813.RC

Enhanced to allow for a public function to createImports and thus jobs from other programs to allow well defined initialization.

20200807.RC

Enhancement for import files that have no header.  The job type definition now allows for the naming each elements based on the delimiter position.

20200615.RC

Refinements for more flexible grouping options.

20200610.RC

  1. A way to group all lines of the import into a single staging record.
  2. The email listener plugin now records and stores the email address of the person who emailed the file in.
  3. An optional URL to call when the job completes.

20200212.RC

Support for JSONPath syntax for parsing batched JSON into individual elements.

20191218.RC

Updates for:
  • Rare issue in the code that parses CSV files, when the file to be parsed uses some non-standard ways of holding data which contains double quotes.
  • Reference to usage guide

20190708.RC

A new checkbox on the Job Type record allows the exported record to be automatically tagged with the Job ID by which it was exported.  This helps make many export plugins unnecessary.

20190522.RC

  1. New field “expected record count” on the job; if the field is populated, and the parsed number of records doesn’t match this value, then the job is stopped with a 'PARSING ERROR' 
  2. Support for XML file formats

20190430.RC

Refinements: Updated to take advantage of new Map/Reduce Utilities feature when trying to import a garbage file.  Prior to this update, such a file could “hang” the process.

 

20190425.RC

Enhanced if a user creates an Import job in the UI, then as soon as the job is created, the referenced file is now automatically moved to the Archive folder (unless it is already there).

20190328.RC

  • When an import job is manually created (in the UI), make sure that user is not placing the file into the folder from where jobs are created automatically, else we end up with duplicate jobs 
  • When an import job is manually created (in the UI), kick off the next step (parsing) immediately

20190227.RC

Refinements for the new delimiter capacities.

20190213.RC

Support for different file (layout) types.

20181228.RC

  1. Fixed bug when RIEM calculated the # of lines of data in the CSV file.  Previously, if a single “cell” had newline characters, it threw counter off.
  2. Added support for CSV files which had more columns of data than headings.  Previously the job just threw an error if that happened.  Now you can control whether it throws an error, or imports the data with auto-generated heading names.

20181210.RC

Additional status code to be leveraged during the import process.

20181119.RC

Minor refinements including reference to documentation.

20180926.RC

Fixed a bug where, if an import file had no rows, the process would loop, continually trying to re-process the same file. 

New feature (specifically for a client), so that you can configure an Export Job Type to regularly check whether there are any records to export, and if there are, automatically create an export job, based on the configured frequency.

20180806.RC

Better status visibility and logging on export jobs.

20180611.RC

First major revision deployed at four different clients.

Obtain Access to Prolecto Record Import/Export Manager (RIEM) Bundle

If this Prolecto Record Import/Export Manager (RIEM) bundle has interest for you, make a request and let's have a conversation.
Be Sociable, Share!

Marty Zigman

Holding all three official certifications, Marty is regarded as the top NetSuite expert and leads a team of senior professionals at Prolecto Resources, Inc. He is a former Deloitte & Touche CPA and has held CTO roles. For over 30 years, Marty has produced leadership in ERP, CRM and eCommerce business systems. Contact Marty to set up a conversation.

More Posts - Website - Twitter - Facebook - LinkedIn - YouTube