This article is relevant if you use NetSuite and you are concerned with managing personal information supporting the European General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA) regulatory requirements.
Background
As the landscape of consumer privacy evolves and global regulatory frameworks are established, organizations utilizing NetSuite must equip themselves with effective tools and strategies to align with legal mandates. This is particularly crucial in the context of major regulations such as the European General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). These regulations empower consumers with the ability to request the deletion of their data from NetSuite ERP databases, a concept often referred to as “The Right to be Forgotten.” In addressing these privacy concerns, the prevalent practice involves obfuscating data – rendering information that can identify an individual as ambiguous or unclear. The objective here may not necessarily mean eliminating records from the NetSuite database but rather surgically altering specific data fields.
Similarly, organizations might adopt a data archiving policy alongside making information less distinct. Such a strategy not only aids in addressing security and privacy issues by reducing the potential areas vulnerable to breaches but also requires careful planning. These efforts aim to effectively expunge data while preserving the integrity of financial and operational records.
To assist organizations in tackling these privacy challenges, NetSuite has introduced a Personal Information (PI) Removal tool. As of the posting of this article, this tool is a relatively new addition to the NetSuite module offerings. In real practical applications for clients we serve, this tool requires supplementation through an additional application to ensure the consistent execution of privacy policies.
Producing a Data Retention Policy Framework for Enterprise NetSuite Organizations
Engaging with a client who belongs to the Oracle NetSuite Billion Dollar Club, boasting annual revenues exceeding $1 billion, our focus was set to proactively address privacy concerns. This client, operating in a high-volume eCommerce sector, aimed to systematically obscure transactional personally identifiable information (PII) at regular intervals. In this pursuit, we evaluated the potential of NetSuite’s Personal Information (PI) Removal tool to drive the objective. However, our analysis revealed that, while functional for single-record obfuscation – ideal for customer service responding to individual requests – it fell short for broader enterprise-level policy implementation. Its limitation was evident in mass updates; the tool’s parameters were not robust enough to enforce an easily comprehensible policy.
The good news is that an API drives the Personal Information (PI) Removal tool (N/piremoval Module). This API opened the door to developing a more comprehensive framework. It allowed us to devise and implement policies systematically and to schedule tasks to modify data within the NetSuite ERP system.
Leveraging the capabilities of the PI API and other NetSuite API tools, we were equipped to create a Data Retention Manager application to meet compliance requirements. The tool was completely built on the NetSuite framework to ensure a cohesive and effective data management strategy.
Considering Data Purging Policies
Following the decision of our client to establish a scheduled policy for data obfuscation, they recognized the necessity to extend this approach to other records for outright removal. This need arose from the utilization of custom record-based batch queues, which are underpinned by a suite of Prolecto license-free utilities we provide. These utilities are instrumental in creating highly reliable system integration and processing programs, as discussed in my 2022 article Contrast Platform vs NetSuite Point-to-Point Integration Options. A critical aspect of managing these batch queues was the imperative to remove completed queued records to maintain system efficiency and possible data privacy security breaches.
In addition to these records, the client’s system was generating a significant volume of integration files within NetSuite’s file cabinet, as discussed in my 2020 article, Solve for High Performance NetSuite Data Exports. These files, destined for a third-party logistics (3PL) fulfillment center, also accumulated over time and contained sensitive, personally identifiable information. The accumulation of these files presented a data storage concern and a potential privacy risk in a security breach.
Therefore, a well-conceived policy must encompass the obfuscation of sensitive data and the systematic purging of unnecessary or completed records and files. This dual approach of obfuscation and purging ensures that the client’s system remains efficient, secure, and compliant with data protection regulations while maintaining the operational integrity of their NetSuite environment. When defined and executed, such a policy would provide a comprehensive framework for data management, aligning with the broader objectives of the organization’s data governance strategy.
Plug-in-Based Data Action Framework
In developing a robust policy framework for data management, we initiated a set of three core plug-ins. These plug-ins are designed to be extendable, ensuring they can adapt to various policy expressions and capacity needs:
- Obfuscate Records: This plug-in is focused on targeting specific record types within NetSuite. It allows for the specification of which fields should be obfuscated. Utilizing NetSuite’s Personal Information (PI) Removal API, this plug-in effectively removes sensitive data elements from the identified records, ensuring compliance with privacy regulations while maintaining the structural integrity of the data.
- Delete Records: This plug-in operates by leveraging a saved search or SQL definition to identify records within the scope of the policy. Once these records are identified, the plug-in systematically deletes them from the system. This function is particularly useful for managing no longer operationally necessary or relevant data, thus optimizing database performance and adhering to data retention policies.
- Delete Files: Similar to the Delete Records plug-in, this module targets files stored in NetSuite’s file cabinet. The plug-in identifies files that need to be deleted based on predefined criteria. This is crucial for managing the accumulation of integration files and other data that may contain sensitive information, ensuring that only necessary and current files are retained in the system.
These plug-ins collectively form a versatile and efficient framework for executing data management policies. They provide the necessary tools to enforce data obfuscation, record deletion, and file management in a controlled and automated manner. This plug-in-based approach streamlines the data management process and ensures that the organization’s data practices remain compliant, secure, and aligned with its operational needs.
Policy Scheduling
With a set of targeted actions defined within the policy framework, the next step is to schedule these actions for execution. Subject to a subsequent article, while not detailed here, our development included an advanced scheduling utility that surpasses the flexibility of NetSuite’s native script scheduler. This enhancement ensures that policy-driven record actions are executed with the desired frequency, providing a more dynamic and responsive approach to data management.
Record Operation and Logging
For policy administrators, robust logging operations are crucial to verify that policy requirements are consistently met. Our framework is designed to carefully log records, indicating which records, files, or field IDs were impacted. Status updates are also recorded, allowing administrators to develop dashboards or other monitoring tools. This capability is vital for smooth operations, offering transparency and accountability. It also serves as a valuable resource for internal auditors to confirm adherence to established policies.
Essential Framework for Data Archiving
This tool transcends its immediate utility, becoming a cornerstone in broader data archiving projects. Although data archiving concerns are less prevalent in the NetSuite environment compared to Tier 1 ERP systems like Oracle EBS and SAP, our tool provides the necessary mechanics to implement an effective data archive policy. Reflecting on my experience from nearly two decades ago, where I led an SAP data archive service aiding Fortune 500 companies to maintain only seven years of data history in their ERP systems, this framework brings similar capabilities to the NetSuite realm. It’s a testament to our commitment to providing sophisticated, enterprise-level data management solutions within the NetSuite ecosystem.
NetSuite’s Personal Information (PI) Removal Nuances
After the development of our application, we proceeded cautiously with the activation of the defined policies, carefully monitoring expected behaviors. During this phase, we started to encounter certain complexities, especially in the performance and capabilities of NetSuite’s Personal Information (PI) Removal API. As we began to utilize the API intensively, we noticed issues related to its performance and certain limitations in its ability to obscure specific fields in allowable records.
These challenges highlighted the need for further refinement and optimization of the API. Recognizing these nuances, we have since been actively collaborating with Oracle NetSuite’s Product and Support teams. Our goal is to enhance and strengthen the API, ensuring it can robustly support high-reliability data retention policies. This ongoing work is critical in ensuring that the PI Removal tool can effectively meet the demands of complex data management tasks within NetSuite, particularly in large-scale and high-frequency environments. Through this collaborative effort, we aim to address the immediate challenges and contribute to the evolution of NetSuite’s data management capabilities, making it more robust and reliable for users who require stringent data privacy and retention protocols.
Note that the NetSuite PI API currently has some limitations on SubList data yet does important work on system notes and workflow history, which is otherwise inaccessible. The capacity to work on those data elements is unique to this API. Because of our plug-in approach, we can overcome existing limitations where NetSuite’s tools give us capacity.
Watch a Prolecto Data Retention Manager Video Demonstration (9:34)
The Prolecto Data Retention Manager is a testament to the expertise and dedication of the Prolecto team. In this video demonstration, I engage in a dialogue with Michelle B., a Business Analyst in our Inventory Practice, to illustrate the tool’s functionalities. Michelle played an important role in leading the charge to gather detailed client requirements and stabilize the app, which were essential in shaping the tool’s development. She collaborated closely with key members of the Prolecto Technology Practice, including Chidi O., Borko M., and Carl Z., who brought their technical acumen to the forefront in implementing the program logic essential for building out the application.
Watch the demonstration here for an in-depth understanding of how the Prolecto Data Retention Manager operates and the value it brings to data management within NetSuite environments:
This video (9:34) provides a comprehensive overview of the tool’s capabilities, reflecting the intricate work and thought process that went into its creation. It’s an excellent resource for anyone interested in understanding the practical aspects of data retention management in NetSuite.
Get the Prolecto Data Retention Manager
At Prolecto, we take pride in our LABS Initiative, where we create and share our intellectual property without license charges with our clients. The Prolecto Data Retention Manager is a prime example of this commitment, offered without any licensing fees. This approach is a cornerstone of our service, reflecting our dedication to adding value through expertise and innovation in the NetSuite ecosystem.
Our clients benefit greatly from our over 15+ years of achievements and experience in NetSuite, where we have continually led the way in business-led systems integration. This expertise leads our clients to realize their ambitions within the NetSuite framework. However, the true value we offer transcends the software itself. It resides in the capability of our people to ideate, design, develop, implement, and support such applications.
We firmly believe that software should be flexible and adaptable to address specific business needs. This philosophy drives our commitment to providing solutions not constrained by existing software paradigms’ limitations. Instead, we encourage a creative approach to modifying and shaping business applications to meet unique requirements. Our aim is to empower our clients with the tools and support they need to optimize their operations within NetSuite, ensuring that their software infrastructure is as dynamic and forward-thinking as their business strategies.
If you found this article relevant, feel free to sign up for notifications to new articles as I post them. If you are ready to tackle your NetSuite privacy-based personally identifiable information challenge or seek to implement a data archiving or purging policy, let’s have a conversation.
Marty Zigman
Holding all three official certifications, Marty is regarded as the top NetSuite expert and leads a team of senior professionals at Prolecto Resources, Inc. He is a former Deloitte & Touche CPA and has held CTO roles. For over 30 years, Marty has produced leadership in ERP, CRM and eCommerce business systems. Contact Marty to set up a conversation.
More Posts - Website - Twitter - Facebook - LinkedIn - YouTube
