Automated removal of records

Patrick Smith Updated by Patrick Smith

The Automated removal of records feature is currently only available as a paid option for Ultimate plan clients.

Automated removal of records allows you to designate, for a given dataset, an age over which all records will be deleted (for example, all data older than one year).

This is especially useful for datasets that are updated incrementally (either via realtime push API or via an incremental data source) that are impractical or expensive to update manually.

And once the option has been activated, it can be used on multiple datasets.

In more detail

Automated removal of records offers several benefits. It frees up space for other records, you don't need to keep track of which specific records to delete, nor do you need to make individual requests to have each record deleted. And once it's set up, this happens automatically and completely.

The feature offers you:

  • Predictability: The feature removes the risk that a dataset may ‘inflate’ too much, which can not only be expensive, but result in your being blocked when you reach your quota, and possibly impact your workspace's performance.
    For example, connected devices often generate large amounts of real-time data. Those datasets can quickly become very large. Automated removal of records enables this type of data to flow through the platform in a controlled, scalable way.
  • Control: In some cases, you may want to limit how much data is available to your users. This feature is a tool to enforce this.
  • Ease of mind: Compared to using the API, this feature requires no need for complex technical implementation. Instead of having to write a script to pick all records not matching the given time window and make a delete request via the Push API, which requires the time and skills of a developer, you only have to define to which dataset you want it to apply to and what your desired time window is.

Setting it up

If you're interested in the option, contact your sales representative for more information. If you have activated this option, your admin should reach out to Opendatasoft's support.

But configuring the feature is easy. For the designated dataset, you define which record field is used as the timestamp value for your data, how long the data should be kept, and how frequently you want us to remove the records. Once enabled on a dataset, for you the behavior is transparent; records over the specified age will be removed from your dataset. The impact is immediate, clean, and easy.

How did we do?

Configuring dataset export


Powered by HelpDocs (opens in a new tab)