Duplicate Data Handling

Duplicate attachment content is shared by multiple attachments.

No duplicates created during attachment copying

The former 1:1 relation 'Attachment-AttachedBlob' has changed to a N:1 relation. When an attachment is copied from one business object to another, the corresponding blob is no longer copied. Instead, the blob is shared by the attachments.

Blob duplicates deletion / prevention

A dedicated column in the AMT_BLOB database table identifies the content of blobvalue. Hash MD5+SHA256 is used as an identifier.

First, hash value is calculated in Java. Once the hash is evaluated, all duplicated blobs are deleted and attachments are relinked to the remaining unique blobs. As a result, the database contains unique blobs only.

• Deduplicate

  This operation can be done using migration tool 'DeduplicateBlobs', using console command 'attachments deduplicate' or using metamodel check for attachments in the repair mode.

• Analyze

  To analyze amount of duplicated blobs it is possible to use console command 'attachments collect duplicates' or metamodel check of attachments in check only mode.

The above described hash calculation process is also used during attachment copying and data import/export.

As another consequence, blobs are not explicitly removed with attachments because they can be shared by multiple attachments. They are removed by a database trigger when there is no attachment using the blob.

Important note:

This blob duplicates prevention introduced with Valuemation 5.1 means that the AMT_BLOB database table is administered by the application's internal processes and thus should no longer be tempered with or referenced manually.