ÃÛ¶¹ÊÓÆµ Experience Manager: Temporary Files under /tmp persist on Standby instance with S3 DataStore
In ÃÛ¶¹ÊÓÆµ Experience Manager (AEM) Cold Standby environments, you would notice unexpected disk space usage on the Standby instance. It happens because the Standby instance generates and accumulates a large number of temporary files in its /tmp
directory. These files follow a specific naming pattern:
/tmp/{random-alphanumeric}#{number}.tmp
To fix this, adjust the standby transfer timeout and perform regular cleanup of temporary files.
Description description
Environments
- ÃÛ¶¹ÊÓÆµ Experience Manager (AEM), v6.5
- TarMK Cold Standby
- S3 Shared DataStore
Issue/Symptoms
In a typical S3 Shared DataStore setup, the Standby instance retrieves blobs directly from S3, not from the primary instance. However, if the Primary is uploading a large asset and the upload to S3 is not yet complete, the standby may request a direct blob transfer from the Primary. These transfers can take time, and if they exceed the value set by the standby.readtimeout parameter, the transfer is interrupted.
If a transfer is interrupted, the temporary file created for the operation is not deleted automatically and remains in the /tmp directory. After the S3 upload completes, the direct blob transfer from primary to standby is no longer necessary and the instances synchronize correctly. However, any temporary files generated before this point still persist.
Example:
/tmp/a4b7c38e9f01d256e3b8f74a2c5e9d1b6a78401cd537e2984bc2519fa6cb813d#1234567890.tmp
Resolution resolution
Here are two options that may help address this issue:
-
Increase the
standby.readtimeout
setting on the standby instance to a value appropriate for your typical file sizes and available network bandwidth. This can help reduce transfer interruptions and minimize orphaned temporary files.Note: Test this change in a staging environment before applying it in production.
-
Use a scheduled task, such as a cron job, to regularly delete temporary files older than a certain period from the
/tmp
directory.