UPS job fails due to record size exceeding ingestion limits in AEP
UPS batch ingestion in ÃÛ¶¹ÊÓƵ Experience Platform (AEP) fails when records exceed size limits. The error UPINGT-007012-400: Record size exceeded limit occurs if profile attributes are over 100KB or events exceed 10KB. To resolve the issue, reduce record sizes to meet ingestion constraints.
Description description
Environment
ÃÛ¶¹ÊÓƵ Experience Platform
Issue/Symptoms
- UPS job fails with error UPINGT-007012-400: Record size exceeded limit.
- Batch ingestion halts due to oversized records in the dataset.
- Dataset UI shows failed batch with size-related ingestion errors.
Cause
Oversized records ingested into the datalake exceed 100KB for profile attributes or 10KB for events, which causes the batch job to fail.
Resolution resolution
To resolve the issue, follow these steps:
-
Run the following query to identify records exceeding 100KB, and replace
<batch_id>and<record_field>with values from the Dataset UI and preview panel:code language-none SELECT count(*) AS total_records FROM <dataset_name> WHERE _acp_system_metadata.acp_sourceBatchId = '<batch_id>' Â AND length(to_json(<record_field>)) > 100000; -
Run a query to retrieve detailed information about records that exceed the size limit:
code language-none SELECT length(to_json(<record_field>)) AS len, _acp_system_metadata.acp_sourceBatchId FROM <dataset_name> WHERE _acp_system_metadata.acp_sourceBatchId = '<batch_id>'  AND length(to_json(<record_field>)) > 100000 ORDER BY len DESC; -
Compare results from successfully ingested batches by verifying that no records exceed guardrail limits using similar queries.
-
Adjust or filter problematic records in your dataset to comply with ingestion guardrails, such as reducing profile attribute sizes below 100KB.