Verifying Log Data

Check for missing log data by either:

• Reviewing recent log performance statistics.
• Looking for gaps in the sequential number reported by each Real-Time Log Delivery software agent.

Log Performance Statistics

The Log Performance page provides a summary view and a breakdown of log delivery failures for up to the last 30 days.

Find out which log files are missing by manually checking for gaps in the sequence number reported by each Real-Time Log Delivery software agent.
Learn more.

Key information:

• Navigate to the Log Performance page by performing the following steps:

  1. Navigate to the Real-Time Log Delivery CDN | Rate Limiting | WAF page. From the main menu, navigate to More and then find Real-Time Log Delivery under Analytics. Select either CDN, WAF, or RL.

  2. Select the desired profile.
  3. Click the Analytics tab from the upper-right hand corner of the page.
• Choose the time period for which log performance statistics will be reported from the upper-right hand corner of the page.

• Log delivery failures are graphed according to the following categories:

  Category	Description
  Bad Certificate	Indicates that the SSL certificate corresponding to the domain where log data is being sent is invalid. Please verify your SSL certificate and then update as needed. There are online tools (e.g., SSL Checker) that analyze your SSL certificate for issues. Log delivery requires a certificate whose trust anchor is a publicly trusted certificate authority (CA). Additionally, the certificate must include a chain of trust for all intermediate certificate(s) and a leaf certificate.
  Connection Time Out	Indicates that the destination server failed to respond in a timely fashion.
  Failed Authentication	Indicates that log delivery failed due to an unauthorized request (i.e., 401 Unauthorized or 403 Forbidden).
  Failed Connection	Indicates that the destination server was unavailable.
  Failing to Deliver	Indicates that log delivery failed for none of the above reasons.

Checking for Sequence Number Gaps

Use the following information when assessing whether there is a gap in the sequential number reported by each Real-Time Log Delivery software agent.

• A software agent's unique ID is reported within the:

  • Log file name (AgentID) - AWS S3, Azure Blob Storage, and Google Cloud Storage only

  • JSON payload (agent-id)

• A software agent's sequence number is reported within the:

  • Log file name (SequenceNumber) - AWS S3, Azure Blob Storage, and Google Cloud Storage only

  • JSON payload (seq-num)
• The sequential number reported for each software agent starts at 0.

• This sequential number resets to 0 at the start of a new day (UTC). The date on which log data was generated is reported within the:

  • Log file name (DateStamp) - AWS S3, Azure Blob Storage, and Google Cloud Storage only
  • JSON payload (date-stamp)
• If a software agent stops running, then it will be assigned a new unique ID.

If log data uses either the JSON Array or JSON Lines log format, then you will be unable to use the JSON payload to check for sequence number gaps. This means that you will be unable to check for sequence gaps when delivering log data to your web server(s), Splunk Enterprise, Sumo Logic, Datadog, or New Relic.

Log File Example

RTLD CDN: On 12/8/2019, the log file naming convention was updated to include the profile ID for your Real-Time Log Delivery configuration.

Let's assume that your AWS S3 bucket, Azure Blob container, or Google Cloud Storage bucket contains the following log files:

In this situation, we can tell that there is missing log data. Specifically, the log entries associated with the following log file are missing: