Help Docs

Log tagging

Log tagging categorizes log data by assigning specific labels or tags to log entries based on predefined rules. This process efficiently filters, manages, and analyzes logs without manually reviewing each log entry. Start monitoring your logs with our log management tool by creating a Log Type and Log Profile, then associating them with a Log Tag for better log organization and analysis.

Use case

Organizations often collect logs from multiple applications, and some logs might contain sensitive data such as email addresses, phone numbers, or other PII. To comply with data protection regulations, they need a systematic way to identify and manage these logs without manually inspecting each entry. Log tagging enables automated identification by applying tags, such as PII, Error, or Security, to relevant logs making it easier to filter and take necessary actions.

For example, if a log entry contains a credit card number, such as:

[2025-02-04 18:22:58] CRITICAL - Unauthorized transaction detected: Card=4111-1111-1111-1111

the CreditCardNumber tag will be automatically applied, enabling organizations to quickly flag and handle these entries.

How it works

You can apply predefined log tagging rules or create custom rules to identify specific log activities. The rule checks the RegEx pattern against your application logs. If a log line matches the pattern, the specified log tags will be applied. Using our powerful query language, you can search, group, and create alerts based on log tags.

Site24x7 provides predefined log tag rules to detect email addresses, phone numbers, social security numbers, credit card numbers, IP addresses, and dates of birth. Custom log tag rules will also be included in the default list.

Note

Currently, log tagging is supported only when logs are collected via Site24x7's server monitoring agent (Windows | Linux). It is not applicable to other log collection methods such as Amazon Lambda, Azure Functions, or API upload.

Steps to add log tagging

Log in to Site24x7 and navigate to Admin > AppLogs > Log Tags. In the top right corner of the page, you will find two buttons:

Note

The Add Log Tag Rule button will be disabled initially. You must first create a log tag group by clicking the Add Log Tag Group button and then associate it with a log tag rule to ensure that each group has at least one log tag rule linked to it.

Add Log Tag Group

Follow these steps to create a log tag group:

  1. Log Tag Group Name: Enter a name for the log tag group.
  2. Description: Provide a brief description of the log tag group.
  3. Select Log Tag Rules: Choose from the available predefined log tag rules. Custom tag rules will also be included in this list. At least one rule is required for log tagging to work. To create a custom rule, use the Add Log Tag Rule option. A maximum of ten rules can be applied to a single log type.
  4. Select Log Types: Select the log types to which you want to apply the log tag rules.
  5. Enable Log Tag Group: Toggle this option to enable or disable the log tag group. When enabled, log tagging rules will be applied to incoming logs.

Under Advanced Settings:

  1. Choose Fields: Select specific fields for log tagging. If no fields are selected, the rule will be applied to the entire log line. If multiple log types are selected in the Select Log Type dropdown above, this dropdown will display each log type along with its corresponding fields for selection.
  2. Log Profiles: This field appears for log types with multiple log profiles. Select the log profile to which the log tag rules should be applied.
  3. Click Save.

Add Log Tag Rule

Follow these steps to create a log tag rule:

  1. Log Tag Rule Name: Enter a name for the log tag rule.
  2. Log Tag Groups: Select the log tag group to which this rule should be applied. The log tagging will be applied to the log types defined in the selected group.
  3. Description: Provide a brief description of the log tag rule.
  4. Matching RegEx: Specify the RegEx pattern to match log entries that should be tagged. Validate your RegEx using our free tool.
  5. Validate Rule with Sample Log: Provide a sample log entry for which the testing should be done.
  6. Add Log Tags: Define the log tags to be applied when a log entry matches the log tag rule. You can add up to three tags.
  7. Enable Log Tag Rule: Toggle this option to enable or disable the log tag rule. When enabled, the rule will be applied to incoming logs.

    Under Advanced Settings:
  8. Keywords: Below are the settings available under the Keywords section:
    1. Keywords to Match: Specify a list of keywords that must be present in the log, along with the matching pattern.
      Note
      • If keywords are not defined, log tags will be applied when the specified RegEx pattern matches the log line.
      • If keywords are defined, log tags will be applied only if both the RegEx pattern matches and the defined keywords are present in the log line.
    2. Keyword Proximity (Characters Before RegEx Match): Define the number of characters before the matching pattern within which keywords should be checked. The proximity is evaluated first, meaning only logs containing the specified keywords within the defined range before the RegEx match will be considered for log tagging.
      Note

      Refer to this section for examples of log tagging using RegEx, keywords to match, and tagging based on keyword proximity.

  9. Actions: You can hash or mask sensitive information to enhance security and compliance. Select the Hash or Mask option from the dropdown to hide sensitive data. If you choose Mask, enter the replacement text in the provided textbox.
  10. Click Save.

Examples of log tagging with RegEx, keywords to match, and keyword proximity

Example 1: Without keywords

  • RegEx Rule: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
  • Sample Log: Marketing mail successfully sent to xyztest@abc.com
  • Keywords: (Not defined)
  • Tag Applied: Yes (because the RegEx matched the email address)

Example 2: With keywords

  • RegEx Rule: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
  • Sample Log: Marketing mail successfully sent to xyztest@abc.com
  • Keywords: marketing
  • Tag Applied: Yes (because both the RegEx matched the email address and the keyword marketing is present)

Example 3: Keywords defined, but a keyword is not present in the log line

  • RegEx Rule: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
  • Sample Log: Newsletter successfully sent to xyztest@abc.com
  • Keywords: marketing
  • Tag Applied: No (because the RegEx matched the email, but the keyword marketing is not present)

Example 4: Keyword proximity defined, but tagging does not apply

  • RegEx Rule: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
  • Sample Log: A promotional email was sent successfully to xyztest@abc.com. Don't miss out on our marketing offers!
  • Keywords: marketing
  • Keyword Proximity: 20 characters
  • Tag Applied: No (because marketing does not appear within the 20-character proximity before the matched email address)

Example 5: Keyword proximity defined; tagging applies

  • RegEx Rule: [a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}
  • Sample Log: Marketing email sent to xyztest@abc.com for promotion.
  • Keyword Proximity: 20 characters
  • Tag Applied: Yes (because marketing appears within the 20-character range of the matched email address)

Edit Log Tag Groups and Rules

You can edit a log tag group and its tags by navigating to Admin > AppLogs > Log Tags. This page displays all log tag groups along with their associated tags.

  • To edit a log tag group, click the pencil icon IAAAAAElFTkSuQmCCnext to the group name.
  • To edit the tags within a group, click the pencil iconIAAAAAElFTkSuQmCCnext to the tags.
  • You can also clone tags using the clone iconclone img.Clicking it opens a new window where you can duplicate existing tag rules, make necessary changes, and assign them to a different group.

Search using a query language and create alerts

Once you have created a log tag group and log tag rules, navigate to the AppLogs search page from the left navigation menu. Here, you can search logs using the powerful query language.

Example query

logtype="ActiveMQ Logs" and  log_tags="Bank"

You can also set up alerts for the created log tags to stay informed about important log events.

Related articles

Was this document helpful?

Would you like to help us improve our documents? Tell us what you think we could do better.


We're sorry to hear that you're not satisfied with the document. We'd love to learn what we could do to improve the experience.


Thanks for taking the time to share your feedback. We'll use your feedback to improve our online help resources.

Shortlink has been copied!