Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: For security purposes, remove the output of Capabilities information from the logs. #13710

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from

Conversation

zhangwt-cn
Copy link

@zhangwt-cn zhangwt-cn commented Mar 20, 2024

User description

Fixes #13648

Description

For security purposes, remove the output of Capabilities information from the logs.

Motivation and Context

fix #13648

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist

  • I have read the contributing document.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

Type

Bug fix


Description

  • Removed the output of session capabilities from log messages in LocalDistributor and LocalNode to enhance security.

Changes walkthrough

Relevant files
Bug fix
LocalDistributor.java
Remove Session Capabilities from Log Output in LocalDistributor

java/src/org/openqa/selenium/grid/distributor/local/LocalDistributor.java

  • Removed the output of session capabilities from the log message during
    session creation.
  • +1/-1     
    LocalNode.java
    Remove Session Capabilities from Log Output in LocalNode 

    java/src/org/openqa/selenium/grid/node/local/LocalNode.java

  • Removed the output of session capabilities from the log message during
    session creation.
  • +2/-2     

    PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    @CLAassistant
    Copy link

    CLAassistant commented Mar 20, 2024

    CLA assistant check
    All committers have signed the CLA.

    Copy link

    PR Description updated to latest commit (0a8c20e)

    Copy link

    PR Review

    ⏱️ Estimated effort to review [1-5]

    1, because the changes are straightforward and involve removing logging of session capabilities in two files for security reasons. The modifications are minimal and localized to logging statements.

    🧪 Relevant tests

    No

    🔍 Possible issues

    No

    🔒 Security concerns

    No


    ✨ Review tool usage guide:

    Overview:
    The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
    When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:

    /review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...
    

    With a configuration file, use the following template:

    [pr_reviewer]
    some_config1=...
    some_config2=...
    
    Utilizing extra instructions

    The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.

    Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.

    Examples for extra instructions:

    [pr_reviewer] # /review #
    extra_instructions="""
    In the 'possible issues' section, emphasize the following:
    - Does the code logic cover relevant edge cases?
    - Is the code logic clear and easy to understand?
    - Is the code logic efficient?
    ...
    """
    

    Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

    How to enable\disable automation
    • When you first install PR-Agent app, the default mode for the review tool is:
    pr_commands = ["/review", ...]
    

    meaning the review tool will run automatically on every PR, with the default configuration.
    Edit this field to enable/disable the tool, or to change the used configurations

    Auto-labels

    The review tool can auto-generate two specific types of labels for a PR:

    • a possible security issue label, that detects possible security issues (enable_review_labels_security flag)
    • a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)
    Extra sub-tools

    The review tool provides a collection of possible feedbacks about a PR.
    It is recommended to review the possible options, and choose the ones relevant for your use case.
    Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example:
    require_score_review, require_soc2_ticket, require_can_be_split_review, and more.

    Auto-approve PRs

    By invoking:

    /review auto_approve
    

    The tool will automatically approve the PR, and add a comment with the approval.

    To ensure safety, the auto-approval feature is disabled by default. To enable auto-approval, you need to actively set in a pre-defined configuration file the following:

    [pr_reviewer]
    enable_auto_approval = true
    

    (this specific flag cannot be set with a command line argument, only in the configuration file, committed to the repository)

    You can also enable auto-approval only if the PR meets certain requirements, such as that the estimated_review_effort is equal or below a certain threshold, by adjusting the flag:

    [pr_reviewer]
    maximal_review_effort = 5
    
    More PR-Agent commands

    To invoke the PR-Agent, add a comment using one of the following commands:

    • /review: Request a review of your Pull Request.
    • /describe: Update the PR title and description based on the contents of the PR.
    • /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
    • /ask <QUESTION>: Ask a question about the PR.
    • /update_changelog: Update the changelog based on the PR's contents.
    • /add_docs 💎: Generate docstring for new components introduced in the PR.
    • /generate_labels 💎: Generate labels for the PR based on the PR's contents.
    • /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

    See the tools guide for more details.
    To list the possible configuration parameters, add a /config comment.

    See the review usage page for a comprehensive guide on using this tool.

    Copy link

    codiumai-pr-agent-pro bot commented Mar 20, 2024

    PR Code Suggestions

    CategorySuggestions                                                                                                                                                       
    Enhancement
    Add a null check for sessionId before logging to prevent potential NullPointerExceptions.

    Consider adding a conditional check to ensure sessionId is not null before logging it.
    This can prevent potential NullPointerExceptions and ensure the log message is meaningful
    even when a session ID is not available.

    java/src/org/openqa/selenium/grid/distributor/local/LocalDistributor.java [588]

     String.format(
    -  "%s. Id: %s", sessionCreatedMessage, sessionId));
    +  "%s. Id: %s", sessionCreatedMessage, sessionId != null ? sessionId : "N/A"));
     
    Maintainability
    Extract log message formatting into a separate method for better readability and maintainability.

    To enhance readability and maintainability, consider extracting the log message formatting
    into a separate method. This approach can simplify the newSession method and make the log
    message construction more flexible and reusable.

    java/src/org/openqa/selenium/grid/node/local/LocalNode.java [497-498]

    -String.format(
    -  "%s. Id: %s",
    -  sessionCreatedMessage, sessionId));
    +formatLogMessage(sessionCreatedMessage, sessionId)
     
    +// Elsewhere in the class
    +private String formatLogMessage(String message, String id) {
    +  return String.format("%s. Id: %s", message, id != null ? id : "N/A");
    +}
    +

    ✨ Improve tool usage guide:

    Overview:
    The improve tool scans the PR code changes, and automatically generates suggestions for improving the PR code. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.
    When commenting, to edit configurations related to the improve tool (pr_code_suggestions section), use the following template:

    /improve --pr_code_suggestions.some_config1=... --pr_code_suggestions.some_config2=...
    

    With a configuration file, use the following template:

    [pr_code_suggestions]
    some_config1=...
    some_config2=...
    
    Enabling\disabling automation

    When you first install the app, the default mode for the improve tool is:

    pr_commands = ["/improve --pr_code_suggestions.summarize=true", ...]
    

    meaning the improve tool will run automatically on every PR, with summarization enabled. Delete this line to disable the tool from running automatically.

    Utilizing extra instructions

    Extra instructions are very important for the improve tool, since they enable to guide the model to suggestions that are more relevant to the specific needs of the project.

    Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify relevant aspects that you want the model to focus on.

    Examples for extra instructions:

    [pr_code_suggestions] # /improve #
    extra_instructions="""
    Emphasize the following aspects:
    - Does the code logic cover relevant edge cases?
    - Is the code logic clear and easy to understand?
    - Is the code logic efficient?
    ...
    """
    

    Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

    A note on code suggestions quality
    • While the current AI for code is getting better and better (GPT-4), it's not flawless. Not all the suggestions will be perfect, and a user should not accept all of them automatically.
    • Suggestions are not meant to be simplistic. Instead, they aim to give deep feedback and raise questions, ideas and thoughts to the user, who can then use his judgment, experience, and understanding of the code base.
    • Recommended to use the 'extra_instructions' field to guide the model to suggestions that are more relevant to the specific needs of the project, or use the custom suggestions 💎 tool
    • With large PRs, best quality will be obtained by using 'improve --extended' mode.
    More PR-Agent commands

    To invoke the PR-Agent, add a comment using one of the following commands:

    • /review: Request a review of your Pull Request.
    • /describe: Update the PR title and description based on the contents of the PR.
    • /improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
    • /ask <QUESTION>: Ask a question about the PR.
    • /update_changelog: Update the changelog based on the PR's contents.
    • /add_docs 💎: Generate docstring for new components introduced in the PR.
    • /generate_labels 💎: Generate labels for the PR based on the PR's contents.
    • /analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

    See the tools guide for more details.
    To list the possible configuration parameters, add a /config comment.

    See the improve usage page for a more comprehensive guide on using this tool.

    @krmahadevan
    Copy link
    Contributor

    @zhangwt-cn - Not everyone would want this to be the default behaviour no? What if this were to be behind a flag, which when flipped will give this behaviour?

    Either way, I would wait to hear from @diemol before you take up the recommendation of obfuscating the capabilities for security reasons.

    @zhangwt-cn
    Copy link
    Author

    @krmahadevan Users should have the capability to produce their own logs, and I believe that the output of sensitive information should not come from the logs of the underlying framework.

    @krmahadevan
    Copy link
    Contributor

    @krmahadevan Users should have the capability to produce their own logs, and I believe that the output of sensitive information should not come from the logs of the underlying framework.

    Sure. But the point of contention is as to what does and what does not denote/represent as sensitive information. For majority of the users, they aren't packing in any sensitive data into the capabilities. The capabilities itself does not have any such information apart from the browser version and the optional flags that are being passed to the underlying driver. So there's no reason as to why this needs to be hidden out from the regular user. It looks to me that you are adding additional stuff into the capabilities object which you wouldn't want to be logged. That was why I am suggesting that this behaviour be controlled via a flag which can be toggled (It should be switched off by default)

    @zhangwt-cn
    Copy link
    Author

    @krmahadevan Users should have the capability to produce their own logs, and I believe that the output of sensitive information should not come from the logs of the underlying framework.

    Sure. But the point of contention is as to what does and what does not denote/represent as sensitive information. For majority of the users, they aren't packing in any sensitive data into the capabilities. The capabilities itself does not have any such information apart from the browser version and the optional flags that are being passed to the underlying driver. So there's no reason as to why this needs to be hidden out from the regular user. It looks to me that you are adding additional stuff into the capabilities object which you wouldn't want to be logged. That was why I am suggesting that this behaviour be controlled via a flag which can be toggled (It should be switched off by default)

    Okay, I understand what you mean. I'll make some adjustments based on your suggestions.

    @krmahadevan
    Copy link
    Contributor

    @zhangwt-cn Please hold on. Lets hear from @diemol before you make any changes. I would like to ensure that you spend more time on this based on his inputs. I was just throwing in my thoughts.

    Copy link
    Member

    @diemol diemol left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Thank you for starting to work on this, @zhangwt-cn!

    I agree with @krmahadevan. Not all users want this to be the default behavior. As an example, this information helps debug as well.

    The fix could be to obfuscate any given capability or a pattern (e.g., wss://admin:admin@org-se...), which should be under a flag and configuration.

    @zhangwt-cn
    Copy link
    Author

    Added a privacyMode attribute to control whether to output the following sensitive information.

    • se:vnc
    • se:cdp

    Users can utilize the privacyMode attribute to control whether or not to output certain types of sensitive information, such as

    FirefoxOptions options = new FirefoxOptions();
    options.setCapability("privacyMode", false);

    If privacyMode is true, then it will display *******, if it is false, it will display the real content.
    @diemol @krmahadevan

    Copy link
    Member

    @diemol diemol left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    I believe this would not work because:

    • privacyMode is not a W3C capability, so it would be filtered and ignored.
    • This change modifies the capability value, and we only want to obfuscate it. The client bindings must use the actual value to connect to the server.

    We probably want a configuration on the server side that enables or disables the obfuscation, following the same pattern we use through flags and options: https://github.com/SeleniumHQ/selenium/tree/trunk/java/src/org/openqa/selenium/grid/distributor/config

    @zhangwt-cn
    Copy link
    Author

    I believe this would not work because:

    • privacyMode is not a W3C capability, so it would be filtered and ignored.
    • This change modifies the capability value, and we only want to obfuscate it. The client bindings must use the actual value to connect to the server.

    We probably want a configuration on the server side that enables or disables the obfuscation, following the same pattern we use through flags and options: https://github.com/SeleniumHQ/selenium/tree/trunk/java/src/org/openqa/selenium/grid/distributor/config

    Okay, I will make some modifications.

    @@ -31,6 +33,16 @@ class SharedCapabilitiesMethods {

    private static final String[] EMPTY_ARRAY = new String[0];

    private static Boolean PRIVACY_MODE = true;

    private static final Set<String> PRIVACY_KEYS;
    Copy link
    Member

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    We now target Java 11+, so it's okay to use Set.of("se:vnc", "se:cdp")

    @zhangwt-cn
    Copy link
    Author

    @diemol Based on your advice, I made the following changes

    public class DistributorOptions {
      public static boolean PRIVACY_MODE = true;
    
      private final Config config;
    
      public DistributorOptions(Config config) {
        this.config = config;
        PRIVACY_MODE = getPrivacyMode();
      }
      public boolean getPrivacyMode() {
        return config
          .getBool(DISTRIBUTOR_SECTION, "privacy-mode")
          .orElse(PRIVACY_MODE);
      }
    }
    public class DistributorFlags implements HasRoles {
     @Parameter(
        names = "--privacy-mode",
        description = "If privacy-mode is true, then sensitive information will display *******, if it is false, it will display the real content. ")
      @ConfigValue(section = DISTRIBUTOR_SECTION, name = "privacy-mode", example = "true")
      private boolean privacyMode = PRIVACY_MODE;
    }

    but I encountered a problem: SharedCapabilitiesMethods cannot access DistributorOptions. It seems that changing dependencies is necessary, but I'm unsure if such changes are allowed, or if you have any better suggestions.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    [🐛 Bug]: Auth info in Logs in selenium grid
    5 participants