Show HN: Nosey Parker, a fast and low-noise secrets detector for textual data https://ift.tt/FHhvU4w

Show HN: Nosey Parker, a fast and low-noise secrets detector for textual data Nosey Parker is an Apache-licensed command-line tool that finds secrets and sensitive information in textual data. It's useful both for offensive and defensive security testing. The big idea: textual content in, hardcoded secrets out. These include things like API keys and passwords. It should do a reasonable job on any textual input. It will recursively scan any directories it is pointed at. It also has special support for scanning the complete contents of Git repositories. The default rules in Nosey Parker have been carefully chosen to minimize false positives. Many API tokens these days have well-specified formats that are amenable to precise matching with regular expressions, and these are the kind of things that the default rules detect. Additionally, its findings are deduplicated. Together, these give much higher signal-to-noise compared to similar tools. Nosey Parker is fast: it can scan 100GB of Linux kernel commit history in just over a minute on my laptop. This speed comes from several factors, but most significantly from using the amazing Hyperscan library for simultaneous matching of all regex rules in a single pass. In comparison with similar tools on large inputs, Nosey Parker is usually 1-3 orders of magnitude faster. Nosey Parker was originally created to help construct a labeled dataset of secrets for machine learning purposes. But it proved surprisingly useful on its own. In the past year, an internal, proprietary version (with added machine learning capabilities) has been regularly used in security engagements at Praetorian. In late 2022, Nosey Parker was reimplemented in Rust, released as open-source, and presented at Black Hat Arsenal. It now supports enumeration and scanning of GitHub repositories by providing just usernames or organization names. It also recently got support for SARIF output, which several other tools understand. https://ift.tt/B3sYrIb March 3, 2023 at 06:10AM