credsweeper.deep_scanner package

Submodules

credsweeper.deep_scanner.abstract_scanner module

class credsweeper.deep_scanner.abstract_scanner.AbstractScanner

Bases: ABC

Base abstract class for all recursive scanners

exception LimitError

Bases: Exception

Decompressed data exceeds configured limit

abstract property config: Config

Abstract property to be defined in DeepScanner

abstract data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Abstract method to be defined in DeepScanner

deep_scan_with_fallback(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate]

Scans with deep scanners and fallback scanners if possible

Parameters:

• data_provider – DataContentProvider with raw data

• depth – maximal level of recursion

• recursive_limit_size – maximal bytes of opened files to prevent recursive zip-bomb attack

Returns: list with candidates

abstract static get_deep_scanners(data: bytes, descriptor: Descriptor, depth: int, limit: int) → Tuple[List[Any], List[Any]]

Returns possibly scan methods for the data depends on content and fallback scanners

static key_value_combination(structure: dict) → Generator[Tuple[Any, Any], None, None]

Combine items by key and value from a dictionary for augmentation {…, “key”: “api_key”, “value”: “XXXXXXX”, …} -> (“api_key”, “XXXXXXX”)

abstract static match(data: bytes | bytearray) → bool

Abstract method for any deep scanner

static read_compressed_with_limit(file: LZMAFile | GzipFile | BZ2File, limit: int) → bytes

Reads data with check limit for single compressed file

recursive_scan(data_provider: DataContentProvider, depth: int = 0, recursive_limit_size: int = 0) → List[Candidate]

Recursive function to scan files which might be containers like ZIP archives

Parameters:

• data_provider – DataContentProvider object may be a container

• depth – maximal level of recursion

• recursive_limit_size – maximal bytes of opened files to prevent recursive zip-bomb attack

scan(content_provider: ContentProvider, depth: int, recursive_limit_size: int | None = None) → List[Candidate]

Initial scan method to launch recursive scan. Skips ByteScanner to prevent extra scan

Parameters:

• content_provider – ContentProvider that might contain raw data

• depth – maximal level of recursion

• recursive_limit_size – maximal bytes of opened files to prevent recursive zip-bomb attack

abstract property scanner: Scanner

Abstract property to be defined in DeepScanner

static structure_processing(structure: Any) → Generator[Tuple[Any, Any], None, None]

Yields pair key, value from given structure if applicable

structure_scan(struct_provider: StructContentProvider, depth: int, recursive_limit_size: int) → List[Candidate]

Recursive function to scan structured data

Parameters:

• struct_provider – DataContentProvider object may be a container

• depth – maximal level of recursion

• recursive_limit_size – maximal bytes of opened files to prevent recursive zip-bomb attack

static structure_size(structure: Any) → int

Calculates approximated size of structure data

credsweeper.deep_scanner.byte_scanner module

class credsweeper.deep_scanner.byte_scanner.ByteScanner

Bases: AbstractScanner, ABC

Implements plain data scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to represent data as plain text with splitting by lines and scan as text lines

static match(data: bytes | bytearray) → bool

Match for any

credsweeper.deep_scanner.bzip2_scanner module

class credsweeper.deep_scanner.bzip2_scanner.Bzip2Scanner

Bases: AbstractScanner, ABC

Implements bzip2 scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data from bzip2 archive and launches data_scan

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/Bzip2

credsweeper.deep_scanner.cpio_scanner module

class credsweeper.deep_scanner.cpio_scanner.CpioScanner

Bases: AbstractScanner, ABC

Implements cpio scanning

ASCII_HEADER = <_struct.Struct object>

BIN_BE_HEADER = <_struct.Struct object>

BIN_LE_HEADER = <_struct.Struct object>

ODC_HEADER = <_struct.Struct object>

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data file from .ar (cpioian) archive and launches data_scan

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures

static walk_cpio(data: bytes, limit: int) → Generator[Tuple[int, str, bytes], None, None]

Processes sequence of cpio archive and yields offset, name and data

credsweeper.deep_scanner.crx_scanner module

class credsweeper.deep_scanner.crx_scanner.CrxScanner

Bases: AbstractScanner, ABC

Implements CRX files scanning with cut-off prefix

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries cut-off header and use ZIP payload

static match(data: bytes | bytearray) → bool

Returns True if prefix match

static zip_extract(data: bytes) → bytes

Extracts zip payload after signature block

credsweeper.deep_scanner.csv_scanner module

class credsweeper.deep_scanner.csv_scanner.CsvScanner

Bases: AbstractScanner, ABC

Implements CSV scanning

CSV_PATTERN = re.compile(b'[^\r\n]{1,8000}[,;\t|\x1f][^\r\n]{1,8000}')

DELIMITERS = ',;\t|\x1f'

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan each row as structure with column name in key

classmethod get_structure(text: str) → List[Dict[str, Any]]

Reads a text as CSV standard with guessed dialect

static match(data: bytes | bytearray) → bool

Check if data MAY be in CSV format

sniffer = <csv.Sniffer object>

credsweeper.deep_scanner.deb_scanner module

class credsweeper.deep_scanner.deb_scanner.DebScanner

Bases: AbstractScanner, ABC

Implements deb (ar) scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data file from .ar (debian) archive and launches data_scan

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/Deb_(file_format)

static walk_deb(data: bytes) → Generator[Tuple[int, str, bytes], None, None]

Processes sequence of DEB archive and yields offset, name and data

credsweeper.deep_scanner.deep_scanner module

class credsweeper.deep_scanner.deep_scanner.DeepScanner(config: Config, scanner: Scanner)

Bases: ByteScanner, Bzip2Scanner, CpioScanner, CrxScanner, CsvScanner, DexScanner, DocxScanner, EncoderScanner, GritScanner, GzipScanner, HtmlScanner, JclassScanner, JksScanner, LangScanner, LexerScanner, LzmaScanner, MxfileScanner, EmlScanner, OdsScanner, PatchScanner, PdfScanner, PickleScanner, PkcsScanner, PlistScanner, PngScanner, PptxScanner, ProtobufScanner, PycacheScanner, RtfScanner, RpmScanner, SquashfsScanner, Sqlite3Scanner, SnkScanner, StringsScanner, TarScanner, DebScanner, XmlScanner, XlsScanner, XlsxScanner, ZipScanner, ZlibScanner, ZstdScanner

Advanced scanner with recursive exploring of data

MEDIA_PATTERNS: Dict[int, List[Tuple[bytes, Pattern | None]]] = {0: [(b'\x00\x00\x00\x0cjP  \r\n\x87\n', None), (b'\x00\x00\x01\x00', None), (b'\x00\x00\x02\x00\x01\x00', None), (b'\x00\x01\x00\x00\x00', None), (b'\x00\x00\x00', re.compile(b'\x00\x00\x00[\x00-\xff]ftyp(3gp[4-9]|M4[ABPV] |qt  |iso[2-9abcm]|mp4[12]|hei[cmsx]|dash|avi[fos]|jx[ls] |mif[12]|avc[1-4]|ccff)')), (b'\x00GITCRYPT\x00', None), (b'\x00asm', None), (b'\x00\x00\xff\xff\x00\x00d\x86', None)], 3: [(b'\x03\x00\x08\x00', None), (b'\x03\xf3\r\n', None), (b'\x03\x02#\x07', None)], 24: [(b'\x18\x00\x00\x00TFL3', None)], 26: [(b'\x1aE\xdf\xa3', None)], 28: [(b'\x1c\x00\x00\x00TFL3', None)], 30: [(b'\x1e\xe7', re.compile(b'\x1e\xe7(\xff\x00|\xff\x01|\x0fB)'))], 32: [(b' \x00\x00\x00TFL3', None)], 36: [(b'$\x00\x00\x00TFL3', None)], 55: [(b"7z\xbc\xaf'\x1c", None)], 56: [(b'8BPS\x00\x01\x00\x00\x00\x00\x00\x00', re.compile(b'8BPS\x00[\x01\x02]\x00\x00\x00\x00\x00\x00'))], 66: [(b'BM', re.compile(b'BM[\x00-\xff]{2}\x00{4}')), (b'BSJB\x01\x00\x01\x00\x00\x00\x00\x00', None)], 67: [(b'CWS', re.compile(b'CWS[\x06-+][^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]'))], 68: [(b'DDS ', re.compile(b'DDS [^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]'))], 70: [(b'FWS', re.compile(b'CWS[\x01-+][^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]'))], 71: [(b'GIF8', re.compile(b'GIF8[79]a[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]')), (b'GOBJ\nMETADATA\r\n\x1a', None)], 73: [(b'II', re.compile(b'II[+*]\x00[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]')), (b'II\xbc\x01', None), (b'ID3', re.compile(b'ID3[\x02\x03\x04]\x00\x00\x00'))], 77: [(b'MM', re.compile(b'MM\x00[+*][^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]')), (b'MZ', re.compile(b'MZ[\x00-\xff]{4,80}?\x00\x00')), (b'Microsoft C/C++ ', re.compile(b'Microsoft C/C[+][+] (program database 2[.]00\r\n\x1aJG\x00\x00|MSF 7[.]00\r\n\x1aDS\x00\x00\x00)')), (b'MTME\x00\x00\x00', None), (b'MThd\x00\x00\x00', None)], 79: [(b'OggS', re.compile(b'OggS[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]')), (b'OTTO\x00', re.compile(b'OTTO\x00[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]'))], 80: [(b'PACK\x00\x00\x00', None)], 82: [(b'RIF', re.compile(b'RIF[FX][\x00-\xff]{4}[ 0-9A-Za-z]{4}[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]')), (b'RIDX\x00\x00\x00', None), (b'Rar!\x1a\x07', re.compile(b'Rar!\x1a\x07(\x00|\x01\x00)'))], 84: [(b'TZif', re.compile(b'TZif[\x00234]\x00{3}'))], 86: [(b'VCLMTF\x01\x00', None)], 88: [(b'XPCOM\nTypeLib\r\n\x1a', None), (b'XFIR', re.compile(b'XFIR[\x00-\xff]{4}[ 0-9A-Za-z]{4}[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]'))], 90: [(b'ZWS', re.compile(b'ZWS[\r-+][^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]'))], 102: [(b'ftyp', re.compile(b'ftyp(isom|MSNV)[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]')), (b'fLaC', re.compile(b'fLaC[\x00-\x06]'))], 103: [(b'gimp xcf', re.compile(b'gimp xcf (file|v001|v002)\x00[^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]'))], 105: [(b'icns\x00', re.compile(b'icns\x00[\x00-\xff]{3}(IC(ON|N#)|ic([hms][#48]|s[bB]|l[48]|p[456]|0[45789]|1[0-4])|is32|s8mk|il32|l8mk|ih32|h8mk|it32|t8mk|sb24|SB24)'))], 116: [(b'ttcf\x00\x01\x00\x00', None), (b'ttcf\x00\x02\x00\x00', None)], 119: [(b'wOF', re.compile(b'wOF[2F][^\x00-\x08\x0c\x0e\x1f\x80-\xff]{0,4096}[\x00-\x08\x0c\x0e\x1f\x80-\xff]')), (b'wvpk', re.compile(b'wvpk[\x00-\xff]{4}[\x02-\x10]\x04'))], 120: [(b'xar!', re.compile(b'xar![\x00-\xff]{2}\x00\x01'))], 127: [(b'\x7fELF', re.compile(b'\x7fELF[\x01\x02][\x01\x02]\x01[\x00-\x12@-\xff]'))], 137: [(b'\x89PNG\r\n\x1a\n', None), (b'\x89HDF\r\n\x1a\n', None)], 147: [(b'\x93NUMPY', None)], 172: [(b'\xac\xed\x00\x05[p-~]', None)], 206: [(b'\xce\xfa\xed\xfe', None)], 207: [(b'\xcf\xfa\xed\xfe', None)], 208: [(b'\xd0\r\xfe\xed', None)], 222: [(b'\xde\x12\x04\x95', None)], 254: [(b'\xfe\xed\xfa\xce', None), (b'\xfe\xed\xfa\xcf', None)], 255: [(b'\xff', re.compile(b'\xff(\xd8\xff[\xdb\xee\xe1\xe0Q]|[\xfb\xf3\xf2])')), (b'\xfftOc\x00\x00\x00', None)]}

property config: Config

Abstract property to be defined in DeepScanner

static get_deep_scanners(data: bytes, descriptor: Descriptor, depth: int, limit: int) → Tuple[List[Any], List[Any]]

Returns possibly scan methods for the data depends on content and fallback scanners

static is_media(data: bytes | bytearray) → bool

Returns True if well-known media format found

property scanner: Scanner

Abstract property to be defined in DeepScanner

credsweeper.deep_scanner.dex_scanner module

class credsweeper.deep_scanner.dex_scanner.DexScanner

Bases: AbstractScanner, ABC

Implements DEX scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data from binary

static match(data: bytes | bytearray) → bool

Checks header magic for various DEX formats and versions

static walk_dex(data: bytes) → Generator[Tuple[int, bytes], None, None]

Processes sequence of DEX file and yields offset and bytes from strings

credsweeper.deep_scanner.docx_scanner module

class credsweeper.deep_scanner.docx_scanner.DocxScanner

Bases: AbstractScanner, ABC

Implements docx scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan DOCX text with splitting by lines

static match(data: bytes | bytearray) → bool

Assume, ZIP prefix and common office files were checked before

credsweeper.deep_scanner.eml_scanner module

class credsweeper.deep_scanner.eml_scanner.EmlScanner

Bases: AbstractScanner, ABC

Implements eml scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan EML with text representation

static match(data: bytes | bytearray) → bool

According to https://datatracker.ietf.org/doc/html/rfc822 lookup the fields: Date, From, To or Subject

credsweeper.deep_scanner.encoder_scanner module

class credsweeper.deep_scanner.encoder_scanner.EncoderScanner

Bases: AbstractScanner, ABC

Implements recursive iteration when data might be encoded from base64

BASE64_PATTERN = re.compile(b'(\\xFF\\xFE|\\xFE\\xFF)?((?:(?P<a>[A-Z])|(?P<b>[a-z])|(?P<c>[0-9/+])|[\\s\\x00\\\\])+(?(a)(?(b)(?(c)(=+|$)|(?!x)x)|(?!x)x)|(?!x)x)|(?:(?P<e>[A-Z])|(?P<f>[a-z])|(?P<g>[0-9_-])|[\\s\\x00\\\\])+(?(e)(?)

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to decode data from base64 encode to bytes and scan as bytes again

static decode(text: str) → bytes | None

Decodes base64 text with cleaning whitespaces. Returns None when the decoding fails

static match(data: bytes | bytearray) → bool

Check if data MAY be base64 encoded with whitespaces (escaping too)

credsweeper.deep_scanner.grit_scanner module

class credsweeper.deep_scanner.grit_scanner.GritScanner

Bases: AbstractScanner, ABC

Implements pak files from grit scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data file from .pak and launches data_scan

static match(data: bytes | bytearray) → bool

Versions 4,5

static walk_pak(data: bytes, limit: int) → Generator[Tuple[int, int, bytes], None, None]

Processes sequence of PAK archive and yields offset, resource id and data

credsweeper.deep_scanner.gzip_scanner module

class credsweeper.deep_scanner.gzip_scanner.GzipScanner

Bases: AbstractScanner, ABC

Realises gzip scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data from gzip archive and launches data_scan

static match(data: bytes | bytearray) → bool

According https://www.rfc-editor.org/rfc/rfc1952

credsweeper.deep_scanner.html_scanner module

class credsweeper.deep_scanner.html_scanner.HtmlScanner

Bases: AbstractScanner, ABC

Implements html scanning if possible

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to represent data as html text and scan as text lines

static match(data: bytes | bytearray) → bool

Used to detect html format. Suppose, invocation of is_xml() was True before.

credsweeper.deep_scanner.jclass_scanner module

class credsweeper.deep_scanner.jclass_scanner.JclassScanner

Bases: AbstractScanner, ABC

Implements java .class scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data from binary

static get_utf8_constants(stream: BytesIO) → List[str]

Extracts only Utf8 constants from java ClassFile

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures - java class

static u2(stream: BytesIO) → int

Extracts unsigned 16 bit big-endian

credsweeper.deep_scanner.jks_scanner module

class credsweeper.deep_scanner.jks_scanner.JksScanner

Bases: AbstractScanner, ABC

Implements jks scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan JKS to open with standard password

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures - jks

credsweeper.deep_scanner.lang_scanner module

class credsweeper.deep_scanner.lang_scanner.LangScanner

Bases: AbstractScanner, ABC

Implements scanning of data if it is a script of some markup language

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to represent data as markup language and scan as structure

static match(data: bytes | bytearray) → bool

Applied in represent_as_structure

credsweeper.deep_scanner.lexer_scanner module

class credsweeper.deep_scanner.lexer_scanner.LexerScanner

Bases: AbstractScanner, ABC

Implements C source scanning with lexical info

EASY_MATCHER = {'.c': <class 'pygments.lexers.c_cpp.CLexer'>, '.cc': <class 'pygments.lexers.c_cpp.CppLexer'>, '.cpp': <class 'pygments.lexers.c_cpp.CppLexer'>, '.cs': <class 'pygments.lexers.dotnet.CSharpLexer'>, '.cxx': <class 'pygments.lexers.c_cpp.CppLexer'>, '.h': <class 'pygments.lexers.c_cpp.CLexer'>, '.hh': <class 'pygments.lexers.c_cpp.CppLexer'>, '.hpp': <class 'pygments.lexers.c_cpp.CppLexer'>, '.hxx': <class 'pygments.lexers.c_cpp.CppLexer'>, '.java': <class 'pygments.lexers.jvm.JavaLexer'>, '.js': <class 'pygments.lexers.javascript.JavascriptLexer'>, '.m': <class 'pygments.lexers.objective.ObjectiveCLexer'>, '.mm': <class 'pygments.lexers.objective.ObjectiveCLexer'>, '.objj': <class 'pygments.lexers.javascript.ObjectiveJLexer'>, '.p6': <class 'pygments.lexers.perl.Perl6Lexer'>, '.pl': <class 'pygments.lexers.perl.PerlLexer'>, '.pl6': <class 'pygments.lexers.perl.Perl6Lexer'>, '.pm': <class 'pygments.lexers.perl.PerlLexer'>, '.raku': <class 'pygments.lexers.perl.Perl6Lexer'>, '.ts': <class 'pygments.lexers.javascript.TypeScriptLexer'>, '.v': <class 'pygments.lexers.hdl.VerilogLexer'>}

GUESSED_LEXERS = (<class 'pygments.lexers.c_cpp.CLexer'>, <class 'pygments.lexers.c_cpp.CppLexer'>, <class 'pygments.lexers.jvm.JavaLexer'>, <class 'pygments.lexers.javascript.JavascriptLexer'>, <class 'pygments.lexers.dotnet.CSharpLexer'>, <class 'pygments.lexers.javascript.ObjectiveJLexer'>, <class 'pygments.lexers.perl.PerlLexer'>, <class 'pygments.lexers.perl.Perl6Lexer'>, <class 'pygments.lexers.javascript.TypeScriptLexer'>)

LEXER_MATCHER = {('.c', '.h'): <class 'pygments.lexers.c_cpp.CLexer'>, ('.cpp', '.hpp', '.cc', '.hh', '.cxx', '.hxx'): <class 'pygments.lexers.c_cpp.CppLexer'>, ('.cs',): <class 'pygments.lexers.dotnet.CSharpLexer'>, ('.java',): <class 'pygments.lexers.jvm.JavaLexer'>, ('.js',): <class 'pygments.lexers.javascript.JavascriptLexer'>, ('.m', '.mm'): <class 'pygments.lexers.objective.ObjectiveCLexer'>, ('.objj',): <class 'pygments.lexers.javascript.ObjectiveJLexer'>, ('.p6', '.pl6', '.raku'): <class 'pygments.lexers.perl.Perl6Lexer'>, ('.pl', '.pm'): <class 'pygments.lexers.perl.PerlLexer'>, ('.ts',): <class 'pygments.lexers.javascript.TypeScriptLexer'>, ('.v',): <class 'pygments.lexers.hdl.VerilogLexer'>}

SUPPORTED_EXTENSIONS = ('.c', '.h', '.cpp', '.hpp', '.cc', '.hh', '.cxx', '.hxx', '.java', '.js', '.cs', '.m', '.mm', '.objj', '.pl', '.pm', '.p6', '.pl6', '.raku', '.v', '.ts')

SUPPORTED_LEXERS = (<class 'pygments.lexers.c_cpp.CLexer'>, <class 'pygments.lexers.c_cpp.CppLexer'>, <class 'pygments.lexers.jvm.JavaLexer'>, <class 'pygments.lexers.javascript.JavascriptLexer'>, <class 'pygments.lexers.dotnet.CSharpLexer'>, <class 'pygments.lexers.objective.ObjectiveCLexer'>, <class 'pygments.lexers.javascript.ObjectiveJLexer'>, <class 'pygments.lexers.perl.PerlLexer'>, <class 'pygments.lexers.perl.Perl6Lexer'>, <class 'pygments.lexers.hdl.VerilogLexer'>, <class 'pygments.lexers.javascript.TypeScriptLexer'>)

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan C code with lexical structures

static get_lexer(text: str, descriptor: Descriptor) → Lexer

Select a lexer for the source text.

A standard lexer is selected by file extension when available. Otherwise, the lexer is detected using the file path, file information, or, as a fallback, the source content itself.

Parameters:

• text – Source text to analyze.

• descriptor – File descriptor containing the extension, path, and file information used for lexer detection.

Returns:

A Pygments lexer suitable for parsing the source text.

static get_lines_semicolon(text: str, lexer: Lexer) → Tuple[List[str], List[int]]

Build logical source lines terminated by semicolons.

Joins consecutive physical lines until a semicolon terminating the current statement is encountered. Semicolons inside strings or other non-terminating lexical contexts do not split the statement.

Comments are extracted as separate logical lines and do not become part of the surrounding source statement.

Returns:

• logical source lines;

• starting line number of each logical line in the original text.

Return type:

A tuple of

static match(data: bytes | bytearray) → bool

Suggestion may be done with file extension, however the

credsweeper.deep_scanner.lzma_scanner module

class credsweeper.deep_scanner.lzma_scanner.LzmaScanner

Bases: AbstractScanner, ABC

Implements lzma scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data from lzma archive and launches data_scan

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures - lzma also xz

credsweeper.deep_scanner.mxfile_scanner module

class credsweeper.deep_scanner.mxfile_scanner.MxfileScanner

Bases: AbstractScanner, ABC

Scanner for drawio diagram

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to get text data from the xml format

static match(data: bytes | bytearray) → bool

Used to detect mxfile (drawio) format. Suppose, invocation of is_xml() was True before.

credsweeper.deep_scanner.ods_scanner module

class credsweeper.deep_scanner.ods_scanner.OdsScanner

Bases: PandasScanner, ABC

Implements xlsx scanning

static match(data: bytes | bytearray) → bool

Assume, ZIP prefix was checked before

credsweeper.deep_scanner.pandas_scanner module

class credsweeper.deep_scanner.pandas_scanner.PandasScanner

Bases: AbstractScanner, ABC

Implements xlsx scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan xlsx text elements for all slides

credsweeper.deep_scanner.patch_scanner module

class credsweeper.deep_scanner.patch_scanner.PatchScanner

Bases: AbstractScanner, ABC

Implements .patch scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan EML with text representation

static match(data: bytes | bytearray) → bool

Match logic in data_scan

credsweeper.deep_scanner.pdf_scanner module

class credsweeper.deep_scanner.pdf_scanner.PdfScanner

Bases: AbstractScanner, ABC

Implements pdf scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan PDF elements recursively and the whole text on page as strings

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures - pdf

credsweeper.deep_scanner.pickle_scanner module

class credsweeper.deep_scanner.pickle_scanner.PickleScanner

Bases: AbstractScanner, ABC

Implements pickle data scanning with pickletools

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts arguments only for safe disassembled pickle

static match(data: bytes | bytearray) → bool

Supports protocols 3, 4 and 5

credsweeper.deep_scanner.pkcs_scanner module

class credsweeper.deep_scanner.pkcs_scanner.PkcsScanner

Bases: AbstractScanner, ABC

Implements pkcs12 scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan PKCS12 to open with standard password

static match(data: bytes | bytearray) → bool

Matched ASN1 structure with exactly size

credsweeper.deep_scanner.plist_scanner module

class credsweeper.deep_scanner.plist_scanner.PlistScanner

Bases: AbstractScanner, ABC

Implements binary plist data scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data file from property list data and scan like a structure

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures - Binary Property List file

credsweeper.deep_scanner.png_scanner module

class credsweeper.deep_scanner.png_scanner.PngScanner

Bases: AbstractScanner, ABC

Implements PNG scanning for text chunks

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan each row as structure with column name in key

static match(data: bytes | bytearray) → bool

Returns True if prefix match

static yield_png_chunks(data: bytes) → Generator[Tuple[int, str, bytes], None, None]

Processes PNG chunks and yields offset, type and data

credsweeper.deep_scanner.pptx_scanner module

class credsweeper.deep_scanner.pptx_scanner.PptxScanner

Bases: AbstractScanner, ABC

Implements pptx scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to scan pptx text elements for all slides

static match(data: bytes | bytearray) → bool

Assume, ZIP prefix and common office files were checked before

credsweeper.deep_scanner.protobuf_scanner module

class credsweeper.deep_scanner.protobuf_scanner.ProtobufScanner

Bases: AbstractScanner, ABC

Implements protobuf (ar) scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data file from protobuf payload and launches data_scan

static match(data: bytes | bytearray) → bool

Simple structure check for whole data

static match_protobuf(data: bytes | bytearray, offset: int, limit: int) → bool

Process data from start to end as simple protobuf chunk

Returns: True when whole chunk was utilized with protobuf structure

static read_wire(data: bytes | bytearray, offset: int) → Tuple[int, int]

Reads wire to detect sizes

Returns: size of wire type (with primitives types), size of data (length-delimited)

static walk_protobuf(data: bytes, offset: int, limit: int) → Generator[Tuple[int, bytes], None, None]

Processes sequence of protobuf and yields offset and data recursive

credsweeper.deep_scanner.pycache_scanner module

class credsweeper.deep_scanner.pycache_scanner.PycacheScanner

Bases: AbstractScanner, ABC

Implements python cache files scanning

MATCHES_SIGNATURES = (b'\r\n\x00\x00\x00\x00', b'\r\n\x01\x00\x00\x00', b'\r\n\x03\x00\x00\x00')

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data from binary

static match(data: bytes | bytearray) → bool

Checks header magic for 3 actual variants

static walk_pycache(data: bytes, limit: int) → Generator[Tuple[int, bytes], None, None]

Yields (offset, bytes) from python cache file

credsweeper.deep_scanner.rpm_scanner module

class credsweeper.deep_scanner.rpm_scanner.RpmScanner

Bases: AbstractScanner, ABC

Implements rpm scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts files one by one from the package type and launches recursive scan

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures

credsweeper.deep_scanner.rtf_scanner module

class credsweeper.deep_scanner.rtf_scanner.RtfScanner

Bases: AbstractScanner, ABC

Implements squash file system scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Scans data as RTF

static get_lines(text: str) → List[str]

Extracts text lines from RTF format

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures - Rich Text Format

credsweeper.deep_scanner.snk_scanner module

class credsweeper.deep_scanner.snk_scanner.SnkScanner

Bases: AbstractScanner, ABC

Implements Snk files scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Scan .snk data for full (private+public) keys

static extract(data: bytes) → None | str

Extracts RSA parameters from .snk file and transform they into JWK representation

Parameters:

data – content of .snk file

Returns:

JWK string if successful format recognized or None if fail

static match(data: bytes | bytearray) → bool

https://github.com/dotnet/runtime/blob/main/docs/project/strong-name-signing.md

credsweeper.deep_scanner.sqlite3_scanner module

class credsweeper.deep_scanner.sqlite3_scanner.Sqlite3Scanner

Bases: AbstractScanner, ABC

Implements SQLite3 database scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts data file from .ar (debian) archive and launches data_scan

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures - SQLite Database

static walk_sqlite(data: bytes) → Generator[Tuple[str, Any], None, None]

Yields data from sqlite3 database

credsweeper.deep_scanner.squashfs_scanner module

class credsweeper.deep_scanner.squashfs_scanner.SquashfsScanner

Bases: AbstractScanner, ABC

Implements squash file system scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts files one by one from tar archive and launches data_scan

static match(data: bytes | bytearray) → bool

Squash file system magic and correctness check

credsweeper.deep_scanner.strings_scanner module

class credsweeper.deep_scanner.strings_scanner.StringsScanner

Bases: AbstractScanner, ABC

Implements known binary file scanning with ASCII strings representations

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Scan binary files for ASCII strings

static get_enumerated_lines(data: bytes) → List[Tuple[int, str]]

Processes binary to found ASCII strings. Use offset instead line number.

static match(data: bytes | bytearray) → bool

Match logic in data_scan

credsweeper.deep_scanner.tar_scanner module

class credsweeper.deep_scanner.tar_scanner.TarScanner

Bases: AbstractScanner, ABC

Implements tar scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts files one by one from tar archive and launches data_scan

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures

credsweeper.deep_scanner.tmx_scanner module

class credsweeper.deep_scanner.tmx_scanner.TmxScanner

Bases: AbstractScanner, ABC

Realises tmX files scanning for values only. Image tags are skipped.

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to represent data as xml text and scan as text lines

static match(data: bytes | bytearray) → bool

Used to detect tm7,tm6,etc. (ThreadModeling) format.

credsweeper.deep_scanner.xls_scanner module

class credsweeper.deep_scanner.xls_scanner.XlsScanner

Bases: PandasScanner, ABC

Implements xls matching

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures

credsweeper.deep_scanner.xlsx_scanner module

class credsweeper.deep_scanner.xlsx_scanner.XlsxScanner

Bases: PandasScanner, ABC

Implements xlsx matching

static match(data: bytes | bytearray) → bool

Assume, ZIP prefix and common office files were checked before

credsweeper.deep_scanner.xml_scanner module

class credsweeper.deep_scanner.xml_scanner.XmlScanner

Bases: AbstractScanner, ABC

Realises xml scanning

XML_FIRST_BRACKET_PATTERN = re.compile(b'^\\s*<')

XML_OPENING_TAG_PATTERN = re.compile(b'<([0-9A-Za-z_]{1,256})')

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Tries to represent data as xml text and scan as text lines

static match(data: bytes | bytearray) → bool

Used to detect xml format from raw bytes

credsweeper.deep_scanner.zip_scanner module

class credsweeper.deep_scanner.zip_scanner.ZipScanner

Bases: AbstractScanner, ABC

Implements zip scanning

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Extracts files one by one from zip archives and launches data_scan

static get_size(data: bytes | bytearray) → int

Evaluate extracted archive size

Returns: size of data or -1 in failure case

static match(data: bytes | bytearray) → bool

According https://en.wikipedia.org/wiki/List_of_file_signatures

credsweeper.deep_scanner.zlib_scanner module

class credsweeper.deep_scanner.zlib_scanner.ZlibScanner

Bases: AbstractScanner, ABC

Implements zlib data inflate and scan

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Inflate data from zlib compressed and launches data_scan

static decompress(limit: int, data: bytes) → bytes

Returns decompressed data by chunks with a limit or exception in unusual cases

static match(data: bytes | bytearray) → bool

Returns True if data looks like deflated data with zlib

credsweeper.deep_scanner.zstd_scanner module

class credsweeper.deep_scanner.zstd_scanner.ZstdScanner

Bases: AbstractScanner, ABC

Implements zstd data inflate and scan

data_scan(data_provider: DataContentProvider, depth: int, recursive_limit_size: int) → List[Candidate] | None

Inflate data from zstd compressed and launches data_scan

static decompress(limit: int, data: bytes) → None | bytes

Decompress zstd compressed data

static match(data: bytes | bytearray) → bool

Returns True if data looks like deflated data with zstd

Module contents