CodeVetter · Repo Unpack · Testing signals
microsoft/markitdown
Tests are pytest-based and live in packages/*/tests. The core package has ~35 test modules anchored by a declarative FileTestVector harness (_test_vectors.py) asserting must_include/must_not_include substrings across 20+ real fixture files in tests/test_files. Coverage spans stream-info guessing, CLI vectors, per-converter edge cases (PDF tables/memory, docx math/images/styles, pptx images/SVG/notes, xlsx images, RSS titles, Outlook ANSI codepages, charset edge cases) and URI handling; remote-URL tests are skipped under GITHUB_ACTIONS. MCP tests drive the real stdio server subprocess and an in-process Starlette app via httpx.
- packages/markitdown/tests/_test_vectors.py defines FileTestVector(filename, mimetype, charset, url, must_include, must_not_include) and two vector sets (GENERAL_TEST_VECTORS with 16 entries, DATA_URI_TEST_VECTORS) that parametrized tests consume.
- test_module_vectors.py parametrizes every vector across test_guess_stream_info, test_convert_local, test_convert_stream_with_hints and more; tests needing live URLs are gated by skip_remote = bool(os.environ.get('GITHUB_ACTIONS')).
- Fixture corpus is real binary files under packages/markitdown/tests/test_files (docx/xlsx/xls/pptx/pdf/msg/epub/zip/html/csv/json/ipynb/jpg/mp3/m4a/wav and edge cases like random.bin, medical-report scans, borderless-table PDFs).
- Edge-case suites are organized per concern: test_pdf_tables.py (1198 lines), test_cu_converter.py (1035 lines), test_pdf_memory.py, test_pdf_masterformat.py, test_docx_math*.py, test_docx_styles.py, test_pptx_images.py, test_xlsx_images.py, test_outlook_msg_ansi.py, test_charset_sample.py, test_undetectable_charset.py, test_csv_line_endings.py, test_csv_blank_runs.py, test_file_paths.py, test_zip_kwargs.py.
- URI/URL parsing is tested in test_module_misc.py: data: URIs (base64, charset attributes, missing mime), file:// URIs including localhost netloc, percent-encoded Windows drive letters, and case-insensitive schemes.
- MCP tests include test_stdio_protocols.py which spawns `python -m markitdown_mcp` as a subprocess and feeds newline-delimited JSON-RPC covering both the legacy initialize handshake and a 2026-07-28 server/discover request, plus test_http_transports.py exercising create_starlette_app in-process with httpx, and test_tool_errors.py for error mapping.
- markitdown-ocr tests cover each enhanced converter (test_pdf/docx/pptx/xlsx_converter.py), inheritance behavior proving the OCR variants subclass the built-ins, and OCR metadata propagation, with fixtures under tests/ocr_test_data.
- CI matrix: tests.yml runs hatch test for packages/markitdown on Python 3.10-3.14 (ubuntu) plus 3.12 on windows-latest and windows-11-vs2026-arm; separate jobs pip-install local packages and run pytest for markitdown-ocr and markitdown-mcp; PYTHONUTF8=1 is set for captured CLI output.
- Coverage config lives in each pyproject ([tool.coverage.run] branch=true, source_pkgs covering src + tests) and hatch envs define `hatch test` (hatch-test env with openai extra dep) and a types env running `mypy --ignore-missing-imports` over src and tests.
- There are no unit tests for the CLI's Azure paths or plugin loading mechanics in the core package, and remote-URL vectors (YouTube/Wikipedia/Bing) are skipped in CI — those converter paths are only exercised by local HTML fixtures. (inferred)