CodeVetter · Repo Unpack · Behavior traces
microsoft/markitdown
Three load-bearing flows were traced end-to-end. (1) CLI file conversion: argparse -> MarkItDown() -> convert -> convert_local -> _get_stream_info_guesses -> _convert -> _handle_output. (2) HTTP/MCP conversion: convert_uri -> requests GET -> convert_response -> _convert, with the MCP tool wrapping it in sanitized ToolErrors. (3) Plugin registration + nested zip/docx conversion showing the recursive and registry-driven design.
- CLI local-file flow: main() parses args -> normalizes extension/mime/charset hints into StreamInfo -> constructs MarkItDown(enable_plugins=args.use_plugins) -> markitdown.convert(filename) -> convert_local -> open(path,'rb') -> _get_stream_info_guesses (mimetypes -> magika -> charset_normalizer) -> _convert -> _handle_output writes markdown to -o file or stdout with errors='replace' encoding.
- Inside _convert, converters are tried in (guess, converter) order; each converter's accepts() runs first under a stream-position assertion, then convert() runs inside try/finally that always seeks the stream back to cur_pos — so every converter sees the stream from the same offset.
- HTTP fetch flow: convert_uri('https://...') -> self._requests_session.get(uri, stream=True) -> raise_for_status -> convert_response parses content-type/content-disposition headers -> iter_content buffering -> _get_stream_info_guesses -> _convert; the default session sends 'Accept: text/markdown, text/html;q=0.9, text/plain;q=0.8, */*;q=0.1'.
- MCP tool flow: client calls tools/call convert_to_markdown{uri} -> handler creates MarkItDown(enable_plugins=check_plugins_enabled()) -> convert_uri -> on success returns result.markdown; on failure the except chain converts UnsupportedFormatException/HTTPError/RequestException/FileConversionException/OSError/ValueError into ToolError messages.
- MCP HTTP startup: main() parses --http/--sse/--host/--port -> for non-localhost hosts prints a security warning to stderr -> create_starlette_app mounts mcp.sse_app() and mcp.streamable_http_app(json_response=True, stateless_http=True) routes under a combined lifespan -> uvicorn.run(app, host, port or 3001).
- Plugin activation flow: MarkItDown(enable_plugins=True) -> enable_plugins() -> _load_plugins() iterates entry_points(group='markitdown.plugin'), entry_point.load() with per-plugin try/except+warn -> plugin.register_converters(self, **kwargs) -> each converter lands in self._converters via register_converter(priority).
- ZIP recursion flow: ZipConverter.convert opens zipfile.ZipFile on the stream -> for each entry reads bytes into BytesIO + a fresh StreamInfo(extension, filename) -> calls self._markitdown.convert_stream (the same MarkItDown instance passed at construction) -> concatenates results under '## File:' headings; failures are swallowed per-member.
- DOCX conversion flow: DocxConverter.convert -> dependency check raising MissingDependencyException -> pre_process_docx (rewrites document.xml: OMML oMath -> LaTeX $...$/$...$ w:t replacements) -> mammoth.convert_to_html with merged style_map -> optional _DocxImages adapter when _image_to_html is overridden -> HtmlConverter.convert_string -> _CustomMarkdownify.
- EPUB conversion flow: EpubConverter.convert -> zipfile -> defusedxml.minidom.parse('META-INF/container.xml') -> locate content.opf -> extract dc:* metadata and manifest/spine -> _resolve_manifest_href per spine item -> HtmlConverter.convert on each zip member stream -> metadata block prepended, title set on result.
- Stdin flow: when no filename is given, main() wraps sys.stdin.buffer.read() in io.BytesIO (because Windows pipe-backed stdin can falsely report seekable()) and calls convert_stream with the hint StreamInfo and keep_data_uris flag.
- LLM image-caption flow: ImageConverter.convert -> exiftool_metadata (subprocess exiftool -json -, version-gated >=12.24 for CVE-2021-22204) -> if llm_client+llm_model kwargs present, _get_llm_description base64-encodes the stream into a data URI and calls client.chat.completions.create(model=..., messages=[...image_url...]).