Text splitter integrations - Docs by LangChain

kept by kotsakis oct 28

LangChain documentation describes text splitters for breaking documents into chunks using strategies based on text structure, length, or document structure like Markdown and HTML.

GitHub - baidu/Unlimited-OCR: Unlimited OCR Works: Welcome the Era of One-shot Long-horizon Parsing.
Baidu releases Unlimited-OCR, a VLM-based OCR that processes entire documents in a single pass using reference sliding window attention to avoid memory overflow.
also on #document-processing, #ai-integration
Don't let the LLM speak, just probe it. - by James Padolsey
By extracting an LLM's hidden state at the last prompt token and training a small MLP, you get a fast, calibrated classifier for any English criterion.
also on #natural-language-processing, #developer-tools
[2603.09906] Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
Reasoning helps LLMs recall simple factual knowledge via a computational buffer effect and factual priming, but hallucinated intermediate facts increase final errors.
also on #natural-language-processing
How to really stop your agents from making the same mistakes (via @garrytan)
LangChain's LangSmith testing platform uses trajectory evals and trace-to-dataset pipelines to prevent AI agents from repeating mistakes, despite its high valuation and funding.
also on LangChain, #ai-integration, #software-engineering, #developer-tools
Technical Deflation
Falling AI costs and rising capability create technical deflation, where building software gets cheaper over time, incentivizing startups to delay development.
also on LangChain, #ai-integration, #software-engineering, #developer-tools