u/Ihtien

I am looking for a tool or pipeline to optimize web fetching for local LLM setups. Currently, when fetching a URL or using web search tools, the scraper often injects 10,000+ tokens of raw text into the context window of the main model. Much of this is boilerplate or irrelevant to the actual prompt, which slows down generation and unnecessarily consumes context limits.

​Is there a self-hosted tool or proxy that implements a preprocessing step? For example a workflow could be:

1. ​Fetch the web page content.

2. ​Use a smaller, faster local model (like Qwen 4b) to extract and summarize only the information relevant to the original prompt (or a prompt created by the main model for this specific search).

3.​Pass this condensed context (e.g., 2000-4000 tokens) to the main, larger LLM (like Qwen 27b)

​OpenRouter Chat implements a similar strategy via Exa Highlights, which returns only the most relevant snippets per page and significantly reduces token usage. It uses way less token in the Web Chat compared to the same model (GLM 5.2) in my OpenWebui setup.

Is there anything (self-hosted) that implements that? I mainly use OpenWebui but if there is another frontend that can implement this easier I'm interested as well. My main use-case for my local LLM setup are research requests that require multiple web searches, so I want to speed this up.

Would it be possible to implement a summarizing step (e.g., using the task model) on top of the web fetch content before injecting the result into the context of the main model? Often, web pages are 10,000+ tokens long, but a lot of the content is not relevant to the actual request.

​It would be useful if the fetched content were pre-processed by a faster model, alongside a prompt specifying the relevant information. This could be either the initial user prompt or a prompt generated by the main model specifically for this web search.

​This would significantly shorten the context for the main model and speed up processing time. OpenRouter implements a similar strategy with Exa Highlights. I noticed that OpenRouter Chat uses considerably fewer tokens than the same model (e.g., GLM 5.2) in Open WebUI because Exa Highlights returns only 2000-4000 characters per web page.

​Would this be a feature that can be added easily? Maybe somehow utilizing the new context summary pipeline? Or is there an external (self-hosted) tool that implements a similar strategy that can already be used for fetching websites in Open WebUI?